gm! It’s Derek & welcome to my blog where I write about random topics that interest me and share some personal experiences that I’ve had the pleasure to be a part of.

This post will be Part 2 of a series I intend to use to cover the unique experience and story behind each ETHGlobal Hackathon I participate in. Note that ETHGlobal Waterloo took place almost exactly a month before ETHGlobal Paris and I’m publishing this retrospectively after having had some down time to reflect!

Waterloo would be the first web3 hackathon I participated in outside of the US and also our first for the 2023 year. Waterloo offered us a chance to step outside our comfort zones, experiment a bit more, and leverage our learnings from previous hackathons to build something more bold and new.

Before the hackathon, we studied the sponsors and decided on building something in service of the security space. And to make our lives even harder, we also wanted to hop on the LLM bandwagon to add sprinkle some AI into our project. To be super clear: nobody on our team had any experience in web3 security or AI/ML. And so, you might now be asking: Why? The answer: Why the hell not? We didn’t need any other reason, really. Hackathons are a place to learn about new tech stacks and push the boundaries on what is possible.

We had no idea what we were getting ourselves into…

To answer this question, I did what I would do at my day job: find users in the space and conduct discovery interviews to uncover pain points in the space we could build against.

A few days before the event, I got a hold of 3 potential users: 2 of which were freelance dApp developers and 1 was a seasoned auditor from Trail of Bits. While this wasn’t a huge sample pool, their opinions and context were precisely what I was looking for. Looking back, I’m genuinely surprised I was able to pull this off on such short notice!

For a dApp developer

In the interviews, I learned that dApp developers spend the majority of their time implementing the business logic for their application rather than on the security side of things. This wasn’t surprising, as the industry norm is to hire external security experts to help with things like an audit for a smart contract. What was surprising, however, was their answer for why they (the dApp developers) did not do more security-related checks and tests themselves:

“I would love to do some of the auditing myself as it would save me time and money, but it’s often quite difficult to: keep up with the latest security best practices, learn how to set up and use various auditing tools, and then finally apply those tools to secure my dApp all by myself.”

Insight #1: dApp developers were actually quite incentivized and willing to do more security-related workflows in their dev loops! However, the technical and knowledge gap required was simply too large for most.

For a smart contract auditor

When talking to the auditor from Trail of Bits, I learned that they would repeat the same tasks and workflows across multiple clients when auditing dApps: setting up the same fuzzing frameworks, running the same vulnerability checkers, and using the same symbolic execution tools over and over again to perform audits. When asked how those tools and frameworks performed and if they had any complaints about those products, the auditor said the same thing that the dApp developers said:

“The learning curve for wielding auditing tools is very steep and the knowledge barrier is quite high. It takes a while to get familiar with it all and, even then, you need to keep up-to-date with the trends since there’s always something new or changing: new protocol upgrades, new chains, new technologies, etc”

Insight #2: Auditors also struggle from the high technical and knowledge barrier that dApp developers face when it comes to securing dApps. These auditors are also often doing the same operations over and over again, spending precious time on set up and redundant tasks instead of doing what they’re paid to do: find vulnerabilities.

Eureka!

These insights, coupled with the alarming fact that more than half of the $3.6B USD lost due to hacks and exploits in 2022 were from audited protocols, led me to decide that I would build something to improve the developer experience of using auditing tools. Since this was a hackathon, I narrowed down our scope to focus solely on dApp developers as our target user base with the thesis that enabling more developers to build with security best practices and tools benefits the entire ecosystem and industry in a net-positive way.

The problem statement for our project would be: how do we make it easier for dApp developers to incorporate a security-first practices earlier in their developer loops?

Our answer was A(i)udit (pronounced A-I-audit): a CLI that took natural language inputs to set up, deploy, and execute smart contract auditing tools and frameworks for an end user.

I invite you to read all the details of my project on Github or the project showcase page, but TLDR: we built a CLI that uses ChatGPT to process natural language intents into actionable proposals that the LangChain framework of AI agents can execute on. Once the AI agent knows what to do, with a high probability, it will pick the tool required and wield it to perform the operation. We also wrote the tools that we wanted the agent to use (see full list).

With A(i)udit, a developer can describe in natural language precisely what they wanted to do - all without the hassle or overhead of learning how to set up the infrastructure and doing it themselves. The only limit was how well the tool would work and how often the AI agent would pick the right tool (the challenges that arose with the latter part here is in the realm of Prompt Engineering that we didn’t dabble too much in). More often than not, A(i)udit was able to perform the ask with only a single prompt! The CLI would print out its thought process, chain of commands, and results from the operation.

After a crazy weekend on less than 10 hours of sleep, we woke up bright and early on demo day and submitted our project at 8:58 am. I still remember the stress that morning waiting for the demo video to upload.

We took a quick breakfast break before demo-ing to all the sponsors and judges we had applied for. While far from perfect, we were very proud of our proof-of-concept to showcase how AI can be used to augment and complement traditional developer workflows. We were able to successfully showcase 3 operations end-to-end:

• Ask ChatGPT to write a smart contract (using some reference implementations), compile, deploy it to a supported chain, and print the deployed address & transaction hash.

• Ask ChatGPT to run a Mythril or Slither test on a smart contract, export the findings to IPFS, and print the results/findings locally.

• Ask ChatGPT to write unit tests (in javascript) or fuzzing cases (in solidity) for your smart contracts.

I will be the first to admit that using LLMs to do these sorts of operations is incredibly flaky and often requires a sanity check from a pair of human eyes (especially when it comes to writing tests). Nonetheless, we wrapped up our demos and headed back to our hotels for a quick lunch and afternoon nap.

While waiting for our food at a nearby poké restaurant - I got an email with the subject: ETHGlobal Finalist - a(i)udit. I literally jumped and celebrated at the restaurant! We grabbed our food and assembled back at the venue to prepare for our demo on stage. Jolted back to life from pure excitement, we inhaled our food and got on stage to present at the closing ceremony. Watch the full presentation here!

In the end, we took home: $3000 for being a ETHGlobal Finalist, $3000 from from Gnosis Chain for the Best AI project, $118 from Polygon for deploying, and $1000 from Filecoin!

This was my first time winning this much prize money and the first time I had placed as a Finalist! I was overjoyed but also genuinely surprised and exhausted. So many emotions and thoughts. Oh, and did I mention that the weekend of the hackathon was also my birthday? This was indeed the best surprise ever.

Thanks for reading this far - this was my longest blog post yet. I prioritized sharing the thought process and motivations behind our hackathon-winning project above the technical implementation and my personal experience/feelings throughout this whole event. Please let me know what kind of content you’d prefer next time on my blog :)