Patrick Collins

https://www.youtube.com/channel/UCMHHNgcezTnfrJUM1vIiZlQ

Fuzz / Invariant Tests | The New Bare Minimum For Smart Contract Security

patrickalphac-old@newsletter.paragraph.com (Patrick Collins) — Thu, 13 Apr 2023 22:24:12 GMT

What is fuzz testing? What are invariant tests? We introduce how to use these tools in Web3 & Solidity and explain why they are essential, especially for security. Every project should have stateful fuzz tests moving forward, and auditors can use understanding invariants to find critical bugs before code is deployed.

Shout-out to Trail of Bits and Horsefacts for all the fuzzing content.

Introduction

Most of the time, hacks come from scenarios you didn’t think about and write a test for.

What if I told you that you could write one test that would check for almost every possible scenario?

Let’s get froggy.

As always, you can watch my video on this subject and view a full sample repo here.

Basic of Fuzzing

What is a Fuzz Test?

Fuzz Testing or Fuzzing is when you supply random data to your system in an attempt to break it.

For example, if this balloon is our system/code, it would involve doing random stuff to the balloon to break it.

Doing random stuff to a balloon — example of fuzz testing

Now, why would we want to do all that?

Let’s look at an example.

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

contract MyContract {
    uint256 public shouldAlwaysBeZero = 0;

    uint256 private hiddenValue = 0;

    function doStuff(uint256 data) public {
        if (data == 2) {
            shouldAlwaysBeZero = 1;
        }
        if (hiddenValue == 7) {
            shouldAlwaysBeZero = 1;
        }
        hiddenValue = data;
    }
}

Let’s say we have this function named doStuff, which takes an integer as input. We additionally have a variable named shouldAlwaysBeZero that we want always to be zero.

The fact that this variable should always be zero is known as our invariant, or “property of the system that should always hold.”

Invariant: The property of the system that should always hold.

Our invariant (also known as property) in this contract is that:

Invariant: `shouldAlwaysBeZero` MUST always be 0

In our balloon example, if we market our balloon as “indestructible,” our invariant might be that “our balloon should never be able to be popped.”

Invariant: `balloon` should never be popped

In DeFi, a good invariant might be:

A protocol must always be overcollateralized
A user should never be able to withdraw more money than they deposited
There can only be 1 winner of the fair lottery

Example in Foundry

Let’s look at a normal unit test in Foundry.

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

import {MyContract} from "../src/MyContract.sol";
import {Test} from "forge-std/Test.sol";

contract MyContractTest is Test {
    MyContract exampleContract;

    function setUp() public {
        exampleContract = new MyContract();
    }

    function testIsAlwaysZeroUnit() public {
        uint256 data = 0;
        exampleContract.doStuff(data);
        assert(exampleContract.shouldAlwaysBeZero() == 0);
    }
}

With this single-unit test testIsAlwaysZeroUnit, we might think our code has enough coverage, but if we look at the doStufffunction again, we can see that if our input is 2, our variable will not be zero.

function doStuff(uint256 data) public {
        // WHAT IS THIS IF STATEMENT???
        // 👇👇👇👇👇👇
        if (data == 2) {
            shouldAlwaysBeZero = 1;
        }
        // 👆👆👆👆👆👆

        // Ignore this one for now
        if (hiddenValue == 7) {
            shouldAlwaysBeZero = 1;
        }
        hiddenValue = data;
    }

This seems obvious with our example function, but more often than not, you’ll have a function or system that looks like this:

function hellFunc(uint128 numberr) public view onlyOwner returns (uint256) {
        uint256 numberrr = uint256(numberr);
        Int number = Int.wrap(numberrr);
        if (Int.unwrap(number) == 1) {
            if (numbr < 3) {
                return Int.unwrap((Int.wrap(2) - number) * Int.wrap(100) / (number + Int.wrap(2)));
            }
            if (Int.unwrap(number) < 3) {
                return Int.unwrap((Int.wrap(numbr) - number) * Int.wrap(92) / (number + Int.wrap(3)));
            }
            if (Int.unwrap(Int.wrap(Int.unwrap(Int.wrap(Int.unwrap(Int.wrap(1)) / Int.unwrap(Int.wrap(Int.unwrap(Int.wrap(Int.unwrap(Int.wrap(numbr)))))))))) == 9) {
                return 1654;
            }
            return 5 - Int.unwrap(number);
        }
        if (Int.unwrap(number) > 100) {
            _numbaar(Int.unwrap(number));
            uint256 dog = _numbaar(Int.unwrap(number) + 50);
            return (dog + numbr - (numbr / numbir) * numbor) - numbir;
        }
        if (Int.unwrap(number) > 1) {
            if (Int.unwrap(number) < 3) {
                return Int.unwrap((Int.wrap(2) - number) * Int.wrap(100) / (number + Int.wrap(2)));
            }
            if (numbr < 3) {
                return (2 / Int.unwrap(number)) + 100 - (Int.unwrap(number) * 2);
            }
            if (Int.unwrap(number) < 12) {
                if (Int.unwrap(number) > 6) {
                    return Int.unwrap((Int.wrap(2) - number) * Int.wrap(100) / (number + Int.wrap(2)));
                }
            }
            if (Int.unwrap(number) < 154) {
                if (Int.unwrap(number) > 100) {
                    if (Int.unwrap(number) < 120) {
                        return (76 / Int.unwrap(number)) + 100 - Int.unwrap(Int.wrap(uint256(uint256(uint256(uint256(uint256(uint256(uint256(uint256(uint256(uint256(uint256(uint256(numbr))))))))))))) + Int.wrap(uint256(2)));
                    }
                }
                if (Int.unwrap(number) > 95) {
                    return Int.unwrap(Int.wrap((Int.unwrap(number) % 99)) / Int.wrap(1));
                }
                if (Int.unwrap(number) > 88) {
                    return Int.unwrap((Int.wrap((Int.unwrap(number) % 99) + 3)) / Int.wrap(1));
                }
                if (Int.unwrap(number) > 80) {
                    return (Int.unwrap(number) + 19) - (numbr * 10);
                }
                return Int.unwrap(number) + numbr - Int.unwrap(Int.wrap(nunber) / Int.wrap(1));
            }
            if (Int.unwrap(number) < 7654) {
                if (Int.unwrap(number) > 100000) {
                    if (Int.unwrap(number) < 1200000) {
                        return (2 / Int.unwrap(number)) + 100 - (Int.unwrap(number) * 2);
                    }
                }
                if (Int.unwrap(number) > 200) {
                    if (Int.unwrap(number) < 300) {
                        return (2 / Int.unwrap(number)) + Int.unwrap(Int.wrap(100) / (number + Int.wrap(2)));
                    }
                }
            }
        }
        if (Int.unwrap(number) == 0) {
            if (Int.unwrap(number) < 3) {
                return Int.unwrap((Int.wrap(2) - (number * Int.wrap(2))) * Int.wrap(100) / (Int.wrap(Int.unwrap(number)) + Int.wrap(2)));
            }
            if (numbr < 3) {
                return (Int.unwrap(Int.wrap(2) - (number * Int.wrap(3)))) + 100 - (Int.unwrap(number) * 2);
            }
            if (numbr == 10) {
                return Int.unwrap(Int.wrap(10));
            }
            return (236 * 24) / Int.unwrap(Int.wrap(Int.unwrap(Int.wrap(Int.unwrap(Int.wrap(Int.unwrap(number)))))));
        }
        return numbr + nunber - mumber - mumber;
    }

This was one of the Cyfrin Security Challenges.

Here, it’s not quite so obvious if there even is an input that will cause a revert. It would be insane to write a test case for every single possible integer or scenario, so we need a programmatic way to find any outlier.

There are two popular methodologies to find these edge cases programmatically:

Fuzz / Invariant Tests
Formal Verification / Symbolic Execution

We will save “Formal Verification” for another video.

In Foundry, you’d write a solidity fuzz test like so:

    function testIsAlwaysZeroFuzz(uint256 randomData) public {
        // uint256 data = 0; // commented out line
        exampleContract.doStuff(randomData);
        assert(exampleContract.shouldAlwaysBeZero() == 0);
    }

Foundry will automatically input semi-random values to randomData and over x number of runs, input them to thedoStuff and check that the assertion holds.

This would be equivalent to writing many tests where randomData had different values, all in one test!

Now I say “semi-random” because the way your fuzzer (in our case, foundry) picks the random data isn’t truly random, and should be somewhat intelligent with the random numbers it picks. Foundry is smart enough to see the if data == 2 conditional, and pick 2 as an input.

Echidna — The real logo

At the moment, I think the trail of bits hybrid echidna is the best fuzzer out there due to its intelligent random number selection, but Foundry’s fuzzer (in my opinion) is easier to write code for at the moment. The echidna logo is also the best logo I’ve ever seen. Even better than the ripped Jesus logo.

Anyways, so if we run our fuzz test, it tells us exactly what input fails our test:

$ forge test -m testIsAlwaysZeroFuzz

Failing tests:
Encountered 1 failing test in test/MyContractTest.t.sol:MyContractTest
[FAIL. Reason: Assertion violated Counterexample: calldata=0x47fb53d00000000000000000000000000000000000000000000000000000000000000002, args=[2]] testIsAlwaysZeroFuzz(uint256) (runs: 6, μ: 27070, ~: 30387)

We can see that it found out if it passed args=[2] to the test, it was able to break our assert(exampleContract.shouldAlwaysBeZero() == 0). So now, we can go back into our code, and realize we need to fix the edge case where data == 2, and now we are safe from the exploit of input data being 2!

Basics of Fuzzing Summary

In summary, to write a fuzz test, we did the following

We understood our **invariant **or “property of our system that must always hold”

2. We wrote a test that would input random values into our function to try to break our invariant

Stateful vs Stateless Fuzzing

Stateless Fuzzing

Now you may notice that there is another scenario where our code could have an issue, and that’s when hiddenValue == 7. In order for this revert to happen, you have to first set hiddenValue to 7, by calling doStuff with the value 7 which sets hiddenValue to 7 and then call this function again.

uint256 hiddenValue = 0;

function doStuff(uint256 data) public {
        // Fixed this part by removing it
        // if (data == 2) {
        //     shouldAlwaysBeZero = 1;
        // }

        // Wait what's this??
        // 👇👇👇👇👇👇👇
        if (hiddenValue == 7) {
            shouldAlwaysBeZero = 1;
        }
        // 👆👆👆👆👆👆👆
        hiddenValue = data;
    }

It takes 2 calls for our invariant to be broken.

Call doStuff with 7
Call doStuff with any other number

Our fuzz test written above will never be able to find this example because as it’s currently written, our test is what’s known as a “stateless fuzz test.” Which is where the state of a previous run is discarded for the next run.

Stateless Fuzzing: Fuzzing/Fuzz Testing where the state of a previous run is discarded for the next run.

An example of 2 stateless fuzz runs

If we go back to the balloon example, stateless fuzzing would be similar to doing something to balloon A for one random attempt to break it, then blowing up a new balloon B and attempting to break it differently.

In the balloon example, you’d never try to break a balloon you already tried to break in the past. This seems a little silly as if our balloon invariant is that “the balloon can’t be popped” we’d want to make multiple attempts on the same balloon.

Stateful Fuzzing

So, in software engineering, we could do “stateful fuzzing” instead. Stateful fuzzing is where the state of our previous run is the starting state of our next run.

Stateful Fuzzing: The state of our previous fuzz run is the starting state of our next fuzz run.

An example of 1 stateful fuzz run, where multiple attempts are made in a run

A single stateful fuzz run would be similar to writing a test with all your assets in the same test.

function testIsAlwaysZeroUnitManyCalls() public {
        uint256 data = 7;
        exampleContract.doStuff(data);
        assert(exampleContract.shouldAlwaysBeZero() == 0);

        data = 0;
        exampleContract.doStuff(data);
        assert(exampleContract.shouldAlwaysBeZero() == 0); // this would fail
    }

This is bad practice btw, don’t write your asserts in the same test, please.

To write a stateful fuzz test in Foundry, you’d use the invariant keyword, and it requires a little more setup.

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

import {MyContract} from "../src/MyContract.sol";
import {Test} from "forge-std/Test.sol";
import {StdInvariant} from "forge-std/StdInvariant.sol";

contract MyContractTest is StdInvariant, Test {
    MyContract exampleContract;

    function setUp() public {
        exampleContract = new MyContract();
        targetContract(address(exampleContract));
    }

    function invariant_testAlwaysReturnsZero() public {
        assert(exampleContract.shouldAlwaysBeZero() == 0);
    }
}

Instead of just passing random data to function calls, a stateful fuzz test (invariant) test will automatically call random functions with random data.

We use the targetContract function to tell Foundry that it can use any of the functions in exampleContract. There is just one function for this example, so it will just call doStuff with different values.

If we run this test, we can see the output as such, and we can see it finds out that if you call doStuff twice (once with the value 7), it will throw an error!

$ forge test -m invariant_testAlwaysReturnsZero

Failing tests:
Encountered 1 failing test in test/MyContractTest.t.sol:MyContractTest
[FAIL. Reason: Assertion violated]
        [Sequence]
                sender=0x000000000000000000000000000000000000018f addr=[src/MyContract.sol:MyContract]0x5615deb798bb3e4dfa0139dfa1b3d433cc23b72f calldata=doStuff(uint256), args=[7]
                sender=0x0000000008ba49893f3f5ba10c99ef3a4209b646 addr=[src/MyContract.sol:MyContract]0x5615deb798bb3e4dfa0139dfa1b3d433cc23b72f calldata=doStuff(uint256), args=[2390]

Wait, what’s an invariant again?

Now, important aside on how Foundry uses the term invariant. As we’ve described, an invariant is a property of the system that must always hold, but Foundry uses the term to mean “stateful fuzzing.” Just keep this in mind.

Foundry Invariant Tests == Stateful Fuzzing
Foundry Fuzz Tests == Stateless Fuzzing
Invariants == Property of the system that must always hold

So in an actual smart contract, your invariant won’t be that a balloon shouldn’t pop or some function should always be zero; it’ll be something like:

New tokens minted < inflation rate
There should only be 1 winner of a random lottery
Someone shouldn’t be able to take more money out of the protocol than they put in

And let me tell you what, at this point, you now know all the basics of Fuzzing! Congratulations! Maybe now you take a break and try writing some tests yourself, and then come back to the video.

This is the new bare minimum

This is the new floor for security in web3. It’s systematic to do, anyone can learn how to do it, and it can save a LOT of headaches.

Understand your invariants
Write stateful fuzz tests for them
Don’t go to an audit before.
If you do, make sure your auditors help you understand your invariants!

Now you understand the basics of fuzzing & invariant tests, you can go use the tools you like! To learn more about advanced stateful fuzz testing, be sure to stay tuned, we have an advanced video coming up soon! Additionally, read the Foundry documentation on the Handler method, as that’s the recommended way to build the most sophisticated stateful fuzz tests.

This article from Horsefacts also does an amazing walkthrough.

Have Fun!

😸😸Follow Patrick!😸😸

Book a smart contract audit: Cyfrin

https://www.youtube.com/channel/UCMHHNgcezTnfrJUM1vIiZlQ

Smart Contract Audit | What it is, and what to expect

patrickalphac-old@newsletter.paragraph.com (Patrick Collins) — Fri, 07 Apr 2023 14:06:15 GMT

A smart contract audit is a time-boxed security-based code review on your smart contract/web3 system. An auditor’s goal is to find as many vulnerabilities as possible and educate the client on ways to improve the security of their codebase moving forward.

What is a Smart Contract Audit?

A smart contract audit is a time-boxed security-based code review on your smart contract/web3 system. An auditor’s goal is to find as many vulnerabilities as possible and educate the client on ways to improve the security of their codebase moving forward.

Auditors use a combination of manual and automated tools to find these vulnerabilities.

Why are smart contract audits so important?

According to a research study by Chainalysis, 2022 was the year the most value was stolen from smart contracts.

Due to the immutability of the blockchain, once a smart contract is deployed, you can’t change it, so you’d better get it right. The blockchain is an adversarial environment, and your protocol needs to be prepared for malicious users.

But even more than just saving your protocol from hacks, an audit can improve your developer’s understanding of code, improving their speed and effectiveness of features moving forward.

Additionally, there is an entire website dedicated to how many hacks happen, and we need to do our best to prevent that list from growing as a community.

Smart Contract Audit Benefits

Find vulnerabilities
Level-up developers
Teach best practices and the most modern tooling

Often, one audit isn’t enough. Many protocols go on a security journey that includes multiple audits and services like:

Formal Verification
Competitive Audits
Bug Bounty Programs

We’ll break these down in a future post.

Smart Contract Auditors

There are a lot of companies that offer smart contract auditing services, like

You can see a more detailed list of the top 7 smart contract auditors here.

Additionally, there are a lot of independent auditors that do great work as well.

Smart Contract Audit Process

A typical audit looks like this:

1. Price & Timeline

A protocol can reach out before or after their code is finished. Ideally, they reach out some time before so the auditor can have enough time to schedule them.

Once they reach out, the teams will discuss how long the audit will take based on the scope and code complexity.

How long the audit will take depends on how many lines of code.
A very rough approximation of how long an audit takes depending on how many source lines of code you have can be found here:

The duration sets the price, and prices can range widely based on the different auditors at the time of writing. At the time of recording, a one-week-long audit can go anywhere from

Duration: Price

- 1 week: $5,000 - $100,000

The range is massive, but I have seen pricing anywhere in this range.

2. Commit Hash, Down payment, Start Date

Auditors need to know exactly what code they are auditing and use the commit hash of your repo to do so.
Once you have a commit hash, you can reach a start date and finalize the price.

3. Audit Begins

Then the audit starts! Your auditors will use every tool in their arsenal to find vulnerabilities in your code.

4. Initial Report

After the period ends, the auditors will give you an initial report that looks like this.

All their findings are listed by severity, usually in formatted into:

High
Medium
Low
Informational / Non-critical / Gas

High, medium, and low represent the severity of the impact and likelihood of each vulnerability.
Informational, Gas, and Non-Critical are findings to improve the efficiency of your code, code structure, and best practice improvement suggestions are not vulnerabilities but ways to improve your code.

5. Mitigation Begins

The protocol’s team will then have an agreed-upon time to fix the vulnerabilities found in the initial audit report. Sometimes, depending on the severity of the findings, this may be long but is often much shorter than the audit itself.

6. Final Report

After the protocol makes the changes, the audit team will do a final audit report exclusively on the fixes made to address the issues brought up in the initial report.

And then, hopefully, the auditors and protocols have had a great experience and will work together to stay secure in the future!

How to get the most out of a smart contract audit

To get the most out of an audit, you should:

Have clear documentation

Robust test suite
Ideally, including fuzz tests
Code should be commented & readable
Modern best practices followed
Communication channel between developer and auditors
Do an initial video walkthrough of the code

However, the most important part of the process is during the audit.

You want to think of you and your auditor as a team to get the best results out of your audit. One of the best ways to do this is to have a dedicated channel where auditors can ask questions to the developers.

Additionally, the more context, documentation, and information they can read, the better. Be sure it’s easy for anyone to walk through your code and understand what it’s supposed to do.

80% of all bugs are due to business logic issues, so the auditors need to understand what the protocol should do more than they should understand the actual code!

Breakdown of bugs between machine auditable and unauditable

Having a modern test suite & tooling can also make it so auditors spend less time fidgeting with your tooling and more time finding issues.

A high-level video walkthrough of your code should be the first thing you and the auditors do together.

After the Audit

We highly encourage you to act on the recommendations of an audit report, we’ve seen too many protocols not take warnings seriously, and that be the attack vector that gets exploited.

Additionally, if you change your codebase, that is now unaudited code, and should not be pushed, no matter how small the change may be. If you change your code, consider getting that piece of code audited.

And often, depending on how much money your protocol will secure, you should consider getting another audit anyway!

What an audit isn’t

One audit does not mean your code is bug-free.

Now here is the thing, an audit does not mean your code is bug-free. It’s a security journey where your team should level up on security.

No matter how experienced an auditor or audit firm is, people at all levels of experience will miss something. On the sad day that happens, get together on an emergency communication channel with your auditors and figure out to remedy the situation quickly.

Insurance is often a good idea for even the most audited protocols.

So with that, now you have a good idea of the smart contract audit process end to end and what to expect. A smart contract audit is more of a security journey between the protocol and the auditors, and having a security-focused mindset doesn’t end even after the audit.

If you’re looking for an audit, be sure to contact the Cyfrin team.

And as always, stay safe out there!

😸😸Follow Patrick!😸😸

Book a smart contract audit: Cyfrin

Twitch Stream Uploads & Shorts

*New to trading? Try crypto trading bots or copy trading on *best crypto exchanges

Join Coinmonks Telegram Channel and Youtube Channel get daily Crypto News

Also, Read

Free Crypto Signals | Crypto Trading Bots
An ultimate guide to Leveraged Token
16 Best Folding Electric Bikes
28 Best Electric Bikes Review
Top 3 Binance Futures Trading Bots

Top 7 Smart Contract Auditors - Patrick Collins - Medium

patrickalphac-old@newsletter.paragraph.com (Patrick Collins) — Fri, 07 Apr 2023 03:16:04 GMT

We look across the industry at the top smart contract auditors in Web3. Smart contract audits are the lifeblood of the Web3 security space and are crucial to ensuring we stay safe.

I am a co-founder and security researcher from Cyfrin, where we do smart contract audits. This is my opinionated take on top smart contract audit firms along with Cyfrin. The only way Web3 scales is if we increase our security, so I want to ensure that other top firms get the exposure they deserve. Auditors left off this list are not necessarily poor choices, I may have just missed them.

What is a Smart Contract Audit

A smart contract audit is a time-boxed security-based code review on your smart contract / Web3 system. An auditor’s goal is to find as many vulnerabilities as possible and educate the client on ways to improve the security of their codebase moving forward.

Auditors use a combination of manual and automated tools to find these vulnerabilities.

However, a smart contract audit does not guarantee your code is bug-free. At the end of the day, your protocol's security is yours and your auditor's responsibility, and sometimes multiple audits or another security tooling might be needed.

A professional audit group can and will give you all the guidance you need to move forward on your security journey so you can feel confident deploying.

Why We Need Smart Contract Audits

Image from Chainalysis

The image above represents the total dollar amount stolen from DeFi for the entire year of 2022, 80% of that value was exclusively in DeFi. According to DeFillama, the current locked value of all DeFi is 50 billion dollars.

This means that over 6% of all DeFi value has been subject to a hack!

If we want blockchain and DeFi to come to the masses, they cannot be worried that there is a 6% chance they will wake up to find their funds exploited. We have to do better than this.

This is why audits are so important:

Keep your protocol safe
Help Web3 gain authority
Educate your team on best practices

That last bullet is what I call the **hidden benefit of smart contract audits. **Getting an audit and working with a group of security and smart contract experts can improve your team's skill in developing protocols. They should give you feedback on vulnerabilities and educate your team on improving. They are security and developer experts, after all!

How to Choose a Smart Contract Auditor

Understand the auditor skill set
Understand your price point
Understand their methodology

Understand the auditor skill set

You want to ensure the auditor has done work in the domain you’re looking for an audit on. If you want an audit on Solana and you work with an EVM specialist group, you’re going to have a bad time.

Maybe you want a DeFi audit, and you’re working with a firm that only understands NFTs; this is also a recipe for disaster.

You having a bad time if you don’t know their skill set

An easy way to see what kinds of projects an auditor can do is to look at a list of their previous audits. Most auditors will have some publicly displayed audits as a showcase of work, and you can decide based on their past if they are right for you.

Understand your price point

Large firms will often cost more than smaller firms and independent auditors. Typically, you can find “cheaper” audits with smaller companies and independent auditors, but the quality can vary.

I’ve seen a **ton **of fantastic solo auditors, though, so don’t discount a solo auditor who wants to audit your protocol.

Pricing varies wildly depending on the auditors, so understand the next point to get the best idea for a price point.

Understand their methodology

Security researcher Tincho walks through his audit methodology

Before locking down any payment, you need to verify the following:

Exactly which auditor(s) will conduct the audit
What tools will they use (fuzzing, formal verification, etc)
If they want a communication channel between your developers and the auditors

80% of hacks are not machine auditable and often come from poor implementation of business logic. Because of this, auditors need as much context as possible in the form of:

Documentation
Q&A Channel with Developers
Code Natspec

If an auditor doesn’t tell you who is doing the audit, what tools they are using, or doesn’t want to have a direct line of communication between your protocol devs and the auditors, there is a good chance they are going to do shoddy work.

Additionally, I think auditors should try to improve your test suite with fuzzing/property/invariant tests, but it’s not required.

How to Prepare for a Smart Contract Audit

How to prepare for a smart contract audit

Imagine two random developers drop 5,000 lines of code you’ve never seen on your desk and tell you that the code needs to be cleaned up and spotless in two weeks.

Developer A tells you what the code should do, it has a test suite, and they say, “feel free to ask me any questions you may have!”
Developer B says, “Don’t talk to me till it’s done.”

Your review of developer A’s code will be 100 times better than developer B. Be like developer A.

To get the most out of an audit, you should:

Have clear documentation
Robust test suite (Ideally, including fuzz tests)
Code should be commented & readable
Modern best practices followed
A communication channel between developers and auditors
You are prepared to do an initial video walkthrough of your code

Additionally, the more context, documentation, and information they can read, the better. Be sure it’s easy for anyone to walk through your code and understand what it’s supposed to do. 80% of all bugs are due to business logic issues, so the auditors need to understand what the protocol should do more than they should understand the actual code!

You can learn more about the smart contract auditing process here.

Top Smart Contract Auditors

Finally!

I wanted to make sure you had all this context before giving you this list because I’ve seen protocols that want an audit for one of two reasons:

Marketing
Security & Marketing

Be like #2. Getting “an” audit and treating all audits the same can be tempting but should **not **be done. You want to treat an audit like you, and your auditor are teaming up to secure your code.

Here it is, my list of top 7 auditors in no particular order, and why I think they are a top auditor.

Cyfrin

https://www.cyfrin.io/#hero

Please keep in mind I am the co-founder of Cyfrin.

As someone who wants to see the success of Web3, I was furious with the state of security in Web3. $3.8B lost in 2022 is a horrifying statistic, so I felt compelled to jump in and help secure DeFi and blockchain.

The Cyfrin team has some of the top engineers and auditors in the space, like:

Hans | #1 Ranked Auditor as of Writing on Code4rena
Alex | Ex-Chainlink Labs Engineer in charge of $5B+ DeFi integrations
0Kage | Code4rena Top Finisher and Experienced FinTech Engineer
Carlos | Code4rena Top Finisher & Expert Solidity Engineer
Gio | Expert Solidity Engineer
Patrick | Most Watched Solidity Education Video(s) of All Time

We thrive on finding as many bugs as possible and finding ways to improve your codebase and test suite.

Web3 security needs a new narrative, and we are excited to push the security space forward. We are a smaller group at the time of writing as we only launched 2 months ago!

You can find a list of notable audits (and skillsets) for Cyfrin here, including the Beanstalk Wells integration and LinkPool.

Trail Of Bits

https://www.trailofbits.com/

I *always *tell people to check out Trail of Bits. They are one of the firms in Web3 security consistently pushing the bar in a practical sense. They don’t just give an audit, they give you all the tools you need to be successful in smart contract security as well.

The Trail of Bits team builds some of the most popular and widely used tools like:

And so many more. They are dedicated to educating the Web3 space as well with tons of free educational content and blogs.

Interview I did with Head of Blockchain Engineering Josselin

Trail of Bits is a large group consistently rated one of the top firms in Web3 for good reason, and I’d definitely classify myself as a fanboy.

You can find a list of notable audits (and skillsets) for Trail Of Bits here, like Uniswap, Yearn, and Compound.

OpenZeppelin

https://www.openzeppelin.com/

OpenZeppelin is another group that constantly pushes the envelope by raising the state of Web3, which is why I’m a massive fan of their work. The OpenZeppelin Contracts is the standard library for solidity that 95% of the rest of Web3 uses and trusts to build their smart contracts.

You should hold onto every report you read from the OpenZeppelin team like gold, as the information they give is some of the best in the business, and their team is constantly raising the bar for security.

OpenZeppelin is a large group used by some of the top protocols in the space, like Aave, Optimism, and Compound.

I really can’t speak highly enough about the skills of this team.

You can find a list of notable audits (and skillsets) for OpenZeppelin here.

https://consensys.net/diligence/

Part of the Consensys team, one of the most well-known groups in Web3 behind projects like Metamask, Infura, and Truffle, their security team is also first-class. They are a large group with a great track record.

The Diligence team is another team that values powerful fuzzing and recently came out with a fuzzer-as-a-service product. To me, this signals that they not only understand security, but they understand trying to scale security throughout all of Web3. You can tell when a group cares when they make tooling & educational material that makes your life better instead of hoarding it all for themselves.

They additionally have formal verification tooling (similar to Trail of Bits) if you want to go the extra mile.

You can see a list of Consensys Diligence audits here, including Aragon, RocketPool, and Fei.

https://spearbit.com/

Spearbit is a decentralized network of security experts that shakes the game up.

Unlike traditional auditing firms, which employ teams of full-time security researchers, Spearbit sources top talent from everywhere in the Web3 ecosystem to assemble the best possible team.

Now you might be thinking, “wait, wouldn’t the quality vary if they have different auditors on different projects?”- however, this hasn’t stopped them from consistently being one of the best in the business.

SpearbitDAO proves the decentralization ethos works, as many top auditors and researchers go solo — so periodically combining them into one group makes them all the better!

You can see a list of SpearbitDAO audits here, including SudoSwap, LooksRare, and ArtGobblers.

https://dedaub.com/

A lesser-known group, I’ve only seen the Dedaub team ship amazing reports, and it was a little confusing to me why so few people know about them.

They are another team that ships more than just security audits, with coding libraries and helpful alpha on social media.

As an ex-Chainlink engineer myself (ex-DevRel technically), I’ve witnessed the good this team can do on an audit.

You can see a list of projects they’ve worked with, including Chainlink, Liquity, and Blur.

https://www.trustindistrust.com/

Trust is a solo auditor consistently at the top of the competitive audit leaderboards and has done fantastic work educating all of Web3. I especially wanted to highlight him to say you don’t always need to go with a firm! Solo auditors can often be cheaper, with as much skill or more than a massive firm.

He has an auditor course, consistently gives beautiful write-ups, and has made a massive impact in keeping Web3 safe by himself!

I had the pleasure of interviewing him, and he gave me all the tips and tricks one would need to move forward and be a successful security engineer in Web3.

Interview with Trust (Or)

You can see a list of Trust audits here, including The Graph and Vagabond.

I wanted to keep the list short because the algorithm likes it like that, but here are some more firms and solo auditors that do a fantastic job. If you think I’m missing any, let me know, and I’ll do an assessment.

Firms

Solo

🐸🐸Follow Patrick!🐸🐸

Book a smart contract audit: Cyfrin

https://www.linkedin.com/in/patrickalphac/

“Damn Vulnerable DeFi” Creator Teaches You How To Audit

patrickalphac-old@newsletter.paragraph.com (Patrick Collins) — Mon, 20 Mar 2023 01:29:36 GMT

We meet up with Tincho, Damn Vulnerable DeFi creator and previous Openzeppelin lead auditor, on what his exact audit process looks like end-to-end. We do a mock audit of ENS to show you how to do it.

We had the pleasure of interviewing Tincho on his **exact **audit process he uses to make web3 more secure.

You can watch the summary video here:

Additionally, if you’d like to watch the full interview, you can find that on my YouTube as well.

Tincho broke down his process for us to get some context. However, one thing he was adamant about was the following:

There is no silver bullet for auditing.

This was something that was repeated throughout our interview. Every auditor is going to learn to approach problems differently, and his methodology might not be the same as one you might invoke. Find what works for you, and go to town!

But here is the summary of how Tincho approaches his audits. We did a mock audit of the ENS protocol to showcase precisely the steps Tincho takes to do an audit.

1. Get Context

Read the documentation.

Read. The. Docs.

Before he even starts looking at code, he makes sure to get an understanding of what the protocol is supposed to do. 80% of all bugs and exploits come from business logic exploits, so if you don’t understand the business logic of the code, it will be impossible to find exploits!

He may also read reports on previous audits, get familiar with similar protocols, etc. Anything to prime himself for the security journey he is about to take.

2. Bring your tools

Tools like: Foundry, Hardhat, Brownie, Apeworx, Remix, Truffle

The next step he takes is to download the code to his local environment and bring his tools. For working with ENS, Tincho git-cloned the GitHub repo and then created a foundry project. ENS is a hardhat-based project, but Tincho has been working with Foundry recently due to its speed, so he spun up a small Foundry project where he would run little tests against the repo.

3. Get your scope

The next step Tincho takes is to use either cloc or solidity metrics to get the scope of the project he is going to audit, and place them into a sheets/excel folder organized by size.

Example Scoping Phase

On the right, he will make a status column so he can keep track of what he has audited. He usually starts from the least complex or the “little legos” and works his way up. Understanding the smaller pieces gives him context on how everything fits together when he gets to the more complex contracts.

Additionally, if your audit calls for only reading specific contracts, you may remove them from your sheet, so you know what you shouldn’t spend your time on. A file like this is especially important when working with teams, as you can know what everyone is working on.

4. Start taking notes on/in the code

Now that you know what the code is doing, you should constantly be thinking, “how can I break this?” or “does this do what it should be doing?”. Tincho will leave comments right in the code to let him know he was there.

The `// e` comments are examples of leaving notes for yourself in the future

Additionally, he has a file that he uses just to dump ideas, thoughts, and keep track of any issues/findings he has.

5. Remember, audits are time-boxed

It’s very easy to fall deep down the rabbit hole on something, and you can always re-read a line of code and re-validate something works the way it can. But you’ll need to remember to come up for air.

6. Keep a line of communication with the developers

Maybe you should have 2 phones too

Developers have worked on this code for much longer than you have and you ever will have. They have more context than you ever will have. Use the developers who wrote the code to your advantage and ask them questions.

Additionally, if you’re a developer and you’re getting an audit, remember, ultimately, you are responsible for your code, so making yourself available to the auditor will make their lives much easier.

7. Audit a lot

The best way to start having a “feel” for vulnerabilities is to do a lot of audits, read audit reports, read post-mortems, and digest as much security-related information as you can.

Every auditor on the planet, at some point, will miss something. Don’t let that stop you. Always focus on doing the best job that you can, and never become content with your skill level.

Always be growing and learning.

As they say:Repetition is the mother of skill.

Some of my favorite resources:

8. Take a lot of time to write your reports

If you find a million vulnerabilities, but you don’t communicate them effectively, you essentially didn’t find any vulnerabilities.

An audit is more than just the report; you are responsible for keeping your client safe and teaching them how to be more secure with their code. So make sure you take the time to write reports that articulate issues so that they can learn.

9. Remember, an audit is more than the report

Leaderboard of rekt.news

I finished the call by asking, “What happens if you audit a contract, and the protocol gets hacked, and you end up on rekt?”.

A question a lot of auditors have that makes them nervous. He responded by saying:

An audit is more than whether or not you find all the bugs
An audit should provide value regardless
Every auditor on the planet who is worth their salt doesn't have a perfect track record

When auditing, you should be not only finding bugs but:

Teaching best practices
Helping them understand attack vectors
Showing effective testing practices

It’s your job to level up the client as a group **and **find bugs. And no matter how amazing of an auditor you are, sometimes a protocol will get hacked. You need to take accountability, learn why you made this mistake, and learn and grow from it. Ideally, this doesn’t happen, but when it does, do your best to help the client with any mitigation.

Summary

There is no one silver bullet to auditing, but hopefully, seeing behind the curtain gives you more context as to what goes on in an audit and how you can start your journey!

😸😸Follow Patrick!😸😸

Book a smart contract audit: Cyfrin