Ours is an age of both data abundance and storage solution scarcity. To store what matters, we must rely on centralized storage options that leave us defenseless against corporate whim and folly. Decentralized, secure, censorship-resistant storage is a necessity in the Web3 era to protect and harness the wealth of human knowledge.

The Banyan Design

Banyan designed a decentralized IPFS pinning service. Our design would match people and the files they want to store with storage providers hosting over IPFS, a distributed file storage system. Programmable financial incentives motivate providers to honor storage deals, and mathematical proofs verify that a provider is actually continuously storing your file.

For a given storage deal, according to our design, storage providers would submit merkle inclusion proofs to Ethereum logs, and oracles execute fast rust binaries to read and verify the proofs against the deal’s Blake3 on-chain root hash. Our decentralized oracles aggregate the results, and report to chain the proof-success rate of the provider. Payments are transferred according to the proof-success rate and deal parameters the counterparties agree to, and your data remains secure, uncensorable, and decentralized.

Verifying the merkle inclusion proofs is the computational backbone of this architecture. Why did we decide to perform this computation off-chain using Chainlink oracle computation instead of just doing it on-chain?

The Cost of Computation

For some operations which are computationally intensive, Ethereum has precompiled contracts. If you want to do a SHA256 hash, the operation is executed by the Ethereum node itself instead of the EVM (Ethereum Virtual Machine), so there is no overhead gas fee. This theoretically allows you to perform all your computation on-chain. Unfortunately, there are only nine precompiled contracts on Ethereum right now, and Blake3 isn’t one of them yet. To verify our Blake3 merkle proof on chain, we would need to perform a large number of Blake3 hashes using a custom Blake3 solidity contract, which could cost hundreds of dollars per proof.

A merkle proof requires hashing a block of bytes, and then successively hashing hashes together until you work your way up to the tree root. For a 1000kb file, this means one hash on a 1024 byte chunk at a price of $1000, followed by log(1000) hashes of hashes at a price of $150. This comes out to approximately $2500 per proof.

For verifying Blake3 merkle proofs and all sorts of other meaningful computational tasks that allow smart contracts to interact with real world data, regular on-chain computation is simply not financially feasible. It can also be dangerous. Engineers trying to reimplement cryptographic primitives like Blake3 is a common cause of errors that lead to hacks.

It is unlikely that Ethereum core maintainers add a precompile for your cryptographic operation of choice, given the burden for them of maintaining cryptographic primitives. Additionally, some precompiled contracts like Blake2s only perform the hash’s compression function, which means the contract still has more work to do to get the final hash.

Assuming a precompile was added for Blake3 that looked like Blake2, where only the compression function was implemented, we can estimate the cost roughly off the current costs for the Blake2s precompile. It would look something like:

log(filesize) * [GBLAKEBASE + GBLAKEWORD * floor(INPUTSIZE / 32)] * num_proofs 

For our same 1000kb file, doing one proof comes out to 575 gas or around $1300 at current exchange rates. Add the memory costs of pushing and popping values off the stack and this isn’t actually as cheap as you might expect.

All of this doesn’t even consider the cost of storing our proofs, which is one of the underemphasized advantages of off-chain computation. To use a solidity Blake3 contract or the theoretical precompile, all proofs must be stored in Ethereum memory, since the EVM can’t access data stored off-chain. Ethereum memory is expensive, especially if you consider Banyan’s use case: storing hundreds of proofs for every single file. Ethereum event logging is around 70x cheaper* as an alternative to storing data for our use case, but since the EVM can’t access log data, it isn’t generally used to store data core to contract logic. But if you are doing your computation off-chain, then you can easily read log data, and save gas on both computing and storage! This same logic applies to any future project using protodanksharding blobs instead of memory.

— — — *The EVM costs 20k gas per 32byte slot to SSTORE and SLOAD * 32 slots = 640k gas. The LOG opcode costs 375gas for signature hash + 375 gas for topic + 8 gas per byte for 1024 bytes = 8.75k gas.

Off-Chain Computation Use Cases

At Banyan, we determined that neither custom contracts nor Ethereum precompiled contracts were a viable on-chain solution, and decided instead to use off-chain oracles with Chainlink. The computational costs of this design would be cheaper than either alternative, with little to no sacrifice in the security of our system. Ultimately, we asked ourselves: Do we need the full economic security behind the Ethereum network to verify this proof or can we get by with less? As long as we make sure the Chainlink nodes are staking more in link than they can make by cheating, and we have a decentralized consensus mechanism like Chainlink OCR (Off Chain Reporting) or Tendermint BFT to catch cheaters, crypto-economic incentives are powerful enough to ensure the validity of our merkle proofs, and our design as a whole would remain decentralized, censorship-resistant, and secure.

Beyond Banyan’s use case, other projects could use off-chain computation to validate signatures from cryptographic schemes other than secp256 curve, or to perform reads and writes on larger data structures stored in log space or protodanksharding blobs that are currently infeasible to store on the EVM alone. Generalized WASM computation is an option too! With off-chain computation, the sky’s the limit.