Storage games
What is this?This is a deep-dive on the Swarm Storage Incentives. Contained within is guidance on how to optimise performance in the "Schelling Game" - the core component of the storage incentives. The article includes details for using tools to optimise this performance. There is NO tl;dr - understanding how to optimise performance REQUIRES knowledge of some Swarm primitives. If you're in for the long haul, read on enthusiastic bee 🐝...PrimitivesStorage (Chunks)When uploading data to S...
Psst, have you heard of Waku?
In 2014, it was hypothesised that a decentralised web3 could result in removing the need to have servers at all. This gave birth to the holy trinity of web3:Contracts: decentralised logicSwarm: decentralised storageWhisper: decentralised messagingFast forward to 2023 and this hypothesis is being actively tested. Contracts have evolved, separating into execution and consensus layers. Swarm spun out of Ethereum Foundation and nears feature completion with it's storage incentives. Whisper h...
Bella Ciao, data silos
What is this?Blockchain sits at the fore-front of novel, technological development, with most software being written by specialists, with little attention given to the end user experience - out-of-date documentation (when the documentation actually exists!) and howtos attempting to guide people through being able to actually use the software. This issue is particularly acute for software at the bleeding edge, that software which is working towards realising the vision of the world computer. N...
Storage games
What is this?This is a deep-dive on the Swarm Storage Incentives. Contained within is guidance on how to optimise performance in the "Schelling Game" - the core component of the storage incentives. The article includes details for using tools to optimise this performance. There is NO tl;dr - understanding how to optimise performance REQUIRES knowledge of some Swarm primitives. If you're in for the long haul, read on enthusiastic bee 🐝...PrimitivesStorage (Chunks)When uploading data to S...
Psst, have you heard of Waku?
In 2014, it was hypothesised that a decentralised web3 could result in removing the need to have servers at all. This gave birth to the holy trinity of web3:Contracts: decentralised logicSwarm: decentralised storageWhisper: decentralised messagingFast forward to 2023 and this hypothesis is being actively tested. Contracts have evolved, separating into execution and consensus layers. Swarm spun out of Ethereum Foundation and nears feature completion with it's storage incentives. Whisper h...
Bella Ciao, data silos
What is this?Blockchain sits at the fore-front of novel, technological development, with most software being written by specialists, with little attention given to the end user experience - out-of-date documentation (when the documentation actually exists!) and howtos attempting to guide people through being able to actually use the software. This issue is particularly acute for software at the bleeding edge, that software which is working towards realising the vision of the world computer. N...
Subscribe to mfw78.eth
Subscribe to mfw78.eth
Share Dialog
Share Dialog
<100 subscribers
<100 subscribers
This morning as the mist rises through the valley above Mestia, yet to reveal the majestic Greater Caucuses, the calendar rolls over marking my one year anniversary at CoW DAO as a "Core Contributor". It's a fitting place to mark this milestone - though it's not my permanent place of abode - it is the place where important milestones have intersected between CoW DAO and myself. It's also quite appropriate considering this village is synonymous with the literal cows that are known to roam its charming medieval streets.
I first heard about CoW Swap in 2021, primarily as a by-product of the Gnosis buyout of the then xDai Chain. xDai Chain always spoke to me as it seemed to be the most logical for on-boarding users - having transaction costs denominated in a known currency, as opposed to having to buy some intermediate token (you don't pay transaction fees on a bank account by purchasing the bank's shares do you?).
CoW Swap offered the prospect of a place where traders can meet, and in the best case, be matched as a Coincidence of Want (COW). This seemed ideal - actual self-custody P2P trading with all the benefits of paying only when your order is executed (i.e. gas-less trading). At a time when gas prices of 100 gwei were considered cheap, the cost for failed swaps through other on-chain venues were not inconsiderable - at which time CoW Swap became integral to some strategies that I was executing surrounding GIP-16: the gnosis chain xdai merge.
During the week that GIP-16 was up for voting it became difficult to know if the "moo" I was hearing was from CoW Swap - or one of my actual bovine neighbours. 🐮
This morning as the mist rises through the valley above Mestia, yet to reveal the majestic Greater Caucuses, the calendar rolls over marking my one year anniversary at CoW DAO as a "Core Contributor". It's a fitting place to mark this milestone - though it's not my permanent place of abode - it is the place where important milestones have intersected between CoW DAO and myself. It's also quite appropriate considering this village is synonymous with the literal cows that are known to roam its charming medieval streets.
I first heard about CoW Swap in 2021, primarily as a by-product of the Gnosis buyout of the then xDai Chain. xDai Chain always spoke to me as it seemed to be the most logical for on-boarding users - having transaction costs denominated in a known currency, as opposed to having to buy some intermediate token (you don't pay transaction fees on a bank account by purchasing the bank's shares do you?).
CoW Swap offered the prospect of a place where traders can meet, and in the best case, be matched as a Coincidence of Want (COW). This seemed ideal - actual self-custody P2P trading with all the benefits of paying only when your order is executed (i.e. gas-less trading). At a time when gas prices of 100 gwei were considered cheap, the cost for failed swaps through other on-chain venues were not inconsiderable - at which time CoW Swap became integral to some strategies that I was executing surrounding GIP-16: the gnosis chain xdai merge.
During the week that GIP-16 was up for voting it became difficult to know if the "moo" I was hearing was from CoW Swap - or one of my actual bovine neighbours. 🐮
For a few months in 2022 I worked for a blockchain organisation with an ambitious goal - the decentralisation / Web3 transition of the travel industry (WindingTree). It was during this time that as a research project, I mapped out Videre, a very basic real-world service marketplace that used Waku to relay bids and asks, which in this context were a domain-specific service (such as a user asking for guesthouses around Mestia, and a guesthouse responding with a bid).
What was actually exchanged with Videre was a user's intent, which true to form like CoW Protocol, was but an EIP-712 signed message (except instead of an ERC-20 being exchanged, it was something analogous to an ERC-721 / ERC-1155 token). It was very exciting getting a glimpse into a world whereby even services could be swapped / traded / purchased - P2P.
A critical component that was missing from this system was a method of discoverability, and composability. How about the case where you want a full holiday, including air travel ticket, taxi hire, guesthouse booking and local guides for an experience to remember? Solving this problem is critical for user experience - needing something like a "searcher" - or a "solver" in CoW Protocol parlance.
From the very start of CoW DAO I have been an active community participant. After becoming part of the Grants Committee as a community member, I also applied for my first grant on New Years Eve 2022/2023. I worked on multiple grants centred around programmatic orders / TWAP, and in doing so developed a closer relationship with the Core Team. I am very thankful for this opportunity as it was instrumental in facilitating skill growth, specialising in smart contracts.
In June 2023 while travelling in Europe, I had the great pleasure of meeting team members in Berlin, and also participated in a Company-wide hackathon in Lisbon. It was at the hackathon in Lisbon that the prospect of joining CoW DAO as a Core Contributor was first raised. The main proposition for joining CoW DAO was to iterate on the smart contracts underpinning CoW's batch auction mechanism. This was interesting, as I could definitely see synergies with CoW's batch auction mechanism and a "marketplace abstraction for intents" - effectively an abstraction of Videre that I was working on the year prior.
I started working as a "Core Contributor" for CoW DAO August 1st, 2023. On this day a year ago, I realised the task before me to dig beyond the background knowledge and learn CoW's product suite, end-to-end. But first there was on-boarding. Time to learn another organisation's workflows, and reluctantly install the tragic trio - Slack, Zoom, and Gmail.
Prior to working in blockchain development, I worked in aviation as a commercial airline pilot/technical pilot for 15 years. This brought with it heavy exposure to bureaucratic organisations with rigid org charts, strict lines of reporting, and high levels of government regulation. Though I tended to work for non-legacy airlines (I preferred less "legacy" airlines as this generally translated to more progressive thinking among management) - I was about to find out that startup culture is nonetheless very different.
When first starting at CoW DAO, it was very difficult to understand what the priorities were, which projects were underway, and what their status was. It kind of didn't help that I was working somewhat in isolation as my smart contract team member was on extended leave, so I ended up "tagging onto the backend team". The lack of being able to see the "trees from the forest" was raised, as I pushed an effort to encourage better project management efforts (notably using the pm repository and associated epics / milestones to define priorities).
Now with some visibility on what projects were in progress, it was time to study the system - end-to-end. Reading and studying manuals / documentation was a normal endeavour.
Having already completed a few grants, I had some experience with the documentation. When diving deep, seeking to understand the system end-to-end, it became apparent that an overhaul of documentation was required. I made this a priority and fortunately was granted the time (as well as time of others) to contribute to a much-needed comprehensive overhaul of developer documentation.
During the pursuit of better documentation, we fortunately managed to get backend resources to document their area (architecturally), which for the first time delivered a digestible explanation of the infrastructure powering Ethereum's first productionised intent-matching engine - prior to this one would have to follow the "code is documentation" principle and be prepared to wade neck deep in the services repository's code hoping to gather some understanding.
Fortunately during COVID (a phrase I'm sure is seldom uttered), as I had nothing better to do with my time when locked in an apartment, I gathered some experience managing Kubernetes clusters and running node infrastructure (Ethereum and Swarm decentralised storage nodes). This was about to prove valuable as I entered the on-call rotation for CoW DAO's infrastructure over the Christmas / New Years period 2023-2024. Being on-call at weird and wonderful hours was somewhat normal from the previous career - having long not adhered to "normal" public holidays.
Many of the on-call rotations I had were spent fighting DDoS attacks on CoW Protocol's API or performing mundane tasks such as weekly releases. One not-so-mundane task was upgrading the antique Nethermind node setup with a 3 week requirement. Working with CoW DAO's sole DevOps engineer we were able to generate reliability data to lobby effectively for the deprecation of legacy Nethermind RPC nodes and migrate CoW's RPC infrastructure to next-generation reth and erigon nodes. We watched on in Evora, Portugal at the CoW team retreat as the Dencun hardfork went live and settlements continued, unaffected.
It was around this time that work for the CoW AMM was starting to mount and I was being exposed more and more to the workings of the elusive Solvers team and the solver competition that underpins CoW's batch auction mechanism.
The development of CoW AMM became a very large co-ordination effort, at times spanning across multiple organisations (CoW DAO and Balancer). It was the smart contract team's responsibility to provide a reference implementation of a CoW AMM (put simply, an AMM that knows how to trade within the batch auction, and which is integrated by solvers as a source of liquidity that a user's order may be settled against within the batch auction).
What became apparent during the development of this product was the hesitation / time frame requirements for external solvers to adequately support new types of liquidity/features of CoW Protocol - leading to an inertia when implementing new features. To bridge this "bootstrapping" problem, the backend added support for publishing CoW AMM JIT (Just-In-Time) liquidity in the auction, therefore exposing CoW AMMs to all lay external solvers for consideration in the batch auction. To meet this requirement in a timely fashion, and reduce the efforts required for a backend implementation, a standard smart contract interface was developed for AMMs to provide the information needed for the backend.
During late May 2024, work began in earnest on the repository layout and migration from Hardhat to Foundry (migrating smart contract development environments) for the core smart contracts. As of the time of writing, we are about to complete the unit test migration, with E2E tests remaining (the current idea is to outsource these, to allow regaining some of the lost time associated with the unforeseen 1 month delay on CoW AMM interface development).
One year on, things are different. Across the market intents are all the hype. CoW DAO has gone from a period of headcount reduction to growth in an increasingly crowded DeFi marketplace. I'll breakdown this section into the state of "CoW DAO's value propositions" and the current state "Within CoW DAO".
CoW Swap is regarded as the OG for intents trading, having pioneered this field years before the competition; 1inch Fushion and UniswapX. As mentioned in background, gas-less trading was a very large value proposition at the time of CoW Swap's launch.
At the point in time of writing this article, gas costs on mainnet are now at least 1, if not 2 orders of magnitude lower as compared to CoW Swap's initial launch. With the advent of intents-based trading competition, lower mainnet gas costs, reduced transaction failures, and the proliferation of cheap L2s with reasonable liquidity, the gas-less value proposition is much reduced.
I have strong conviction on the academic value proposition of batch auctions, but find that this concept is not well understood, difficult to communicate, and difficult to observe (being backed with quantifiable data that there is indeed better execution versus competition). It may provide benefits to users including trade aggregation, and possible P2P/COW (Coincidence of Wants). Unfortunately the current state is such that only 2 solvers are able to actually compute COWs and there is little incentive to solvers to actually settle a COW (marginal return on complexity requirements).
CoW Swap is a highly recognised app within the established Ethereum community, with high loyalty offering some advanced features not found in other DeFi dapps. I still love the "moo" to this day.
The above notwithstanding, I am not a new user to CoW Swap. CoW Swap does a great job of showcasing access to CoW Protocol features, but user growth is strongly coupled to those features.
Documentation as it exists now, while significantly better than a year ago, is not perfect and gaps remain. Off the top of my head, a critical section that is missing is "How to build a solver", which was an article that was removed as it was grossly out-of-date and therefore no longer usable. This is an often asked question on Discord and smoothing this with tutorials / documentation would contribute greatly in reducing the barrier of entry for new solvers - further enhancing the solver competition and therefore adding value back to the end user.
The DevOps team has undergone expansion - allowing for a significant degree of tech debt elimination, migrating the clusters to supported versions of Kubernetes, and having a pro-active approach to cost reduction. The additional head count here has been instrumental. The current infrastructure stack makes use of:
General architecture
AWS EKS (managed Kubernetes)
AWS RDS (postgres database)
AWS KMS (private key management)
AWS Cloudfront (DDoS protection)
ElasticSearch (kibana)
Grafana / Prometheus
Per-network architecture
RPC nodes (formally Nethermind, now reth + erigon + Arbitrum's geth adaptation) with custom-rolled NGINX proxy for RPC calls within the cluster.
Autopilot
Driver (Gnosis solvers)
Driver (External solvers)
Watch-tower (programmatic orders)
While much technical debt has been eliminated, infrastructure assumptions remain a key issue for CoW DAO and there has been little to no movement on systems architectural debt on the road to decentralisation (solver-driver co-location efforts had started prior to my time commencing at CoW DAO).
The batch auction competition is relatively robust, but in its current form requires:
A significant bond and/or KYC (competition constraint) (out of 24 solvers, only 1 solver is fully bonded, and 1 solver has a reduced bond - the rest rely on the CoW DAO or Gnosis bonding pools)
Overcoming a significant technical barrier (no simple "how to write a solver" with an end-to-end example) (competition constraint)
High human co-ordination costs for the Core Team (external solver team assessment, allow-listing / vouching, addition of API to backend, solver payouts / accounting) (resource constraint)
High inertia for solvers adopting new features of CoW Protocol
The above constraints represent significant costs to CoW DAO in the form of bonding pool capital requirements and operational expenses administering the competition. Competition subsidies have mostly been alleviated with the passage of CIP-48.
Finally, the work on contract migrations are well and truly underway - which is great as this is the premise on which I came aboard CoW DAO. To be fair, all the exposure to date in other areas of CoW DAO has been instrumental in providing a well-formed view of CoW DAO's product suite, its pain points, growth opportunities, and value propositions - hopefully providing some insight on how best to weave this altogether. So...
Broadly speaking, I would advocate for CoW DAO to be split into 3 initiatives:
A Generic Intents Protocol - a public good, becoming a defacto and/or formalised standard, free for all of the Ethereum ecosystem to build on.
CoW Protocol - initially specialise in ERC-20 batch auctions - becoming a reference implementation on the generic intents protocol.
CoW Swap - a competitive front-end providing connectivity to CoW Protocol.
Generic Intents Protocol
Target market: Cross ecosystem protocols / developers - NOT end-user facing. Resource requirements: smart contract / backend / integrations engineers / community engagement / user experience
We should seek to leverage CoW DAO's OG intents capabilities and elevate this to the level of an Ethereum standard (formalised, or defacto). Related elements for an intents marketplace should be rationalised to reduce duplicate work across the ecosystem, fostering collaboration as opposed to outright competition. Having standardised infrastructure for processing intents reduces infrastructure development costs across the industry and prioritises market logic / value proposition building.
The intents protocol would:
Leverage Permit2 for a smooth user experience when signing an intent.
Encourage the development of an ERC-721/ERC-1155 capable version of Permit2, able to be utilised across the ecosystem opening additional use cases.
Enforce pre and post execution hooks (with optional contextual callbacks).
Enforce basic requirements for the intent (validFrom / validTo).
Provide a security abstraction for marketplaces to assert an economic level of security (such as by token locks in the event of cross-chain bridging, bonding pools for the case of batch auctions, etc.) by solvers.
Provide a nullfier abstraction so that marketplaces can have flexibility as to how an intent may be nullified.
Standardised EIP-712 compliant metadata
CoW Protocol would become a "reference marketplace" implementation on the generic intents protocol, retaining its unique characteristics as an ERC-20 batch auction marketplace, but inherit properties of the generic intents protocol including:
Bonding pools can be asserted using the security abstraction at (5)
Pre and post hooks are no longer required in CoW Protocol and can be removed (3) (with their execution now guaranteed)
By implementing CoW Protocol as a handler allows for progressively upgrading the implementation of the batch auction mechanism, allowing for the marketplace to adapt and move faster with reduced friction (e.g. no need to worry about ERC-20 approval migration with the use of Permit2).
Metadata, an often fraught area of CoW Protocol's specification can be migrated to a friendly EIP-712 type (7).
As the reference marketplace, take an opinionated stance on the flow of intents, with these being gossiped in a decentralised mempool (eliminating centralisation of the current order book).
By encouraging cross-ecosystem development and standard adoption, we:
Enjoy long-term reduction of costs associated with maintaining systems as many intent marketplaces would share similar infrastructure, and therefore lead to scales of economy and reduced waste on duplicated effort.
Unify efforts among the ecosystem for improving UX associated with signature handling and allowance management.
Reduces boilerplate for CoW Protocol, allowing the team to focus on the batch auction value proposition.
This effort would squarely place CoW DAO at the centre of the intents ecosystem, contributing to growth of the market and not just market share.
Target market: DeFi protocols and trading venues requiring superior execution (NOT end-user facing).
Resource requirements: smart contract / backend / integrations engineers / data analytics / solvers / community engagement
We should seek to focus CoW Protocol on integrations and improving the efficacy of the solver competition, primarily with the motivation to increase competition among the solvers. Initiatives for this include:
Adjust the protocol architecture to reduce the financial barrier of entry for solvers (a concrete examples being trade handlers, eliminating shared internal buffers).
Provide a comprehensive tutorial of "How to write a solver", providing an end-to-end example in Python / JavaScript / Rust.
Implement a "devnet" - our own "ephemeral chain" that periodically forks from Ethereum mainnet, on which solvers may practice execution and integrators may test on - creating an environment that is actually representative of production (and we can test new features).
Amalgamate all CIPs into a single "tome" describing the rules of the auction. All subsequent CIPs would be a diff of this "tome" - delivering a significant increase in readability of the rules - increasing compliance and simplifying governance review processes.
Provide a "leaderboard" of solvers on the devnet (and other chains), allowing for solvers to progressively build their bond with 3rd parties contributing to the bond on their behalf - with returns for the solver being distributed among the bonding pool.
Ensure that all data required for rebuilding the auction and verifying the outcome is available on a data availability layer to allow for punishing a malicious/erroneous solver.
Publish the scoped project "LP Book" - a standard for DeFi protocols to expose their liquidity and invariants - eliminating the bootstrapping issues for supporting additional features / liquidity sources within the batch auction. This would be a public good and reduce technical barriers for new solvers (and projects wanting to add liquidity).
Increase data analytics capacity with the primary goal being to measure execution between CoW Protocol's batch auction and other trading venues.
Explicitly track flow referral and distribute a revenue share to the flow referrer. This would provide a mechanism whereby CoW Swap could become self-sustainable, and allow allocating funds to incentivise venue integration, similar to the method that was used for bootstrapping the solver's competition.
Significantly reduce co-ordination costs for the batch auction by:
Facilitating permissionless entry / exit of the batch auction (no need to update APIs, vet teams etc).
Automate accounting epochs (solvers payouts, flow referral) with "oracles" running to update the accounting roots for payouts / dues (at each epoch, oracles with a stake run the accounting program and vote on an accounting root).
Investigate leader-less (or consensus-based) auction method to eliminate the central role of the autopilot.
The above roadmap seeks to focus efforts on improving / building out the solver competition to ensure that it is as robust and distributed as possible so as to give the best opportunity for batch auctions to live up to their academic merit. It also seeks to reduce expense and human co-ordination costs for the batch auction, freeing resources for CoW DAO which may be redirected towards partnership / chain expansion.
Target market: Retail users, sophisticated users**
Resource requirements**: front-end, user experience, retail marketing
By CoW Protocol explicitly tracking referral value, CoW Swap would be able to earn revenue from CoW Protocol, allowing CoW Swap to start a path towards independence. Additional growth streams here could be UX-smoothed cross-ecosystem dapps (hooks) which would potentially provide other CoW Swap revenue streams.
Additionally this therefore provides a method to encourage additional trading venues / integrations - facilitating healthy competition among user interfaces and reducing single-channel risk to CoW Protocol.
Reflecting on the past year, the strides we’ve made at CoW DAO are considerable. Despite the initial challenges, we've established a solid foundation, addressing gaps in documentation, enhancing infrastructure, and expanding our team. The solver competition is becoming more robust, and our focus on improving user experience and reducing barriers to entry is yielding positive results. CoW DAO stands at the precipice of innovation within the Ethereum ecosystem.
Looking ahead, the future is bright. Playing to our strengths and formalising a Generic Intents Protocol and streamlining CoW Protocol, we are poised to lead the way in decentralisation and interoperability. The proposed initiatives will not only enhance our offerings but also foster a more inclusive and competitive ecosystem. As we continue to build and grow, the vision of a more efficient, user-friendly, and interconnected DeFi ecosystem becomes increasingly attainable. Here's to another year of growth, innovation, and collective success at CoW DAO!
For a few months in 2022 I worked for a blockchain organisation with an ambitious goal - the decentralisation / Web3 transition of the travel industry (WindingTree). It was during this time that as a research project, I mapped out Videre, a very basic real-world service marketplace that used Waku to relay bids and asks, which in this context were a domain-specific service (such as a user asking for guesthouses around Mestia, and a guesthouse responding with a bid).
What was actually exchanged with Videre was a user's intent, which true to form like CoW Protocol, was but an EIP-712 signed message (except instead of an ERC-20 being exchanged, it was something analogous to an ERC-721 / ERC-1155 token). It was very exciting getting a glimpse into a world whereby even services could be swapped / traded / purchased - P2P.
A critical component that was missing from this system was a method of discoverability, and composability. How about the case where you want a full holiday, including air travel ticket, taxi hire, guesthouse booking and local guides for an experience to remember? Solving this problem is critical for user experience - needing something like a "searcher" - or a "solver" in CoW Protocol parlance.
From the very start of CoW DAO I have been an active community participant. After becoming part of the Grants Committee as a community member, I also applied for my first grant on New Years Eve 2022/2023. I worked on multiple grants centred around programmatic orders / TWAP, and in doing so developed a closer relationship with the Core Team. I am very thankful for this opportunity as it was instrumental in facilitating skill growth, specialising in smart contracts.
In June 2023 while travelling in Europe, I had the great pleasure of meeting team members in Berlin, and also participated in a Company-wide hackathon in Lisbon. It was at the hackathon in Lisbon that the prospect of joining CoW DAO as a Core Contributor was first raised. The main proposition for joining CoW DAO was to iterate on the smart contracts underpinning CoW's batch auction mechanism. This was interesting, as I could definitely see synergies with CoW's batch auction mechanism and a "marketplace abstraction for intents" - effectively an abstraction of Videre that I was working on the year prior.
I started working as a "Core Contributor" for CoW DAO August 1st, 2023. On this day a year ago, I realised the task before me to dig beyond the background knowledge and learn CoW's product suite, end-to-end. But first there was on-boarding. Time to learn another organisation's workflows, and reluctantly install the tragic trio - Slack, Zoom, and Gmail.
Prior to working in blockchain development, I worked in aviation as a commercial airline pilot/technical pilot for 15 years. This brought with it heavy exposure to bureaucratic organisations with rigid org charts, strict lines of reporting, and high levels of government regulation. Though I tended to work for non-legacy airlines (I preferred less "legacy" airlines as this generally translated to more progressive thinking among management) - I was about to find out that startup culture is nonetheless very different.
When first starting at CoW DAO, it was very difficult to understand what the priorities were, which projects were underway, and what their status was. It kind of didn't help that I was working somewhat in isolation as my smart contract team member was on extended leave, so I ended up "tagging onto the backend team". The lack of being able to see the "trees from the forest" was raised, as I pushed an effort to encourage better project management efforts (notably using the pm repository and associated epics / milestones to define priorities).
Now with some visibility on what projects were in progress, it was time to study the system - end-to-end. Reading and studying manuals / documentation was a normal endeavour.
Having already completed a few grants, I had some experience with the documentation. When diving deep, seeking to understand the system end-to-end, it became apparent that an overhaul of documentation was required. I made this a priority and fortunately was granted the time (as well as time of others) to contribute to a much-needed comprehensive overhaul of developer documentation.
During the pursuit of better documentation, we fortunately managed to get backend resources to document their area (architecturally), which for the first time delivered a digestible explanation of the infrastructure powering Ethereum's first productionised intent-matching engine - prior to this one would have to follow the "code is documentation" principle and be prepared to wade neck deep in the services repository's code hoping to gather some understanding.
Fortunately during COVID (a phrase I'm sure is seldom uttered), as I had nothing better to do with my time when locked in an apartment, I gathered some experience managing Kubernetes clusters and running node infrastructure (Ethereum and Swarm decentralised storage nodes). This was about to prove valuable as I entered the on-call rotation for CoW DAO's infrastructure over the Christmas / New Years period 2023-2024. Being on-call at weird and wonderful hours was somewhat normal from the previous career - having long not adhered to "normal" public holidays.
Many of the on-call rotations I had were spent fighting DDoS attacks on CoW Protocol's API or performing mundane tasks such as weekly releases. One not-so-mundane task was upgrading the antique Nethermind node setup with a 3 week requirement. Working with CoW DAO's sole DevOps engineer we were able to generate reliability data to lobby effectively for the deprecation of legacy Nethermind RPC nodes and migrate CoW's RPC infrastructure to next-generation reth and erigon nodes. We watched on in Evora, Portugal at the CoW team retreat as the Dencun hardfork went live and settlements continued, unaffected.
It was around this time that work for the CoW AMM was starting to mount and I was being exposed more and more to the workings of the elusive Solvers team and the solver competition that underpins CoW's batch auction mechanism.
The development of CoW AMM became a very large co-ordination effort, at times spanning across multiple organisations (CoW DAO and Balancer). It was the smart contract team's responsibility to provide a reference implementation of a CoW AMM (put simply, an AMM that knows how to trade within the batch auction, and which is integrated by solvers as a source of liquidity that a user's order may be settled against within the batch auction).
What became apparent during the development of this product was the hesitation / time frame requirements for external solvers to adequately support new types of liquidity/features of CoW Protocol - leading to an inertia when implementing new features. To bridge this "bootstrapping" problem, the backend added support for publishing CoW AMM JIT (Just-In-Time) liquidity in the auction, therefore exposing CoW AMMs to all lay external solvers for consideration in the batch auction. To meet this requirement in a timely fashion, and reduce the efforts required for a backend implementation, a standard smart contract interface was developed for AMMs to provide the information needed for the backend.
During late May 2024, work began in earnest on the repository layout and migration from Hardhat to Foundry (migrating smart contract development environments) for the core smart contracts. As of the time of writing, we are about to complete the unit test migration, with E2E tests remaining (the current idea is to outsource these, to allow regaining some of the lost time associated with the unforeseen 1 month delay on CoW AMM interface development).
One year on, things are different. Across the market intents are all the hype. CoW DAO has gone from a period of headcount reduction to growth in an increasingly crowded DeFi marketplace. I'll breakdown this section into the state of "CoW DAO's value propositions" and the current state "Within CoW DAO".
CoW Swap is regarded as the OG for intents trading, having pioneered this field years before the competition; 1inch Fushion and UniswapX. As mentioned in background, gas-less trading was a very large value proposition at the time of CoW Swap's launch.
At the point in time of writing this article, gas costs on mainnet are now at least 1, if not 2 orders of magnitude lower as compared to CoW Swap's initial launch. With the advent of intents-based trading competition, lower mainnet gas costs, reduced transaction failures, and the proliferation of cheap L2s with reasonable liquidity, the gas-less value proposition is much reduced.
I have strong conviction on the academic value proposition of batch auctions, but find that this concept is not well understood, difficult to communicate, and difficult to observe (being backed with quantifiable data that there is indeed better execution versus competition). It may provide benefits to users including trade aggregation, and possible P2P/COW (Coincidence of Wants). Unfortunately the current state is such that only 2 solvers are able to actually compute COWs and there is little incentive to solvers to actually settle a COW (marginal return on complexity requirements).
CoW Swap is a highly recognised app within the established Ethereum community, with high loyalty offering some advanced features not found in other DeFi dapps. I still love the "moo" to this day.
The above notwithstanding, I am not a new user to CoW Swap. CoW Swap does a great job of showcasing access to CoW Protocol features, but user growth is strongly coupled to those features.
Documentation as it exists now, while significantly better than a year ago, is not perfect and gaps remain. Off the top of my head, a critical section that is missing is "How to build a solver", which was an article that was removed as it was grossly out-of-date and therefore no longer usable. This is an often asked question on Discord and smoothing this with tutorials / documentation would contribute greatly in reducing the barrier of entry for new solvers - further enhancing the solver competition and therefore adding value back to the end user.
The DevOps team has undergone expansion - allowing for a significant degree of tech debt elimination, migrating the clusters to supported versions of Kubernetes, and having a pro-active approach to cost reduction. The additional head count here has been instrumental. The current infrastructure stack makes use of:
General architecture
AWS EKS (managed Kubernetes)
AWS RDS (postgres database)
AWS KMS (private key management)
AWS Cloudfront (DDoS protection)
ElasticSearch (kibana)
Grafana / Prometheus
Per-network architecture
RPC nodes (formally Nethermind, now reth + erigon + Arbitrum's geth adaptation) with custom-rolled NGINX proxy for RPC calls within the cluster.
Autopilot
Driver (Gnosis solvers)
Driver (External solvers)
Watch-tower (programmatic orders)
While much technical debt has been eliminated, infrastructure assumptions remain a key issue for CoW DAO and there has been little to no movement on systems architectural debt on the road to decentralisation (solver-driver co-location efforts had started prior to my time commencing at CoW DAO).
The batch auction competition is relatively robust, but in its current form requires:
A significant bond and/or KYC (competition constraint) (out of 24 solvers, only 1 solver is fully bonded, and 1 solver has a reduced bond - the rest rely on the CoW DAO or Gnosis bonding pools)
Overcoming a significant technical barrier (no simple "how to write a solver" with an end-to-end example) (competition constraint)
High human co-ordination costs for the Core Team (external solver team assessment, allow-listing / vouching, addition of API to backend, solver payouts / accounting) (resource constraint)
High inertia for solvers adopting new features of CoW Protocol
The above constraints represent significant costs to CoW DAO in the form of bonding pool capital requirements and operational expenses administering the competition. Competition subsidies have mostly been alleviated with the passage of CIP-48.
Finally, the work on contract migrations are well and truly underway - which is great as this is the premise on which I came aboard CoW DAO. To be fair, all the exposure to date in other areas of CoW DAO has been instrumental in providing a well-formed view of CoW DAO's product suite, its pain points, growth opportunities, and value propositions - hopefully providing some insight on how best to weave this altogether. So...
Broadly speaking, I would advocate for CoW DAO to be split into 3 initiatives:
A Generic Intents Protocol - a public good, becoming a defacto and/or formalised standard, free for all of the Ethereum ecosystem to build on.
CoW Protocol - initially specialise in ERC-20 batch auctions - becoming a reference implementation on the generic intents protocol.
CoW Swap - a competitive front-end providing connectivity to CoW Protocol.
Generic Intents Protocol
Target market: Cross ecosystem protocols / developers - NOT end-user facing. Resource requirements: smart contract / backend / integrations engineers / community engagement / user experience
We should seek to leverage CoW DAO's OG intents capabilities and elevate this to the level of an Ethereum standard (formalised, or defacto). Related elements for an intents marketplace should be rationalised to reduce duplicate work across the ecosystem, fostering collaboration as opposed to outright competition. Having standardised infrastructure for processing intents reduces infrastructure development costs across the industry and prioritises market logic / value proposition building.
The intents protocol would:
Leverage Permit2 for a smooth user experience when signing an intent.
Encourage the development of an ERC-721/ERC-1155 capable version of Permit2, able to be utilised across the ecosystem opening additional use cases.
Enforce pre and post execution hooks (with optional contextual callbacks).
Enforce basic requirements for the intent (validFrom / validTo).
Provide a security abstraction for marketplaces to assert an economic level of security (such as by token locks in the event of cross-chain bridging, bonding pools for the case of batch auctions, etc.) by solvers.
Provide a nullfier abstraction so that marketplaces can have flexibility as to how an intent may be nullified.
Standardised EIP-712 compliant metadata
CoW Protocol would become a "reference marketplace" implementation on the generic intents protocol, retaining its unique characteristics as an ERC-20 batch auction marketplace, but inherit properties of the generic intents protocol including:
Bonding pools can be asserted using the security abstraction at (5)
Pre and post hooks are no longer required in CoW Protocol and can be removed (3) (with their execution now guaranteed)
By implementing CoW Protocol as a handler allows for progressively upgrading the implementation of the batch auction mechanism, allowing for the marketplace to adapt and move faster with reduced friction (e.g. no need to worry about ERC-20 approval migration with the use of Permit2).
Metadata, an often fraught area of CoW Protocol's specification can be migrated to a friendly EIP-712 type (7).
As the reference marketplace, take an opinionated stance on the flow of intents, with these being gossiped in a decentralised mempool (eliminating centralisation of the current order book).
By encouraging cross-ecosystem development and standard adoption, we:
Enjoy long-term reduction of costs associated with maintaining systems as many intent marketplaces would share similar infrastructure, and therefore lead to scales of economy and reduced waste on duplicated effort.
Unify efforts among the ecosystem for improving UX associated with signature handling and allowance management.
Reduces boilerplate for CoW Protocol, allowing the team to focus on the batch auction value proposition.
This effort would squarely place CoW DAO at the centre of the intents ecosystem, contributing to growth of the market and not just market share.
Target market: DeFi protocols and trading venues requiring superior execution (NOT end-user facing).
Resource requirements: smart contract / backend / integrations engineers / data analytics / solvers / community engagement
We should seek to focus CoW Protocol on integrations and improving the efficacy of the solver competition, primarily with the motivation to increase competition among the solvers. Initiatives for this include:
Adjust the protocol architecture to reduce the financial barrier of entry for solvers (a concrete examples being trade handlers, eliminating shared internal buffers).
Provide a comprehensive tutorial of "How to write a solver", providing an end-to-end example in Python / JavaScript / Rust.
Implement a "devnet" - our own "ephemeral chain" that periodically forks from Ethereum mainnet, on which solvers may practice execution and integrators may test on - creating an environment that is actually representative of production (and we can test new features).
Amalgamate all CIPs into a single "tome" describing the rules of the auction. All subsequent CIPs would be a diff of this "tome" - delivering a significant increase in readability of the rules - increasing compliance and simplifying governance review processes.
Provide a "leaderboard" of solvers on the devnet (and other chains), allowing for solvers to progressively build their bond with 3rd parties contributing to the bond on their behalf - with returns for the solver being distributed among the bonding pool.
Ensure that all data required for rebuilding the auction and verifying the outcome is available on a data availability layer to allow for punishing a malicious/erroneous solver.
Publish the scoped project "LP Book" - a standard for DeFi protocols to expose their liquidity and invariants - eliminating the bootstrapping issues for supporting additional features / liquidity sources within the batch auction. This would be a public good and reduce technical barriers for new solvers (and projects wanting to add liquidity).
Increase data analytics capacity with the primary goal being to measure execution between CoW Protocol's batch auction and other trading venues.
Explicitly track flow referral and distribute a revenue share to the flow referrer. This would provide a mechanism whereby CoW Swap could become self-sustainable, and allow allocating funds to incentivise venue integration, similar to the method that was used for bootstrapping the solver's competition.
Significantly reduce co-ordination costs for the batch auction by:
Facilitating permissionless entry / exit of the batch auction (no need to update APIs, vet teams etc).
Automate accounting epochs (solvers payouts, flow referral) with "oracles" running to update the accounting roots for payouts / dues (at each epoch, oracles with a stake run the accounting program and vote on an accounting root).
Investigate leader-less (or consensus-based) auction method to eliminate the central role of the autopilot.
The above roadmap seeks to focus efforts on improving / building out the solver competition to ensure that it is as robust and distributed as possible so as to give the best opportunity for batch auctions to live up to their academic merit. It also seeks to reduce expense and human co-ordination costs for the batch auction, freeing resources for CoW DAO which may be redirected towards partnership / chain expansion.
Target market: Retail users, sophisticated users**
Resource requirements**: front-end, user experience, retail marketing
By CoW Protocol explicitly tracking referral value, CoW Swap would be able to earn revenue from CoW Protocol, allowing CoW Swap to start a path towards independence. Additional growth streams here could be UX-smoothed cross-ecosystem dapps (hooks) which would potentially provide other CoW Swap revenue streams.
Additionally this therefore provides a method to encourage additional trading venues / integrations - facilitating healthy competition among user interfaces and reducing single-channel risk to CoW Protocol.
Reflecting on the past year, the strides we’ve made at CoW DAO are considerable. Despite the initial challenges, we've established a solid foundation, addressing gaps in documentation, enhancing infrastructure, and expanding our team. The solver competition is becoming more robust, and our focus on improving user experience and reducing barriers to entry is yielding positive results. CoW DAO stands at the precipice of innovation within the Ethereum ecosystem.
Looking ahead, the future is bright. Playing to our strengths and formalising a Generic Intents Protocol and streamlining CoW Protocol, we are poised to lead the way in decentralisation and interoperability. The proposed initiatives will not only enhance our offerings but also foster a more inclusive and competitive ecosystem. As we continue to build and grow, the vision of a more efficient, user-friendly, and interconnected DeFi ecosystem becomes increasingly attainable. Here's to another year of growth, innovation, and collective success at CoW DAO!
No activity yet