A Primer on US Elections

The US Presidential Elections typically runs every 4 years, of which members of each state will vote for various representatives at the federal-level (e.g president, vice, senator rep) and state-level.

For the presidential election, the winner is not the person with the most votes by person across the country but by the number of “electoral college” votes given to each state based on population. Some states have a strong tendency to vote in favour for specific parties each election, having converged to a bipartisan system involving the “Republican” and the “Democratic” political parties. The last time that a third-party presidential candidate has won a state was back in 1968, relating to the support of states to police racial segregation at the state-level by the presidential candidate George Wallace of the American Independent Party.

Many states vote in favour of the same party for their presidential candidate again and again in consecutive elections, but with the gap between each election, votership and party alignment changes, sometimes “swinging” from Republican to Democrat or vice-versa. Many interested observers tend to bring greater attention to these “swing states” as battlegrounds that presidential candidates will dedicate more effort such as rallies and media events, to help swing votership their way. This is done to elect a sole “nominee” of the political party, or ultimately the presidential candidate that wins; for the former, “primaries” and “caucuses” are held by political parties to choose the presidential candidate that the party puts forward for their campaign.

Tailwinds from On-Chain Capital

Undoubtedly, this year has seen on-chain prediction markets grow substantially from the many accessible markets and venues to take directional exposure in the 2024 US Presidential Elections that aim to be as efficient as possible.

In prior years, many would use macro instruments (Bitcoin, Ethereum) to get indirect exposure to political decisions and elections due to the lack of liquidity in prediction markets at the time - indirect as it often didn’t lead to complete correlated risk / reward of the impact on the market. Investors for certain asset companies or companies would benefit or suffer depending on certain policies or trends, but it wasn’t the end-all-be-all.

The isolation of outcomes has lead to more distinct markets, allowing for greater risk tailoring that help to either increase directionality to a certain future that a trader is believing to occur or to hedge against it.

Though Polymarket wasn’t one of the first (many years ago were Augur and Gnosis / Azuro, and then Omen, Manifold), it happens to be a matter of right time, right place for them to capitalise on the considerable amount of on-chain capital that has been eager to play in the upcoming elections.

It was even the case in which larger players like Robinhood released market products to trade the election results throughout the run-up, where events throughout the campaign helped to drum up volume.

Robinhood

@RobinhoodApp

Presidential election contracts are rolling out to Robinhood customers.

Update your Robinhood app, and check out go.robinhood.com/election. Stay tuned for daily updates leading up to election day.

562

3:00 PM • Oct 28, 2024

Most prediction markets work by allowing users to buy YES or NO binary futures on any of the possible candidates relating to the outcome of the result. A pair of YES and NO binary futures on a candidate can be minted for $1, where specifically one of the futures will payout $1 and the other $0 based on the resolved outcome of the election. These futures can then be sold on the same platform to others, allowing people to have directional exposure of betting YES on Kamala or NO on Trump etc. This could be held until the time of expiry (when the election results are officially released) or sold before then for a return.

Putting ethics aside for the example, a bet YES on Trump winning the presidential election before his Pennslyvania rally on July 13th could then be sold a day after for a positive ROI to trade around the event, effectively allowing for people to be paid for their skin-in-the-game interest even in the run-up to the elections.

Arbitrage in Prediction Markets

One of my favourite posts comes from a couple years ago from Dan Robinson:

A nice thought experiment, it puts into perspective that there is no definitive price for each token individually but that there are mechanisms that can be used to deduce their prices in relation to each other in an efficient market. Token A can be trading at 0.75 USDC and token B can be trading at 0.25 USDC and some traders may consider it a ‘fair price’ whereas others may consider it ‘mispriced’ and participate. Information asymmetry and beliefs leads to divergences in perceived fair value of each token for which that comes together to form a ‘market price’ as there are more participants.

A previous tagline used years ago for prediction markets was that they helped to pool together the “wisdom of crowds” but with varying barriers to entry and types of audiences led to extreme skew in the types of entrants involved. The failure comes where experts are not themselves the representative trading participants in these markets, being overshadowed by speculative traders that add noise.

Different CEX and DEX prediction market venues e.g Polymarket, Drift, PredictIt, Kalshi, will vary in odds on the same equivalent prediction due to a number of factors such as market access, membership, and behaviour of traders that can greatly affect how representative the prediction market is.

The sample that trades on a specific venue’s prediction market may not be representative to e.g the votership of an election, or the production team involved in the making of a documentary on who Satoshi could be. There are some cases were one can be more informed than others, but there always exist a risk, especially on such a scale, that the lack of information ends up becoming a tail risk you can’t accurately account for.

scan

@sxcnxxx

Not Helpful

Polymarket allows a vastly larger number of people to participate compared to traditional polls, offering broader insights. More importantly, it aggregates collective predictions about who will win, rather than simply reflecting who individuals plan to vote for.

2

11:56 PM • Oct 23, 2024

Polymarket allows a vastly larger number of people to participate compared to traditional polls, offering broader insights. More importantly, it aggregates collective predictions about who will win, rather than simply reflecting who individuals plan to vote for. This distinction makes it a unique and valuable indicator of perceived outcomes rather than direct voter intent.

With this, there was much variance to be had between venues. Ignoring exchange or gas costs, if 1 USDC can be minted for 1 YES and 1 NO token for a choice where one eventually resolves to 1 USDC and the other to 0 USDC and equivalently on other venues, then there is that certainty that can be used to find arbitrages.

Many venues gave different odds throughout. A YES for Trump on one venue could be $0.60 but a NO bet for Trump could be valued at $0.30 on another, effectively cancelling each other out in terms of sentiment and netting a total $1 risk-free when both markets resolve as expected (often termed the expected value) - a form of cross-venue arbitrage when this expected value is common with others. By purchasing it for “cheap” ($1 of value at $0.90), it can then be sold for something more closer to a $1.

Pricing In or Out Tail Risk

Tail risks such as the potential for a third party candidate to win or the chance of no outcomes to be correct are possible, but incredibly unlikely, so it is up to one to decide how they price this in terms of expected value.

A YES for Trump and a NO for Trump will have a total expected value of $1 at the time of expiry but there are things that can make that not be the case e.g if the venue somehow chooses not to go ahead with paying out i.e counterparty risk. For most venues, it is expected that they will pay out and thus one can assume a total expected value prior to the expiry of $1 (by considering this to be priced at $0 to happen), but one can assume otherwise.

It is possible to consider a YES for Trump as a NO for Kamala (the lead candidates representing each political party) but that is not exactly equivalent. Rather, a YES for Trump is a NO for Kamala, a NO for Jill Stein (Green Party nominee), a NO for Chase Oliver (Libertarian Party nominee) and a NO for every other candidate, but one can choose to price third party candidates at $0 and consider arbitrage opportunities conditional on this.

Bebop

@bebop_dex

We will know the winner soon.

There's still time back your guess by trading $TRUMPWIN or $HARRISWIN tokens via Bebop, on @base, @arbitrum and @ethereum.

Upon Elections results, price of one will go to $1 and the other to $0.

Wintermute

@wintermute_t

The future of prediction markets is already here –decentralized, accessible, and free from barriers

OutcomeMarket, developed by Wintermute, has been demonstrating this since launch as the first prediction market contract built in a truly permissionless way

We’re excited to see

24

9:52 AM • Nov 4, 2024

As mentioned before, the last time that a third-party presidential candidate has won a state was back in 1968, but was related to a controversial policy at the time that won a third party candidate, George Wallace, the winning candidate in a number of states.

Arbitrage opportunities lie in also buying a portfolio of positions of YES for one candidate and YES for the others that ultimately cancel out to $1 but for other candidates / successful outcomes, they may not be as liquid.

Frontrunning the News

One of the interesting points to mention in regards to decentralised nature of the prediction market is that you can always sell your positions at any time before the closing of the market, contrasting the experience normally with centralised bookmakers e.g Fanduel, Draftkings where a bet is held by the sole bettor until expiry. A common notion that many highlight for decentralised prediction markets is that if the prediction market resolves in your favour, then you’ll make money, but that is one part of the story of how to win.

Joey Krug

@joeykrug

On prediction markets and “manipulation”:

These markets are similar to the stock market in some ways. Is the stock market priced perfectly? No. Is it priced relatively accurately? Yes. If you think you can beat it, and you can beat it consistently, you're essentially

Shayne Coplan 🦅

@shayne_coplan

Given the latest New York Times article on Polymarket, this seems like a good time to make clear:

• Polymarket is strictly non-partisan. We get told we're Dem operatives and MAGA, depending on the day. Unfortunately the story is much less juicy, we're just market nerds who

72

8:48 PM • Oct 25, 2024

Prediction markets are a very pure form of market, you make money if you’re right, and lose it if you’re wrong. And that’s why they work.

Purchasing a YES binary option at $1 that resolves as YES will payout $1 but it doesn’t offer anything additional. Purchasing a YES binary future at $0.90 that then resolves to $1 makes money, but so is purchasing a YES binary future at $0.80 that you then sell for $0.90 a week later.

The better mental model here is that if you make money if the ‘market’ believes that your prediction is ‘more right’ at the time of expiry or when you come to sell the future. Trading before and after a note-worthy event that the market deems positive in your favour can prove to be profitable so finding venues to offload inventory was part of the upkeep.

One feature of prediction markets that I think gets understated (and often gets misconstrued) is the fast rate of convergence to the correct outcome in the presence of definitive evidence to rule the outcome correct, operating faster than the oracle to resolve towards the right answer. The value that can be captured from the definitive $1 payoff is a known premium that can be earned when certain of a specific result thus exists an arbitrage.

A strategy I commonly observed were using bots or live sources to pick up official information as quickly as possible to trade on as a form of latency arbitrage, similar to the story of the Blanc Brothers, finding indicative information faster (please don’t) and trading upon that:

Their methods involved tapping into the telegraph wires outside of stations with their own magnets to intercept communications. The brothers then decoded messages related to financial transactions and stock prices by translating the Morse code. This gave them a sneak peek at breaking financial news up to a day before it reached Paris and moved markets there.

You just have to be the first to pick up asymmetric information, automatic or manual much like the French trader.

Akin to this is insider trading, but I digress; bots would trade on market events related to election news sure to spike some change and frontrun market news to enter a position prior to any updates and sell shortly after by taking on short-term inventory risk gambling on the direction of the price movement.

Preempting important price-moving events by following campaign calendars from each party and candidate, particularly events in “battleground” states, helped in monitoring risk tolerance and catch any bugs live in the code under stress - primaries, conventions and some caucuses with set dates were good for this, before eventually volume picked up with the mimetic nature of things e.g the attempted assassination of Trump, “Kamala is brat”, the euthanisation of Peanut the Squirrel and Fred the Raccoon and how news outlets have a tendency to bring such things back to the presidential race.

Post-Election Outlook for Prediction Markets

Arbitrage trading, whether it be via delta neutral market making (market making and then taking back the position or equivalent on another venue to close out the arbitrage) or simple take-takes, naturally benefits from more volume, variance and changes in prices happening for more arbitrage opportunities to exist.

jez (blast era)

@izebel_eth

prediction markets about to have their last 2bn+ volume month for the next 4 years lmfao

Nick Tomaino

@NTmoney

Prediction markets just had their first $2B+ volume month

Meanwhile legacy media credibility is collapsing

Skin in the game or STFU

774

3:13 AM • Nov 4, 2024

For a future that eventually will expire and close out, this undoubtedly sets a cap on the time that the market can really exist for, so demand needs to be sustained for the next cohort of markets for prediction markets to be successful. The volatility of the US elections market each occasion is becoming more and more events-driven, being greatly emotion and speculative in nature where logic goes out the window (both in price and on CT threads) to potentially take opportunities on, though it’s hard to say whether other prediction markets will receive the same level of attention, much like the shiny-object-syndrome volumes seen on DeFi Kingdoms, Friendtech and LooksRare.

Lamenting on specific venues for those interested, CEXs (think prediction market platforms like PredictIt, Kalshi, Robinhood, as well as more retail-focused betting platforms such as Betfair, Bet365 and Shuffle) were relatively easy to integrate and utilise after KYC, whereas on-chain prediction market DEXs (Polymarket, Drift BET, etc.), especially ones that spun up specifically to capture election-specific volume, were less popular routes for some strategies due to their liquidity / taker volume.

Polymarket was our predominant on-chain venue to make and arbitrage against, but the infrequency of bets, many taking single naked positions and holding until expiry, and limits on users (no US users), sometimes didn’t make arbitrages sensible unless facilitating some of this flow OTC for friends.

Stat Arb

@quant_arb

Revealing this one (yes on left, no on right)

Stat Arb

@quant_arb

Found some big APR arbs

60

1:33 PM • Nov 5, 2024

There is also the incoming regulatory air by other commissions to crack down on some of these venues as gambling and limit users further.

French regulators are looking into Polymarket after a massive surge in US election betting

Polymarket garnered attention after a trader revealed to be a French national had placed bets worth $30 million on a victory for Donald Trump.

https://www.businessinsider.com

Personally, I believe that other real-world prediction markets won’t come close in terms of popularity, speculation and action as the US elections. I instead point towards more targeted use cases of prediction markets that I think will prove to be more useful in localised markets (i.e applications) such as that of futarchy (e.g MetaDAO) or more customised trading experiences (@euphoria_fi).

Overall, it was pretty fun.

Feel free to DM me (@0xTaker) and reach out about anything trading, MEV or whatnot - always happy to chat. I would also like to thank Nate (@0xPerp) for a lot of insightful feedback - this guy is the real person to look at for news-based trading and prediction markets.

Independent searchers and searcher teams build automated systems, sometimes referred to as searcher bots, to execute strategies that take advantage of these opportunities. Some opportunities are unique or esoteric in nature, leading to the development of one-off monolithic custom designs.

This piece proposes a modular framework for searcher (bot) design, an architecture that allows many strategies to be ran off the same services and be easily maintainable by teams, taking inspiration from trading systems within traditional finance. A brief look into the current state of searcher design from open-source examples and how it may develop in the future is also offered, given the landscape for searcher design is often distorted due to the proprietary nature of these systems.

A number of topics are explored including:

• The Current Landscape

• The Current State of Searcher Design

• The General Architecture of Traditional Trading Systems

• Building a Modular Searcher Execution Platform

• The Potential Futures of Searcher Design

The Current Landscape

Opportunities on-chain come in many forms, driven by the activity of users who create this opportunity but don’t capture it themselves. This is generally labelled as MEV (or maximal extractable value) which often times falls into one of the three categories: frontrunning, backrunning or sandwich attacks.

Searchers will employ strategies to extract MEV and profit. A strategy can be to arbitrage USD-pegged stablecoin prices using MakerDAO and Curve or spotting unhealthy borrow-lend positions on Abracadabra to liquidate.

Some strategies revolve around one-off events (e.g arbitraging Otherside land claims) whereas others, though longer-term, may be tightly coupled with the unique implementation of the underlying protocol (e.g rebase farming on OHM forks). The non-uniformity of smart contract interfaces and esoteric underlying mechanisms that power these protocols have largely driven these strategies to be implemented as single-purpose programs that can be quite difficult to extend or toilsome to adapt.

Other opportunities are more frequent with a much larger ability to profit, drawing searchers to the potential payoff of these opportunities. Such an example is the DEX / DEX atomic arbitrage strategy. The top MEV winner in 2022 made over $3m in a single arbitrage by back-running large swap transactions during the Nomad bridge exploit, with many stories alike. Given the large competition in the market, many will build more thought-out trading systems for these types of opportunities, with speed and execution performance being a large focus.

More lucrative trading strategies come from taking on additional risk in some way; a common form is that of inventory risk, having exposure to assets that can potentially depreciate (or appreciate) in value for the duration it is being held in one’s portfolio. From mass minting and hodling NFTs to memetokens in sandwich attacks, these types of strategies come with a risk that the price of the asset can fall through and not profit. CEX / DEX arbitrage is too an example, in which the settlement on-chain varies to that of off-chain such that the searcher (or trader) involved faces some exposure to risk in between the time the trade is being settled on each venue. Every single trade by users can create an arbitrage opportunity to capitalise on, driving competition and optimisations: from strategic gas reductions at the contract-level, to custom node clients to win latency wars.

Different opportunities are dominated by various players, some big and some small, but all in that position generally have a notion to avoid giving away too much sensitive information that could be used by others to gain an edge or reverse engineer their strategy. As such, code is generally only open sourced when an organisation or project is wishing to push usage or that the underlying strategy is no longer profitable to continue running (a reverse survivorship bias).

Where solo searchers and smaller players tend to focus on the long-tail of opportunities, institutional trading firms and larger-sized MEV teams are more capable to capitalise on the short-tail ones with higher capacity, with the well-versed domain-knowledge optimisations, production capability and, overall, money to fund essential resources to continue and improve the execution of strategies.

The Current State of Searcher Design

Searchers are generally developed in three phases:

1. Strategy generation. Opportunities are spotted and the strategies to extract MEV from them are ideated upon. A simple proof-of-concept may be made to test part of the strategy as a sanity check.

2. Searcher implementation. Code is written that will listen to or poll for events, and upon meeting a certain trigger condition, constructs a transaction or bundle to exploit the opportunity.

3. Execution strategy. The transaction or bundle is then sent off to be executed on-chain in some fashion. This transaction supply chain is spoken about more here.

(Searcher implementation will be analysed here in this article, with strategy generation potentially being covered in a future article.)

As mentioned, the core of a searcher (bot) is as follows:

1. Listen or poll for events.

2. Check for a trigger condition.

3. Formulate some action to exploit opportunity.

4. Propagate action out to be executed on-chain.

Searchers can be implemented in most languages, but most tend to be programmed in Rust, TypeScript, JavaScript or Python to run intense computation and network calls with as little overhead as possible - developer support in the form of libraries e.g Hardhat, ethers-rs, ApeWorX etc. reduces development time substantially. Smart contracts, written in Solidity, Vyper or Huff, may be used to run some of this logic on-chain, but there needs to be some activation call to the smart contract to execute one of its function. There may be certain reasons why someone may choose to write their logic inside of a smart contract despite the upfront deploy cost. This will be touched upon later.

It is first worth showing various examples of how a searcher may be implemented for various strategies:

• Fuse by Rari Capital: Liquidator Bot: Fuse is a money market for borrowing and lending assets. If a user chooses to lend some collateral and then borrow against this, they can be at risk of liquidation by the protocol if their collateral to debt ratio falls under a certain threshold - this usually leads to the collateral being sold cheaper than market rate, leading to an arbitrage opportunity. This specific bot is written completely in JavaScript. For a set interval of time, it will make various RPC calls to fetch pool users and compute whether any of their positions are unhealthy (trigger) and how it would be best liquidated (action) - this is all done before each liquidation is sent off over RPC as its own transaction. The Fuse FuseSafeLiquidator contract is a periphery (helper) contract that offers a method that allows a bot to liquidate a position without needing to code their own smart contract - this is utilised to avoid the user having to implement their own smart contract that would have been the recipient of the flash loan.

• Yield Protocol Liquidator: Yield Protocol is a protocol that enables interest rate swaps on the yield of money markets - it does this by creating fixed yield tokens (fyTokens) of assets, and allowing one to purchase these fyTokens with the original asset (getting it at a discount) or through minting new fyTokens by borrowing it against some collateral - the latter presents an arbitrage opportunity should the borrow position become unhealthy due to not closing the borrow position after maturity. This specific bot is written in Rust, with an execution contract in Solidity. On each new block, the searcher bot will update its running list of borrowers for the fyDAI-ETH market and the health of their positions, before then checking to see which positions are unhealthy to then trigger an auction for that position. It then tries to participate in the auction by flash-loaning the funds from Uniswap V2 into the smart contract to then purchase the collateral.

• goblinmode: Goblintown is an NFT collection that received traction due to its clever marketing - with it’s success, the team behind the project released another collection that spun out off the Goblin theme known as McGoblinBurger. Holders could claim a McGoblinBurger NFT using their Goblintown NFT if it hadn’t been used to claim before. An arbitrage strategy from Anish Agnihotri revolved around the usage of NFTX, an automated market maker for NFTs that allows users to also flash loan NFTs, like Goblintown, as long as a user pays back the loan with interest (6% in Goblintown’s case). It is as follows: flash loan all Goblintown NFTs that hadn’t claim their McGoblinBurger, claim, and pay the interest with own capital - this would be profitable if sale price of the McGoblinBurger NFT is greater than that of the NFTX loan + gas. This strategy is implemented in Solidity, but requires a caller to execute it - this can either be done manually or by an off-chain script (bot) that makes the trigger call with the necessary inputs.

• Bundle Generator: DEX mispricing is one of most common opportunities that arise on-chain. By considering each asset as a node of a graph and each Uniswap-like pool as weighted edges, finding a cycle that starts from one asset and loops back to the asset with more of that asset is an arbitrage opportunity. This repository is an example of such, doing so in Rust. Several smart contracts are interfaced with, such as for batch querying multiple pairs.

Various templates exist online as well, offering a boilerplate to develop searchers on. These help speed up the process of implementation.

• degenbot - Python classes to aid rapid development of Uniswap V2 & V3 arbitrage bots on EVM-compatible blockchains

• Artemis - A framework developed by Paradigm for writing MEV bots in Rust, with a focus on simplicity, modularity and speed.

• Atomic Arbitrage - A base example of a bare implementation of an arbitrage bot written in Go.

• Qilin - A general purpose bot to provide modular components for new searchers, written in Rust.

• Hummingbot - An open source framework written in Python to run automated CeFi and DeFi trading strategies on any exchange or blockchain.

The design of many of these open-sourced implementations fall into the category of monoliths, in which there is cohesion between different components of the code under the context that it is specialised for a specific strategy e.g liquidating assets from Aave or performing generalised sandwich attacks. They can be used to validate a strategy with speed because of their leanness and ease in utilising context-specific optimisations to gain a greater edge than general competitors. Monolithic approaches do have their potential drawbacks to consider. Below, a few are spoken about.

Smart Contracts and Their Immutability

A searcher may choose to deploy a smart contract that can be called to execute some of the logic behind a strategy on-chain. This can be for several reasons, but most boil down to the attribute of atomicity.

One may choose to make a number of RPC calls to a node to fetch some on-chain data to then construct the transaction that they would like to make, but supposing that a block has passed during the round trip back, this data is likely to have become outdated. A smart contract can be implemented that, when called, performs this query without leaving the node and performs the rest of the strategy logic. It is also the case that because a transaction can only be sent to a single contract, atomicity for multiple contract calls is difficult for an EOA (externally owned account) unless a smart contract is used to perform these operations under one execution.

Deploying a smart contract relies on an initial upfront cost for gas to deploy the contract on-chain. For some strategies, this may be a relatively insignificant price to pay, with some opportunities returning this deployment cost in a single successful opportunity e.g DEX / DEX arbitrage, but for long-tail opportunities, this may not be the case. If the deployment cost of a contract onto the Ethereum blockchain is $150 in Ether, but the average return of some long-tail MEV strategy only returns $3 per day, it’ll take 50 days to breakeven, which may not be appropriate - this is also under the assumption that this strategy remains as profitable over the 50 days. This is also not helped by a contract having bugs or needing to be optimised further, of which the immutability of a smart contract must mean that a new contract would have to be deployed and switched to.

Context abstraction with a generalised execution contract can be used to minimise the number of contract deployments that have to be made across multiple strategies, elevating strategy logic from the smart contract to the off-chain part of the searcher. An on-chain execution engine can make arbitrary calls (requiring just to, value, data encoded in some format) meaning that any strategy can utilise that smart contract for their on-chain execution. One can also utilise such a contract to aggregate calls into a single transaction to amortise the base fee cost that is incurred on every transaction. Some dApps also feature this capability for non-technical users to utilise e.g Furucombo and DeFi Saver.

There are various open source examples of generalised execution contracts, including multicall or weiroll, of which the contract will act as a proxy for the calls. Other generalised execution contracts such as that of entrypoint contracts seen in the ERC-4337 supply chain take in meta-transactions that are each done by the respective account that has signed for that meta-transaction. These examples can also be adapted to utilise atomic constructs such as flash loans to have access to large enough capital to extract as much MEV as optimally possible.

It should be noted that generalised execution contracts do impose additional overheads in gas during the execution of any calls to offer their functionality, coming at the benefit of only having to pay the one-time fixed deployment cost. After a strategy has been validated live after multiple successes, it may make sense to scale the strategy and implement a custom smart contract with specialised logic to bring the gas cost down further. It is also the case that certain paradigms e.g conditional branching, runtime queries as parameters etc. may not be supported by certain generalised smart contract implementations.

Difficulty in Utilising Bundle-Level Optimisations

Given the composability offered by DeFi primitives and protocols, it is natural for teams running multiple strategies at times to have transactions or trades that conflict. A searcher running a DEX / DEX arbitrage strategy finding profitable cycles may have one leg of their trade that conflicts with that of a liquidation bot selling off a large amount of UNI to ETH. This can be minor in impact, causing additional bips in slippage to the other transactions, to one or more transactions reverting if they overlap.

Synchronisation across multiple strategies by a service called a bundler could be leveraged to find a bundle of transactions that offers the “best” execution when considering the transactions as a whole, utilising a number of optimisation tricks and techniques to achieve what is deemed as best. This can be done in a trustless or a trusted manner with access to private keys.

(The term “bundler” is one that is already used to describe a service that groups 4337 user-operations into a single transaction, but here the term is used to consider a generalised version of this that takes user operations, transactions, intents etc. and output a bundle of transactions.)

As mentioned, a number of tricks can be considered to provide optimisations at the bundle-level before it is sent off to a block builder.

• Internalisation of Trades: Trades that go in opposite directions can be internalised and be matched within two strategies to forego the usage of the underlying swap protocol (e.g Uniswap) and protocol fees as a result.

• Order Rerouting: If two transactions or trades are using the same pool and going in the same direction, it may make sense to reroute one to another venue to avoid causing slippage to the other transaction.

• Transaction Ordering: Transactions can be ordered in such a way that minimises gas cost but upholds the success of transactions. After asserting an ordering, it is also possible add additional transactions to capture any residue MEV.

The Cold Start Problem

The development of MEV strategies and their implementation is a time-consuming process. As a strategy lessens in profit, newer strategies may be considered for additional streams of revenue. Minimising the time to deploy a new strategy is crucial to capitalise on opportunities quickly, for which it makes sense to avoid duplicating code and repeating processes. This prompts the use of reusable components known as services (or on more technical terms, microservices).

Several examples, not exhaustive, can be thought of:

• A transaction propagation service: Transaction propagation is a key component of one’s MEV strategy, interfacing with RPCs, sequencers, own nodes etc. to broadcast a transaction so that it can be executed on-chain. This can be abstracted into its own service i.e a transaction gateway that handles the management of this.

• A DEX smart order router: Many strategies (from liquidators, keepers, and so on, to more traditional strategies such as NAV trading, mean reversion, momentum trading) require the liquidating of funds from one token to another e.g to the native token to pay for gas fees, to a stablecoin to neutralise (additional) inventory risk etc. A DEX smart order router taps into many decentralised exchanges to provide an optimal routing for these transactions. One could use a public DEX aggregator but these are limited to routing orders off top-of-the-block state. An inline smart order router allows one to simulate pending mempool transactions and swaps to find context-adjusted paths.

• EVM simulation suite: For one, it is not at all appropriate to test a strategy on a testnet like you would with a dApp. Testing live with a mainnet fork or EVM simulation helps to reduce worries for how the strategy will execute under certain conditions. A simulation suite can help standardise this process, catching some of the edge case of issues seen previously e.g weird ERC20 tokens.

Modularising logic from searcher bots can help to make development for future strategies easier, leveraging optimisations that have already been explored and implemented prior.

New infrastructure can be modularised on-the-go when deemed fit as new intricate problems arise, but such a methodology requires experimentation for us to build out a full suite of services for trading on-chain. We can look towards trading architectures used in traditional finance as inspiration for a proposed framework.

The General Architecture of Traditional Trading Systems

The culmination of modern trading systems comes from the work of many years of research, development and engineering in electronic trading. They largely follow the requirements of strategies and the traditional trade lifecycle.

One such depiction comes from the book Professional Automated Trading by Eugene A. Durenard.

The traditional trade lifecycle can be simplified to such:

• Exchange sends events to connected trading systems

• Trading system processes events to generate the set of actions it wants to perform on the exchange to maximise returns

• Actions are sent back to the exchange to be executed

Following the structure described in Durenard’s book, when an event comes in from the electronic communication network (ECN) layer:

1. It is semantically processed through an interfacing translation layer as to tell the trading system what has happened.

2. The resultant event data is used to update a local state representation of the market (e.g a price aggregator like an orderbook).

3. It is then shared with the brains of the trading system: the trading agents. Trading agents run in parallel to find trades or interactions that would ultimately give them a better return. These can be atomic in nature (arbitrage) or over time (trading).

4. Trade decisions are checked through a control layer before reaching the order management system (OMS), involving automated risk management and human control.

5. The OMS layer takes orders and finds ways to best execute them. In traditional finance, such execution algorithms include TWAP and VWAP. For similar orders that are done on the same exchanges, orders are aggregated through the order aggregator to reduce brokerage commission or other fees.

6. The translation layer handles the semantics of the trade into the necessary syntax over the required communication channel to relay the trade to the exchange.

7. The ECN layer returns a consequential acknowledgement and soon delivers a fill event back to us to let us know of a completed trade. For aggregated orders, these fills are then disaggregated.

8. Orders are also shared with middle and back-office layers for post-trade analysis that can help improve execution performance.

Building a Modular Searcher Execution Platform

As institutions get more involved within the space of DeFi, more sophisticated architectures compared to that of one-off searcher bots will begin to appear for those that wish to have better control over their execution on-chain.

If we consider translations of the traditional design over to DeFi, we can develop a framework that mimics such, a prediction into what searchers in the future can look like.

1. Collectors are used to collect events from a series of different sources (such as the public mempool, Order Flow Auctions, Alt Mempool etc.), extracting the semantic of events from the syntactic messages that are sent across the network or from the event source. A collector may exist for capturing emitted Uniswap swap events, another for listening to new Aave lend-borrow positions, all to allow for one strategy that liquidates Aave positions into Uniswap (for each strategy or pair there may be a unique Collector). The name “Collectors” comes from Artemis by Frankieislost at Paradigm.

2. Data aggregators take data from collectors and saves it into a representation that is semantically useful for the trading system e.g an orderbook that aggregates all central limit order books (CLOBs) on-chain. This can be for specific strategies or a class of strategies where modularity comes into play e.g the same graph of pool reserves from top-of-the-block state can be shown to trading agents running strategies on DEXs. Capturing the state of a protocol and being able to update it through events means that the cost of having to construct the full state upon load is amortised. For example, one can keep track of Uniswap V2 pools and their reserves by first retrieving an indexed version of reserves (e.g through an indexer like Subsquid, Transpose, The Graph to name a few) and then applying transformations based on the simulated swaps that occur.

3. Searchers / Executors (Trading Agents) - The logic behind a searcher’s strategy is what would be defined as the trading agent. They take the state of protocols provided to them and make a decision on whether any interactions in the form of a protocol interaction should be made. Under the context of a lending protocol, using the state of positions under that protocol, we can run through some logic checking for positions that are unhealthy. For any that are, we can package the operation to liquidate the position with another that sells off the collateral for our debt token back. We can also consider this under the context of inter-block liquidity provisioning (LP) in which our trading agent will decide if we should pull our liquidity from a certain protocol. By separating the logic of strategies away from the execution logic, they can utilise the optimisations from the rest of the system. We can also look to have agents work together e.g parallelised negative cycle detection on subgraphs of a saved DEX reserve graph, however, some synchronisation will need to occur at some point, often done at the bundler-level.

4. Automated Risk Management is something that helps to sanity check the decisions made by searchers. Tooling such as EVM simulations can be used as a simple smoke test for whether the gas costs will be covered, are too high, is there any potential profit to be extracted, Salmonella etc. We also consider the risk of the position as well e.g how will that trade change our portfolio allocation? What exposure will we have? Should we trade the profits of an atomic arbitrage strategy all back into stablecoins? How much is an acceptable amount of profit to actually consider executing this transaction?

5. Bundlers are very alike to that of a trading system’s order management system. They take in user transactions or operations that are translated into relevant order-management actions inside of a “transaction vessel” that is constructed in such a way that achieves “best” execution when considering all transactions in bulk. A smart order management system can utilise a number of techniques to reduce the cost of execution and manage conflict resolution of transactions. Operation aggregation is one of these techniques, relying on either account abstraction or full trust (access to all keys) to function, amortising the base fee cost introduced by EIP1559. Another is internalisation of trades i.e finding Coincidence-of-Wants that forgo an intermediary (and therefore associated fees). Bundlers can also take note of other chains and DEX aggregators to offer better order routing through considering inter-chain venues.

6. Gateways can be thought of as the translation layer of a trading system, providing the software client to process incoming and outgoing messages to and from the network. It abstracts away the network layer from the rest of the trading system, focusing on transmission of data.

7. Block builders are actors introduced by proposer-builder separation to offload the work of block construction away from validators. Given that blockchains packages transactions into blocks and not individually as a normal off-chain exchange would, block builders help to aggregate transactions into this form of blocks. A searcher can choose to broadcast their bundle to multiple existing builders, but one can also make their own block builder to process blocks for their own transactions and have more control over execution. The work involved with this includes having to receive orderflow from external sources (user transactions, OFA, Private RPC) to have a greater chance of having your block picked as the most valuable block in terms of block rewards to the validator. Though this role may be specific to current times (under mev-boost), we may see this role transform into that of gas-maximising bundlers in systems such as SUAVE.

8. Middle office contains sideline services that are off the critical path, but are still essential to the improvement of strategies. These can include:

   • Performance Analysis (Slippage, Executed Price, Reverts, Bundle Execution)

   • Competitor Analysis (Did a revert happen because of another searcher? Priority Gas Auction Relative Performance, How many other searchers are doing the same strategy etc.)

   • Latency Benchmarking (How is latency across the transaction supply chain; am I getting data fast enough or should I switch / integrate more data providers? etc.)

The Potential Futures of Searcher Design

A modular microservice architecture is just one design that the design space for searchers can potentially gravitate to, but there are many ways that searcher design can evolve in the future, all not mutually exclusive, and the potential new use cases for drawing attention to it.

Here are a few speculations:

• Searcher design will become more alike to supply chain infrastructure design. After seeing the vertical integration of searchers further down the transaction lifecycle by creating builders, it is likely that we’ll see searchers also look up towards services that are more user-facing e.g 4337 bundlers, paymasters etc. Such services are closely related to the components that searchers would have built (block builders, DEX order routers etc.) where it would make sense to consider turning some of their own services into products that accept external orderflow. We’ll also see current supply chain infrastructure take features observed in searcher design to find ways to better their performance. One good example of this is credibly neutral searching by block builders such as BuildAI, using AI to capture any residue MEV and return some additional profit back to the bundle sender, and ProtoRev by the Skip Protocol team.

• Creation of B2B MEV-as-a-service desks or DeFi execution platforms. In traditional finance, execution is key, but waiting several months to a year to create the needed trading infrastructure internally to deliver on this can be an obstacle. As such, some institutions e.g hedge funds choose to go with third-party systems so that they can deploy their strategies now e.g Systemathics, Bloomberg Terminal etc.. Rather than financial institutions choosing to hire their own engineers to build out their own DeFi execution platform, it may make sense for them to use a third-party agency execution platform such that they can roll out their MEV or on-chain HFT trading strategies quickly. Similar to above, it is likely that we’ll see searchers consider this as a potential product opportunity, given that the infrastructure is very alike. Several execution platforms already exist for trading on centralised exchanges e.g Talos, Caspian, Elwood but only a few have done such for trading on-chain e.g Eulith, Dexible, Vektor. These services can then make use of their position to vertically scale up the supply chain, with such volumes being visibly significant on-chain. We can see this also being applied to protocols that require (MEV) automation services as MEV-as-a-service desks (e.g Gelato, Keeper Network, Propeller Heads etc.).

• The space for searcher design will be experimented with more out in the open, as initiatives such as SUAVE, Anoma, CoWSwap utilise searchers for good. User-benefitting solutions such as orderflow auctions and RFQ systems have driven a positive narrative for searchers to be framed as automatic agents that can help the user benefit more through offering better price quotes or reclaiming MEV. Though the name may differ i.e executors, solvers etc., the design is generally similar. Given this, open-sourcing these bots is a positive sum for users and it is likely that we will see initiatives that drive the development of these searchers out in the open to drive collaborative innovation. Artemis from Paradigm is one of the great initiatives that is propelling this forward.

• There will be also be more searcher/infrastructure-related tooling built out to measure the effectiveness of initiatives and optimisations as we get closer and closer to the efficiency frontier in MEV extraction. Tooling such as performance analysis dashboards (slippage, executed price), competitor analysis etc. that analyse certain components of a searcher’s setup can be positioned as products that are utilised by other searchers. A number of solutions such as Eigenphi, Zeromev and Flashbots Explorer have helped in illuminating the dark forest, and it is likely that we’ll continue to see more dashboards that analyse or measure each component of the MEV supply chain, especially as participants, particularly searchers, become more involved.

Conclusion

This article layouts a modular framework that searchers can implement for their MEV trading architecture, acting as a foundation for adaptations and products that may take inspiration from some of its components. By considering the current monolithic paradigm of searcher design, it is possible to consider a modular approach as done so here.

Also discussed is a number of speculations into searcher design can evolve in the future, following from considering a modular approach. Given the initiatives that have managed to frame searcher bots as agents that can deliver additional benefits to the user, it is likely that searcher design will be discussed more and more openly in a collaborative manner to get as close to the theoretical limits of MEV.

Acknowledgments

We would like to thank Quintus Kilbourn, Ankit Chiplunkar, William Robinson, Dan Marzec and Ezon for their comments and discussions on the initial drafts of this article.

You can find us on Twitter at 0xTaker and its_mobeen - we’re both open to any conversations so please do reach out.

The Traditional Trade Lifecycle

Traditional limit order book exchanges host two-sided auctions that represent the market of some asset (with respect to the value of another asset). Sellers create asks for the price they want to sell at, while buyers create bids to buy at a certain price. When sellers and buyers cross, the intents are matched and orders are generated, providing information that can be useful for predicting market trends.

New ask and bid listings, along with information about the last trade price and size, are emitted as events/messages to connected clients, such as market makers and hedge funds. These clients generate a trade decision, be it an order placement or cancellation, based on the information received. The decision is then sent back to the exchange venue to be executed.

To simplify, the steps involved can be bullet-pointed as follows:

• Exchange sends events to connected trading systems

• Trading system processes events to generate the set of actions it wants to perform on the exchange to maximise returns

• Actions are sent back to the exchange to be executed

The Trade Lifecycle On-Chain

Trading strategies executed on-chain typically follow a similar flow:

• Trigger propagation: A node (or broadcaster) propagates either a pending transaction received from a user or a block of transactions proposed as the block proposer to connected peers and systems. This triggers the strategy to know when to run.

• Strategy execution: Trading systems process events to generate a set of operations/transactions they wish to perform on the blockchain to maximise returns.

• Transaction propagation: Transactions are propagated around the blockchain network or through other means (e.g PBS supply chain) to be received and executed by the next block proposer.

A trading strategy may consider working off a pending transaction (e.g., sandwiching, backrunning, etc.) or a proposed block (e.g., interest rate pushing borrow position into unhealthy position, rebase hopping) as its trigger event. Latency can play a monopolizing role in accessing the opportunity based on the time remaining to bid.

Solutions such as orderflow auctions (OFAs) and encrypted mempools aim to democratise both types of opportunities respectively by running sealed bid auctions off-chain and having arbitrageurs specify their intent to participate through bidding instead of latency-related investment.

Defining Performance Metrics

To remain competitive, trading firms monitor latency as one of their key performance indicators (KPIs). There are many sources of latency that exist in traditional trade lifecycles, including:

• Order validation by the exchange

• Exchange connectivity bandwidth

• Types of operations on the hot code path

• Latency from I/O

Similar sources of latency can be identified in blockchain-based systems as metrics to optimise. Below, I define four such metrics:

• Trigger propagation latency: the time it takes for a trigger event to be sent from a node, block proposer, or off-chain service, and then received by the trading system.

• Tick-to-trade: a metric borrowed from traditional finance. It measures the time it takes for a trading system to receive the trigger and then generate the trade decisions (transaction or transactions as a bundle) to execute.

• Transaction propagation latency: the time it takes for a transaction or bundle to be sent from the trading system and received by the next block proposer (pre-PBS), the block builder (PBS), the rollup sequencer (layer 2) or auction decider (OFA).

• Supply chain latency (PBS): the time it takes for a transaction or bundle, upon receipt by the builder, to be validated, included in the built block, and then sent across a relayer to the validator. This metric is recorded from the time of receipt by the builder to when the validator receives the block.

Various approaches are being looked into to minimise each. For example, minimising the number of peer-to-peer (P2P) hops on the network layer reduces the latency induced by intermediate nodes that have to validate a transaction before rebroadcasting it. For traditional finance system engineers, this should bring to mind several techniques for optimising the network layer, such as co-location and adapted clients for networking, among others. Some research is being looked at to reduce latency further up the supply chain as well, for example, optimistic relays.

Conclusion

This piece aims to discuss the possible segmentation of latencies that exist in the trade lifecycle for on-chain trading. These latencies can be defined as metrics that can be examined more closely. A number of optimisation techniques can be applied to each segment, which can be further discussed.

Acknowledgements

I would like to thank mempirate for comments and discussions on my earlier drafts.

I enjoy discussing topics such as this and am always happy to chat and collaborate on writing future pieces together.

My Twitter can be found here.