Early Research

We have come a long way since the birth of MEV, discovering its existence and uncovering its intricacies. When I say we have come a long way I mean research wise, cause all of this stuff is still in its infancy. MEV hasn’t really surfaced meaningfully before 2020, as most transactions (like basic transfers) were not generating any MEV. It’s only in 2017 with the advent and more extensive use of smart contracts that we really started to see some nascent MEV. While flying under the radar and little understood at that time, there wasn’t too much studies about the topic. It’s only in 2020 that research around the subject began to surface, notably with the Ethereum is a Dark Forest post by Paradigm that caught much of the attention. If you want to learn more about MEV and its intricacies I would recommend these two great reads: Encrypted Mempools and the MEV book. 

2020 is also the year that Flashbots came to life. One of the pioneers in extensive MEV research. They went further than just researching by also acting on their findings and trying to develop software that would mitigate MEV. First with MEV-geth for miners (pre-merge), then with MEV-boost for validators (post-merge). Their main goal is to prevent MEV leading to harmful network centralisation. Cause that is one of the big issues with MEV - next to worsened user experience (because a swapper can get sandwich attacked for example): an increased risk of validator centralisation due to more efficient validators having access to better hardware and better algorithms crowding out the less efficient ones. This requires computing power and beefy hardware which favours centralisation and prevents everyone from running such a strategy and node. The superior profits of those organisations could then get invested in more research and algorithm optimisation further driving and deepening the gap between sophisticated and unsophisticated validators. This is the main raisons d’être of Flashbots’ software: to democratise MEV by allowing every validator to plug in this sidecar tool and tap into searchers and builders that will extract MEV.

As a sidenote, the EF plans to enshrine a very similar mechanism with ePBS.

Ethereum Transaction Lifecycle

Past & Current - Public Mempool

To understand why MEV is actually a user’s choice, it is important to understand the lifecycle of a transaction (currently under MEV-boost). Here is a brief summary: you interact with a dapp (that will take care of constructing the transaction) and a wallet (allowing you to cryptographically sign the tx). The tx will then get relayed to a public RPC - most users will just use the default RPC set by their wallet. This RPC will then broadcast the tx to the p2p network and land in the public mempool of validating nodes. In that same mempool, searchers (bots running algorithms scanning all transactions  and try to extract MEV by re-ordering, censoring or inserting transactions) will create bundles and will submit them to the builder with a bribe (bid). The builder will then assemble multiple bundles together, as well as any transactions in the mempool to make the best possible block with the highest value. That block gets submitted with the bid of the builder to the validator in the hope that it gets chosen by the validator - MEV-boost is currently set-up for the validator to just select the block with the highest builder bid. Important to note is that you have numerous searchers and block builders competing against each other and hence why they will probably bid away most of the MEV that they could extract. This can be nuanced as you have different types of MEV, some more competitive than others (competitive MEV vs non-competitive MEV). This paper provides a good overview of these different types. One additional actor in this transaction supply chain is the middlemen between the validator and the block builder. That middleman is the relayer which will handle the bid payment from the builder to the validator. The purpose of this middleman is also in large part to avoid the validator stealing all of the MEV from the block builder. The validator will then ultimately propose the block to the network (composed of searcher bundles and other tx from the block builder). The outcome of this flow is that most of the value and MEV from a user-generated transaction go to the validator.

Current & Future - Private Order Flow

There is a way for the user (originator of the tx) to extract much of this value for himself and also avoid the bad externalities from MEV at the same time. This can be done by taking an alternative transaction supply chain. We are witnessing the birth of a multitude of those systems - most of which include some kind of order flow auction (OFA) to sell your good MEV to the highest bidder, while avoiding the bad one. To learn more about OFAs I recommend this article. The bottom line here is that most of the value and MEV generated by a user’s transaction will also flow back to the originator of that transaction. An important caveat to make here is that not all types of MEV are harmful as we saw - see appendix for a brief overview of main types of MEV. Some forms are even useful for DeFi and dapps: correct price reflection on an AMM (arbitrage MEV) or effective liquidation system for borrowing and lending protocols (liquidations MEV). Thus not all forms of MEV are harmful and undesirable for the user originating the transaction. It is important to make that distinction as not all forms of MEV will lead to a worsened user experience and needs to be mitigated. That’s why some private order flows construct it in a way to prevent harmful MEV (MEV blocker part) while allowing non-harmful MEV like backrunning for example (MEV share part). Some order flows that enable both these parts are Flashbots protect and MEV blocker. These tools will route your transaction via a private RPC that will submit it to a private mempool. Some information about your transaction is revealed with searchers so that they can backrun you. The registered builder(s) will then create the most profitable block with the protected transactions and the backrun bundles. You are thus trusting these searchers and builders to not exploit any harmful MEV out of your transaction.

If you are performing a swap it is important that you set your slippage appropriately as MEV protection and slippage protection are two distinct things !

Maximize your transaction value as a user

How can you take this peaceful and improved order flow path as a user? All you have to do is replace the default RPC of your wallet and maintain a custom RPC from an existing OFA (like Flashbots Protect or MEVBlocker). This will ignite an entirely different flow for your transactions and will prevent them from going to the public mempool, where extractive searchers are waiting to exploit your txs. The pretty neat thing with this is that you can also exert control over your transactions by manually configuring this custom RPC URL. In that sense it is possible to maintain aspects like: what information of my transaction will be divulged, which builders can be involved, do I want any revert protection or MEV refund (and if yes which percentage) etc. The extent of this configurability will depend on the tool that you are using. How I look at it is that the parameters you set are a trade-off between different aspects like privacy, monetary, latency and trust. Revert protection is a no-brainer and should be included as it has no downsides (no gas fees paid if your transaction reverts). Now for the other ones:

• Latency and trust: if you involve a higher number of block builders the likelihood of your transaction being picked up and confirmed quickly will increase. The downside is that you are trusting a larger set of block builders to not screw you over by still allowing predatory MEV to happen.

• Privacy, monetary and latency: the more information you obfuscate about your transaction, the harder it will be for your transaction to generate any MEV opportunity and thus the MEV payout to you or the validator will be lower. This also has an impact on the latency as this will imply a low or non-existent bribe to the validator and thus reduced likelihood of it getting taken up promptly. Related to this is the distribution of the MEV between the validator and the user. The refund determines the distribution of the searcher's payment among different addresses if your transaction is bundled. The less you leave on the table for validators the harder it will be for your transaction to get confirmed in a short time span. Suppose that you set it to 100 so that you want to get 100% of the MEV opportunity back as a user. That transaction (if it contains MEV) will take more time to get as one where you only demanded 10%. That is because 90% of the tx MEV will be used to bid for its inclusion to the validator and hence will get included in a block more quickly. So to conclude, the more that is paid to the originator, the less likely inclusion becomes in the next block. Another way to reduce latency is to set a higher priority fee when broadcasting the transaction. So in case you are in a hurry tip your validator. Those set of parameters are what I refer to as value. Value can thus be different to different people. So it is up to you to make the trade-off you want and select whichever configuration suits your priorities.

Network effects

One thing is sure though, you want to choose a route that is not riddled with sharks or other vicious MEV extracting animals. Hence why I do think it is important to make users aware of which alternatives exist. For now, the latency is likely worse than the public mempool, but this can change as more people adopt their tool and order flow - and which is also one of the reasons I am putting this post out. More users flocking to those private order flows could also ignite a virtuous flywheel. As less and less people use the general public mempool, the less opportunity for MEV extraction there will be. As most users would use OFAs, the MEV opportunities would increase and latency would get better as the number of transaction increases and validators would be incentivised to take those up as there are no other juicy alternatives. That’s also why I would argue against OFA fragmentation, we should consolidate and route our transactions to as few order flows as possible to achieve the greatest user experience - from a latency, privacy and financial perspective! Even though we might not have to come this far as we are seeing Blocknative came out with Transaction Boost which is a private RPC aggregator ‘that gives wallets and applications maximum observability over their users’ transactions’. This will allow users to participate in multiple OFAs at once - MEV Blocker, MEV-Share etc or send directly to the builders. I think you can start to see the picture here: More transactions = More MEV opportunities = More validator incentives (if part of MEV opportunity is shared with them) = Better Latency.

Concluding thoughts

So I hope that this post has made it clear that you as a user can manage MEV and this it is somewhat of a choice. I would like to finish off by saying that the users execute transactions and they should thus be the beneficiary of all of the value those generate - this is by no means a biased take as I am myself a validator. So choose wisely which (order) route you will take anon.

Through this I also want to covey that crypto and blockchain is all about empowering people. Let’s improve the user experience and onboard all those who wish to be. We can make a change and are the sole deciders of our faith. The social layer is the layer everything else is built on!

Appendix

Most common types of MEV

Harmful MEV

• Sandwiching: Exploiting retail users on decentralized exchanges, particularly targeting transactions with high slippage tolerance, using a strategy that involves creating a bundle with transactions both before and after the user's transaction to maximize profits. This type of MEV is considered harmful as it extracts value from retail users through worse execution.

• Generalized Frontrunning: Bots simulate the outcome of pending transactions in the mempool, identifying profitable opportunities and frontrunning those transactions by replacing original transaction details with their own, submitting them with higher gas fees to ensure execution. This form of MEV is considered harmful as it can negatively impact the fairness and integrity of transaction execution.

Not Harmful MEV

• Arbitrage: Searchers capitalize on mispricing across different trading venues, with a focus on decentralized exchanges (DEX). This involves taking advantage of pricing differences and is generally considered a legitimate form of MEV.

• Liquidations: Searchers execute backrunning strategies on on-chain lending protocols, taking advantage of positions near the liquidation threshold, thereby capitalizing on market movements and potential forced liquidations. Liquidations are often viewed as a necessary and constructive form of MEV in the decentralized finance (DeFi) ecosystem.

• Backrunning: Searchers integrate their MEV extracting transaction immediately after another user's transaction, optimizing their position in the transaction order. Backrunning, when executed in certain contexts like liquidations, may not be inherently harmful and can contribute to the efficiency of the market.

Early Research

We have come a long way since the birth of MEV, discovering its existence and uncovering its intricacies. When I say we have come a long way I mean research wise, cause all of this stuff is still in its infancy. MEV hasn’t really surfaced meaningfully before 2020, as most transactions (like basic transfers) were not generating any MEV. It’s only in 2017 with the advent and more extensive use of smart contracts that we really started to see some nascent MEV. While flying under the radar and little understood at that time, there wasn’t too much studies about the topic. It’s only in 2020 that research around the subject began to surface, notably with the Ethereum is a Dark Forest post by Paradigm that caught much of the attention. If you want to learn more about MEV and its intricacies I would recommend these two great reads: Encrypted Mempools and the MEV book. 

2020 is also the year that Flashbots came to life. One of the pioneers in extensive MEV research. They went further than just researching by also acting on their findings and trying to develop software that would mitigate MEV. First with MEV-geth for miners (pre-merge), then with MEV-boost for validators (post-merge). Their main goal is to prevent MEV leading to harmful network centralisation. Cause that is one of the big issues with MEV - next to worsened user experience (because a swapper can get sandwich attacked for example): an increased risk of validator centralisation due to more efficient validators having access to better hardware and better algorithms crowding out the less efficient ones. This requires computing power and beefy hardware which favours centralisation and prevents everyone from running such a strategy and node. The superior profits of those organisations could then get invested in more research and algorithm optimisation further driving and deepening the gap between sophisticated and unsophisticated validators. This is the main raisons d’être of Flashbots’ software: to democratise MEV by allowing every validator to plug in this sidecar tool and tap into searchers and builders that will extract MEV.

As a sidenote, the EF plans to enshrine a very similar mechanism with ePBS.

Ethereum Transaction Lifecycle

Past & Current - Public Mempool

To understand why MEV is actually a user’s choice, it is important to understand the lifecycle of a transaction (currently under MEV-boost). Here is a brief summary: you interact with a dapp (that will take care of constructing the transaction) and a wallet (allowing you to cryptographically sign the tx). The tx will then get relayed to a public RPC - most users will just use the default RPC set by their wallet. This RPC will then broadcast the tx to the p2p network and land in the public mempool of validating nodes. In that same mempool, searchers (bots running algorithms scanning all transactions  and try to extract MEV by re-ordering, censoring or inserting transactions) will create bundles and will submit them to the builder with a bribe (bid). The builder will then assemble multiple bundles together, as well as any transactions in the mempool to make the best possible block with the highest value. That block gets submitted with the bid of the builder to the validator in the hope that it gets chosen by the validator - MEV-boost is currently set-up for the validator to just select the block with the highest builder bid. Important to note is that you have numerous searchers and block builders competing against each other and hence why they will probably bid away most of the MEV that they could extract. This can be nuanced as you have different types of MEV, some more competitive than others (competitive MEV vs non-competitive MEV). This paper provides a good overview of these different types. One additional actor in this transaction supply chain is the middlemen between the validator and the block builder. That middleman is the relayer which will handle the bid payment from the builder to the validator. The purpose of this middleman is also in large part to avoid the validator stealing all of the MEV from the block builder. The validator will then ultimately propose the block to the network (composed of searcher bundles and other tx from the block builder). The outcome of this flow is that most of the value and MEV from a user-generated transaction go to the validator.

Current & Future - Private Order Flow

There is a way for the user (originator of the tx) to extract much of this value for himself and also avoid the bad externalities from MEV at the same time. This can be done by taking an alternative transaction supply chain. We are witnessing the birth of a multitude of those systems - most of which include some kind of order flow auction (OFA) to sell your good MEV to the highest bidder, while avoiding the bad one. To learn more about OFAs I recommend this article. The bottom line here is that most of the value and MEV generated by a user’s transaction will also flow back to the originator of that transaction. An important caveat to make here is that not all types of MEV are harmful as we saw - see appendix for a brief overview of main types of MEV. Some forms are even useful for DeFi and dapps: correct price reflection on an AMM (arbitrage MEV) or effective liquidation system for borrowing and lending protocols (liquidations MEV). Thus not all forms of MEV are harmful and undesirable for the user originating the transaction. It is important to make that distinction as not all forms of MEV will lead to a worsened user experience and needs to be mitigated. That’s why some private order flows construct it in a way to prevent harmful MEV (MEV blocker part) while allowing non-harmful MEV like backrunning for example (MEV share part). Some order flows that enable both these parts are Flashbots protect and MEV blocker. These tools will route your transaction via a private RPC that will submit it to a private mempool. Some information about your transaction is revealed with searchers so that they can backrun you. The registered builder(s) will then create the most profitable block with the protected transactions and the backrun bundles. You are thus trusting these searchers and builders to not exploit any harmful MEV out of your transaction.

If you are performing a swap it is important that you set your slippage appropriately as MEV protection and slippage protection are two distinct things !

Maximize your transaction value as a user

How can you take this peaceful and improved order flow path as a user? All you have to do is replace the default RPC of your wallet and maintain a custom RPC from an existing OFA (like Flashbots Protect or MEVBlocker). This will ignite an entirely different flow for your transactions and will prevent them from going to the public mempool, where extractive searchers are waiting to exploit your txs. The pretty neat thing with this is that you can also exert control over your transactions by manually configuring this custom RPC URL. In that sense it is possible to maintain aspects like: what information of my transaction will be divulged, which builders can be involved, do I want any revert protection or MEV refund (and if yes which percentage) etc. The extent of this configurability will depend on the tool that you are using. How I look at it is that the parameters you set are a trade-off between different aspects like privacy, monetary, latency and trust. Revert protection is a no-brainer and should be included as it has no downsides (no gas fees paid if your transaction reverts). Now for the other ones:

• Latency and trust: if you involve a higher number of block builders the likelihood of your transaction being picked up and confirmed quickly will increase. The downside is that you are trusting a larger set of block builders to not screw you over by still allowing predatory MEV to happen.

• Privacy, monetary and latency: the more information you obfuscate about your transaction, the harder it will be for your transaction to generate any MEV opportunity and thus the MEV payout to you or the validator will be lower. This also has an impact on the latency as this will imply a low or non-existent bribe to the validator and thus reduced likelihood of it getting taken up promptly. Related to this is the distribution of the MEV between the validator and the user. The refund determines the distribution of the searcher's payment among different addresses if your transaction is bundled. The less you leave on the table for validators the harder it will be for your transaction to get confirmed in a short time span. Suppose that you set it to 100 so that you want to get 100% of the MEV opportunity back as a user. That transaction (if it contains MEV) will take more time to get as one where you only demanded 10%. That is because 90% of the tx MEV will be used to bid for its inclusion to the validator and hence will get included in a block more quickly. So to conclude, the more that is paid to the originator, the less likely inclusion becomes in the next block. Another way to reduce latency is to set a higher priority fee when broadcasting the transaction. So in case you are in a hurry tip your validator. Those set of parameters are what I refer to as value. Value can thus be different to different people. So it is up to you to make the trade-off you want and select whichever configuration suits your priorities.

Network effects

One thing is sure though, you want to choose a route that is not riddled with sharks or other vicious MEV extracting animals. Hence why I do think it is important to make users aware of which alternatives exist. For now, the latency is likely worse than the public mempool, but this can change as more people adopt their tool and order flow - and which is also one of the reasons I am putting this post out. More users flocking to those private order flows could also ignite a virtuous flywheel. As less and less people use the general public mempool, the less opportunity for MEV extraction there will be. As most users would use OFAs, the MEV opportunities would increase and latency would get better as the number of transaction increases and validators would be incentivised to take those up as there are no other juicy alternatives. That’s also why I would argue against OFA fragmentation, we should consolidate and route our transactions to as few order flows as possible to achieve the greatest user experience - from a latency, privacy and financial perspective! Even though we might not have to come this far as we are seeing Blocknative came out with Transaction Boost which is a private RPC aggregator ‘that gives wallets and applications maximum observability over their users’ transactions’. This will allow users to participate in multiple OFAs at once - MEV Blocker, MEV-Share etc or send directly to the builders. I think you can start to see the picture here: More transactions = More MEV opportunities = More validator incentives (if part of MEV opportunity is shared with them) = Better Latency.

Concluding thoughts

So I hope that this post has made it clear that you as a user can manage MEV and this it is somewhat of a choice. I would like to finish off by saying that the users execute transactions and they should thus be the beneficiary of all of the value those generate - this is by no means a biased take as I am myself a validator. So choose wisely which (order) route you will take anon.

Through this I also want to covey that crypto and blockchain is all about empowering people. Let’s improve the user experience and onboard all those who wish to be. We can make a change and are the sole deciders of our faith. The social layer is the layer everything else is built on!

Appendix

Most common types of MEV

Harmful MEV

• Sandwiching: Exploiting retail users on decentralized exchanges, particularly targeting transactions with high slippage tolerance, using a strategy that involves creating a bundle with transactions both before and after the user's transaction to maximize profits. This type of MEV is considered harmful as it extracts value from retail users through worse execution.

• Generalized Frontrunning: Bots simulate the outcome of pending transactions in the mempool, identifying profitable opportunities and frontrunning those transactions by replacing original transaction details with their own, submitting them with higher gas fees to ensure execution. This form of MEV is considered harmful as it can negatively impact the fairness and integrity of transaction execution.

Not Harmful MEV

• Arbitrage: Searchers capitalize on mispricing across different trading venues, with a focus on decentralized exchanges (DEX). This involves taking advantage of pricing differences and is generally considered a legitimate form of MEV.

• Liquidations: Searchers execute backrunning strategies on on-chain lending protocols, taking advantage of positions near the liquidation threshold, thereby capitalizing on market movements and potential forced liquidations. Liquidations are often viewed as a necessary and constructive form of MEV in the decentralized finance (DeFi) ecosystem.

• Backrunning: Searchers integrate their MEV extracting transaction immediately after another user's transaction, optimizing their position in the transaction order. Backrunning, when executed in certain contexts like liquidations, may not be inherently harmful and can contribute to the efficiency of the market.