As promised in the previous update, this one will cover the post-merge Ethereum client architecture in depth. With the progress made during the Amphora Interop event, the specification is now close to final 🎉
Before we dive into The Merge, though, a short update on the difficulty bomb!
On AllCoreDevs #124 (recording, tweets), we agreed to push back the difficulty bomb, set to go off in December 2021, to June 2022. To do so, we will have a network upgrade, Arrow Glacier, which contains a single EIP, EIP-4345, that pushes back the bomb.
Arrow Glacier is scheduled for block 13,773,000, expected on December 8th, 2021.
We debated various options for the Ice Age pushback on AllCoreDevs. June was chosen because we had reasonable confidence that The Merge could happen before, and we wanted to avoid having to organize another difficulty bomb pushback right before it.
Of course, The Merge is independent from the bomb: it will require a separate network upgrade and is activated based on a PoW total difficulty threshold. This means that we do not need to "wait" for the difficulty bomb to go off in order to transition Ethereum to proof of stake. Similarly, if we encountered issues with the transition, we could decide to push back the bomb once more.
Hopefully, Arrow Glacier will be the last network upgrade on PoW Ethereum 🤞🏻 Onto The Merge!
The architecture for The Merge leverages the fact that Ethereum already has battle-tested clients built for both its execution (Eth1) and beacon (Eth2) chains. Given that these exist, it makes sense to keep using them.
At a high-level, at The Merge, clients will switch from following PoW to following PoS to determine Ethereum's latest valid block. Aside from that, most of the clients' functionality, and, more importantly, the EVM, its state, and how it executes transactions, will stay the same.
Post-merge, the current Eth1 and Eth2 clients respectively become the execution and consensus layers (or engines) of Ethereum. This means that node operators of either Eth1 or the Beacon Chain clients will need to run the "other half" of the stack to have a fully validating node. Danny Ryan has great diagrams illustrating this. They've been minted as NFTs, with the proceeds going towards the engineers and researchers working on The Merge.
The above shows what a full Ethereum client will look like post-merge. Let's use this as our starting point to dive into each component!
Today, Beacon Nodes are tasked with coming to consensus on empty (from end users' standpoint!) blocks. These contain pieces of consensus-related information, called Operations, such as attestations, the deposit contract root and validator slashings/exits, but do not contain any "transactions" in the Eth1 sense (e.g. sending ETH or interacting with smart contracts). The Merge is where this changes.
As The Merge happens, Beacon Nodes will watch the current PoW chain and wait for it to hit a predefined total difficulty threshold, called TERMINAL_TOTAL_DIFFICULTY. Once a block with total difficulty >= TERMINAL_TOTAL_DIFFICULTY is produced, it will be considered the final PoW block. Subsequent blocks will be produced and attested to by the validators on the Beacon Chain.
To do this, Beacon Nodes will communicate with their execution engine (formerly the Eth1 client) and ask it to either produce or validate ExecutionPayloads. These payloads are the post-merge equivalent of Eth1 blocks. They contain information such as the parent's hash, the state root, the base fee, and a list of transactions to execute. Once these have been produced or validated, the Beacon Node will share them with other nodes on the p2p network.
To establish communication between the consensus and execution layers, a new set of JSON RPC endpoints is introduced: the Engine API.
The Engine API is the communication interface between the consensus and execution layers. It is exposed on a separate port than the execution layer's public JSON RPC API. For simplicity, calls to the API are always initiated by the consensus layer and the API introduces only three methods: engine_executePayload, engine_forkchoiceUpdated and engine_getPayload. Let's go over what each of these does:
engine_executePayload asks the execution layer to validate that an ExecutionPayload conforms to all protocol rules.
After receiving a payload via this call, the execution layer will either return VALID/INVALID or, if it still has not synced to the tip of the chain, SYNCING. Because a block's validity is dependent on its parents' validity, if the execution layer is missing historical data to assess the payload's validity, it will fetch it from the network.
engine_forkchoiceUpdated is how the consensus layer informs the execution layer of a new head and finalized block on the network. In the case where the consensus layer needs the execution layer to produce a new ExecutionPayload on top of the latest head block, it will pass a payloadAttributes field along with this call.
ThepayloadAttributes field contains information relevant for the execution engine to produce an ExecutionPayload, specifically the timestamp, random and feeRecipient (formerly coinbase) values. Upon receiving this call, the execution layer will update its head, sync as needed, and, if required, begin building an ExecutionPayload with the values from payloadAttributes.
engine_getPayload
... and that's it! With these three new endpoints, the consensus and execution layers can communicate information about the state of the chain and transactional payloads. Now, let's dive into how the execution engine works some more.
As stated above, the execution engine is what becomes of Eth1 clients after The Merge. At that point, anything related to consensus is removed from their purview. Their main focus becomes state management, as well as block creation and validation, which is modified slightly. The bulk of the changes are described in EIP-3675.
First, The Merge will require some change to the block format. Several fields, which are only relevant for PoW and not PoS, will be set to 0 (or their data-structure's equivalent). These are fields which either relate to mining (difficulty, mixHash, nonce) or to ommers (ommers, ommersHash), which are not possible in PoS. The length of extraData will also be capped to 32 bytes on mainnet.
Second, because issuance will only happen on the Beacon Chain post-merge, the execution layer will stop processing block and uncle rewards. That being said, the execution engine will still be in charge of handling transaction fees. Indeed, as it creates an ExecutionPayload, the execution engine ensures that all transaction senders can pay at least the current baseFeePerGas and that any extra fees are sent to feeReceipient. Note that feeReceipient refers to a "legacy" Ethereum address, and not a Beacon Chain validator.
Third, once PoS has taken over from PoW, execution engines will stop gossiping blocks. This means deprecating the NewBlockHashes (0x01) and NewBlock (0x07) handlers at the p2p layer. Again, the execution layer will still be in charge of syncing the network state, gossiping transactions and maintaining its transaction pool.
The following diagram, again by Danny Ryan, shows the execution layer dropping PoW and relying on the Beacon Chain as The Merge happens.
We've now covered all the core components of how clients process blocks and communicate internally post-merge. Now, let us briefly touch on the various "edges" of the system.
As the first diagram in this post shows, post-merge, both the execution and Beacon Chain layers participate in a p2p network. Aside from block gossiping being deprecated on the execution layer side, everything else in the p2p network remains unchanged: Beacon Nodes will gossip attestations, slashings, etc. and the execution layer will share transactions, sync state, etc. each on their independent p2p network.
Similarly, the user APIs for both the Beacon Chain and the execution layer will remain independent, with the exception of the newly created Engine API.
The one component which will live across both layers is sync. Several syncing strategies are being developed for every possible edge case pre and post merge. These are still being refined and tested, and may be the subject of a future deep dive.. 👀
Since Amphora, the focus has been on specification refinements and devnet testing. Over the coming weeks, expect the specifications to have settled to a point where we do not expect any more major modifications.
In the meantime, the Pithos testnet is up and running with more client combinations being tested daily, and a community call is scheduled for next week for infrastructure & tooling providers to get up to speed on The Merge. See you then 👋🏻
ExecutionPayloadengine_forkChoiceUpdatedThis is how, when a validator must produce a block, it gets a valid one from its execution engine. Other nodes will then, upon receiving that block from the p2p layer, call engine_executePayload to assess its validity.
Tim BeikoProtocol Guild Pledge
Important caveats: This piece assumes context about Protocol Guild and narrowly focuses on its economic implications. For an introduction to PG, see Cheeky Gorilla’s EDCON talk. For a more philosophical read on its role, see Trent Van Epps’s article on Capital and Enclosure in Software Commons.A rough, quantitative framing of Protocol Guild’s mission is to make contributing to Ethereum L1 R&D economically rational on a risk-adjusted basis, while avoiding capture. With our pilot concluded and ...
Tim BeikoAllCoreDevs Update 016 ⛓
Welcome to another AllCoreDevs update 👋 Over the past six months or so, teams have been focused on Dencun, the next Ethereum upgrade, as well as some EIP process tweaks. This post covers both these topics, and briefly touches on planning for the upgrade.TL;DR 👀Dencun testnets 🔜The upgrade is in its final testing stages, with testnet upgrades expected to begin early next year.There’s more than danksharding in Dencun: EIP-7569 has the full list, and this post summarizes every included EIP ✅M...
Tim BeikoAllCoreDevs 2024 Recap & 2025 Outlook
As promised in my recent Pectra update, here’s a follow-up post covering this year’s changes to AllCoreDevs and what’s on the horizon for 2025 and beyond!ACD Process ImprovementsThe first small but important process change of 2024 came with Reth publishing their team’s preferences for the Pectra hard fork. Other execution and consensus layer client teams quickly adopted this written approach. While we had previously attempted to gather async input on upgrade scoping, Pectra marked the first t...
As promised in the previous update, this one will cover the post-merge Ethereum client architecture in depth. With the progress made during the Amphora Interop event, the specification is now close to final 🎉
Before we dive into The Merge, though, a short update on the difficulty bomb!
On AllCoreDevs #124 (recording, tweets), we agreed to push back the difficulty bomb, set to go off in December 2021, to June 2022. To do so, we will have a network upgrade, Arrow Glacier, which contains a single EIP, EIP-4345, that pushes back the bomb.
Arrow Glacier is scheduled for block 13,773,000, expected on December 8th, 2021.
We debated various options for the Ice Age pushback on AllCoreDevs. June was chosen because we had reasonable confidence that The Merge could happen before, and we wanted to avoid having to organize another difficulty bomb pushback right before it.
Of course, The Merge is independent from the bomb: it will require a separate network upgrade and is activated based on a PoW total difficulty threshold. This means that we do not need to "wait" for the difficulty bomb to go off in order to transition Ethereum to proof of stake. Similarly, if we encountered issues with the transition, we could decide to push back the bomb once more.
Hopefully, Arrow Glacier will be the last network upgrade on PoW Ethereum 🤞🏻 Onto The Merge!
The architecture for The Merge leverages the fact that Ethereum already has battle-tested clients built for both its execution (Eth1) and beacon (Eth2) chains. Given that these exist, it makes sense to keep using them.
At a high-level, at The Merge, clients will switch from following PoW to following PoS to determine Ethereum's latest valid block. Aside from that, most of the clients' functionality, and, more importantly, the EVM, its state, and how it executes transactions, will stay the same.
Post-merge, the current Eth1 and Eth2 clients respectively become the execution and consensus layers (or engines) of Ethereum. This means that node operators of either Eth1 or the Beacon Chain clients will need to run the "other half" of the stack to have a fully validating node. Danny Ryan has great diagrams illustrating this. They've been minted as NFTs, with the proceeds going towards the engineers and researchers working on The Merge.
The above shows what a full Ethereum client will look like post-merge. Let's use this as our starting point to dive into each component!
Today, Beacon Nodes are tasked with coming to consensus on empty (from end users' standpoint!) blocks. These contain pieces of consensus-related information, called Operations, such as attestations, the deposit contract root and validator slashings/exits, but do not contain any "transactions" in the Eth1 sense (e.g. sending ETH or interacting with smart contracts). The Merge is where this changes.
As The Merge happens, Beacon Nodes will watch the current PoW chain and wait for it to hit a predefined total difficulty threshold, called TERMINAL_TOTAL_DIFFICULTY. Once a block with total difficulty >= TERMINAL_TOTAL_DIFFICULTY is produced, it will be considered the final PoW block. Subsequent blocks will be produced and attested to by the validators on the Beacon Chain.
To do this, Beacon Nodes will communicate with their execution engine (formerly the Eth1 client) and ask it to either produce or validate ExecutionPayloads. These payloads are the post-merge equivalent of Eth1 blocks. They contain information such as the parent's hash, the state root, the base fee, and a list of transactions to execute. Once these have been produced or validated, the Beacon Node will share them with other nodes on the p2p network.
To establish communication between the consensus and execution layers, a new set of JSON RPC endpoints is introduced: the Engine API.
The Engine API is the communication interface between the consensus and execution layers. It is exposed on a separate port than the execution layer's public JSON RPC API. For simplicity, calls to the API are always initiated by the consensus layer and the API introduces only three methods: engine_executePayload, engine_forkchoiceUpdated and engine_getPayload. Let's go over what each of these does:
engine_executePayload asks the execution layer to validate that an ExecutionPayload conforms to all protocol rules.
After receiving a payload via this call, the execution layer will either return VALID/INVALID or, if it still has not synced to the tip of the chain, SYNCING. Because a block's validity is dependent on its parents' validity, if the execution layer is missing historical data to assess the payload's validity, it will fetch it from the network.
engine_forkchoiceUpdated is how the consensus layer informs the execution layer of a new head and finalized block on the network. In the case where the consensus layer needs the execution layer to produce a new ExecutionPayload on top of the latest head block, it will pass a payloadAttributes field along with this call.
ThepayloadAttributes field contains information relevant for the execution engine to produce an ExecutionPayload, specifically the timestamp, random and feeRecipient (formerly coinbase) values. Upon receiving this call, the execution layer will update its head, sync as needed, and, if required, begin building an ExecutionPayload with the values from payloadAttributes.
engine_getPayload asks the execution layer to return its best ExecutionPayload whose build process was previously initiated in an associated call to engine_forkChoiceUpdated.
This is how, when a validator must produce a block, it gets a valid one from its execution engine. Other nodes will then, upon receiving that block from the p2p layer, call engine_executePayload to assess its validity.
... and that's it! With these three new endpoints, the consensus and execution layers can communicate information about the state of the chain and transactional payloads. Now, let's dive into how the execution engine works some more.
As stated above, the execution engine is what becomes of Eth1 clients after The Merge. At that point, anything related to consensus is removed from their purview. Their main focus becomes state management, as well as block creation and validation, which is modified slightly. The bulk of the changes are described in EIP-3675.
First, The Merge will require some change to the block format. Several fields, which are only relevant for PoW and not PoS, will be set to 0 (or their data-structure's equivalent). These are fields which either relate to mining (difficulty, mixHash, nonce) or to ommers (ommers, ommersHash), which are not possible in PoS. The length of extraData will also be capped to 32 bytes on mainnet.
Second, because issuance will only happen on the Beacon Chain post-merge, the execution layer will stop processing block and uncle rewards. That being said, the execution engine will still be in charge of handling transaction fees. Indeed, as it creates an ExecutionPayload, the execution engine ensures that all transaction senders can pay at least the current baseFeePerGas and that any extra fees are sent to feeReceipient. Note that feeReceipient refers to a "legacy" Ethereum address, and not a Beacon Chain validator.
Third, once PoS has taken over from PoW, execution engines will stop gossiping blocks. This means deprecating the NewBlockHashes (0x01) and NewBlock (0x07) handlers at the p2p layer. Again, the execution layer will still be in charge of syncing the network state, gossiping transactions and maintaining its transaction pool.
The following diagram, again by Danny Ryan, shows the execution layer dropping PoW and relying on the Beacon Chain as The Merge happens.
We've now covered all the core components of how clients process blocks and communicate internally post-merge. Now, let us briefly touch on the various "edges" of the system.
As the first diagram in this post shows, post-merge, both the execution and Beacon Chain layers participate in a p2p network. Aside from block gossiping being deprecated on the execution layer side, everything else in the p2p network remains unchanged: Beacon Nodes will gossip attestations, slashings, etc. and the execution layer will share transactions, sync state, etc. each on their independent p2p network.
Similarly, the user APIs for both the Beacon Chain and the execution layer will remain independent, with the exception of the newly created Engine API.
The one component which will live across both layers is sync. Several syncing strategies are being developed for every possible edge case pre and post merge. These are still being refined and tested, and may be the subject of a future deep dive.. 👀
Since Amphora, the focus has been on specification refinements and devnet testing. Over the coming weeks, expect the specifications to have settled to a point where we do not expect any more major modifications.
In the meantime, the Pithos testnet is up and running with more client combinations being tested daily, and a community call is scheduled for next week for infrastructure & tooling providers to get up to speed on The Merge. See you then 👋🏻
Tim BeikoProtocol Guild Pledge
Important caveats: This piece assumes context about Protocol Guild and narrowly focuses on its economic implications. For an introduction to PG, see Cheeky Gorilla’s EDCON talk. For a more philosophical read on its role, see Trent Van Epps’s article on Capital and Enclosure in Software Commons.A rough, quantitative framing of Protocol Guild’s mission is to make contributing to Ethereum L1 R&D economically rational on a risk-adjusted basis, while avoiding capture. With our pilot concluded and ...
Tim BeikoAllCoreDevs Update 016 ⛓
Welcome to another AllCoreDevs update 👋 Over the past six months or so, teams have been focused on Dencun, the next Ethereum upgrade, as well as some EIP process tweaks. This post covers both these topics, and briefly touches on planning for the upgrade.TL;DR 👀Dencun testnets 🔜The upgrade is in its final testing stages, with testnet upgrades expected to begin early next year.There’s more than danksharding in Dencun: EIP-7569 has the full list, and this post summarizes every included EIP ✅M...
Tim BeikoAllCoreDevs 2024 Recap & 2025 Outlook
As promised in my recent Pectra update, here’s a follow-up post covering this year’s changes to AllCoreDevs and what’s on the horizon for 2025 and beyond!ACD Process ImprovementsThe first small but important process change of 2024 came with Reth publishing their team’s preferences for the Pectra hard fork. Other execution and consensus layer client teams quickly adopted this written approach. While we had previously attempted to gather async input on upgrade scoping, Pectra marked the first t...
Subscribe to Tim Beiko
Subscribe to Tim Beiko
Share Dialog
Share Dialog
>200 subscribers
>200 subscribers
No activity yet