OptimumIntroducing Optimum: Building the Missing Memory Layer for Web3
Blockchain technology has been heralded as the foundation of a new decentralized internet, now referred to as Web3. But as this "world computer" grows, it’s becoming increasingly clear that something fundamental is missing: a high-performance memory layer. Memory—the component that enables computers to store, access, and update data efficiently—is a key element of every computer on earth. From your laptop to a supercomputer, all rely on memory. This architecture, first outlined by John von Ne...
OptimumBattle of the Codes: RLNC vs Reed-Solomon & Fountain Codes
AuthorsKishori Konwar (Optimum), Muriel Médard (Optimum/MIT), Nicolas Nicolaou (Optimum)AbstractSending data over shaky or delayed connections often means it gets lost or delayed. The old way to deal with this was simply to resend anything that didn’t make it (like the TCP protocol does). But this approach has problems, especially on crowded networks where resending things just adds to the traffic jam and leads to further delays. Erasure codes offer a different solution. They work by adding e...
OptimumIntroducing OptimumP2P: Implementation and Benefits
OptimumP2P is coming. Currently in private testnet with a group of leading Ethereum validators and node operators, OptimumP2P boosts the speed and efficiency of communicating data between nodes. It leverages Random Linear Network Coding, a data coding technique our Co-founder Prof. Muriel Medard developed at MIT. RLNC is built specifically for achieving high throughput at scale on decentralized networks. To give a better understanding of how OptimumP2P works and its effects on blockchain ecos...
High-performance memory for the world computer.
Subscribe to Optimum
Subscribe to Optimum
High-performance memory for the world computer.
OptimumIntroducing Optimum: Building the Missing Memory Layer for Web3
Blockchain technology has been heralded as the foundation of a new decentralized internet, now referred to as Web3. But as this "world computer" grows, it’s becoming increasingly clear that something fundamental is missing: a high-performance memory layer. Memory—the component that enables computers to store, access, and update data efficiently—is a key element of every computer on earth. From your laptop to a supercomputer, all rely on memory. This architecture, first outlined by John von Ne...
OptimumBattle of the Codes: RLNC vs Reed-Solomon & Fountain Codes
AuthorsKishori Konwar (Optimum), Muriel Médard (Optimum/MIT), Nicolas Nicolaou (Optimum)AbstractSending data over shaky or delayed connections often means it gets lost or delayed. The old way to deal with this was simply to resend anything that didn’t make it (like the TCP protocol does). But this approach has problems, especially on crowded networks where resending things just adds to the traffic jam and leads to further delays. Erasure codes offer a different solution. They work by adding e...
OptimumIntroducing OptimumP2P: Implementation and Benefits
OptimumP2P is coming. Currently in private testnet with a group of leading Ethereum validators and node operators, OptimumP2P boosts the speed and efficiency of communicating data between nodes. It leverages Random Linear Network Coding, a data coding technique our Co-founder Prof. Muriel Medard developed at MIT. RLNC is built specifically for achieving high throughput at scale on decentralized networks. To give a better understanding of how OptimumP2P works and its effects on blockchain ecos...
Share Dialog
Share Dialog
>800 subscribers
>800 subscribers
Written by Aayush Rajasekaran & Har Preet
Singh Test environment created by Santiago Paiva & Alejandro Bergasov
OptimumP2P is a data propagation library that uses Random Linear Network Coding (RLNC)
We ran identical simulations to prior Ethereum research, and observed over 2x faster message arrival times than Gossipsub
We also ran experiments on real-world infrastructure, publishing 6 MB messages on a network of 36 nodes using first Gossipsub, and then OptimumP2P
OptimumP2P's average latency in these experiments was over 2x better than Gossipsub
We also observed that OptimumP2P scaled to larger message sizes (10 MiB and above), while Gossipsub started to fail to deliver messages under these circumstances
Improvements to gossip benefit all levels of a blockchain ecosystem; scaling, validator profitability, user experience, and enabling new use cases that need low latency to thrive
In peer-to-peer networks, nodes need to disseminate information to each other, a task known as gossip. Gossipsub is a widely-used algorithm to perform gossip in decentralized peer-to-peer networks, such as blockchains. Notably, Gossipsub is used to disseminate new beacon blocks on the Ethereum network.
Random Linear Network Coding (RLNC) is a form of network coding that has been studied in various settings, including gossip in peer-to-peer networks. Encoding messages to be gossiped using RLNC leads to faster propagation time (lower latency), and reduced duplicated transmissions (lower bandwidth usage). Indeed, it has been shown that incorporating RLNC into gossip leads to optimal behavior. OptimumP2P is a new RLNC-based fast data propagation protocol for any blockchain, starting with Ethereum.
We present the results of some simulations and experiments that we have run, comparing Gossipsub and OptimumP2P. The results confirm the theory: OptimumP2P is over twice as fast as Gossipsub, which does not utilize any network coding.
We used the Ethereum tool Ethshadow to simulate gossip in an Ethereum-like network. This tool is built on the general-purpose simulation library Shadow. We built off the work of prior Ethereum research, running the same simulations as them with 1,000 nodes, 20% of which have the bandwidth of 1Gbps/1Gbps and 80% of which have 50Mbps/50Mbps. The publisher always has 1Gbps/1Gbps in order to get consistent simulation results. The latencies between pairs of nodes are based on real-world geographic locations.
We first ran a simple experiment, in which a single publisher publishes a single message. We varied message sizes from 128KB to 4096 KB, and observed significantly faster arrival times for all message sizes, as shown below. We remark that our observed performance of Gossipsub matches the results here, which give us confidence in our reproducibility.
We then ran simulations in which a single publisher published multiple messages (up to 64), with each message having a size of 128 KB. Once again, we observed notably faster arrival times in all cases.
Given the promising results in our simulation, we decided to test things out on real-world infrastructure. Our experiments were run on a cluster of 36 geographically distributed nodes, connected to each other. We compared the result of publishing 6 MB messages, at a rate of 2 messages per second. OptimumP2P achieved an average latency of 1.17 seconds for message delivery, compared to Gossipsub's average latency of 2.35 seconds.
We also varied the message size, from 2 MB to 10 MB. We observed that OptimumP2P scales to larger messages significantly better than Gossipsub, with the performance gap widening as message size increased. We also note that at 10 MB, Gossipsub no longer successfully delivered messages to nodes, while OptimumP2P continued to perform with consistently low latency. This behavior is characteristic of congestion, where delay increases highly non-linearly with load. Watch below as we run two of these tests.
We also measured the standard deviations of the latencies, and observed that Gossipsub’s standard deviation was twice as much as in the case of OptimumP2P. This is significant, because it is known from queueing theory that a greater amount of variation leads to overall worse performance for any system.
Our results with OptimumP2P confirm what is already well-established in the literature: there are significant performance improvements when Random Linear Network Coding (RLNC) is incorporated into gossip systems. The impact of these improvements is wide-ranging. Protocols like Ethereum can benefit from faster propagation of larger blocks and blobs, allowing them to accommodate increasing demand for blockspace and maintain high performance during periods of network congestion. Individual validators benefit from higher staking APY through more timely proposal and attestation of more profitable blocks, while also seeing lower operational costs from more efficient resource usage. Additional downstream benefits exist at the application and end user levels, which we'll explore in future pieces.
We hope you’re equally excited about the work Optimum is doing! To follow along as testnet progress continues, please follow us here on X and join the community on Discord.
Scale Any Chain
Written by Aayush Rajasekaran & Har Preet
Singh Test environment created by Santiago Paiva & Alejandro Bergasov
OptimumP2P is a data propagation library that uses Random Linear Network Coding (RLNC)
We ran identical simulations to prior Ethereum research, and observed over 2x faster message arrival times than Gossipsub
We also ran experiments on real-world infrastructure, publishing 6 MB messages on a network of 36 nodes using first Gossipsub, and then OptimumP2P
OptimumP2P's average latency in these experiments was over 2x better than Gossipsub
We also observed that OptimumP2P scaled to larger message sizes (10 MiB and above), while Gossipsub started to fail to deliver messages under these circumstances
Improvements to gossip benefit all levels of a blockchain ecosystem; scaling, validator profitability, user experience, and enabling new use cases that need low latency to thrive
In peer-to-peer networks, nodes need to disseminate information to each other, a task known as gossip. Gossipsub is a widely-used algorithm to perform gossip in decentralized peer-to-peer networks, such as blockchains. Notably, Gossipsub is used to disseminate new beacon blocks on the Ethereum network.
Random Linear Network Coding (RLNC) is a form of network coding that has been studied in various settings, including gossip in peer-to-peer networks. Encoding messages to be gossiped using RLNC leads to faster propagation time (lower latency), and reduced duplicated transmissions (lower bandwidth usage). Indeed, it has been shown that incorporating RLNC into gossip leads to optimal behavior. OptimumP2P is a new RLNC-based fast data propagation protocol for any blockchain, starting with Ethereum.
We present the results of some simulations and experiments that we have run, comparing Gossipsub and OptimumP2P. The results confirm the theory: OptimumP2P is over twice as fast as Gossipsub, which does not utilize any network coding.
We used the Ethereum tool Ethshadow to simulate gossip in an Ethereum-like network. This tool is built on the general-purpose simulation library Shadow. We built off the work of prior Ethereum research, running the same simulations as them with 1,000 nodes, 20% of which have the bandwidth of 1Gbps/1Gbps and 80% of which have 50Mbps/50Mbps. The publisher always has 1Gbps/1Gbps in order to get consistent simulation results. The latencies between pairs of nodes are based on real-world geographic locations.
We first ran a simple experiment, in which a single publisher publishes a single message. We varied message sizes from 128KB to 4096 KB, and observed significantly faster arrival times for all message sizes, as shown below. We remark that our observed performance of Gossipsub matches the results here, which give us confidence in our reproducibility.
We then ran simulations in which a single publisher published multiple messages (up to 64), with each message having a size of 128 KB. Once again, we observed notably faster arrival times in all cases.
Given the promising results in our simulation, we decided to test things out on real-world infrastructure. Our experiments were run on a cluster of 36 geographically distributed nodes, connected to each other. We compared the result of publishing 6 MB messages, at a rate of 2 messages per second. OptimumP2P achieved an average latency of 1.17 seconds for message delivery, compared to Gossipsub's average latency of 2.35 seconds.
We also varied the message size, from 2 MB to 10 MB. We observed that OptimumP2P scales to larger messages significantly better than Gossipsub, with the performance gap widening as message size increased. We also note that at 10 MB, Gossipsub no longer successfully delivered messages to nodes, while OptimumP2P continued to perform with consistently low latency. This behavior is characteristic of congestion, where delay increases highly non-linearly with load. Watch below as we run two of these tests.
We also measured the standard deviations of the latencies, and observed that Gossipsub’s standard deviation was twice as much as in the case of OptimumP2P. This is significant, because it is known from queueing theory that a greater amount of variation leads to overall worse performance for any system.
Our results with OptimumP2P confirm what is already well-established in the literature: there are significant performance improvements when Random Linear Network Coding (RLNC) is incorporated into gossip systems. The impact of these improvements is wide-ranging. Protocols like Ethereum can benefit from faster propagation of larger blocks and blobs, allowing them to accommodate increasing demand for blockspace and maintain high performance during periods of network congestion. Individual validators benefit from higher staking APY through more timely proposal and attestation of more profitable blocks, while also seeing lower operational costs from more efficient resource usage. Additional downstream benefits exist at the application and end user levels, which we'll explore in future pieces.
We hope you’re equally excited about the work Optimum is doing! To follow along as testnet progress continues, please follow us here on X and join the community on Discord.
Scale Any Chain
No activity yet