Rollups are horizontally scaling the application layer of Ethereum. But as computation is sharded across different rollups, the interoperability of applications running on these rollups becomes fragmented, significantly impacting user utility. Moreover, today rollups are operated by centralized servers that decide on which transactions to include and in what order (aka sequencing). Fragmented interoperability and sequencing centralization are some of the biggest challenges facing rollups today as they undermine the core benefits of running applications on Ethereum in the first place.
The Espresso Sequencer is a decentralized shared sequencing layer designed to solve these challenges for Ethereum rollups. As we advocated in a previous post, a shared decentralized sequencing layer has the potential to improve the overall decentralization and interoperability of Ethereum’s rollup-centric future.
Back in November, we released the Americano testnet, our first demonstration of HotShot, the consensus protocol underpinning the Espresso Sequencer. Today we published our paper on HotShot and are announcing our second major milestone and testnet: Doppio.
HotShot Recap
HotShot is a fast-finality consensus protocol designed to reach the same scale as Ethereum’s validator set — not just in theory, but in practice, as restaking will enable Ethereum validators to use their staked ETH to participate in operating the protocol. HotShot offers a property called optimistic responsiveness, which is the ability to confirm transactions as fast as the network will allow. We take advantage of this property by using a CDN-like architecture to boost performance when network conditions are favorable, achieving “Web2 performance with Web3 security”.
As a consensus protocol, HotShot is optimized for the use-case of lazy sequencing, where the nodes participating in the consensus protocol only need to agree on an ordering of available transactions — they do not need to execute these transactions or run a virtual machine. Moreover, given this relaxation, while the nodes must collectively ensure the availability of transaction data, there is no strict requirement that all nodes must receive all the data. This allows for data sharding that can help massively scale the communication efficiency and throughput of consensus without compromising on availability.
Data Sharding in Doppio
Data sharding with guaranteed availability, also known as verifiable information dispersal (VID), is one of the major additions to HotShot in the new Doppio release. In Doppio, each new block of transaction data is both stored by high-bandwith nodes and broadcasted to a small randomly sampled committee instead of the entire validator set. The committee generates a threshold signature on a hash of the data. All other nodes receive only this hash and signature, which convinces them of the data’s availability. Indeed, if fewer than 25% of the nodes are corrupt, then the probability a valid signature is created on unavailable data is negligibly small. The consensus protocol enables all nodes to reach agreement on an ordering of these hashes as references to available data.
An adversary may adaptively try to bribe members of these committees. To defend against this, every node participating in consensus may additionally receive an erasure code share of each transaction block’s data such that any 25% subset of the nodes are able to cooperatively recover the full data. The parameters of this erasure code can be tuned such that the overall amount of data that needs to be broadcasted per block stays constant even as the number of validators scale to tens of thousands of nodes. This capability has been implemented in Jellyfish, the cryptographic library maintained by Espresso Systems, and will be integrated into HotShot in our next release. This three-tiered data availability design, which we call Tiramisu, is explained in a separate post.
View Synchronization & Signature Aggregation
Two other major improvements to HotShot featured in the Doppio release include a new view synchronization subprotocol and signature aggregation for quorum certificates.
In leader-based consensus protocols that operate in a partially synchronous network, the participating nodes may temporarily have different views of the correct leader, preventing progress. A core part of the protocol is a view synchronization mechanism, sometimes called a Pacemaker, which ensures that (after GST) all nodes will enter the same view for a sufficiently long period of time. The previous version of HotShot implemented this mechanism using a naive “textbook” method called exponential backoff, which could lead to unbounded delays in the worst case. The new version adapts and integrates the Naor-Keidar protocol, which maintains very low communication and latency against static adversaries except with small probability, and still bounded latency with adaptive budget-constrained adversaries.
Leaders in HotShot collect many signatures from the other nodes participating in consensus. These signatures form a quorum certificate (QC) that is then broadcasted back to all the nodes. Signature aggregation enables the leader (or third-party) to compress these signatures into a much smaller collective signature that can be broadcasted instead, achieving asymptotically linear instead of quadratic communication. This is especially beneficial in a heterogeneous network where a few powerful servers can perform this aggregation step. When every server has identical bandwidth and computational resources this may not be better than peer-to-peer broadcast, but in the context of scaling consensus to thousands of nodes this will not realistically be the case. HotShot implements signature aggregation on high-bandwidth nodes, which are already leveraged to optimistically boost communication efficiency.
Rollup integration: Polygon zkEVM Stack
Finally, one of the most exciting features of Doppio is an end-to-end integration of the Espresso Sequencer with a fully functional zk-rollup, specifically a fork of the Polygon zkEVM. We have released a short demonstration of the end-user experience for submitting transactions to Polygon zkEVM fork running over the decentralized Espresso Sequencer. Doing so via MetaMask is identical to the experience of submitting transactions to Polygon’s rollup today. When on mainnet this integration, the gas costs of transactions are significantly lower as HotShot is relied upon for data availability, saving the need to store rollup transaction data directly on Ethereum (though this is still an option for those who wish). Read more about the architecture of our integration in the Espresso Sequencer documentation, and read more about the integration in our announcement here.
What’s Brewing: More Rollup Integrations, OP stack Support, and More
Over the coming months, we will be continuing to implement this architecture, improving on our benchmarks and adding benchmarks for the peer-to-peer network. We will also be implementing randomized committees for data availability and beginning early on-boarding of nodes.
Most notably, we are beginning to onboard rollups and rollup-as-a-service companies who are excited about the combination of decentralization, performance, and support for interoperability that the Espresso Sequencer can offer their users. We are developing an SDK and documentation in collaboration with an early set of partners as we support their integration work. We are thrilled to be working with Injective as they develop Cascade, their SVM rollup; Spire, who unveiled today their layer 3 rollup-as-a-service offering; AltLayer, who will offer the Espresso Sequencer as an option to developers. If you are developing a rollup or working on a rollup-as-a-service, and you are interested in this type of collaboration, please reach out to us here.
In addition to this work supporting these rollup teams, we will also be building our next integration with the OP Stack. We are thrilled to be collaborating with Caldera in order to develop and deploy an OP Stack rollup that will see transactions sequenced by the Espresso Sequencer. Our proposal to Optimism’s Leader Election Proof-of-Concept RFP was recently accepted, so we are also thrilled to be contributing to the Optimism Collective through that work. If you are building on the OP Stack and interested in getting early access to integrate with the Espresso Sequencer, we would love to hear from you here.
Summary
In summary, key highlights of this Doppio release of the Espresso Sequencer include:
- Efficiency improvements to HotShot (read paper here)
- Data committee sharding (first two layers of Tiramisu)
- View synchronization based on Naor-Keidar
- Signature aggregation of quorum certificates
- End-to-end integration with the Polygon zkEVM stack