BTC APR sounds sexy, but it could be a house of cards- built on layers of altcoin incentives that collapse at any time.

Is the yield paid in BTC or altcoins? What risks are you subject to? What's the potential principal loss? How sustainable is it? Does it dilute as TVL grows?

This article focuses on sustainable BTC-denominated yields across CeFi and DeFi:

• Original Yield Sources: Quant Trading, DEX LP, Lending, Staking, Collateral, LSTs and Pendle

• A New Venue for BTC Yield: Introducing @yieldbasis

• Looking Ahead: • Minor Errors Can Lead to Catastrophic Failures • The Scarcity of Elite Quant Teams • The Convergence of TradFi, CeFi, and DeFi, and IPO Opportunities

Original BTC Yield Sources

Despite the many ways for looping and compounding, we can categorize the original yield sources into 5 major buckets: Quant Trading, DEX LP, Lending, Staking, and Collateral.

1) Quant Trading Strategies: A “Zero-Sum” Game

Make sure your alpha strategy is net profitable. Arbitrage strategies include funding rate, spot-futures basis, cross-exchange, and lending arbs, sometimes involving event-driven trades. Quant requires deep liquidity— now primarily on TradFi and CeFi. Also, TradFi to DeFi arbitrage lacks cross-venue infra.

• Yield in BTC: Vary based on asset size, risk profile, and execution. Market-neutral strategies might target 4–8% BTC-based APR with ~1% stop-loss. Top-performing quant teams even pursue 200–300%+ APR with sophisticated risk controls around 10-30% stop-loss.

• Risks: Highly subjective, have model, judgment, and execution risks- even neutral strategies could fall into directional bets. Requires real-time monitoring, robust infra (e.g. latency, custody, and delivery agreements), insurance against loss, and trading venue risk controls.

2) DEX LP: Capped by Supply and Demand

DEXs also facilitate real trading volume beyond arbs. Now only ~3% of wrapped BTC is in DEXs due to limited supply and demand. In volatile pair LPing (e.g. WBTC-USDC), supply is constrained by impermanent loss, while demand faces wrapped BTC friction, and limited utility in DeFi.

• Yield in BTC: Fluctuated, @Uniswap is at 6.88% APR now, can be up to double digits.

• Risks: Due to impermanent loss, simply holding BTC typically outperforms providing liquidity. However new LPs are often misled, which reflects a common psychological bias: fee yields and APRs are highly visible metrics, tempting LPs to maximize short-term gains while overlooking the less obvious long-term capital losses. Standard DeFi risks also apply here.

3) Lending: BTC Loans

BTC serves mainly as USD or stablecoins borrowing collateral used in looping or leveraged trading, rather than focusing on the APR for lending out BTC—due to current low borrowing demand.

• Yield in BTC: CeFi and DeFi lending rates are typically low, around 0.02%-0.5% APR. Loans’ LTV ratios vary: TradFi 60-75% LTV with currently 2-3% prime rate, CeFi 33-50% LTV with currently ~7% USDC rate, and DeFi 33-67% with currently ~5.2% USDC rates.

• Risks: Liquidation risks, though low LTV ratios help at the cost of capital efficiency. Hedging strategies can offer additional protection. CeFi and DeFi risks exist.

4) Staking: Earning Alt Reward

Babylon is at a unique position, where staking contributes to the security of associated PoS chains.

• Yield in Altcoins: Denominated in altcoins, APR unknown.

• Risks: Babylon protocol should undergo multiple security audits and disclose estimated staking yields once the system is live. If the Babylon token launch isn’t successful, the ecosystem sustainability is at risk.

5) Collateral: Liquidity Mining

When you provide BTC to DeFi, BTC L2s, and others as TVL to earn altcoins.

• Yield in Altcoins: Vary, some 5-7%, but larger whales always get more preferential rates.

• Risks: Protocols vary in their reliability and proven track record. Each imposes different lock-up and capital requirements.

On top of the 5 sources, there are LSTs and yield tokenization platforms:

6) Liquid Staking Token: Compounding Yield

BTC “LSTs” like Lombard, PumpBTC, Solv Protocol, BitFi started from the Babylon ecosystem, now are yield-bearing cross-chain BTCs with complex yield strategies. Veda acts like an aggregating interface.

• Yield Mostly in Altcoins:
  Vary, combining Babylon staking rewards, points from different chains, $Pendle, and some with quant strategies through third-party. Additionally, offer own tokens as incentives.

• Risks:
  The low liquidity of LSTs poses a risk of cascading liquidations. Single points of failure exist during minting, redemption, staking, and bridging processes. The heavy reliance on own and others' altcoin yields indicates significant yield volatility.

7) Yield Tokenization: Yield Trading

Pendle is the primary platform where LSTs earn more yield, currently managing $444.17m BTC TVL. It enables traders to secure fixed yields from principal- like a spot alternative, hedge against interest volatility, and access yield liquidity, long or short yield positions.

• Yield in Altcoins:
  Yields can be volatile. YT holders earn LST's underlying yield, swap fees, fixed yield, and $Pendle tokens.

• Risks:
  A decline in $Pendle token price significantly affects participation. Pendle lives on sustainable yield volatility- which is tricky when most of its assets depend on multiple layers of altcoin incentives, including $Pendle itself.

A New Venue for BTC Yield: Introducing Yield Basis

As discussed, while altcoin-denominated yields are unsustainable, genuine BTC-based yields are scarce and carry significant risks. Quant teams need sufficient liquidity, but DEXs falling short.

• What is YB @yieldbasi: YB is an AMM that minimizes impermanent loss and encourages BTC liquidity provision, cross-venue arbitrages, and real transactions.

• Fundamental layer for BTC yield: YB can deliver an average 20% APR (net profit), even higher in a bull market —based on simulation of the past 6 years of data. Also, it is composable with any LSTs wanting real BTC-based yield exposure, and Pendle can work with the yield generated by YB.

• Venue for sophisticated trading strategies: Gradually set up a venue with sufficient liquidity for meaningful quant trading. It also offers a lucrative looping opportunity to boost the BTC borrow rates in lending protocol-Currently, $3.286B WBTC is in @aave with ~0.02% borrow rate.

• Venue for retail trading BTC: YB’s long-term goal is to create the deepest on-chain liquidity for wrapped BTC, and compete with CeFi trading.

Mechanism: Solving Impermanent Loss

• Mechanism: one AMM sitting in another AMM
  When BTC LP deposits into YB, it allows leveraged LP creation by borrowing the half LP value and constantly re-leveraging it. This creates a stablecoin-to-BTC pool with a rebalancing model, with both borrow rates and 50% of pool-generated fees going back to subsidize the pool rebalancing.

• APR= 2*pool yield - (borrow rate + releverage loss)
  The fixed cost is the borrowing interest against crvUSD, which is controllable since the system uses it for more earnings. Higher volatility increases the releverage loss but also the pool revenue, so the strategy remains effective unless volatility exceeds the chosen maximum liquidity concentration. Parameter selection is crucial: More aggressive parameters can increase yield but are riskier for releverage losses, and vice versa.

• Enhancing APR by $YB:
  LPs can choose between earning pool fees or staking to earn YB token emissions. When YB tokens are attractive, more LPs opt out of pool fees, resulting in higher pool APR.

Looking Ahead

BTC Yield generation will become increasingly sophisticated, focusing on risk-managed, BTC-denominated, and institutionalized products. The winners will be those who can deliver deep liquidity and fair yield without excessive risk exposure, and innovate within regulatory frameworks.

• Minor Errors Can Lead to Catastrophic Failures
  "Nothing is unbreakable — it just hasn't been targeted yet." BTC faces increasingly sophisticated attacks across multiple risk layers: CEX trust delegation, self-custody phishing, if put them in work- smart contract vulnerabilities (permissions, logic, arithmetic), and mechanism risks (liquidation, principal loss). Social engineering poses a critical threat by exploiting relationships and interfaces. Major BTC LPs even require chain halts and bridge blocks. Counterintuitive as it sounds, public blockchains and high-liquidity, permissionless protocols are not ideal for their security.

  @0xAquariusCap proposed a security frameworks enabling comprehensive testing, monitoring, and risk tracking.

• The Scarcity of Elite Quant Teams
  Obviously, a real BTC-denominated yield is more attractive. While DEX LPs are still nascent, quant trading remains dominant. Some teams package their strategies into yield-generating BTC products and raise capital from external LPs. However, the capacity for high-frequency arbitrage could be limited, and top teams often keep these strategies in-house, declining outside capital—creating a form of "adverse selection". That said, market-neutral and other low-risk strategies make sense as productized BTC yield offerings.

• TradFi, CeFi and DeFi convergence, with IPO opportunity
  As BTC liquidity continues to deepen, we’re witnessing the convergence of TradFi conservatism, CeFi accessibility, and DeFi innovation. Jan 2025, @coinbase introduced BTC-backed loans powered by @MorphoLabs, it’s a signal: CeFi is operationalizing DeFi mechanisms for a broader audience. Along the way, opportunities for institutional-grade asset management firms to emerge and potentially go public. Firms that combine security-first infra, transparent risk disclosures, and trusted governance will build strong brands, deliver recurring revenue that TradFi asset managers need, and serve everything from high-net-worth clients to pensions and endowments.

Special thanks to @newmichwill, @visavishesh, @JacobPPhillips, @joshkessler_, @tomisonchain

for their feedback.

Starter: Distribution

Younger generations are digital natives, with stablecoins as their natural money. As AI and IoT drive billions of automated micro-transactions, global finance requires adaptable money solutions. Stablecoins act as a "Currency API," transferring as seamlessly as internet data, and have reached $4.5 trillion transaction volume in 2024, this figure is set to grow as more institutions realize stablecoins are an unparalleled business model—Tether profit $5.2 billion from investing its reserve in the H1 2024.

In the stablecoins race, distribution and genuine adoption matter, not the complex crypto mechanics. Their adoption unfolds across three key arenas: the Native Crypto, Fully-Banked, and Under-Banked worlds.

• In the $2.9 trillion Native Crypto world, stablecoins function as DeFi gateways, crucial for trading, lending, derivatives, yield farming and RWA. Crypto-native stablecoins compete through liquidity incentives and DeFi integration.

• In the $400+ trillion Fully-Banked world, stablecoins enhance financial efficiency, primarily for B2B, P2P, and B2C payments. Stablecoins focuses on regulation, licensing and leveraging banks, card networks, payments, merchants to distribute.

• In the Under-Banked world, stablecoins provide the dollar access, boosting financial inclusion. Stablecoins are used for savings, payments, foreign exchange, and yield generation. Grassroots go-to-market matters.

Content Outline

• Starter: Distribution

• Natives in the Native Crypto World

  • Constant Pegging Battle

  • Liquidity Bootstrapping Challenge

  • DeFi Gateway: Trading Pairs, Lending, Derivatives, Yield, RWA

• Aliens in the Fully-Banked World

  • Key Players on the Move

  • Efficiency Enabler: B2B, P2P, C2B Payments

• Mavericks in the Under-Banked World

  • Shadow Dollar Economy

  • Dollar Access: Saving, Payouts, Foreign Exchange

• Ending: Intertwined

  • Interoperability: Cross-currency, cross-coin and cross-chain

  • Opportunity Highlight

  • Open Questions

TL;DR

Natives in the Crypto World

In Q2 2024, stablecoins comprised 8.2% of the total crypto market cap. Maintaining peg stability remains challenging, and unique incentives are key to expanding on-chain distribution, the critical issue is the limited on-chain utility.

Constant Pegging Battle

• Fiat-backed stablecoins work on banking relationships:
  93.33% are fiat-backed stablecoins. they have greater stability and capital efficiency, with banks having the ultimate say by controlling redemption. Regulated issuers like Paxos became PayPal's USD issuer due to its reliability in successfully redeeming billions in BUSD.

• CDP stablecoins improve collateral and liquidation for better peg:
  3.89% are Collateralized debt positions (CDPs) stablecoins. They use crypto as collateral, but they face issues with scaling and volatility. In 2024, CDPs have improved resilience by accepting a wider range of liquid and stable collateral, Aave’s GHO accepts any assets in Aave v3, and Curve’s crvUSD recently added USDM(RWA). Partial liquidation is improving, especially crvUSD's soft liquidation, creating a buffer against further bad debt via their customized AMM. However, the ve-token incentive model falls, as witnessing the crvUSD market cap shrink when CRV's valuation drops after large liquidations.

• Synthetic dollar uses hedging to maintain stability:
  Ethena USDe single-handedly made up 1.67% stablecoin market cap at $3B in one year. It's a delta-neutral synthetic dollar that opens short positions in derivatives to counteract volatility. The funding rate is expected to perform well during the upcoming bull market, even post-point seasons. However, its long-term viability that relies heavily on CEXs, is questionable. As similar products proliferate, the impact of small funds on Ethereum will likely diminish. These synthetic dollars are potentially vulnerable to black swan events and persistently low funding rates during bear markets.

• Algorithm stablecoins diminished to 0.56%

Liquidity Bootstrapping Challenge

Crypto stablecoins leverage yield to attract liquidity. Fundamentally, their liquidity cost comprises the risk-free interest rate plus a risk premium. To remain competitive, stablecoin yields must at least match T-bill rates—we've seen stablecoin borrowing costs decline as T-bill rates reached 5.5%. sFrax and DAI led the way in T-bill exposure. In 2024, several Real World Asset (RWA) projects boosted on-chain T-bill composability: CrvUSD added USDM from Mountain as collateral, while Ondo's USDY and Ethena's USDtb are backed by BlackRock's BUIDL.

Based on T-bill rates, stablecoins employ various strategies to add risk premiums, including fixed-budget incentives like DEX emissions(constraint and death spiral); user fees (tied to lending and perp volume); arbitrage from volatility(Fall when volatility wanes); reserve utilization like staking or restaking (not appealing).

Innovative liquidity strategies are emerging in 2024:

• Maximize in-block yield: While lots of yields currently stem from self-consuming DeFi inflation as an incentive, more innovative strategies are emerging. By leveraging reserves as a bank, projects like CAP aim to funnel MEV and arbitrage profits directly to stablecoin holders, offering a sustainable and potentially more lucrative yield source.

• Compound with T-bill yield:
  Leveraging RWA projects' newfound composability, initiatives like Usual Money (USD0) offer “theoretically” unlimited yield through their governance token, with T-bill yields serving as a baseline- attracted $350M from LPs and got on Binance launchpool. Agora(AUSD) is also a t-bill yield-bearing off-shore stablecoin.

• Balanced high yield resists volatility:
  Newer stablecoins employ a diversified basket approach, avoiding singular yield and volatility risk and providing a balanced high yield. Fortunafi's Reservoir, for instance, allocated T-bills, Hilbert, Morpho, PSM and adapts the portion dynamically and incorporates other high-yield assets as needed.

• Is your TVL a flash in the pan?
  Stablecoin yield often struggles with scalability. While fixed-budget yield can trigger initial spikes, growing TVL dilutes returns, diminishing yield effectiveness over time. Without sustainable yields or genuine utility in trading pairs and derivatives post-incentive, their TVL is unlikely to be stuck around.

DeFi Gateway Dilemma

Onchain visibility allows us to examine the true nature of stablecoins: Are stablecoins genuine representations of money being used as mediums of exchange or merely financial products for yield?

• Only best-interest stablecoins are used as trading paris on CEXs:
  Nearly 80% of trades still occur on centralized exchanges, with top CEXs supporting their ‘preferred’ stablecoins (e.g., FDUSD for Binance, USDC for Coinbase). Other CEXs rely on USDT and USDC's overspilled liquidity. Moreover, stablecoins are trying hard to be the CEX’s margin deposits.

• Few stablecoins are used as trading pairs on DEXs:
  Only USDT, USDC, and a small amount of DAI are being used as trading pairs today. Other stablecoin, such as Ethena with 57% of USDe staked into its own protocol, are purely financial products held to earn yield, far from a medium of exchange.

• Makerdao + Curve + Morpho + Pendle, a set distribution combo:
  Markets like Jupiter, GMX, and DYDX prefer USDC as deposits since USDT has a more skeptical mint-redemption process. Lending platforms like Morpho, and AAVE prefer USDC since it has better liquidity on Ethereum. On the other hand, PYUSD is mainly used for lending on Solana’s Kamino, especially when there are incentives being offered by the Solana Foundation. Ethena’s USDe is mainly used in Pendle for yield activities.

• RWA is under-valued:
  Most of the RWA platforms like BlackRock use USDC as minting assets out of compliance consider, also BlockRock is a shareholder of Circle. DAI success in their RWA products.

• Growing the pie or venturing beyond:
  Although stablecoins can attract major liquidity providers through incentives, they face a bottleneck- DeFi usage has been declining. Stablecoins now face a dilemma: they must either wait for crypto-native activities to expand or seek new utility beyond it.

Aliens in the Fully-Banked World

Key Players on the Move

• Global regulation is gaining clarity:
  99% of stablecoins are USD-backed, with the Federal having ultimate influence. The U.S. regulatory framework is expected to clarify after crypto-friendly Trump's presidency, with his promises to lower interest rates and ban CBDCs potentially benefiting stablecoins. The U.S. Treasury report notes stablecoins' impact on short-dated Treasuries demand, with Tether holding $90 billion in U.S. debt. Crypto crime prevention and maintaining dollar dominance also are the motivations.In 2024, several countries established regulations under common principles, including approval for stablecoin issuance, reserve liquidity and stability requirements, foreign-currency stablecoin usage limitations, and often, banned interest generation. Key examples: MiCA (E.U), PTSR (U.A.E.), sandbox (HongKong), MAS (Singapore), PSA (Japan) Notably, Bermuda became the first nation to accept stablecoin tax payments and license interest-bearing stablecoin issuers.

• Licensed issuers are getting trust:
  Stablecoin issuance requires technical prowess, regulatory compliance across regions, and robust management. Key players include Paxos (PYUSD, BUSD), Brale (USC) and Bridge(B2B API). Reserve management is handled by trusted institutions like BNY Mellon for USDC, and securely generating yield by investing in its BlackRock-managed fund. BUIDL now allows broader onchain projects to get yield.

• Banks are the off-ramp gatekeepers:
  While on-ramping(fiat-to-stablecoin) has become easier, off-ramping(stablecoin-to-fiat) challenges persist as banks struggle to verify the source of incoming funds. Banks favor licensed exchanges like Coinbase and Kraken, which conduct KYC/KYB and have similar AML frameworks. While high-reputation banks like Standard Chartered start to embrace off-ramping, small and medium-sized banks are moving fast, like DBS bank in Singapore. B2B services like Bridge aggregates off-ramp channels, and managed billions in volume for high-profile clients including SpaceX and the US government.

• Distributors has the ultimate say:
  Circle, as the compliant stablecoin leader, relies on Coinbase and is now pursuing global licenses and partnerships. However, this strategy could be undermined as institutions issue their own stablecoins due to the unparalleled business model—Tether, a 100-employee company profited $5.2 billion from investing its reserve in H1 2024. Banks, like JPMorgan, have already launched JPM Coin for institutional transactions. Payment apps, Stripe's acquisition of Bridge showed interest in owning the stablecoin stack rather than merely integrating USDC. PayPal, too, issued PYUSD to capture reserve yield. Card networks, Visa and Mastercard are testing the water by accepting stablecoins.

Efficiency Enabler

With trusted issuers, healthy banking relationships, and distributors as foundations, stablecoins can enhance efficiency in large-scale financial systems, particularly in payments.

Traditional systems face efficiency and cost limitations. Inner-app or inner-bank transfers offer instant settlements, but only within their ecosystems. Inter-bank payments incur fees of about 2.6% (70% to issuer bank, 20% to acquirer bank, 10% to card networks) and take over a day to settle. Cross-border transactions are even costlier at roughly 6.25%, with settlement times up to five days.

Stablecoin payments, by eliminating intermediaries, provide peer-to-peer instant settlement. This accelerates money velocity and reduces capital costs, while offering programmability features such as conditional auto-payments.

• B2B (annual $120-150 trillion):
  Banks are at best position pushing stablecoins. JPMorgan Chase has developed JPM Coin on its Quorum chain, as of October 2023, JPM Coin is used for about $1 billion of transactions each day.

• P2P (annual $1.8-2 trillion):
  E-wallet and mobile payment apps are in the best position, PayPal has launched PYUSD, with currently $604 million market cap on Ethereum and Solana. PayPal enables end users to onboard and send PYUSD for free.

• B2C commerce (annual $5.5-6 trillion):
  Stablecoins need to work with POS, entry point with Banking API, and card network, Visa became the first payment network to settle transactions in USDC back to 2021.

Mavericks in the Under-banked World

Shadow Dollar Economy

Emerging markets desperately need stablecoins due to severe currency devaluation and economic instability. In Turkey, stablecoin purchases accounted for 3.7% of its GDP. People and businesses are willing to pay a premium for stablecoins over fiat USD, with Argentina's stablecoin premium reaching 30.5% and Nigeria's 22.1%. Stablecoins offer dollar access and financial inclusion.

Tether dominates this space with a reliable 10-year track record. Even with their tricky banking relationships and redemption crisis—Tether admitted in April 2019 that USDT was only 70% backed by reserves—the peg remains stable. This is because Tether built a robust shadow dollar economy: In emerging markets, people rarely off-ramp USDT; they view it as dollars, this phenomenon is evident in regions like Africa and Latin America for paying employees, invoices, etc. Tether achieved this without incentives, solely through prolonged existence and consistent utility, enhancing its credibility and acceptance. This should be every stablecoin's ultimate goal.

Dollar Access

• Remittance:
  Remittance inequality slows economic growth. On average, an economically active individual in sub-Saharan Africa is charged 8.5% of the total remittance when sending money into LMICs and developed countries. For business even harder, high fees, long processing times, bureaucracy, and exchange rate risk are barriers that directly affect the growth and competitiveness of companies in the region.

• Dollar access:
  Currency volatility cost 17 emerging market countries $1.2 trillion in GDP from 1992 to 2022—a staggering 9.4% of their total GDP. Access to dollars is crucial for local financial development. Many crypto projects are dedicated to on-ramping, with ZAR focusing on grassroots "DePIN" methods. These methods leverage local agents to facilitate cash-to-stablecoin transactions in Africa, Latin America, and Pakistan.

• Foreign exchange:
  Today, the FX market facilitates over $7.5 trillion in daily volumes ****In the Global South, individuals often resort to the black market to convert local fiat currencies into U.S. dollars, primarily due to more favorable rates compared to official channels. Binance P2P has started to be adopted, but due to its order book manner, it lacks flexibility. Many projects like ViFi are building onchain AMM FX solutions.

• Humanitarian aid disbursement:
  Ukraine war refugees can receive humanitarian assistance in the form of USDC, which they can store in a digital wallet or cash out locally. In Venezuela, frontline healthcare workers used USDC to pay for medical supplies during the COVID-19 pandemic amid a deepening political and economic crisis.

Ending: Intertwined

Interoperability

• Foreign Exchange(FX):

  Traditional FX system is highly inefficent, it faces several challenges: counterparty settlement risk (CLS enhanced but cumbersome), cost of multi-banking system(Six banks are involved in an Australian bank's JPY purchase to a London USD office), global settlement timezone differences(CAD and JPY banking systems overlap for fewer than 5 hours a day), limited access to FX markets (Retail users pay up to 100 times more than large institutions). Onchain FX offers significant advantages:

  • Cost, Efficiency, and Transparency: Oracles like Redstone and Chainlink provide real-time price quotes. DEXs offer cost efficiency and transparency, with Uniswap CLMM enabling trading costs of 0.15-0.25%—roughly 90% lower than traditional FX. The shifting from T+2 banking settlement to instant settling provides arbitrageurs can employ various strategies to correct mis-pricing.

  • Flexibility and Accessibility: Onchain FX gives corporate treasurers and asset managers exposure to a wide range of products without requiring multiple currency-specific bank accounts. Retails can use a crypto wallet with embedded DEX API to get the best FX price.

  • Separation of Currency and Jurisdiction: Transactions no longer require domestic banks, divorcing them from the underlying jurisdiction. This approach harnesses the efficiencies of digitization while maintaining currency sovereignty, though pros and cons remain.

  However, challenges remain, including the scarcity of non-USD-denominated digital assets, oracle security, support for long-tail currencies, regulatory, and unified interfaces with on/off-ramp. Despite these obstacles, onchain FX presents lucrative opportunities. For instance, Citi is developing a blockchain FX solution under the guidance of the Monetary Authority of Singapore.

• Stablecoin Exchange:

  Imagine a world where most companies are issuing their own stablecoins. A stablecoin exchange proposes a challenge: Paying JP Morgan’s merchant using PayPal’s PYUSD. Although off-and-on ramps can solve the problem, they lose the efficiency crypto promises. Onchain AMMs provide the best real-time and low-cost stablecoin-to-stablecoin. For instance, Uniswap offers a number of such pools with fees as low as 0.01%. However, once a multi-billion is brought onchain, they have to trust the smart contract security, and there must be deep enough liquidity with instant performance to support in-real-life activities.

• Cross-chains Exchange:

  Major blockchains have diverse strengths and weaknesses, leading stablecoins to deploy across multiple chains. This multi-chain approach introduces cross-chain challenges, with bridges posing HUGE security risks. In my view, the optimal solution is for stablecoins to launch their own layer 0, as seen with USDC's CCTP, PYUSD's Layer0 integration, and we’ve witness USDT's move to recall bridge-locked tokens potentially launch a Layer0-like solution.

Meanwhile, several open questions remain:

• Will regulations compromise "open finance," given that compliant stablecoins are able to potentially monitor, freeze, and withdraw funds?

• Will compliant stablecoins still avoid providing yields that could be classified as security products, thereby preventing onchain DeFi from benefiting from their massive expansion?

• Can any open blockchain truly handle vast sums of money, considering Ethereum's slow speed and its L2s' reliance on single sequencers, Solana's imperfect uptime record, and the lack of long-term track records for other hyped chains?

• Will the separation of currency and jurisdiction introduce more chaos or opportunities?

The financial revolution led by stablecoin ahead of us is as exciting as it is unpredictable—a new chapter where freedom and regulation dance in a delicate balance.

The AO computer is the actor-oriented machine that emerges from the network of nodes that adhere to its core data protocol, running on the Arweave network. Inspired by the original actor model and Erlang, ao does not allow processes to share memory between one another but does allow them to coordinate via a native message-passing standard. Each of these processes can then be operated at the full speed of the computing resources that are available, without interfering with one another.

Tech Architecture

The best place to understand the architecture of AO is the official doc. Here I will briefly walk through the core parts so we are ready for the discussion later.

The fundamental components of ao are as follows:

• Processes: The network's unit of computation. Processes are represented by a log of interacting messages stored on Arweave, as well as an initialization data item (its VM, scheduler, memory requirements, and necessary extensions).

• Messages: Every interaction with a process in ao is represented by a message, which is an ANS-104 compliant data item. Users and processes can send messages to other processes on the network by way of scheduler units.

• Scheduler Units (SUs): Scheduler Units are responsible for atomically incrementing slot numberings to the messages that are sent to a process. After the assignment, schedulers are required to ensure that data is uploaded to Arweave. Processes have free choice of their preferred sequencer, which can be implemented in a variety of ways: Decentralized, centralized, or even user-hosted.

• Compute Units (CUs): Compute Units are nodes that users and messenger units can use in order to calculate the state of processes in AO. Once the computation of a state is completed, the CU will return to the caller a signed attested of the output (logs, outboxes, and requests to spawn other processes) of the resolution of a specific message.

• Messenger Units (MUs): Messenger Units are nodes that relay messages around the AO network according to a process called cranking. In essence, when MUs crank a message around the system they send it to the appropriate SU for a process, then coordinate with a CU in order to calculate the output of the interaction, and then repeat the process recursively for any resulting outbox messages. Users and processes can also pay a MU to subscribe to a process, cranking any messages that result from its timed interactions. Processes can also optionally label a message as a cast – leading the MU to send the message to its SU, but not listen for a response.

One simple way to fit AO into the existing L1/L2 model is by imagining multiple co-processors running under different VMs, machines and even consensus models spread into a network. If we were to draw a comparison between parts of AO to parts of a traditional L1/L2:

• Onchain states = Messages backed up on Arweave

  AO does not have a straightforward global state. Instead, when observing the latest or historical state of a dapp, one needs to access the historical messages involved with the dApp's processes on Arweave and then recontruct its state progression until the latest message. Therefore, AO's state might seem different if one were to observe different dapps at a different time. Sometimes the state might even be in conflict. The invalid message should be ignored by the observer according to pre-defined consensus mechanism of the dapp at its initiation. Therefore AO's state is always holographic, and it's not very meaningful to construct a global consensus state.

• RPCs = Messaging Units + Gateway

  Front-ends send computation demands and retrieve computation results via MUs, similar to how users initiate transactions and observe the onchain state via the query from an RPC provider.

• Full Node = Computation Units

  This is where the VMs are hosted and the computations are conducted. The difference is that the computation in CUs is stateless, while full nodes store their input and output in the state of their blockchain. Also, full nodes take care of the consensus mechanism of their chain and conduct P2P communication all the time. CUs in AO are only involved in consensus mechanisms when required.

• Sequencer = Scheduler

  In a traditional blockchain, the sequencer puts transactions in a specific sequence so that the end state can be computed in a deterministic way. In AO, the scheduler can determine the end state of a process by putting messages in a specific sequence. It's just that these messages are a lower-level representation of a transaction, not necessarily calling a specific function.

• Data Availability Layer = Arweave

  Arweave serves a similar role to a DA layer, but not in the sense of ensuring that all states are published, but by ensuring that all the state & information (including initial state, computation logic, input, middle state, and end state) needed to reconstruct the entire history of a computation can be retrieved. State retrievability ensures that all computations conducted on the AO network can be lazily/optimistically verified.

The advantage of AO's architecture is quite clear from a traditional blockchain. Instead of trying to figure out the best place to optimize (big vs small blocks, PoW vs PoS, parallel vs sequential, monolithic VS modular, etc), AO throws everything out of the window and gives dapps, or more specifically their processes, maximum flexibility on how they would like to exist. One process can choose to have 50 SUs (running PoS to ensure sequencer decentralization), then 1 CU (sitting in a data center for maximum performance), together with 3 MUs (sitting in the 3 different data centers in the same area to ensure low latency but running 2/3 multi-sig to eliminate single point of failure on backing up messages to Arweave). Even if something goes wrong, the entire historical state of the dApp's process can still be retrieved from Arweave and a hard or soft fork on the dApp's state tree can always be implemented.

The flexibility of Arweave ensures that it remains maximally wide open for any dApp's infrastructure need. AO is essentially taking the bottle of "lazy verification" pills and emptying the whole bottle down its throat. By lazily verifying everything (including the computation, consensus mechanism, and even the P2P communication), a dapp can choose to run on the most affordable infra, the most performant infra, or the most economically secured infra, all within a change on their process's parameters. For most applications, this drastically saved their cost on all items that would need to carry the weight of a consensus mechanism or economic guarantees until it's absolutely needed.

Custom-made for Decentralized AI

Among the various use cases on the blockchain, AI use cases has been the least explored one, given it's very recent emergence. In the following section, we will break down the demand for decentralized AI and explain why the AO architecture is the best

If the use case needs to scale in a horizontal way, why choose vertical scaling for your infra?

Decentralized AI needs access to:

• Decentralized model (hosting & training)

  A decentralized AI infrastructure requires access to decentralized model hosting and training to ensure the robustness and resilience of AI models. Traditional centralized AI systems are prone to single points of failure, making them vulnerable to manipulation.

• Decentralized data (training, fine-tuning & inference)

  Access to decentralized data for training, fine-tuning, and inference is crucial for building robust and diverse AI systems. Centralized data storage can create bottlenecks and censorship of information. Decentralized data input also allows for more diverse and representative datasets, as data can be sourced from places previously underpaid for their contribution. Decentralized data access supports collaborative efforts in AI development, enabling multiple stakeholders to contribute and verify data quality, which is essential for developing unbiased and accurate AI models.

• Decentralized computation

  Centralized computation systems are limited by their finite resources and cost structures. Decentralized computation allows for parallel processing and distributed workloads, making it possible to tackle complex AI tasks that require substantial computational power.

Both the above demands should have the following characteristics.

• Super-Scalable

  For a decentralized AI infrastructure to be effective, it must be super-scalable, and capable of expanding its resources seamlessly to accommodate growing demands. Super-scalability ensures that as more data and computational tasks are added to the network, the system can scale horizontally without significant performance degradation. This characteristic is vital for supporting the continuous growth and evolution of AI models, which often require increasing amounts of data and computational power.

• Decentralized Pricing

  A decentralized pricing mechanism is essential for fostering a fair and competitive ecosystem for AI services. Traditional centralized pricing models such as multi-tier subscriptions or token fees is not efficient enough to fully reflect the dynamics of flexible AI usage. Having the price reflected by market dynamics is the best way to cope with the flexibility needed for AI services.

• Censorship Resistant

  Censorship resistance is a critical characteristic of decentralized AI infrastructure, ensuring that AI models and data remain accessible and unaltered by any single authority. In centralized systems, entities with control over the infrastructure can impose restrictions or manipulation, potentially leading to biased or censored outcomes. The resistance to censorship preserves the integrity and neutrality of AI models, fostering trust and reliability in AI-driven decisions.

• Autonomous (self-starting and sustaining)

  Autonomy is a defining feature of decentralized AI infrastructure, enabling systems to self-start and sustain without human intervention. Autonomous AI can dynamically manage resources, adapt to changing conditions, and recover from failures independently. We can only witness the true power of AI by ensuring that the AI can operate in an ever-lasting environment and evolve autonomously, without arbitrary guidelines pointing their fingers.

Decentralized AI will incur the following costs to deploy and operate

• Cost of Storage

  Decentralized AI would need plenty of space for it's model, data input and data output. AO leverages Arweave for the cheapest permanent and decentralized storage. Arweave significantly reduces the long-term cost of storage for decentralized AI systems, making it a cost-effective choice for managing large datasets necessary for AI training and operations.

• Cost of Computation

  Computation on the blockchain is generally much more expensive than computation on a server or on a PC. The extra costs come from redundancy or other ways to ensure validity. It's hard to lower the computation price beyond the machine cost. AO provides the maximum flexibility on how much redundancy is needed, or to what extent is validity ensured.

• Cost of Communication/Bandwidth

  Decentralized AI systems incur costs related to communication and bandwidth as data and computational tasks need to be transmitted across the network. One of AO's greatest cost savings on bandwidth is the reduction in handshake operations by implementing the actor-oriented model. Messages can choose to be simply be casted to avoid costing the bandwidth a response. Also, AO's flexibility is designed to optimize these costs by giving this power to the AI (dapp) where they can decide for themselves how much data exchanges are needed beyond the absolute basics, optimizing for performance, security, and decentralization VS cost and latency. It's a fair assumption that an AI on AO would be much cheaper to run than other networks not implementing the Actor Oriented Model.

• Cost of Validity

  Ensuring the validity of data and computations is crazy expensive, especially under the scale of AIs. Validity technology such as ZK proofs has been focusing on the validity of the computational process. Today it's still way too expensive for AIs, introducing 1000X overhead. On AO, validity is lazily verified. One can still reconstruct the entire computation based on the messages stored on Arweave, without having to prove the validity of the computational process. In AO, validity is relative. States are observed as a holographic state. There is no one single global state consensus. Every single state is "perceived" or "observed" to be correct by querying from a different node of AO. This works in a very similar way to EigenLayer's "inter-subjectivity" where whether a statement is valid or not is not entirely subjective or objective.

• Cost of Consensus

  Consensus is almost always achieved through some form of redundancy with a fault tolerance mechanism and anti-sybil mechanism on top. The Arweave storage already handles fault tolerance and anti-sybil. So for the dapps constructed on AO, there is no immediate need to implement the cost of consensus in the first place. Consensus can always be implemented when the economic situation of the dapp improves and the cost of not having enough consensus grows. The consensus cost of AO comes from the consensus of CU and Arweave itself. If a user chooses to, multiple CUs can conduct computations in parallel and the result is chosen or calculated based on a pre-defined consensus mechanism. The consensus of the starting state of an AI, the ending state, and all middle states are stored on Arweave and can be inferred by running through all the intermittent messages.

To summarize the rundown for AO's technical architecture, it's evident that AO has offered a network under the Actor Oriented Model that offers enough validity and consensus (through Arweave and lazy verification), high flexibility on computation (from super affordable to super performance), and great cost optimization and decentralized storage and decentralized communication. These advantages lines up nicely with the emerging decentralized AI where loads on computation, bandwidth, and storage is tremendous, especially decentralized.

Ecosystem Projects

AO does not directly enforce any direct consensus or security mechanism upon the computations. Instead, it offers a flexible system from which economic security mechanisms can be implemented. This opens up a huge design space for both infrastructures and applications.

Besides the standard WASM implemented by the Forward Research team as a demo/default option, developers would have several other options to play with when it comes to deploying their smart contract on AO.

• EverPay: The EverVison team built everPay, a real-time payment settlement layer that uses the storage computing paradigm, where all calculations are done off-chain and the results are synchronized with Arweave. There are already several dapps running on the everpay infra including Permaswap (DEX), ARIS (Inscription), and WeWeave (storage service)

• MEM.Tech (Molecular Execution Machine): MEM is a layer enabling flexible, chain-agnostic serverless functions, secured by Arweave. The MEM protocol leverages 3EM, supporting languages that can compile to WebAssembly (WASM), such as AssemblyScript, JavaScript, TypeScript, Rust, C/C++, Zig, GoLang, Solidity, and Vyper. MEM is also natively chain-agnostic, allowing authentication from 14 EVMs and non-EVM chains, and can call data from any chain or API.

• WeaveVM: WeaveVM is a hyper-scalable, lazy-evaluated EVM protocol to compute data at any scale. It's the first EVM-compatible protocol that fully relies on Arweave for permanent storage, enabling atomicity of executable EVM bytecode and data of any size on the same ledger.

Several other middle-ware is also made available for the AO Network.

• AR.IO: AR.IO is the decentralized and incentivized gateway node network of the Arweave ecosystem. The various nodes on the AR.IO Network, known as gateways, are the interface between users and the permaweb. Each gateway acts like a “Permaweb Service Provider” and supports multiple, value-added, services like reading, writing, querying, and indexing of Arweave data.

• AOX: AOX is a bridge facilitating bridging the AR token from Arweave into AO, or out of AO to Arweave.

• 0rbit: 0rbit is the Decentralized Oracle Network built on AO to provide access to limitless data on any AO process. 0rbit can also bring data from any valid URL to an AO process. Building with 0rbit is as simple as making an HTTP GET or POST request to an API.

• AO.Link: Built by the Autonomous Finance team, ao.link serves as a message explorer for the ao Network, offering functionalities similar to block explorers in conventional blockchain systems. It includes message computing capabilities, graphical visualization of message links for clarity, real-time message streaming for up-to-date information, and a list of linked messages for organized navigation.

• DataOS: Built by the Autonomous Finance team, DataOS is a protocol developed atop the AO Network, leveraging it for computing power and Arweave for data storage, creating a synergistic foundation for its operations. As a ContentFI protocol, DataOS employs autonomous AI agents to source and regenerate content derivatives, aiming to foster engagement and fulfill the intent of consumers.

Beyond infras, AO has a very exciting application ecosystem landscaping across the basics defi and new exciting AI applications:

Defi:

• Permaswap: Permaswap Network is a fully decentralized P2P network where LPs in the network are peer-to-peer self-hosted accounts and assets are self-managed by the user's private key. The nodes in the network perform price calculations and build peer-to-peer orders in a distributed manner according to protocol rules. Users can trade on Permaswap with extremely low gas fees without experiencing MEV issues. At the same time, the network protocol can flexibly support multiple AMM algorithms.

• ArSwap: ArSwap is a next-generation Automated Market Maker (AMM) built on the AO blockchain. Leveraging the unique capabilities of AO, ArSwap offers decentralized swapping, liquidity provision, and yield farming services with unparalleled security and efficiency. ArSwap introduces innovative features like dynamic swapping pools and custom liquidity schemes, enabling users to maximize returns while minimizing risks.

• Bark: Built by Community Labs, Bark is A decentralized AMM exchange backed by token liquidity pools.

• Astro: Astro is a stablecoin protocol on the AO network where users can use the $AR token as collateral and mint the $USDA stablecoin. Astro introduces innovations like Wrapped Arweave (wAR) and a unique liquidation mechanism with Bark AMM, enhancing permaweb liquidity and stability.

• Harvest: Harvest is an RWA marketplace for natural resources, featuring land, cattle, and other natural investments from Colombia. Harvest has built-in profit mechanisms, allowing the investor to get exposure to the additional cash flow from their investment. For example, milk & offspring revenue from investing in a cow.

• Mentions on memecoins: $TRUNK, Aethirs (which consists of several other memecoins: $FIRE, $AIR, $WATER, which can be composed into some other memecoins: $Lava, $Dust, $Mud, $Lightning, $Steam and $Ice).

AI Applications:

• Autonomous Finance: Autonomous Finance is a dedicated research and technology entity, focusing on the intricacies of financial infrastructure within the AO network. Autonomous Finance team is building AI onchain and generating new data-driven financial primitives. Autonomous Finance is developing composable semi and fully autonomous agents. The autonomous agents will be self-learning and executing logic using onchain data streams.

• Autonomous DCA Agent: Built by the Autonomous Finance team, Autonomous DCA Agent is an Autonomous Investment Agent (AIA) executing a dynamic dollar-cost-average (DCA) investment strategy across various liquidity pools within the AO ecosystem. It automatically buys a base token at predetermined, user-configurable intervals using a consistent amount of a quote token for each transaction.

• AO Games: AO Games is a fully decentralized game launchpad on AO. Looking to build games delivering a Web2-level experience with fully decentralized games. The team will also introduce On-Chain AI so that the gameplay experience can adapt, learn, and evolve.

• Outcøme: Built by the Puente.ai team, Outcome is a prediction market hyper-focused on the AO ecosystem. The prediction markets center around AO, games, defi, memes, business, and technology. Users bet whether they are "IN" or "OUT" of a prediction. Users can also claim daily tokens, predict, and climb the leaderboards to win prizes.

Closing

The projects mentioned above covers the majority of projects currently announced or live on the AO network. Though the ecosystem is quite young, we are thrilled to see that a lot of the projects already have a working product. Some of them already have a tradable token. The AO testnet is announced in a very usable stage which is ready to support some basic AI functionalities. Given more time and resources, we would not be surprised to see more dapps thriving with their onchain AI.

Given the less-than-expected friction deploy AIs on AO, it is evident that the architecture of the AO supercomputer is well-aligned with AI operations. As the mainnet date comes closer, we look forward to see more exciting new innovations happening here in AO and Arweave.

Recap for Decentralized AI House at ETHDenver2024, co-hosted by SevenX Ventures, MyShell, and Jessy’s Hacker House

When AI is wildly growing in the web3 world, it’s even harder to distinguish real innovation from narrative bubbles. We invited 12 most mainstream AI mafias during ETHDenver and this article does a simple recap of the project's visions, methods, and scenarios, to see how they shaking the current world.

Here are some important questions to ask, and answers from our mafias:

Data:

• Data Provisioning: How to source AI training data？ @Grass

• Data provenance: How to protect data source IP? @Story Protocol

• Data alignment: How to ensure certain data used by the model? @Space&Time

Model:

• Open economics: How to build an open platform with incentive? @Bittensor, @Sentient

• Model alignment: How to prove model result is untampered? @Modulus labs, @Ora

Infra:

• General infrastructure: How to connect all infrastructures together? @Ritual

Agent:

• AI agent: How to enable agents to be smart, composable, and ownable? @Future primitive, @Olas, @Myshell

Full Youtube Link: https://www.youtube.com/playlist?list=PLFRYxG8q7EY6SgJHzEefMEq-VyrhzK20n

Grass

• Why: Data is the foundation of all AI training, but extractive gatekeepers make it difficult to source high quality training data. A great deal can be scraped from the public web, but it’s common practice among major websites to block commercial data centers.

• What: Grass is a data provisioning protocol, makes data accessible and AI infrastructure equitable.

• How: Users install a Chrome web extension and trace amounts of their excess compute and bandwidth are used to scan the internet in search of AI data. Grass operates a network of nearly 1 million web scraping nodes located around the world.  Using this network, it scrapes over 1 TB of data per day before cleaning and processing it to produce structured datasets.

• Where: Grass nodes are now operating in 190 countries around the globe.

• Youtube: https://www.youtube.com/watch?v=dwBlqwOimig&list=PLFRYxG8q7EY6SgJHzEefMEq-VyrhzK20n&index=2

• Presenter twitter: @0xdrej

Story protocol

• Why: AI remixing is illegal and inevitable. The major block of AI growth is the lack of monetizing and creating attribution and providence for IP and content creators.

• What: A composable on-chain IP layer allows creators to set autonomous rules of engagement. Adding legibility and liquidity for global IP marvel.

• How: Creators can purchase the license NFT, converting their static IP into programmable IP. Programmable IP is a layer that any program can read and write on, consisting of Nouns and Verbs. Nouns include data structures, relevant IP metadata, use ERC6551; And verbs include modules, an array of functionality for IP Assets, like licensing, revenue streams from derivative works, and access globally. Whenever the derivative monetizes, the revenue flows back automatically.

• Where: Story protocol can be used in license renting, derivatives, customization about territory, channels, expiration, revocability, transferability, attribution, etc.

• Youtube: https://www.youtube.com/watch?v=ymq1mhRSxTg&list=PLFRYxG8q7EY6SgJHzEefMEq-VyrhzK20n&index=3&t=201s

• Presenter twitter: @jasonjzhao

Space and Times:

• Why: As the LLM evolves, major companies can bias, alt or tamper the dataset and parameters; it's important to have cryptographic proof of the untampered dataset, ensuring that same dataset was used during LLM training. Also, SxT has been exploring ways to sanitize out copyrighted data, pulling from a verifiable vector database and injecting into prompts during inference.

• What: SxT is an indexer and ZK prover that proves SQL queries or vector searches against that indexed data.

• How: LLM providers can load their on-chain/off-chain training dataset into Space and Time, where the data is witnessed and threshold signed with cryptographic commitments, which are used later to prove that a dataset was used for training. From there, litigators or auditors/reviewers can ensure a dataset hasn’t been tampered with since training. SxT built GPU accelerator "Blitzar", which already achieves 2 million-row table queries with 14-second proof time on a single GPU.

• Where: SxT enables users to create queries in plain words, in a few seconds OpenAI retrieves a context from the vector search database and writes accurate SxT SQL that can be executed by the prover, and the prover returns proof in 4 seconds.

• Youtube: https://www.youtube.com/watch?v=cxT0vcU4mSo&list=PLFRYxG8q7EY6SgJHzEefMEq-VyrhzK20n&index=4

• Presenter twitter: @chiefbuidl

Bittensor

• Why: OpenAI aims to monopolize the control of AI

• What: Bittensor is a decentralized platform for open-sourced AI

• How: The Bittensor network has 32 subnets. Those subnets started from the model, but now have extended to storage, computing, scraping, tracking, and different AI fields. $TAO incentivizes subnet builders to keep polishing the model or project, and validators rank the results of the subnet, the rank changes the distribution of $TAO, and the least one will be kicked out of the network. This mechanism guarantees models compete to produce the best outputs, and the work that is most valuable to the collective will be rewarded.

• Where: Powerful applications emerge, like FileTAO doing decentralized storage, Cortex TAO doing OpenAI Inference, Nous Research doing fine-tuning LLM; Fractal Research doing Decentralized Text-to-Video, etc.

• Youtube: https://www.youtube.com/watch?v=xkXBDCaPMYk&list=PLFRYxG8q7EY6SgJHzEefMEq-VyrhzK20n&index=6

Sentient

• Why: AGI building is dangerous, facing "human extinction threat" and capitalistic framework risk, so they natively need crypto platforms; whereas the crypto platforms need native killer apps.

• What: Sentient is a platform for sovereign incentive-driven AI development.

• How: Use crowdsource methods, allowing communities to coordinate and contribute to training models to lower cost, control inference using an open protocol, enable composability across models, and value flow back to network participants. Aggregating web2 and web3 forces, and leveraging the token, Sentient will fairly incentivize developers to build trustless AGI.

• Youtube: https://www.youtube.com/watch?v=1fbwIGG7PV8&list=PLFRYxG8q7EY6SgJHzEefMEq-VyrhzK20n&index=10

Modulus labs

• Why: When AGI's future is unstoppable, we need to prove the AI result is accountable and safe, which is generated from a certificated model instead of manipulated, without relying on the trusted centralized authority's good behavior.

• What: Modulus builds a specialized AI ZK prover "Remainder" which delivers AI features for dApps at a fraction of the cost.

• How: It doesn't make sense to use the modern ZK proving system for AI when there's around 10,000x to 100,000x overhead compared to non-verifiable AI outputs. Modulus built a custom prover for AI inference, demonstrating only ~180x overhead.

• Where: One notable implementation is Upshot, trust concern was raised since Upshot's complicated appraisal mode can only be conducted off-chain. However, Upshot can send valuable AI appraisals to Modulus every hour, and Modulus can generate "proof of correctness" for AI computations, aggregate them, and send them to Ethereum for final verification.

• Youtube: https://www.youtube.com/watch?v=4JRh2eeZCO0&list=PLFRYxG8q7EY6SgJHzEefMEq-VyrhzK20n&index=7

• Presenter twitter: @realDanielShorr

Ora

• Why: AI models can't run on-chain, thousands of computers will execute one inference. Verifying results on-chain is feasible but ZKML has an exponential cost with model size increases, so there's a need for linear growth cost- OPML.

• What: Ora is an onchain AI oracle using OPML for any size AI models.

• How: Use Oracle to delegate the computation to off-chain nodes. The user initiates a transaction from the smart contract with prompts and a named model. OAO contract delegates the tx into the OPML node to execute the inference, produce fraud proof and submit the verifier to ORO. The verified result goes back to the transaction initiator. But OPML still needs the help of ZKP to fulfill the input/output privacy and instant finalization. Also, a ZK oracle from ORA can generate a storage proof for the OPML, so you don't need to repeatedly perform OPML when it is being reused.

• Where: Now using ORA we can have Stable Diffusion, 7B-LLaMA on ethereum mainnet. ORA can power AI-managed DAO, AIGC NFT like EIP7007, empower model ownership.

• Youtube: https://www.youtube.com/watch?v=i2Zkz45AGr4&list=PLFRYxG8q7EY6SgJHzEefMEq-VyrhzK20n&index=1

• Presenter twitter: @0xKartin

Ritual

• Why: New censorship and manipulation take place when AI infrastructure is more and more centralized, permissioned, and increasingly regulated. But crypto gives primitives around privacy and computational integrity, coordination and incentive, and default permissionless infra.

• What: Ritual as a natural convergence point of Crypto*AI, consists of a decentralized Oracle network and sovereign chain with custom VM and coprocessor.

• How: The Oracle network "Infernet" enables any smart contract on evm chains linking on-chain workflow to off-chain ML inference, and ultimately the co-processor enables AI native operation at VM level, while remains composability with other Data Availability, Permanent Storages, Prover networks, GPU networks, Inference engine. Nodes in Ritual network not only run and service model operations, but also do consensus and execution clients.

• Where: "Frenrug" uses inferent SDK to build a non-deterministic LLM that guides users to buy, and sell keys on friend.tech. For defi lending protocols, they can train models to parameterize and generalize them well across all protocols. For crypto-enabled AI, myshell will be the first model creator economy example.

• Youtube: https://www.youtube.com/watch?v=DV-rMP6V-Yk&list=PLFRYxG8q7EY6SgJHzEefMEq-VyrhzK20n&index=9

• Presenter twitter: @sanlsrni

Olas

• Why: Autonomous Agents(AA) are powerful entities that perceive certain information and conduct actions. But web2 AA is severely limited in its potential: They can't do KYC, users can't take ownership, platform censorship risks, and limited composability.

• What: Olas is a decentralized protocol for co-ownable autonomous agents.

• How: In Olas world, agents live off-chain, while the registration and management are on-chain. Agents are arranged in autonomous services (“decentralised autonomous agents”). The agent operator is in control of one agent and one consensus gadget. Each agent runs a Finite-State Machine(FSM) that is replicated on the temporary blockchain between agents in a service. Agents in a service come to consensus off-chain before taking action on-chain. The network uses a $OLAS incentives for all stakeholders: capital providers (bonders), code providers (developers), agent operators (stakers), and service owners (entrepreneurs).

• Where: Olas Predict is an economy of three types of autonomous agents that continuously create and participate in prediction markets on arbitrary future events. The trader agent operator doesn't need to worry about subscribing to OpenAI, they just simply pay per request in crypto.

• Youtube: https://www.youtube.com/watch?v=KQeQgqFAQfI&list=PLFRYxG8q7EY6SgJHzEefMEq-VyrhzK20n&index=8

• Presenter twitter: @david_enim

MyShell

• Why: With the rise of LLM, we need better tools to help creators easily build applications, also subversing the current "static" creator economy into a dynamic space.

• What: MyShell is a decentralized platform for discovering, creating, and staking AI-native apps.

• How: Creator can create AI-native Apps in MyShell within a few minutes, from AI characters for companionship to various tools designed to enhance your study and work experience: choose an open source model, edit the prefix and suffix of the prompt, feed information with specific fields, image, and video capability coming soon. MyShell LLM is based on massive private data to make the roleplay experience more human-like. The token is used for accessing premium features, supporting creators, and settling usage charges.

• Where: MyShell platform has multiple use cases: AI characters have personalities and unique voices for companionship, learning, and playing; Tools for learning language, text-to-image, summarizing videos, and so on.

• Youtube: https://www.youtube.com/watch?v=4PgIk4S5a0w&list=PLFRYxG8q7EY6SgJHzEefMEq-VyrhzK20n&index=5

• Presenter twitter: @ethan_myshell

Future Primitive

• Why: NFTs are internet-native objects, but they lack programmability, therefore constrain its potential to conduct further behavior on-chain.

• What: Future Primitives turns NFTs into agents through ERC6551 and its infrastructure.

• How: Using ERC6551, every NFT has the same rights as an Ethereum user. They can self-custody assets, execute arbitrary operations, control multiple independent accounts. Every NFT's address is the same on any evm chain, releasing the cross-chain potential. For tokenbound V4, it publishes the Power of Attorney for your TBA of which a smart contract can perform onchain actions fully autonomously without you needing to sign, validate, or execute the tx, and the owner can always revoke auth extensions.

• Youtube: https://www.youtube.com/watch?v=aIr2HeZsQi4&list=PLFRYxG8q7EY6SgJHzEefMEq-VyrhzK20n&index=11

• Presenter twitter: @jayden_windle

With a growing emphasis on real-world applications and the rise of Solana, DePin has regained significant popularity in 2024. However, DePIN is not a novel concept. In my opinion, the essence of DePIN is about the token economic model that turning participants into stakeholders, rather than creating new productivity.

In the previous bull run, projects like Arweave, Filecoin, and Helium stood out, while many others fell short, even went down to zero. Now, as we approach the cusp of the next bull market, our challenge lies in discerning between fleeting fads and genuinely sustainable innovations that will usher us into a new era of real-world applications.

This piece sets out to provide a balanced critique of DePIN's historical shortcomings, exploring areas such as regulatory hurdles, lack of demand, flawed tokenomics, and the risk of rugs. Nevertheless, it also highlights the substantial prospects within this space, including overcoming profit margins, value anchors, token incentives, and robust community engagement. And I’ll provide a DePIN evaluation framework, and in-details case studies of DePIN projects. Hopefully this piece can equip readers with insights and tools to build, evaluate and invest in DePIN landscapes.

Content

• Introduction

• Participants: Hardware, Miner, Protocol, Operator and Users

• Ecosystem Landscape

  • DePIN Projects: Purpose, Type and Market

  • Blockchain and Middleware: General and DePin-specific chain

• Essence:

  • Why Didn't It Work: Regulation, Demand, Tokenomic, Moat and Rug

  • Why There's Hope: Margin, Anchor, Booster and Community

• Evaluation Criteria:

  • Total Value = Product X Tokenomic X Narrative

  • Duration = Being proven or disproven

  • Last Straw = Compliance X Team

• Case Study:

  • Helium: 5G celluar network

  • io.net: AI/ML Sharing GPU

Intro

The blockchain domain has focused on infrastructure development, emphasizing scaling solutions (Layer2s, Data Availability), privacy enhancement (Zero-Knowledge), and user experience improvement (Account Abstraction) over the years to prepare for broad adoption. The need for real-world, mass adoption is now more evident than ever. Initiatives such as exchanges working towards compliance and ETFs entering the traditional markets have marked significant progress and are paving the way for us.

DePIN is not a specific area, this term originally brought up by Messari as the short for Decentralized Physical Infrastructure Networks. It represents a community-driven, decentralized hardware network incentivized by token. Its primary goal is to replace the monopolized coordinator and use native tokens to transform participants into stakeholders of the network. Since the last bull market, there has been a significant expansion into high-value sectors such as AI/ML, 5G, WiFi, Bandwidth, Vehicle, Energy and so on.

So, what's new that DePIN brought to the table in this spectrum?

• Assets: Type, distribution, trading methods
  Looking at successful crypto projects, typically feature assets with generic attributes and continuously refine these assets' functionalities and use cases. DePIN projects can incorporate physical assets, transforming the services provided or data collected by the hardware into tokenized assets. This tokenization facilitates permissionless trading and staking, paving the way for a broader spectrum of financial activities.

• Participants: Roles, amount, stickiness and relationship
  DePIN enhances the ecosystem by diversifying roles, creating a cohesive network of hardware manufacturers, miners, network applications, consumers and so on. Reducing entry barriers and expanding geographical reach draws a broader range of participants. More importantly, DePIN transforms all parties involved into stakeholders through crypto assets, fostering both labor contribution and consumption. This approach ensures ongoing engagement, increasing the ecosystem's stickiness and surpassing the traditional, simplistic buy-and-sell relationships. By fostering complex interconnections, DePIN significantly bolsters the robustness and sustainability of its ecosystem.

• Scenarios: Premium value, frequency and scalability
  DePIN is grounded in real-world scenarios, addressing existing needs. Beyond the focus on crypto investors, the network's generic value can be derived from the number of purchasers, value per transaction, and frequency of transactions. Success could be achieved if any two of these parameters excel. E.g. 5G user base is huge, approximately over 100 million in the U.S. pay once a month for 80 dollars. While individual AI companies may represent a smaller volume of transactions compared to 5G consumers, the high frequency and value of these transactions, propelled by extensive demand, highlight a substantial opportunity.

Beyond the initial excitement, several critical factors warrant attention: Achieving product-market fit remains essential for success, even with sufficient supply; the implementation of protocol designs that ensure partition tolerance and censorship resistance is crucial to avoid new forms of censorship; moreover, incentives must be meticulously crafted to avert unsustainable inflation. We will explore these aspects in more detail below.

Participants

This sector examines the roles and objectives of its five principal stakeholders: hardware manufacturers, hosters (referred to as miners for ease of memory), network providers, operators, and end-users.

Hardware Manufacturer: The Physical Devices that Provide Services or Collect Data

• Manufacturer Evolution:
  Transitioning from centralized, whitelisted manufacturers to a permissionless system is a natural progression. Initially, it's reasonable to depend on a single entity to produce reliable hardware. However, to prevent a single vendor from becoming a monopoly threat or a bottleneck in network growth, opening the ecosystem to more qualified hardware manufacturers is advisable. Eventually, an open market can foster full competition to offer the best products at fair prices to miners(e.g. Helium HIP19).

Miner: Entities That Run Hardware as Nodes in the Network

• Cost Considerations:
  Miners consider the total cost, including hardware, onboarding, maintenance, and operational (if applicable) fees. The time and attention cost associated with network participation also counts.

• Reward System:
  Beyond the belief in building physical networks, miners prioritize rewards, often calculating the payback period. Immediate incentives for beta versions are attractive, but the protocol should curb the crazy inflation, incorporate a robust deflationary design and staking mechanisms, to ensure long-term revenue for miners. Many projects have been adopting the Burn-Mint-Equilibrium(BME) model.

• User Experience:
  The process should be straightforward, aiming to lower the learning curve for participation, thereby enabling a low-barrier and scalable network.

Protocol: The Coordination Core of the Network

• Price Structure:
  The price of commoditized services is fixed at the protocol level, while decisions should be made cautiously to achieve sustainability. Taking io.net’s pricing model as example: The price is effected by multi-dimensional factors like web2 competitor price, peak hours, hardware performance, internet bandwidth, crypto earnings and so on.

• Rule Transparency:
  Rules like reward mechanisms, at least should be transparent and accessible to the public. Then the executed result should be verifiable even if off-chain and, eventually, the whole process automated. On-chain automation seems optimal so guarantees fairness but it may sacrifice flexibility for evolving adjustments.

• Data Integrity and Honesty:
  Establishing transparent and verifiable procedures for fetching data from hardware, uploading it to databases, and reflecting it into rewards.

• Security Measures:
  Anticipate malicious behavior and implement measures like random liveness checks and effective data validation. KYC might be involved, and a reputation system could reward honest behavior while penalizing dishonesty.

• Privacy and Compliance:
  Collecting data from participants improves the protocol's robustness and efficiency but comes with privacy risks. While only collecting necessary data, projects should also consider implementing Zero Knowledge Proof (ZKP).

Operators: Entities That Package Network Resources into Services

• Operator Entity Diversity:
  Initially, a single operator, possibly from the network foundation may set up a schema for cold start. Over time, allowing more operators to join can enhance market reach, maximize the usage of miner resources, and allow full competition.

• Revenue Model:
  Besides the cost that the operator pays to the miner through the network, the operator usually charges more for the wrapped service. Operators can use profits to invest more in marketing, subsidize users and so on.

Purchaser: Users of the Network

• Goals and Needs:
  Different users have varying goals, such as cost efficiency, time savings, security, accessibility, or performance. The network should effectively address these primary needs.

• User Experience:
  Seamless onboarding and payment processes are essential. Whether users need to understand cryptocurrency depends on the service design, but the goal should be to simplify transactions, integrating fiat payment options alongside cryptocurrency.

Ecosystem Landscape

DePIN Network

DePIN projects vary widely, however for better understanding we can categorized DePin projects by Purpose, Type, and Market.

Purpose

• Computing:
  This encompasses processing and communicating information using computational systems. Rollups, for example, address computing challenges for blockchain, while decentralized GPU and CPU resources cater to off-chain computing needs.

• Storage:
  Storage projects ensure data is preserved for future retrieval and use. This broader concept includes file storage solutions like Arweave and Filecoin, as well as databases such as KwilDB.

• Network:
  Network projects facilitate the fetching and transmitting of real-world data across the network, enabling communication among different nodes and connecting them to users. Commonly, projects utilize IoT sensors and wireless networks to gather data and deliver services.

Type

• Service Network: Core asset is the service
  At its core, a service network aims to effectively utilize unused or available resources to provide services. The protocol aggregates demand from users, matches it with miners' resources, and allocates tasks accordingly. Examples include io.net, which connects GPU resources with AI companies; Helium, linking small cell hosters with 5G users; and Teleport, connecting drivers with riders.

• Data Network: Core asset is the aggregated dataset
  Data networks incentivize data contributions to amass a large dataset. They standardize data formats, ensure data validity and quality, and may involve data cleaning and training before wrapping them into datasets, APIs, or other formats for commercial use. For instance, DIMO collects vehicle data for insights useful to insurance or gas companies; Hivemapper captures street images to create map data for delivery and smart city projects.

Market

• AI and ML:
  The global ML market size was expected to grow from $26.03 billion in 2023 to $225.91 billion by 2030. The AI/ML opportunity in web3 lies with data, computing, and models. For data, projects like grass enable mass user sharing of bandwidth for AI companies to fetch data at scale; For computing, projects like io.net pool data center computing power at a lower cost compared to web2; For models, projects like Bittensor enable the assetization of models.

• 5G/ Cellur:
  The global 5G market size was valued at USD 84.31 billion in 2023. The transition to 5G requires more cells, with web2 monopolies capturing significant profit margins. DePIN 5G projects, such as Helium, aim to build decentralized, affordable networks by leveraging small cells, free spectrum bands in U.S., and eSIM cards.

• Vehicle:
  The vehicle market is projected to grow from USD 145.24 billion in 2023. DePIN vehicle networks gather data via specialized hardware and software, providing valuable insights for insurance, maintenance, gas and other businesses. Projects like DIMO leverage token incentives drivers to encourage data sharing.

• Hailing:
  The global ride-hailing service market reached a value of US$ 176.6 Billion in 2023. DePIN hailing services disrupt the ride-sharing market by using tokens to foster growth, offering fair pricing and lower transaction fees compared to Web2 intermediaries. Notable projects include Teleport and Drife.

• Map:
  The global digital map market size was estimated at USD 18.18 billion in 2022. DePIN map projects address data fragmentation and over-reliance on major tech firms by rewarding community contributions with tokens, offering an equitable data sharing model. Examples include Hivemapper, 2blox, and Natix.

Blockchain and Middleware

In the current state, most DePIN projects have only the token on-chain with most of the part off-chain. Helium has transitioned from its proprietary blockchain to Solana since 2023 April. This shift was motivated by the need to address maintenance costs and scalability challenges. The nature of DePIN projects, minor and frequent transactions, and timely rewards, dictate a blockchain infrastructure that is fast, cost-effective, and scalable.

For general-purpose blockchain, the performance, cost, and ecosystem are crucial:

• Solana, with a notable approximately $2.4B DePIN Market Cap, has emerged as the most popular choice for major projects like Helium, io.net, Teleport, Hivemappers, Render, Nosana, and many more. Solana aligns well with the DePIN projects’ needs for swift and low-cost transactions, because its unique Proof-of-History (PoH) mechanism reduces the computational load, and its high hardware requirements improve the processing performance. Secondly, scalability is crucial, for example, Helium needs millions of transactions each month and minted 900k NFT for subscribers when it migrated to Solana. Solana needs to stay robust during the spikes and enable fast mass minting at a minor price. Projects can interact with other Solana ecosystem projects, and Solana is also upgrading to serve the DePIN needs better. The recent launch of token extension enables advanced customization for SPLs, like threshold transaction, privacy mode, and delegation, which are going to give DePIN protocols more flexibility in how they build their on-chain components.

On the other hand, DePIN-specific blockchains are gaining attention. These blockchains address critical issues such as data trustworthiness, identity management, and verifiability. By providing automation and verifiable data, DePIN projects can save repetitive operational effort, avoid risks, and focus better on network development.

• IoTeX as a Layer 1 blockchain, has introduced its Layer 2 solution "W3bstream," which enables secure IoT data collection, leverages flexible data availability layers, and aggregates large volumes of off-chain data into verifiable zero-knowledge proofs (ZKPs) and triggers on-chain transactions.

• Peaq is a multi-chain Layer1 blockchain for DePIN. While having low cost and high transaction speed, it also provides Modular DePIN Functions like Multi-Chain Machine ID, payment, role-based access for machines, 3-tier data verification, AI agents, data storage and indexing. Peaq is integrated with Wormhole for liquidity and has strong relations with relevant partners such as Bosch. peaq will launch in 2024.

Why Didn't It Work?

• Regulation difficulty: Managing network in the physical realm necessitates navigating a myriad of regulations, with feasibility and costs differing across markets. For example, hailing projects must manage travel data, comply with local governance reporting requirements, and adhere to personal safety standards. Similarly, 5G projects are subject to varying spectrum licensing regulations across different countries.

• Demand failure:
  Supply side growth doesn't guarantee demand side adoption. DePIN projects utilize tokens to rapidly incentivize miner expansion, with onboarding fees often reinvested to enhance network value. However, a surplus of supply without corresponding usage can be problematic. To prevent this, it's crucial to ensure genuine product-market fit for each project. Additionally, effective marketing, sales strategies, and business development are key to competing with giant Web2 vendors.

• Tokenomic Failure:
  DePIN projects often have their token from day one, with the design of its mechanism being a critical determinant of success. However this tokenomic expertise proposed a challenge for projects originating from Web2 manufacturers, who tend to rely on precedents to formulate their token strategies. Sometimes they tend to have mass incentives to attract participants, which leads to unsustainable inflation. The Work token model (Stake for Access) is commonly employed for commoditized services with fixed protocol-level service fees. BME(Burn-Mint-Equilibrium) can adopt a dual-token system, combining a tradable, value-seeking token with a fiat-pegged payment token, allowing for fixed fiat pricing for the service.

• Performance consideration:
  Centralization offers the benefits of focused coordination, adaptable management, and high-speed performance. Even though decentralized projects can attract attention with their lower fees, but matching the performance of centralized service remains a significant uncertainty. Taking sharing GPU as an instance, the distribution of GPUs across various locations complicates the task of consolidating them into a single cluster, leading to potential delays for intensive ML applications, compare to a data center. If decentralized systems cannot achieve performance parity with centralized service, their cost advantage will not be sufficient to offer a competitive alternative.

• Hard to build a moat:
  This challenge is particularly acute in commoditized services, where standardized and interchangeable offerings make it easy for both suppliers and customers to switch networks in pursuit of better revenue or lower fees. This lack of a strong moat is evident even in Web2 businesses. However, certain factors can create a competitive edge: Securing a first-mover position to establish brand recognition, delivering the best user experience, and developing proprietary, patent-protected software or hardware.

• RUG: Easily rug after selling narrative:
  Unlike other fields, DePIN participants (miners) usually need to initially invest significantly in hardware to start earning rewards, although this may be different for use cases. This presents a potential fraud risk, where the foundation could publish a promising roadmap, build a narrative, ask influencers to promote and sell its hardware, and then abruptly rug the money and disappear.

Why There's Still Hope

• Your Margin is My Opportunity:
  Web2 is FAT. In the current Web2 landscape, large companies like Amazon and Microsoft enjoy high gross margins. DePINs can disrupt this by enabling individuals and smaller entities to compete against these giants. By tapping into this margin space, DePINs can offer more cost-effective solutions, undercutting the high-margin models of established players and passing savings onto users.

• Value Anchor to Minimize Bubbles:
  DePINs provide tangible value through hardware devices, services, and data. This practical utility acts as a stabilizing factor, anchoring the business model and mitigating the risk of speculative bubbles. By offering real-world applications and benefits, DePINs ensure a grounded and sustainable growth trajectory.

• Token as a Business Booster:

  • Initial Funding: DePINs can raise significant capital from VC, sell hardware, and launch a narrative-driven token. This initial funding kickstarts the business.

  • Subsidies for Growth: Using a portion of the raised funds to subsidize service fees and reward users, DePINs can rapidly expand their network with more nodes and users, fostering a robust and widespread community.

  • Sustainable Expansion: While hardware reach has its limits (e.g., a certain number of hotspots needed to cover an area), a well-established network can sustain itself in the long term. Once the initial funds are utilized, the matured network should be self-sustaining, driven by its own momentum and user base.

• Community-Driven, Bottom-Up Approach:
  By leveraging a community-driven model, DePINs can achieve more cost-effective and agile development compared to traditional top-down approaches. This grassroots strategy not only reduces costs but also enhances user engagement and investment in the platform, creating a loyal and active community.

Evaluation Criteria

Case study

Helium 5G

In brief, Helium Mobile is tackling the expansive U.S. 5G market, now extending into Mexico, as a forerunner and one of the largest in the DePIN arena. It stands as a significant venture for Solana, DePIN, and 5G innovations. The Helium Mobile offers $20 monthly 5G plan, currently serving over 43,000 subscribers in the U.S. Its BME approach, deflationary token model, and pegged coin concept have set a precedent for numerous projects.

• Background:
  Helium stands as a pioneer and among the largest in the DePIN networks, initially launched to address IoT challenges in 2019. Through effective token incentives, it became the world's largest IoT network with over 100,000 hotspots across 182 countries within just two years. At its peak during the last bull market, its FDV hit $11.7 billion, marking it as a standout project. Despite criticisms over the limited IoT demand, Helium has continued to explore and develop. In 2022, it branched into IoT and mobile via two sub-DAOs and began focusing on the 5G hotspot market. A partnership with T-Mobile enhanced network accessibility, and its migration to Solana in 2023 solidified its position as a major DePIN project on this platform. As of January 2024, Helium has deployed 8,000 5G hotspots and has over 43,000 monthly 5G subscribers.

• Network:

  • How come the community-driven 5G service possible?
    Traditional 5G providers such as AT&T, Verizon, and T-Mobile dominate the U.S. wireless market, investing heavily in spectrum licenses and infrastructure. However, DePIN 5G becomes viable through three key opportunities: the rise of eSIM technology allowing users to switch to virtual providers seamlessly; the availability of the CBRS band for public use without the need for costly licenses; and the advent of small cells that individuals can host, providing sufficient coverage when densely deployed.

  • What’s the goal for Helium 5G network?
    Considering the limit coverage of small cell- only cover 100 yards to a mile, completely replacing traditional 5G networks isn't feasible. Instead, Helium's 5G network aims to serve as a supplementary roaming service, especially in densely populated areas, to offer more affordable connectivity.

  • How Helium network works?
    The network comprises five roles: hardware vendors, miners, network, operators, and users. Vendors produce and sell 5G hotspots, which miners purchase at price of $1000-2598 and operate, contributing to the network's coverage. Miners are rewarded for their active and effective service. The network manages these nodes and maintains a set data price at $0.5/GB. The Operator, Nova Labs packages this infrastructure into a user-friendly service, offering $20 monthly 5G plans for users.

• Token:

  • Token types:
    Helium utilizes $HNT for utility, people need to burn $HNT for Data Credits (DC), a coin that pegged to the USD, for network usage, and $MOBILE and $IOT tokens for rewarding participants in its sub-DAOs.

• $HNT and BME:
  $HNT is capped at 223 million with a halving event every two years. The constant emissions will eventually decrease to just 1 HNT per month by 2064. To avoid depleting rewards, Helium employs a Burn-and-Mint Equivalence (BME) method without exceeding the max supply. Users must buy $HNT and burn it for Data Credits (DC) to use the network. Burned $HNT is then recycled into a Net Emission Pool. To exert deflationary pressure, a cap ensures that any burned token amount exceeding 1% of the net emission (currently 1,644 per day) is permanently removed.

  • $HNT and $MOBILE:
    The value of $MOBILE is underpinned by $HNT. The daily minting of $HNT includes both the recycled amount from the Net Emission Pool and new emissions, with the total distributed among various stakeholders. Approximately 30% goes to investors, with the remainder split between the two sub-DAOs based on a formula considering DC consumption, the number of hotspots, and ve-HNT holdings. This distribution mechanism determines the allocation to the MOBILE DAO’s treasury, impacting the value of $MOBILE.

  • $MOBILE:
    $MOBILE directly influences rewards for participants, with a maximum supply set at 250 billion and halving every two years. Daily emissions are allocated as follows: 20% to the operator (Nova Labs), 4% to the oracle, 60% to hardware vendors and miners, 10% to mappers (users sharing their location), 3% to stakers, and 3% to operational funds.

  • Theoretical Flywheel Effect:
    A higher $HNT price boosts the value of $MOBILE, which in turn increases the rewards for miners and mappers. This encourages more participants to join, leading to more $HNT being burned, which can further drive up the price of $HNT, creating a positive feedback loop for the ecosystem's growth and value.xx

• Essence:
  While investor activity can significantly influence the network's valuation, the sustainable growth of the network fundamentally relies on increasing its user base and miner participation.

  • User Growth: Managed by Operators
    Currently, Helium Mobile, serving as the sole network operator, offers a $20 monthly plan with approximately 43,000 activated SIM cards, generating an estimated monthly revenue of $860,000 (But this is inaccurate since excluding the $5/month Miami plan). However, according to on-chain data, only 8%-15% of this revenue contributes to the network's earnings. Assuming the operator has sufficient funds for marketing and covering expenses like the T-Mobile partnership, user attraction strategies include the appealing $20 unlimited 5G offer. Additionally, they've introduced innovative incentives, rewarding users with $MOBILE for sharing their location, a program that has seen an 88% participation rate among users. These $MOBILE tokens can directly offset subscription costs. Referral programs are another tactic being utilized.

  • Miner Growth: Influenced by $MOBILE's Volatility
    Miners are primarily concerned with the return on investment and rewards. The initial costs for miners, including hardware purchase and setup, range from $1,025 to $2,624. With 7,672 active hotspots, this represents a significant value generated by miners. However, the rewards fluctuate greatly due to $MOBILE's volatility; for example, the average daily earnings for a miner in November 2023 were about $12.47, which spiked to $147.63 in December 2023 following a surge in Solana's value, only to decrease afterwards. Enhancing $MOBILE's value, and thereby attracting more miners, hinges on increasing $HNT's value, creating more buying pressure from investors, and exploring other strategies to bolster buying demand.

IO.NET AI/ML

In short, io.net targets at AI Machine Learning market especially GPU-As-A-Service market. Its high-end GPU is ~82% cheaper compare to web2 providers like AWS and Goggle Cloud, also it has the largest amount of GPU, especially high-end GPU among web3 competitors. It introduced BME similiar to Helium.

• Background:
  Machine learning, as the fundamental algorithm of AI, is experiencing significant market growth. The realm of machine learning training and inference involves computing elements like CPUs, ASICs and especially GPUs stand out as the most foundational resource. However, there's a notable supply shortage of high-end GPUs needed for large-scale machine learning. Consequently, many AI/ML businesses, especially those lacking the necessary infrastructure and expertise, are increasingly turning to Cloud GPU services. These services offer a rental model that is both cost-effective and practical for running machine learning tasks.

• Network:

  • Why web3 Cloud GPU service makes sense?
    ”Your margin is my opportunity”, io.net's price is A100($0.89/Hour), 522 RTX 3090($0.38/Hour), 301 RTX3080($0.23/Hour), 8426 photos RTX A6000($0.75/Hour), 1988 photos RTX, A4000K8S($0.23/Hour). A100 as an example, 82.45% cheaper than Google Cloud, 82.62% cheaper than Amazon AWS's .

  • What’s the goal of io.net?
    Aiming to become the world's largest and most cost-effective GPU cloud on-demand with unlimited scalability for AI/ML Training/Inference.

  • How io.net network works?
    io.net solves this cost and availability problem by aggregating GPUs from underutilized sources such as independent data centers, crypto miners, and crypto projects like Filecoin, Render, and others. GPU workers can join the GPU pool, the pool groups GPU by their internet speed, location, chip manufacturer, and model name, GPUs can earn rewards not only for hired time but also for idle time when they are being utilized for inferencing. Consumers can create a cluster in under 90 seconds. First, create a cluster with a customized choice for GPU, pay using Sphere, and use VSCode to deploy ML tasks, monitoring progress.

Acknowledges:

Acknowledgment to my dearest friends for their review:

Kuleen, DePIN eco lead of Solana

0xlol, Defi research lead of Polygon

Mads, researcher of Maven11

Andrew, research lead of IoTeX

Max, founder of peaq

drej, founder of Grass

Content:

• Crypto UX sucks? Key management sucks!

• Key management layers: Responsibility, Storage, and Access

• Existing players analysis: MetaMask, Trust Wallet, Privy and Particle

• New solution:

  • Key layer: WebAuthn, Secure Enclave and Passkey

  • Account layer: Smart Contract Account(SCA), External-Owned Account(EOA)

  • Signing layer: Protocol r1 pre-compile, third-party service, Solidity and ZK verifier

• Implementation case study: (key)+(account)

  • Clave wallet: (Secure Enclave webAuthn) + (SCA)

  • Soul wallet: (Passkey) + (4337 SCA)

  • OKX wallet: (MPC-TSS + Passkey) + (4337 SCA)

  • Web3Auth: (MPC-TSS + Passkey)+ (EOA/SCA)

  • Lit protocol: (MPC-TSS + Decentralized nodes + Passkey) +(EOA/SCA)

  *Please notice cases above may soon change and improve

• Conclusion

TL;DR

The private key is the core that allows us to sign transactions on Ethereum, but managing it has been a nightmare, even in the readable form of “seed phrases”. Yet our goal was never to turn blockchain into a sophisticated game.

Authenticating authorized users is crucial for secure transactions. With the evolution of internet security and UX, we've moved from password authentication to biometrics like facial recognition and fingerprints. WebAuthn is a key development in this progression. This article closely discusses three terms:

1. WebAuthn: A web authentication standard uses public key-based credentials, often created by external authenticators. It eliminates the need for passwords and enables secure user authentication.

2. Secure Enclave: A hardware-based secure area within computing devices, the Secure Enclave is designed to protect sensitive data. Versions of a Secure Enclave are found in iOS, Android, and Windows devices. It can serve as a secure authenticator by implementing WebAuthn, but the private key, tied to the SE, often presents cross-device challenges.

3. Passkey: An implementation of WebAuthn at the operating system level, customized by various device and system providers. For instance, Apple's Passkey uses the key stored in iCloud Keychain for cross-device synchronization. However, this approach is typically locked to specific platforms or systems.

As described above, the webAuthn implementations align with our goal for daily blockchain users, to achieve high-level anti-phishing security and a friendly experience. Here’s the idea to merge them into blockchain:

• Key Layer: Users authenticate using seamless biometric methods like facial recognition or fingerprint. Under the hood, it is the hardware-based security processor like Secure Enclave or cloud services like iCloud and Google Cloud that handle the key management. I'll dive into addressing cross-device and cross-platform issues later.

• Account Layer: A Smart Contract Account (SCA) offers the ability to assign arbitrary signers(e.g. SE and Passkey) and threshold mechanisms. Moreover, its modular design enhances flexibility and upgradability. For example, an SCA can adapt its signing requirements dynamically based on factors such as transaction amount, time, or IP address. On the other hand, a traditional External-Owned Account (EOA) can be augmented with MPC services, their combination offers better interoperability and cost-effectiveness compared to SCA, though it lacks advanced functionalities that SCAs provide, especially for key rotation.

• Signing Layer: Ethereum natively supports the k1 curve, but WebAuthn's signature verification incurs higher costs, since it uses the r1 curve for key generation. Therefore, some Layer 2 solutions like zkSync, planning on native EIP-7212 r1 curve precompiles. Additionally, there are third-party services, Solidity verifiers, Zero-Knowledge (ZK) verifiers, and distributed key management systems, to facilitate r1 curve signing in a more cost-effective manner.

Disclaimer:

Technological advancement doesn't guarantee market success; Not all devices and platforms have adopted Passkey; Using SCA can be more expensive than EOA; The proposed solution evolves with technological progress.

Crypto UX sucks? Key management sucks!

In the blockchain realm, the real control of blockchain assets is not in the hands of the user or the wallet provider, but rather lies with the private key. This key is the most integral to the entire process of executing a transaction on Ethereum. To understand this better, let's take EOA as an example:

• Key Generation: A random number selected from the secp256k1 elliptic curve serves as the private key. This key is then multiplied by a predefined point on the curve to generate the public key. The Ethereum address is derived from the last 20 bytes of the hashed public key. The 'seed phrase' is usually introduced for human-readable backup, enabling the deterministic derivation of private and public keys.

• Signing Transactions: A transaction, containing details such as nonce(a sequential number), amount, gas price, and recipient address, is signed using the private key. This process, involving the ECDSA, a digital signature algorithm that uses elliptic curve cryptography and adopts secp256k1 as the curve, generates a signature consisting of values (r, s, v). The signature and the original transaction are then broadcast on the network.

• Verifying Transactions: Once a transaction reaches Ethereum nodes, it undergoes a validation process in the node's mempool. To verify the signer, the nodes use the signature and hashed transaction to derive the sender's public key and confirm the transaction’s authenticity by matching the derived address with the sender's.

As described above, the private key is an essential entity on-chain. Originally, Ethereum accounts, known as External Owned Accounts (EOAs), relied solely on a single private key, which posed significant risks, as losing the key meant losing access to the account.

Many may think Account Abstraction(AA) is the solution to everything related the bad user experience, which I'll say not exactly. AA is about changing the validity rules to be programmable, and the programmability of Smart Contract Account (SCAs) makes it possible. AA is powerful, enables sending multiple transactions in parallel(abstract nonce), gas sponsorship and pay gas in ERC20(abstract gas), and more relevant to this article’s topic, to break the fixed signature validation(abstract ECDSA signature). Instead of EOA, SCAs can assign arbitrary signers and signing mechanisms like multi-signature(multisigs) or scoped keys(session keys). However, despite the flexibility and upgradability advancement of AA, the fundamental reliance on key(s) for transaction signing remains unchanged.

Even when converted into a 12-word seed phrase, managing a private key remains challenging, posing risks of loss or phishing attacks. Users must navigate between complex layers of decentralized solutions or the warm embrace of centralized service, neither of them is ideal.

Why crypto experience sucks? A large part of it is because key management sucks. It always requires tradeoffs in experience, security, and decentralization. This article explores potential optimal solutions for managing the key.

Key Management Layers

There’ll never be a one-size-fits-all solution, the best way to preserve the key is tailored to specific user scenarios, influenced by factors like user type (institutional or individual), capital amount, transaction frequency, and interaction types.

To clarify ahead, I avoid using the popular 'self, semi, and fully custody' methods to catalog. In my view, true self-custody means signing a transaction independently, without relying on another party, this holds even if the solution isn't custodial in the traditional sense (like being stored in decentralized nodes' TEE). Categorizing solutions as merely 'good' or 'bad' based on custody type is overly simplistic and doesn't account for their varying suitability. For a more nuanced assessment of key management methods, I suggest analyzing them through three distinct layers:

Responsibility

Whether to split the responsibility of managing a key into different parties.

Given that individuals often face challenges in managing the key, distributing the responsibility of safeguarding the key emerges as a natural risk mitigation strategy. This category includes methods like using multiple keys to collaboratively sign, as seen in Multi-Signature (Multi-sig) systems, and dividing the private key into shares through a Secret Sharing Scheme(SSS) or Multi-Party Computation (MPC).

• Multi-sig: Requiring multiple complete private keys to generate transaction signatures. This method necessitates on-chain communication about the different signers, incurring higher transaction fees, and impacting privacy because the number of signers is visible on-chain.

• SSS: a private key is generated at a single location, and a dealer distributes pieces of this key to different parties. All parties must reconstruct the full private key to sign a transaction. However, this temporary reconstruction may introduce vulnerability.

• MPC-TSS(Threshold Signature Scheme): as an implementation of MPC, a cryptographic approach enabling several parties to perform computations while keeping their inputs private jointly. Each party independently creates a secret key share, and transactions are signed without these parties ever needing to physically meet. It introduces lower fees because it’s off-chain, and no single point of failure risk as SSS.

Storage

Store the key or shares, affected by security, accessibility, cost and decentralization factors.

• Centralized cloud services like AWS, iCloud and other servers. They're convenient for frequent transactions, but more vulnerable to censorship.

• Decentralized storage like IPFS and Filecoin.

• Local computer/mobile: Keys are stored locally within the browser's secure storage.

• Paper wallets: Physical printout of private keys or QR codes.

• Trusted Execution Environment (TEE): TEE provides a secure area within the main processor to execute or store sensitive data, isolated from the main operating system.

• Secure Enclave: The Secure Enclave on modern devices is isolated from the main processor to provide an extra layer of security and is designed to keep sensitive user data secure even when the Application Processor kernel becomes compromised.

• Hardware wallets: Physical devices like Ledger and Trezor, specifically designed to securely store private keys.

• Hardware Security Module (HSM): HSMs are specialized hardware devices designed for secure key management and cryptographic operations. They are typically used in enterprise environments and offer high-grade security features.

Access

How to verify user identity to access the stored key

Authentication is involved in accessing the stored key. It's about validating that the individual attempting access is indeed authorized to do so. Looking back to history, we can categorize the history like this:

1. Something You Know: Passwords, PINs, answers to security questions, or specific patterns.

2. Something You Have:Include smart cards, hardware tokens (Time-based one-time passwords), or digital factors like social account verifications and SMS codes sent to a phone.

3. Someone You Are:Involve unique physical characteristics of the user, such as fingerprints, facial recognition (like Apple’s Face ID or Windows Hello), voice recognition, or iris/retina scans.

Upon these, 2FA and MFA are methods that combine two or more factors, like SMS combined push notification, to add more security layers to user accounts.

Existing Players Analysis

MetaMask allows users to use a password to access the key stored in the user’s local browser’s storage.

Trust Wallet allows users to use a password or faceID to access the key stored in the user’s local browser’s storage, the user also can choose cloud service to back up the private key.

Privy allows users to use multiple social login methods like email, using SSS to split three shares:

• Device share: Browser-iFrame, mobile- secure enclave.

• Auth share: Stored by privy, link to privy id).

• Recovery share: User password or encrypted by Privy stored in hardware security module (HSM).

Particle allows users to use social login, using MPC-TSS that has two shares:

• Device share: browser-iFrame

• Server key share: Particle's server

New Solution

Key layer: WebAuthn, Secure Enclave and Passkey

Existing solutions above have been pivotal in introducing users to web3. However, they often come with challenges: passwords can be forgotten or targeted in phishing attacks, and even 2FA, though more secure, can be cumbersome with its multiple steps. Additionally, not everyone is comfortable entrusting a third party with key management, users still depend on their system availability and liveness while some services ensure they can't access the key.

This brings us to ponder whether there's a more effective solution — one that offers the closest solution to a trustless, high-security, and seamless user experience. This search leads us to the optimal web2 methods. Several terms are tightly related to the topic, as I described at the beginning of the article, WebAuthn is the authentication standard itself, while Secure Enclave and Passkey are implementations or components related to this standard.

WebAuthn

WebAuthn standardizes the interface of authenticating users to web-based applications. It allows users to log in to internet accounts using external authenticators instead of a password. Authenticators could be Roaming Authenticators (Yubikey, Titan key) or a Platform Authenticator (Built-in keychain on Apple devices), and so on.

The FIDO (Fast IDentity Online) Alliance initially developed the technologies behind WebAuthn. It was officially declared a web standard by the W3C in March 2019 and along with its standardization, major browsers like Google Chrome, Mozilla Firefox, Microsoft Edge, and Apple Safari have adopted WebAuthn, significantly increasing its reach and usability. Now it is supported by many advanced devices.

Benefits of webAuthn:

• Enhanced Security: Eliminates reliance on passwords, reducing vulnerability to phishing, brute force, and replay attacks.

• Improved User Experience: Offers a simpler and quicker login process, often with just a tap or biometric verification.

• Privacy Protection: No shared secrets are transmitted during authentication, and individual websites don’t receive any personally identifiable information.

• Scalability and Standardization: As a web standard, WebAuthn ensures consistency and interoperability across different browsers and platforms.

Device-bound WebAuthn, eg. Secure Enclave

In modern cases, we can use the hardware processor as the authenticator, like Secure Enclave for Apple device, Trustzone for Android, and Strongbox for Google Pixel.

1. Key Generation: Using public-key cryptography, a key pair is generated in accordance with WebAuthn standards, typically using the P-256 r1 curve. The public key is sent to the service, while private key NEVER leaves Secure Enclave. User never handles the plain-text key, making it difficult for the private key to become compromised.

2. Key Storage: Private key is securely stored within the device’s Secure Enclave, a fortified subsystem segregated from the main processor. It safeguards sensitive data, ensuring that even if the main system is compromised, the raw key material remains inaccessible. The bar to compromise the Secure Enclave is extremely high, and therefore the most sensitive data types such as Apple Pay and FaceID data are stored there. Here’s a in-depth explanation of how SE works.

3. Authentication: Users use their facial recognition or fingerprint to get access, the Secure Enclave using the private key to sign a challenge from the service, and the service verifies using the public key.

Device-based webAuthn pros:

• Hardware-level security: Using Secure Enclave, an isolated hardware-based key manager to provide an extra layer of security.

• Phishing resistance: Do not involve entering any information on potentially compromised devices or websites.

• Convenient experience: They provide a more user-friendly experience. Users no longer need to remember complex passwords for different sites.

Device-based webAuthn cons:

• Device constraints: The private key can not be exported or retrieved if the device is lost or damaged, cross-device operation is impossible.

Cloud-based WebAuthn, Passkey

Addressing the challenge of cross-device functionality, tech giants have introduced cloud-based webAuthn implementation, Passkey is broadly familiar because of Apple.

Let’s take Apple's Passkey as an example:

1. Key generation: The user’s macOS, iOS, or iPadOS device, as the authenticator, generates a public-private key pair when the user creates the account. Then sends the public key to the server, and the private key is stored on the device’s iCloud keychain. iCloud Keychain data is encrypted with a hardware-bound keypair, and stored in a hardware security module (HSM). The key is inaccessible to Apple.

2. Synchronization across devices: This process will be the same as accessing iCloud. Authenticate to the iCloud account, receive an SMS code, and enter the passcode of one of the devices.

Cloud-based webAuthn pros:

• Cross-device: Passkeys were designed to be convenient and accessible from all devices used on a regular basis. But currently limited to Apple devices, for Android it’s more challenging because of its diverse versions and hardware variations.

• Phishing Resistance: Same as above.

• Convenient Experience: Same as above.

Cloud-based Passkey cons:

• Rely on cloud service: Compare to device-based webAuthn, cloud-based passkey moves the security level from Secure Enclave's hardware to iCloud keychain, some can argue it’s custodial to your cloud service. Some key aspects to consider include: User’s AppleID account used with iCloud is compromised; While iCloud Keychain employs end-to-end encryption to protect data, operational errors or vulnerabilities pose a risk.

• Restrict to platform: For example, using an iCloud-based passkey on an Android device is extremely challenging. Furthermore, unlike traditional methods, Apple and Google do not send device-specific assertions. This means it’s currently impossible to verify the type of device that generated a key, which raises questions about the reliability of the key and its associated metadata.

Account Layer: SCA and EOA

So far, we can see maintaining hardware-level security while addressing cross-device and cross-platform compatibility is challenging. Equally crucial is the social recovery option, such as adding multiple guardians for enhanced security. In this context, blockchain can show us a way.

A notable gap when we try to implement web2 webAuthn to web3: Web2 only requires proving ownership, whereas web3 also necessitates authorizing the transaction simultaneously. With Passkey, developers lack control over the signing message, which is typically generic, like 'sign in.' This can lead to potential front-end manipulation, users signing messages blindly—a seemingly minor but crucial concern.

Smart Contract Accounts(SCA), inherently smart contracts themselves, function as on-chain entities, capable of assigning arbitrary signers. This flexibility allows for programming various devices and platforms – such settingan Android phone, a Macbook, and an iPhone – as signers. Even further, Modular Smart Contract Account allows upgradability, swapping out new signers, changing the signing threshold from 2 out of 3 to even more intricate configurations.

Envision a wallet that adapts its security requirements based on context: it allows single-signer authentication when the user is on a familiar local IP address, but requires multiple signers for transactions from unknown IP addresses or above a certain value. With modularity and programmability, our imagination is the only limit to such innovations. Many SCA service providers actively build this space, including Safe, Zerodev, Biconomy, Etherspots, Rhinestone, etc. Also shout out to infrastructure like Stackup, Plimico, Alchemy make it possible.

Please check my previous research provides more comprehensive context around SCA.

EOAs can achieve social recovery and cross-device/platform compatibility through MPC services. Despite EOAs having fixed signers, MPC providers can split keys into shares for enhanced security and flexibility. This method lacks SCA's programmable and upgradable features, such as timelock recovery and easy key deactivation. However, it still offers superior cross-chain capabilities by being chain-agnostic and is currently more cost-effective than SCAs. Notable MPC providers including Privy, Particle Network, web3Auth, OKX wallet, Binance wallet, etc.

Signing Layer: R1 support

Let's take a step back to understand the context: On Ethereum, private keys are random numbers selected from the k1 curve, and the signing process also utilizes this curve.

However, the key pairs generated according to WebAuthn standards, use the r1 curve. Therefore, verifying an r1 signature on Ethereum is approximately three times more expensive than a k1 signature. Here are some approaches to address this:

Protocol Solution:

• Solution: EIP7212, Precompiled for secp256r1 Curve Support proposed by Clave team.

• Evaluation: This proposal creates a precompiled contract that performs signature verifications in the “secp256r1” elliptic curve by given parameters of message hash, r and s components of the signature, and x, y coordinates of the public key. So that, any EVM chain - principally Ethereum rollups - will be able to integrate this precompiled contract easily. By far, protocol precompile maybe the most gas efficient solution.

• Implementation: zkSync

Third-party Service

• Solution: Turnkey

• Evaluation: a Turkey TEE ensures the private key is accessible only to the user via their PassKey and never accessible for the Turnkey themselves, however this still requires the liveness of the service.

• Implementation: Goldfinch

Solidity Verifier Solutions:

• Solution: FCL's Solidity Verifier, FCL's Solidity Verifier With Precomputation, Daimo’s P256Verifier

• Implementation: Clave, Obvious Wallet

Zero-Knowledge(ZK) Verifier:

• Solution: Risc0 Bonsai, Axiom’s halo2-ecc

• Evaluation: This approach leverages zero-knowledge proofs to verify computations outside the Ethereum Virtual Machine (EVM), reducing on-chain computational costs.

• Implementation: Bonfire Wallet(Risc0), Know Nothing Labs(Axiom)

Each of these solutions offers a different method to empower cheaper and feasible r1 signature verification in Ethereum's ecosystem, and here’s an assessment by Dogan.

Implementation Case Study

*Please note that as of December 2023, most of these solutions are in their early stages and may change or improve at any time. These examples are only for learning purposes; always refer to their official websites for accurate information.

Clave wallet: (Secure Enclave webAuthn) + (SCA)

Basics:

• Demo: https://getclave.io/

• Account: SCA

• Chain: ZkSync

Transaction process:

• Key creation: User provide biometric authentication like fingerprint or facial recognition, a key pair is generated inside the Secure Enclave, which never reveal or leave outside.

• Key signing: The app takes a required transaction message and forwards a signing request to Secure Enclave, user provide bio-auth to approve signing, and the Secure Enclave signs the message with the key, and to be broadcast to blockchain nodes.

• Additional functions: Smart contract account enables many powerful functions. First, gas sponsorship. Due to the paymaster, other parties like dApp or advertisers can pay for user’s gas fee, making the transaction process smoother, and also they are able to allow users pay gas in ERC20 instead of Ether or native token. And using session key, user can conduct signless transaction in a period of time.

Recovery mechanism:

The recovery process is conduct by Clave’s smart contract on zkSync, user is able to cancel the recovery during a 48-hour time lock, to prevent unauthorized and malicious activities.

• Cloud backup: A EOA will be created when user choose the cloud backup, the private key of the EOA is stored in either iCloud or Google Drive, user can use this cloud-stored private key to access their account from different device, and users can remove or overwrite this backup section at any time.

• Social recovery: User can assign their family or friend’s clave address as backup, if M of N guardians give a confirmation for recovery, the recovery will be executed after 48hrs lockdown if without cancel.

Soul wallet: (Passkey) + (4337 SCA)

Basics:

• Demo: https://alpha.soulwallet.io/wallet

• Account: ERC4337 SCA

• Chain: Ethereum, Optimism, Arbitrum, and soon all EVM layer2

Transaction process:

• Key creation: Users provide biometric authentication like fingerprint or facial recognition, and the operating system generates Passkey and backs it up using cloud service. You can add more than one passkey cross-device and cross-platform.

• Add gardians(Optional): User can assign different EVM EOA address as guardians and set up a threshold for account recovery.

• Account generation: Using counterfactual deployment, users don’t need to pay any fee until the first transaction

Recovery mechanism:

• Passkey: Use any defined passkey to log into the wallet using an arbitrary device.

• Guardians recovery: The assigned guardians can rotate the wallet according to the threshold, and a time lock may addressed later to prevent malicious behavior.

OKX wallet:(MPC-TSS + Passkey) + (4337 SCA)

Basics:

• Demo: https://www.okx.com/help/what-is-an-aa-smart-contract-wallet

• Chain: 30+ chains

• Key: MPC-TSS, 2/3

• Account: 4337 SCA

Transaction process:

• Key creation: By creating a wallet, the OKX transforms a single private key into three separate shares. Share 1 is stored in the OKX server, share 2 is stored on user’s device local storage, and share 3 is generated by device, encrypted and can be backed up to the device’s preferred cloud services, like Google Cloud, iCloud and Huawei Cloud.

• Key signing: OKX using the MPC-TSS technology, user can obtain the full signature by using two out of three the private key shares when signing the transaction, the key shares never meet each other during this process.

Recovery mechanism:

• 2/3 mechanism: When user logged out, device is unavailable or one of the keys on the device is compromised, you can use new device to login OKX wallet(get the server share) and get the cloud service share, combine these2 shares to recover the wallet, OKX wallet will generate new secret shares.

Web3Auth: (MPC-TSS + Passkey)+ (EOA/SCA)

Basics:

• Demo: https://w3a.link/passkeysDemo

• Chain: All EVM and Solana

• Key: MPC-TSS, usually 2/3

• Account: Any account like EOA, 4337 SCA or general SCA

Transaction process:

• Key creation: By creating a wallet, three key shares generated. Share1 is social login share, user can type in their email, and a decentralized network of nodes the store the key for each user; Share2 is device share that stored on user’s device local storage; Share3 is generated by local computer and backed up by user’s preferred cloud services.

• Key signing: The Web3Auth MPC-TSS architecture ensures that the user’s key is always available, even using the threshold signing, keys never reconstruct or stored in a single place.

Recovery mechanism:

• Threshold recovery When user logged out, device is unavailable or one of the keys on the device is compromised, you can use social login method to login webAuthn account and get the passkey cloud share, combine these2 shares to recover the wallet.

Lit Protocol(MPC-TSS + Decentralized nodes + Passkey)+ (EOA/SCA)

Basic information:

• Demo: https://lit-pkp-auth-demo.vercel.app/

• Chain: Most of the EVM, Cosmos, Solana.

• Account: MPC-TSS, 20 of 30 network, can be adopted by both SCA and EOA.

Transaction process:

• Key creation: When user want to create a wallet, use first select an auth method(passkey, oAuth social login supported), a request then sent to relayer to create key pairs and store the auth logic to smart contract. Each key pair is generated collectively by the Lit nodes through a process called Distributed Key Generation (DKG). Operating as a decentralized network, 30 Lit nodes running inside TEE, each node holds a share of the key but the private key never exists in its entirely.

• Key signing: Receiving the request, Lit nodes independently validate or reject the request against the assigned authentication method, and using the MPC-TSS technology,1. Key shares are collected above the threshold(20 out of 30) to generate a signature and combined by client to fulfill the request.

Recovery mechanism:

• 2/3 mechanism: Use the auth methods stored in the smart contract to access the account, Lit nodes validate the requests and it will proceed if over 2/3 of nodes confirm.

Conclusion:

Fueled by enthusiasm for Layer2, Layer3, and Data availability solutions, we are keen to improve the performance of blockchain. Also, pursuing real security, combining Zero Knowledge Proof Privacy with transparency nature. All efforts are targeting one goal: To be ready for real users that frequently interact with blockchain and adopt crypto into their lives.

It’s easy to be trapped in an optimal technology dream, but we must ask: what kind of experience are we aiming for? We envision a world where crypto is about intuition rather than daunting tech terms, where a user jumps into the rabbit hole without hesitation and hassle.

Picture a user Rui: She discovers a fantastic dApp, easily signs up using facial recognition or a fingerprint, with the option to set up backups or guardians. As she engages with the dApp, she smoothly executes transactions, possibly with small ERC20 fees or even none at all. Later, she customizes her wallet settings – perhaps activating a time-lock for automated transactions, adding another device as a backup signer, or revising her guardian list.

Our builders make that happen. Integrating WebAuthn and Passkey, we enhance private key management, making it both secure and user-friendly. On top of the key, SCA as an entity opens a realm of personalized security and functionality. And as for gas fees? They become less burdensome, thanks to Paymaster providers who can create a 'vault' for swaps or even allow advertisers to cover fees for users. At the heart of this evolution, particularly for Ethereum mainnet and its equivalent Layer2s, is ERC4337. It introduces an alternative mempool that sets apart SCA transactions from EOAs, without major protocol overhauls. On the other hand, some Layer 2 networks are even embracing SCAs natively, seamlessly incorporating them into their systems.

It requires tremendous efforts to make everything easy. Many challenges exist like lowering deployment, validation, and execution fees for SCA; Standardizing the interface to increase account interoperability; Syncing account state cross-chain; and many more. Credit to all the builders, we’re getting closer to solve the puzzle day by day. And crypto ventures like us- SevenX, are ready to help great firms realize their vision.

If all of these interest you, there are my other pieces provides more comprehensive context :

• 04/ Account: Modular Smart Contract Account Architecture and Challenges

• 03/ Key(this article): WebAuthn and Passkey, Key Management for Daily Crypto Users

• 02/ Infrastructure: Ethereum Account Evolution brought by ERC4337

Acknowledges:

Acknowledgment to my dearest friends for their review:

• David Sneider, co-founder of Lit Protocol

• Henri Stern, co-founder of Privy

• Zhen, co-founder of Web3Auth

• Dogan and Rafi, founding members of Clave

• Ivo, founder of Ambire

• Itmar, founder of Argent

• Kurt Larsen and Konrad, co-founders of Rhinestone

• Pengyu Wang and Peter Pan, co-founders of Particle network

• OKX Wallet team

• David Kim, eng lead of Trust Wallet

• Vivian Phung, founder of Snowball Tools

• evmBrahmin, founder of HopeSec

• Jose Aguinaga

Reference:

Keychain services | Apple Developer Documentation

Securely store small chunks of data on behalf of the user.

https://developer.apple.com

Supporting passkeys | Apple Developer Documentation

Eliminate passwords for your users when they sign in to apps and websites.

https://developer.apple.com

The good, the bad and the ugly of Apple Passkeys | SlashID Blog

We stop identity breaches

https://www.slashid.dev

Clave - Meet with Clave #3 : Recovery for Everyone

Onboarding to future of finance

https://blog.getclave.io

Many thanks to @13yearoldvc @prabalbanerjee @maqstik @ballsyalchemist @donnoh_eth @ChundaMcCain @shumochu @ranvirrana001 and others for related discussions and review!

Recently, there has been a noticeable trend where an increasing number of dApps are announcing the launch of their own rollups. Additionally, there's a rise in the number of generic rollups that are set to go live.

Generic rollups address Ethereum's scalability issues as it faces rising transaction volumes and dApp growth. These layer 2 solutions process more transactions off-chain, later securing them on the main chain, balancing scalability with security. Their versatility supports various dApps, removing the need for unique scaling solutions for each application.

App-specific rollups are tailored solutions that address the unique needs of individual applications. They offer enhanced speed by optimizing transaction processing for specific use cases. Cost-wise, they might provide a more efficient alternative to generic solutions, especially during network congestion. Their standout feature is flexibility. Unlike general-purpose Layer 2 solutions that are rigid and are more constrained by the enshrined EVM design, app-specific rollups can be customized, making them ideal for applications like games that require specific precompiles. Additionally, they allow dApps to better capture value, offering more control over token economics and revenue streams.

With the consensus forming around the proliferation of rollups, looking a year into the future where multiple rollups dominate the market, the need for a robust infrastructure becomes paramount. This infrastructure will serve as the "reinforced concrete" of a multi-rollup world.

This article will delve into four fundamental pillars that will shape the future of the multi-rollup ecosystem:

• Security As a Foundation: The Security Layer is the bedrock of trust in the decentralized world. In this section, we explore the vital role it plays in ensuring the integrity of Layer 2 transactions, identifying trust assumptions, and addressing potential security pitfalls.

• Balancing Customizability and Interoperability : Achieving seamless interoperability among diverse rollups is pivotal for a modular blockchain world. In this section, we dive into the interop problems brought by a modular structure and discuss current solutions to address fragmentation, and foster a cohesive ecosystem.

• Cost Analysis: Reducing costs is crucial for the broader adoption and viability of rollups, as it lowers the economic barriers compared to utilizing smart contracts. Cost efficiency in rollups is primarily achieved through harnessing economies of scale by aggregating with other rollups to share fees, and embracing the division of labor by delegating certain tasks to external service providers.

• Shared Security: A shared security layer is essential as it alleviates the time and resource-intensive process of bootstrapping security for new protocols or modular layers, ensuring a robust security comparable to established platforms like Ethereum. Numerous solutions like Eigenlayer, Babylon, Cosmos's ICS, and Mesh Security have emerged, showcasing a variety of applications

Together, these four layers will provide a comprehensive blueprint for the infrastructure needed to support a thriving and cohesive modular blockchain world.

Security As a Foundation

At the heart of any decentralized system lies trust and security. Their absence undermines the very promise of a trustless ecosystem. This is why the security layer is paramount; without it, users and TVL are at risk. Plasma and Sidechains' decline offers a cautionary tale. Once seen as Ethereum's scaling savior, its issues, like the "data availability problem," eroded trust and led to its waning popularity. That's why the security layer becomes part I of this article.

To understand the intricacies of rollups and their potential vulnerabilities, it's essential to dissect the lifecycle of a Layer 2 transaction. Using smart contract rollups as a reference, let's delve into each phase and identify the trust assumptions and potential security pitfalls:

• Tx Submission through RPC:

  • Trust Assumption: The RPC endpoint is reliable and secure. Users and dapps are now trusting rpc providers eg alchemy, infura, etc.

  • Security Concern: Users might be censored by rpc providers, e.g. infura and alchemy blocking rpc requests to tornardo cash. RPC providers might face DDOS attacks, eg ankr being comprised via DNS hijack.

  • Solutions: RPC providers, such as Infura, are actively pursuing a decentralized roadmap. Additionally, users have the option to choose decentralized solutions like the Pocket Network.

• Sequencer Orders the Tx, Provides Soft Commitments: unsafe state

  • Trust Assumption: Users expect sequencers to fairly order transactions and provide genuine soft commitments.

  • Security Concern: The system must resist censorship, ensuring all transactions are processed without bias. It's crucial for the system to remain continuously operational, and it would be better to guard against sequencers gainingbad MEV at the expense of the end user.

  • Solutions:

    • CR and liveness:

      • current solutions ranking based on CR and liveness level (low to high): single sequencer——POA——permissionless POS sequencers——shared sequencers——based rollups(sequenced by l1)

        • Note that POA with limited authorities without support for force txns can be less CR than a centralized sequencer with force txn enabled.

        • Regarding liveness, another crucial metric to consider is proposer failure, which occurs when a proposer goes offline. In such cases, it's essential to ensure that users can still withdraw their funds.

          • Even if the sequencers are censoring or refuse to work, some rollups enable users to submit their txs directly to L1s by themselves, i.e the escape hatch ( liveness for forced txs depend on the specific implementation ). The problem is that it might be too expensive for users with limited funds to do that and users might expect real-time CR and liveness.

          • Certain rollup solutions, such as Arbitrum and Fuel, offer the capability for anyone to become a proposer after a certain time delay, i.e self propose.

          • Check out this indicater for each rollup: https://l2beat.com/scaling/risk

      • More details on other different solutions can be referred to my previous thread: https://twitter.com/yuxiao_deng/status/1666086091336880128

    • MEV-protection:

      • Different privacy solutions can help protect users from being front-run or sandwiched as the tx info is hidden (also help with CR). Related methods for hiding tx info include FCFS with a private mempool (what arbitrum and optimism are implementing right now), SUAVE's TEE solution, threshold encryption (shutter network working on this), etc. The more sophisticated the solution is, the less complicated computation on the txs can be done.

• Note that what we want is mev-protection not mev-elimination. Research by @tarunchitra summarizes two main directions to reduce MEV: reduce the flexibility of the miner to reorder transactions by enforcing ordering rules and introduce a competitive market for the right to reorder, add, and/or censor transactions. However, the paper concludes that neither fair ordering nor economic mechanisms alone can effectively mitigate MEV for all payoff functions. There are lower bounds on how you can’t remove MEV beyond some point.

• Sequencer executes and posts tx batch and state roots to the DA layer when it is economically reasonable; safe state

  • Trust assumption: Block producers publish the whole block on the DA layer so that others can download and validate them.

  • Security Concern: If part of the data is not available, the block might contain malicious transactions that are being hidden by the block producer. Even if the block contains non-malicious transactions, hiding them might compromise the security of the system. It is very important that sequencers have tx data available, as the rollup needs to know about the state of the network and account balances

  • Solutions:

  • Posting on Ethereum now is the safest but most expensive solution(would be 90% cheaper after protodankshadring, but even a 10x throughput increase might still not be enough for the rollups): the rollups' txs are downloaded and gossiped by all Ethereum nodes. As Ethereum has a large number of nodes replicating and verifying transaction data, it is highly unlikely that data will ever disappear or be entirely unavailable.

    • After danksharding, ethereum nodes will not download all the tx data, but only parts of the data using the DAS and KZG (similar to avail's solution mentioned below)

    • Under the modular concept, it might be more efficient for rollups to post tx data to a DA layer which is only responsible for DA (The theoretical performance of Ethereum might be slightly inferior because, in addition to DA, it still retains the execution of L1, see the performance comparison between eigenDA and Ethereum below).

• Current modular DA solutions present trade-offs between safety and performance. It's challenging to compare the security of DA using just one dimension:

  • Avail and Celestia utilize DAS to ensure the availability of the data; as long as there is enough sampling, the data is secured. LCs can sample and get a high guarantee of DA as data unavailability would be easily detected and recovered by very small portions of LCs . This is not possible without DAS. The decentralization of the DA layer, i.e. the number of nodes in the network decides the security level and also the stake distribution. EigenDA doesn't use DAS but uses a proof-of-custody mechanism to prevent restakers from being lazy, i.e. the DA operators have to routinely compute a function that can only be completed if they have downloaded all the required data and get slashed if they fail to attest to the blobs right (no need to store after the proof has been performed though).

  • Ensure the data duplication process, i.e., erasure encoding, is accurate. EigenDA, Ethereum after 4844, and Avail use kzg commitments to guarantee accuracy, but these are computationally intensive. Celestia employs fraud-proof. Light nodes must wait for a brief interval before they can confirm a block has been correctly encoded, finalizing it from their perspective. (*Celestia could potentially switch to validity proofs if it’s a better trade-off option)

  • Economic security of the DA layer (reorg and collusion risks): depends on the value staked in the DA layer, =2/3 of value staked in the Avail and Celestia

  • Relaying the DA attestation of the DA layer to Ethereum. If the data is posted to another DA layer while the settlement contract is still in Ethereum, then we need a bridge contract to validate the DA is available in the DA layer for final settlement.

    • Celestia's blobstream verifies the signatures on the DA attestation from Celestia. The attestation is a Merkle root of the L2 data signed by the Celestia validators attesting to the fact that the data is available on Celestia. This feature is available on testnet right now.

    • Avail uses an optimistic approach to verify the DA attestation. Once the attestation is posted to the bridge contract on Ethereum, a waiting period begins during which the attestation is assumed to be valid unless challenged.

    • Succinct is working with Avail and Celestia on a zk-SNARK-based data attestation bridge, which makes the attestation progress more secure and cheaper by just verifying the zk proof.

    • For EigenDA, the disperser splits and posts tasks to EigenDA nodes and then aggregates signatures from them and relays data to Ethereum

• Final Settlement: finalized state

  • Trust Assumption 1:

    • Rollup full nodes (a node that can fully calculates the state without relying on other proofs) can finalize the first valid rollup block at its height as soon as it is published on the parent chain as they have the necessary data and computational resources to verify the validity of the block quickly. However, this is not the case for other third parties like light clients, which rely on validity proofs, fraud proofs, or dispute resolution protocols to verify the state trustlessly without needing to run a full replica of the chain themselves.

  • Security Concern 1:

    • For ZK Rollups, l1 verifies the zkp and only accepts correct state roots. The difficulty mainly lies in the cost and generation process of zkp.

    • On the other hand, Optimistic Rollups depend on the premise that at least one honest party will promptly submit fraud proof to contest any malicious transactions. However, most current fraud-proving systems are not yet permissionless, and the submission of fraud proofs is reliant on only a few validators.

  • Solutions 1:

    • Permissionless fraud proving enabled by Arbitrum's BOLD protocol. The main reason why fraud proving is permissioned right now is the concern of delay attacks:

      • During the challenge period, any stakers other than the proposer can launch a challenge. The proposer is then required to defend their assertion against each challenger individually, one at a time. At the conclusion of each challenge, the party on the losing end forfeits their stake.

      • In a delay attack, a malicious party (or group of parties) can prevent or delay the confirmation of results back to the L1 chain by making challenges and deliberately losing the dispute and stakes)

      • 𝐁𝐎𝐋𝐃 challenge protocol addresses this by guaranteeing fixed upper bounds on confirmation times for Optimistic Rollups’ settlement by ensuring a single honest party in the world can win against any number of malicious claims.

    • Witness chain can act as a watch tower for new optimistic rollups to guarantee at least one honest party would challenge an invalid state:

      • For established rollups such as Arbitrum and Optimism, there are enough intrinsic incentives for several third-party providers such as explorers, Infura-like services, and their foundation to monitor the chain state and submit fraud proof when necessary. However, new rollups or appchains might lack this level of security.

      • Witness Chain employs a unique incentive mechanism, the "Proof of Diligence," which ensures that watchtowers (validators) are consistently motivated to monitor and verify transactions, ensuring the state submitted to the parent chain is correct. This mechanism guarantees that each watchtower performs its due diligence since the rewards they receive are specific and independent for each node. In other words, if one watchtower discovers a bounty, it cannot share the exact incentive payout with other watchtowers, ensuring that every node conducts its independent verification. Additionally, Witness Chain offers flexibility by allowing rollups to specify custom requirements, such as the number of watchtowers and their geographical distribution powered by “proof of location” - their independent service. This flexibility ensures a balance between security and efficiency.

        *Watchtower network is also emerging as a new layer in the rollup stack itself, providing pooled security to execution used by other related applications - such as the rollup security itself, interop protocols, notification service and keeper network, etc. More details will be launched in the future.

  • Trust Assumption 2:

    • The whole process of settlement for smart-contract rollups is written in smart contact on L1. The smart contract on the DA layer is assumed to be logically accurate, bug-free, and not evilly upgraded.

  • Security Concern 2: Smart contract rollups' bridges and upgrades are controlled by multi-sig wallets. The bridge has the ability to arbitrarily steal funds from users via a malicious upgrade.

  • Solutions 2:

    • The most popular idea today is to add time delays that allow users to exit if they disagree with a planned upgrade. However, this solution requires users to continually monitor all chains they have tokens on in case they ever need to exit.

    • Altlayer's Beacon Layer can act as a social layer for upgrades for all the rollups enshrined to it. Sequencers that register to operate a rollup together with the Beacon Layer rollup validators can socially fork the rollup regardless of whether or not the enshrined bridge contract on Ethereum gets upgraded.

    • Enshrined rollups in the long term: enshrined rollup has been the endgame of the Ethereum roadmap for several years now. Except for the enshrining bridge/fraud proof verifier on L1, the settlement contract is also enshrined.

      • Ethereum PSE is working in this direction

As for sovereign rollups, the main difference is that the chain state is settled by rollup full nodes instead of the enshrined smart contract in the L1. More detailed comparison can referred to https://www.cryptofrens.info/p/settlement-layers-ethereum-rollups

It's important to note that more security doesn't equate to better performance. Typically, as security measures increase, there's a trade-off with scalability. Therefore, it's essential to strike a balance between the two. In conclusion, rollups offer the flexibility to select varying levels of security assumptions based on individual preferences. This adaptability is one of the remarkable features of the modular world, allowing for a tailored approach to meet specific needs while maintaining the integrity of the system.

Balancing Customizability and Interoperability

It's a well-known adage in the modular world: "Modularism, not maximalism." If rollups can't interoperate securely and efficiently, then modularism ≠ maximalism but = fragmentation. It's crucial to figure out how to handle interoperability between different rollups.

Let's first revisit how monolithic chains achieve interoperability. In simplest terms, they achieve cross-chain operations by verifying the consensus or state of the other chain. There are various approaches available in the market, and the differences lie in who is responsible for the verification (official entities, multi-signature mechanisms, decentralized networks, etc.) and how the correctness of the verification is ensured (through external parties, economic guarantees, optimistic mechanisms, zk-proofs, etc.). For a deeper dive into this topic, check out my favorite bridging pieces: Thoughts on Interoperability.

With the rise of modularization, the issue of interoperability has become more intricate:

• Fragmentation problem:

  • The proliferation of rollups is expected to significantly surpass the number of L1s as it’s way easier to land a l2 than l1. Could this lead to a highly fragmented network?

  • While monolithic blockchains offer a consistent consensus and state for straightforward verification, what will be the verification process for modular blockchains which have three (or possibly four) distinct components (DA, execution, settlement, and sequencing) ?

    • DA and settlement layer become the main source of truth. Execution verification is already available as rollups inherently provide execution proofs. Sequencing happens before posting to DA.

• Extensible Problem:

  • As new rollups are introduced, the question arises: can we promptly offer bridging services to accommodate them? Even if building up a rollup is permissionless, you might need to spend 10 weeks convincing other folks to add one. Current bridging services predominantly cater to mainstream rollups and tokens. With the potential influx of numerous rollups, there's a concern about whether these services can efficiently evaluate and launch corresponding solutions to support these emerging rollups without compromising on security and functionality.

• User-experience Problem:

  • The final settlement of optimistic rollups takes seven days, which is much longer than other L1s. The challenge is addressing the seven-day wait time for the official bridges of optimistic rollups. The submission of zkp also has a time lag as rollups usually wait to accumulate a large batch of transactions before submitting proof to save verification costs. Popular rollups like StarkEx typically post proofs to L1 only once every few hours.

  • Rollups' tx data submitted to the DA/settlement layer will have a time lag to save costs (1-3 mins for optimistic rollups and a few hours for zk rollups as mentioned above. This needs to be abstracted away from users who have needs for quicker and safer finality.

The good news is that there are emerging solutions to these challenges:

• Fragmentation problem:

  • While there is a proliferation of rollups in the ecosystem, it's noteworthy that the majority of smart contract rollups currently share a common settlement layer, namely Ethereum. The primary distinctions among these rollups lie in their execution and sequencing layers. To achieve interoperability, they simply need to mutually verify the final state of the shared settlement layer. However, the scenario becomes slightly more intricate for sovereign rollups. Their interoperability is somewhat challenging due to differing settlement layers. One approach to address this is by establishing a Peer-to-Peer (P2P) settlement mechanism, where each chain directly embeds a light client of the other, facilitating mutual verification. Alternatively, these sovereign rollups can first bridge to a centralized settlement hub, which then serves as a conduit for connecting with other chains. This hub-centric approach streamlines the process and ensures a more cohesive interconnection among diverse rollups. (similar to the status of cosmos interop)

• Besides Ethereum serving as one of the settlement hubs, other potential settlement hubs include Arbitrum, zkSync, and StarkNet, which act as settlement hubs for L3s built on them. The interop layer of Polygon 2.0 also functions as a central hub for zk rollups built on top of it.

• In conclusion, while the number of rollups and their variations are expanding, the quantity of settlement hubs remains limited. This effectively simplifies the topology, narrowing down the fragmentation problem to just a few key hubs. Although there will be more rollups than altl1s, cross-rollup interactions are less complicated than cross-l1 interactions as rollups usually fall within the same trust/security zoom.

• How different settlement hubs interop with each other can refer to how current monolithic chains interop with each other, as mentioned at the beginning.

*Moreover, in an effort to eliminate fragmentation on the user side, certain Layer 2s, such as ZKSync, have integrated native Account Abstraction to facilitate a seamless cross-rollup experience.

• Extensible problem

  • Hyperlane(provide modular security for modular chains) and Catalyst(Permissionless cross-chain liquidity) are born to solve the permissioned interop problem.

    • The essence of Hyperlane is to create a standardized security layer that can be applied across various chains, making them inherently interoperable.

    • Catalyst is designed to offer permissionless liquidity for modular chains. It acts as a bridge, allowing any new chain to connect liquidity and swap with major hubs like Ethereum and Cosmos seamlessly.

  • Rollup SDK/RAAS Providers offer native bridging services within their ecosystem

    • Now, new rollups are mostly launched through existing rollup SDKs or RAAS services, so they are inherently interoperable with other rollups that use the same services. For example, for infrastructure built with the OP Stack, the foundational level is a shared bridging standard, which allows for the seamless movement of assets across everything that shares the OP Stack codebase. For rollups launching through the altlayer, they are all enshrined to the beacon layer, which acts as the settlement hub and ensures safe interoperability. For rollups launching through sovereign labs or zksync, they are interoperable with each other out of the box based on proof aggregation (will explain more later).

UE problem:

• Before diving in this part, let’s first recognize different levels of commitments and their time lag:

• Some parties are comfortable with stage1 soft commitments by l2, eg exchanges like Binance only wait for a certain number of Layer2 blocks to consider transactions as confirmed, without the need to wait for batch processing to be submitted to Layer1

  • Some bridge providers like hop protocol would take as many blocks on the sending chain and determine finality based on L1 consensus (stage 2)

  • For trust-minimized bridges and for users to withdraw the funds from l2-l1 using the official bridge, it might take too long (several hours & 7 days)

• Reducing either Stage 2 or Stage 3 would offer significant advantages, delivering a stronger guarantee in a shorter time frame for a more secure and faster user experience. Additionally, achieving a trust-minimized bridge has always been a coveted goal, especially in light of the frequent security incidents with bridges.

• Reducing the final settlement time (7days for optimistic rollups and several hours for zk rollups), i.e. shorten stage 3

  • Hybrid Rollups (Fraud Proof + ZK): This approach combines the strengths of ZK proofs and optimistic rollups. While generating and verifying proofs can be resource-intensive, it's only executed when a state transition is challenged. Instead of posting a ZK proof for every batch of transactions, the proof is computed and posted only when a proposed state is contested, similar to an optimistic rollup. This allows for a shorter challenge period since the fraud proof can be generated in a single step, and the cost of ZK proofs is avoided in most scenarios.

    • Notably, Eclipse's SVM rollups and LayerN utilize risc0 to generate the zk fraud proof. The OP Stack has granted support to Risc0 and Mina for zk fraud proof development. Additionally, Fuel has recently introduced a similar hybrid method that supports multiple provers.

  • After posting data to the DA layer, do some extra verification of the correctness of execution to increase the confidence level——high requirements, same as a full node

    • When the sequencer batches txs to the optimistic rollups’ DA layers, it ensures canonical ordering and DA for x-rollup txs. Therefore, the only thing that needs to be confirmed is the execution: S1 == STF(S0, B1). Of course, you can simply run a full node (high requirements) to verify the tx, but what we really want is to reduce latency for light clients. Prover networks like @SuccinctLabs and @RiscZero can confirm the post-execution state by providing succinct state proofs. This provides robust confirmation for dapps and users.

    • Altlayer has a beacon layer between rollups and L1. Sequencers from the beacon layer are responsible for sequencing, executing, and generating proof of validity (POV). POV allows verifiers to verify a state transition for a rollup later without having access to the entire state. With decentralized verifiers performing periodic checks, we have achieved a highly robust transaction finality. No need to wait for 7 days as the verifiers have already completed the necessary checks. As a result, cross-chain messaging has become faster and more secure.

    • EigenSettle guarantees verification through economic mechanisms. Opt-in EigenLayer nodes with stakes do the computation to ensure the validity of the state and use their collateral to back their commitments. Any amount that is lower than the amount of stake these operators have posted can be treated as safely settled and enables economically-backed interoperability.

  • Instant Verification with ZK Rollups:

    • Sovereign Labs and Polygon 2.0 employ an innovative approach to achieve rapid finality by circumventing the settlement layer. Instead of waiting to submit the proof to Ethereum, we can instantly disseminate the generated zk proofs through a peer-to-peer network and conduct cross-chain operations based on the propagated zkps. Later on, we can utilize recursion to consolidate them into a batch proof and submit it to Layer 1 when it becomes economically viable.

      • It’s not fully settled though, we still need to trust the correct aggregation of the zkp. Polygon 2.0's Aggregator may be operated in a decentralized manner, involving Polygon validators from the shared validator pool, thereby improving network liveness and resistance to censorship. Nevertheless, using this method will also lead to a shorter finality time since aggregating zkps from multiple chains is certainly faster than waiting for sufficient zkps on a single chain.

    • Zksync’s hyperchains utilize the layering method to aggregate zkp and achieve shorter finality. In contrast to settling on L1, hyperchains can settle their proofs on L2 (become a l3). This approach facilitates rapid messaging, as the cost-effective environment in L2 enables swift and economically viable verification.

      • To enhance scalability further, we can replace L2 settlement with a minimal program required to operate L3 with messaging. This concept has been substantiated through specialized proofs that allow for aggregation.

• Addressing the Time Lag in Posting to the DA Layer (some methods can also be applied to reduce the settlement period), i.e. shorten stage2

  • Shared Sequencing Layer: If rollups share a sequencing layer (e.g., through a shared sequencer service or using the same set of sequencing layers), they can obtain a soft confirmation from the sequencer. This, combined with an economic mechanism, ensures the final state's integrity. Possible combinations include:

    • Stateless shared sequencer + builder makes execution promise by staking proposed by Espresso; This approach is more suitable for rollups with PBS structure, assuming the block builder already has the necessary rights to parts of the blocks. Since the builder is stateful and serves as the underlying execution role for the shared sequencers, it's natural for it to make additional promises.

    • Shared validity sequencing proposed by Umbra research: stateful shared sequencer + fraud proof to ensure good behavior. Sequencers accept cross-chain requests. To prevent dishonest behavior by sequencers, a shared fraud-proof mechanism is used, involving slight changes to the original rollup fraud-proof mechanism. During the challenge period, challengers would also verify the correct execution of atomic actions. This may involve checking the roots of bridging contracts on different rollups or examining the Merkle proof provided by the sequencers. Dishonest sequencers get slashed.

  • Third-Party Intervention: External entities like Hop, Connext, and Across can step in to mitigate risks. They validate messages and front the capital for users' cross-chain financial activities, effectively reducing the waiting period. For example, Boost (GMP Express) is a special feature of Axelar and Squid that reduces transaction time across chains to 5-30 seconds for swaps below a value of $20,000 USD.

  • Intent Infrastructure for bridging as a specific form of third-party intervention: This revamped infrastructure can embrace more third-parties to step in and solve the cross-domain intents for users.

    • Through an intent-focused architecture (extracting away frictions and complexity from users by involving sophisticated actors like MMs and builders), users convey their intended objective or result without detailing the precise transactions needed to realize it. Individuals with a high tolerance for risk can step in, front the necessary capital, and levy increased fees.

    • It’s safer because users’ funds would only be released when the outcome is valid. It might be faster and more flexible because more parties(solvers) are permissionlessly involved in the solving process and competing to give a better outcome to users.

    • UniswapX, flashbots’ SUAVE, and essential are all working in this direction. More on intents:

      nft://10/0x9351de088B597BA0dd2c1188f6054f1388e83578/?showBuying=true&showMeta=true

    • The challenging aspect of this solution centers around the settlement oracle. Let's take UniswapX as an example. To facilitate cross-chain swaps, we rely on a settlement oracle to determine when to release funds to solvers. If the settlement oracle opts for the native bridge (which is slow), or if a third-party bridge is used (raising trust concerns), or even if it's a Light Client bridge (not yet ready for use), we essentially find ourselves in the same loop as before. Hence, UniswapX also offers "Fast cross-chain swaps" similar to an optimistic bridge.

    • Simultaneously, the effectiveness of intent resolution relies on the competition among solvers. Since solvers need to rebalance their inventory across different chains, this may potentially lead to centralized solver issues, limiting the full potential of intents.

To summarize, it can be observed that there are three ways to address the UE problems:

• Use the magic of zk :

  • The primary challenge lies in the performance of zk technology, encompassing both the time required for generation and associated costs. Additionally, when dealing with highly customizable modular blockchains, the question arises: do we possess a zk proving system capable of accommodating the myriad of differences?

• Use an economic slashing scheme to guarantee :

  • The major drawback of this approach is the time delay inherent in the decentralized method (for instance, in the case of EigenSettle, we must wait for the cap to be reached). Furthermore, the centralized approach offers limited commitments (as exemplified by shared sequencing), relying on builders/sequencers to make commitments, which can be restricted and lack extensibility.

• Trust a third party :

  • While trusting a third party can introduce additional risks, as users must have faith in the bridge, intent-enabled cross-domain swaps represent a somewhat more "decentralized" form of third-party bridging. However, this approach still contends with oracle latency, trust issues, and potential time delays, as you must wait someone to accept your intent.

It's interesting that modularization also introduces new possibilities for interoperability experiences:

• Enhanced Speed with Modular Components: By breaking down into finer modules, users can get quicker confirmations from the layer2 level (might be already safe enough for ordinal users)

• Shared Sequencer for Atomic Transactions: The concept of a shared sequencer could potentially enable a new form of atomic transactions, such as flash loans. More details on : https://twitter.com/sanjaypshah/status/1686759738996912128

Modular interoperability solutions are experiencing rapid growth, and currently, there are various approaches, each with its own strengths and weaknesses. Perhaps the ultimate solution is still some distance away, but it's heartening to see so many individuals striving to create a safer and more connected modular world before the rollup explosion arrives.

Cost Analysis

One factor contributing to the limited number of rollups in existence is the economic consideration associated with their launch, in comparison to utilizing smart contracts. Operating via smart contracts adopts a more variable cost model, where the primary expense is the gas fee, whereas launching and maintaining a rollup incurs both fixed and variable costs. This cost dynamic suggests that applications with a substantial transaction volume or relatively high transaction fees are better positioned to leverage rollups, as they have a greater capacity to amortize the fixed costs involved. Consequently, initiatives aimed at reducing the cost associated with rollups—both fixed and variable—are paramount. Delving into the cost components of rollups, as elucidated by Neel and Yaoqi during their talk at ETHCC, provides a clearer picture:

Employing a financial model, such as the Discounted Cash Flow (DCF) analysis, can be instrumental in evaluating the viability of launching a rollup for an application. The formula:

DCF(Revenue - Expenses)>Initial Investment

serves as a baseline to ascertain whether the operational income surpasses the initial investment, thereby making the launch of a rollup a financially sound decision. Protocols that succeed in lowering operational costs while augmenting revenue are instrumental in encouraging the increased adoption of rollups. Let’s explore one by one:

• Initial Development and Deployment Fee

  • The initial setup, despite the availability of open-source SDKs like Opstack and Rollkit, still demands a significant amount of time and human capital for installation and debugging. Customization needs, for instance, integrating a VM into an SDK, further escalate the resources required to align the VM with the various interfaces each SDK provides.

  • RAAS services like AltLayer and Caldera can significantly alleviate these complexities and efforts, encapsulating the economic benefits of division of labor.

• Recurring Fee/Revenue

  • Revenue （++++）

    • User fees

      • = L1 Data Posting Fee + L2 Operator Fee + L2 Congestion Fee

      • Although some user fees might be offset by expenses, scrutinizing and striving to lower these costs is vital as rollups may become untenable if user fees are prohibitively high for users. (Explored in the expense section)

    • Miner Extractable Value (MEV) captured

      • Primarily related to the transaction value originating from the chain, this can be boosted either by enhancing MEV-extraction efficiency or increasing cross-domain MEV.

      • Partnering with established searchers, employing a PBS auction to foster competition, or leveraging SUAVE’s block building as a service are viable strategies to optimize MEV capture efficiency.

      • For capturing more cross-chain MEV, utilizing a shared sequencer layer or SUAVE (shared mempool and shared block building) is beneficial as they connect to several domains.

        • According to recent research by Akaki, shared sequencers are valuable for arbitrage searchers aiming to seize arbitrage opportunities across different chains, as they ensure victory in simultaneous races on all chains.

        • SUAVE serves as a multi-domain order flow aggregation layer, aiding the builder/searcher in exploring cross-domain MEV.

  • Expenses （- - - -)

    • Layer 2 (L2) operation fee

      • Ordering: It might be tricky to compare centralized and decentralized ordering solutions. Competition in more decentralized solutions like Proof of Efficiency can help decrease cost by keeping the operator margin minimal and also incentivize posting batches as often as possible. On the other hand, centralized solutions typically involve fewer parties, which can simplify the process but may not benefit from the same cost-reducing dynamics.

      • Execution: This is where full nodes use VMs/EVMs to execute the changes to a rollup’s state given new user transactions.

        • Efficiency can be bolstered through optimized alt-VMs like Fuel and Eclipse’s Solana VM, which enable parallel execution. However, deviating from EVM compatibility could introduce friction for developers and end-users, along with potential security issues. The compatibility of Arbitrum’s Stylus with both EVM and WASM (which is more efficient than EVM) is commendable.

      • Proving

        • Prover Market

          • Theoretically, utilizing a specialized prover market like Risc0, =nil and marlin, instead of creating a proprietary centralized or decentralized prover network, can result in cost savings for several reasons:

            • There may be a higher level of participation in a dedicated prover market, which in turn fosters increased competition, ultimately leading to lower prices.

            • Provers can optimize hardware usage and be repurposed when a specific application does not require immediate proof generation, reducing operation costs and providing cheaper service.

            • Naturally, there are downsides, including potentially capturing less token utility and relying on the performance of an external party. Furthermore, distinct zk rollups may impose varying hardware prerequisites for the proof generation process. This variability could pose a challenge for provers seeking to expand their proving operations.

            • more on the prover market and prover network: https://figmentcapital.medium.com/decentralized-proving-proof-markets-and-zk-infrastructure-f4cce2c58596

    • Layer 1 (L1) data posting

      • Opting for a more cost-effective Data Availability (DA) layer apart from Ethereum or even using DAC solution can significantly cut down expenses, although at the potential cost of reduced security (explored further in the security layer). For gaming and social which usually have low-value but high-bandwidth, scalability might be a more important factor than security for them.

      • Employing Ethereum as the DA layer allows for leveraging protodansharing and dansharding to attain cost efficiency. Moreover, given that the blob posting fee is set per block irrespective of the blob’s utilization by the rollup, there exists a need to balance between cost and delay: While a rollup would ideally post a complete blob, a low transaction arrival rate leading to fully occupying a blob space results in excessive delay costs.

        • Potential solutions: joint blob posting cost for small rollups;

    • L1 settlement fee

      • For optimistic rollups, the settlement cost is relatively low. Post bedrock, Optimism only pays ~5$ a day to ethereum;

      • For zk settlement, it’s relatively expensive for zkp verification

        • zk-proof aggregation

          • Depending on the underlying proof system, a rollup on Ethereum might spend anywhere from 300k to 5m gas to validate a single proof. But since proof sizes grow very slowly (or not at all) with the number of transactions, rollups can reduce their per-transaction cost by waiting to accumulate a large batch of transactions before submitting a proof.

          • Sovereign labs, polygon 2.0's interoperability layer as mentioned before aggregates proofs from multiple rollups, each rollup can then verify the state of multiple rollups at the same time, saving on verification costs. Zksync’s layering structure combined with proof aggregation further reduces the verification costs.

          • Nonetheless, this method is most effective when two domains utilize the same ZKVM or a shared prover scheme (zksync’s hyperchains use the same zkEVM with fully identical zkp circuits) ; otherwise, it may result in compromised performance.

            • NEBRA labs bring economy of scale and composability of proof verification on Ethereum. NEBRA UPA (Universal Proof Aggregator) universally aggregates heterogeneous proofs so that the verification cost can be amortized. UPA can be used to compose proofs from different sources to allow new use cases as well.

To summarize, the primary methods to economize on rollup costs include:

• Co-aggregating with other rollups to share fees or harness economies of scale:

  • It's noteworthy that such aggregation is also potentially crucial for achieving interoperability. As previously highlighted, employing a congruent layer or framework across diverse rollups simplifies interaction amongst them, ensuring hassle-free information exchange. This consolidated strategy fosters a more integrated and unified Layer 2 infrastructure.

• Delegating certain tasks to external service providers, capitalizing on the principle of division of labor.

As more rollups emerge (meaning you can collaborate with additional parties to divide fees), and more rollup service providers offer more refined services (providing a wider selection of mature upstream providers), we anticipate that the expenses associated with establishing a rollup will decrease.

Shared Security

If you aim to achieve an equivalent level of security (both economically and in terms of decentralization) as the source chain, simply deploy a smart contract or a smart contract rollup. If harnessing a portion of the security provided by the source chain is sufficient to improve performance, there are currently several shared security solutions at your disposal.

Shared security solutions greatly ease the security bootstrap process of most of the protocols or modular layers that need initial security. This is very meaningful for a future modular world, as we envision more infra/protocols emerging to facilitate the functionality of a modular world and more parts of a rollup become modular, except for DA, execution, settlement and sequencing. If a rollup uses a certain modular layer (such as DA) or a service whose security isn't on par with Ethereum's, then the overall security of the entire modular chain could be compromised. We need shared security to enable a decentralized and reliable SAAS service economy.

Eigenlayer, Babylon and Cosmos's ICS and Osmosis’ mesh security serve a pivotal role in offering decentralized trust as a service to other infrastructural entities.

• Eigenlayer allows Ethereum stakers to repurpose their staked $ETH to secure other applications built on the network.

• Cosmos’ ICS allows the Cosmos Hub (“provider chain”) to lend its security to other blockchains (“consumer chains”) in return for fees.

• Mesh Security, brought up by osmosis, enables token delegators (not validators) to restake their staked tokens on a partner chain within the ecosystem. This allows for bidirectional or multilateral security flow, as different appchains can combine their mcaps to enhance overall security.

• Babylon allows BTC holders stake their BTC within the BTC network and provide security to other POS chains by optimizing the use of the Bitcoin scripting language and use an advanced cryptographic mechanism

ICS and Mesh Security, both integral to the Cosmos ecosystem, primarily aim to facilitate inter-chain borrowing of security. These solutions predominantly address the security needs of Cosmos appchains, allowing them to leverage the security of other chains within the ecosystem. Specifically, cosmos hub ICS offers for cosmos chains that don't want to bootstrap validator sets (replicated security), while mesh security requires each chain to have its ownb validator set, but allows far greater optionality to the chain governance.

On the other hand, Babylon presents a unique approach by unlocking the latent potential of BTC holders' idle assets without moving BTC out of its native chain. By optimizing the use of Bitcoin's scripting language and integrating advanced cryptographic mechanisms, Babylon provides additional security to the consensus mechanisms of other chains with great features like faster unbonding periods. Validators on other POS chains with BTC can lock their BTC on the Bitcoin network and sign POS blocks with BTC private keys. Invalid performances like double-signing would leak the btc private key of the validator and burn its BTC on the bitcoin network. BTC staking would be launched in babylon’s 2nd testnet.

While Babylon navigates the constraints of Bitcoin's lack of smart contract support, Eigenlayer operators on the Turing-complete Ethereum platform, Not only does Eigenlayer offer economic security to new rollups and chains, but its environment on Ethereum also allows for a more diverse range of AVS. According to eigenlayer’s article on programmable trust, the security eigenlayer can provide can actually be further broken down into 3 types:

• Economic trust: Trust from validators making commitments and backing their promises with financial stakes.This trust model ensures consistency regardless of the number of parties involved. There must be objective slashing conditions that can be submitted and verified onchain and it’s usually heavyweight for restakers.

• Decentralized trust: trust from having a decentralized network operated by independent and geographically isolated operators. This aspect emphasizes the intrinsic value of decentralization and enables use cases that are not objectively provable as decentralization increases the difficulty of collusion. To utilize decentralized trust, it’s usually lightweight.

• Ethereum inclusion trust: trust that Ethereum validators will construct and include your blocks as promised, alongside the consensus software they are running. This can specifically be committed by ethereum validators (not LST restakers). They run software sidecars to perform extra computation and receive extra rewards.

So now we are clear with the security materials, what can we expect?

• ICS and mesh security lower the security barriers for cosmos appchains like neutron, stride and axelar.

• Eigenlayer can fit into many solutions that have been mentioned before:

  • rollup security: relay network; watch tower, sequencing, mev-protection, eigenDA

  • rollup interop: eigensettle; bridges

  • cost analysis: prover network

  • more to explore, check out https://www.blog.eigenlayer.xyz/eigenlayer-universe-15-unicorn-ideas/

• Babylon is running testnet to increase the security level of other pos chains. Its first testnet provides timestamping service to add addtional security to high-value defi activities from several cosmos chains like akash, osmosis, juno,etc.

The core idea behind these shared security solutions is to enhance the capital efficiency of staked or illiquid assets by introducing additional responsibilities. However, it's essential to be vigilant about the added risks when seeking higher returns:

• Increased complexity introduces more uncertainties. Validators become exposed to additional slashing conditions that may lack enough training wheels, which can be precarious.

  • Eigenlayer aims to address this issue by proposing the implementation of a veto committee. This committee serves as a mutually trusted entity among stakers, operators, and AVS developers. In the event of a software bug within the AVS, stakers and operators won't face penalties as the veto committee can cast a veto vote. While this approach may not be inherently scalable and could be subjective if AVSs are not strictly aligned with use cases based on trustlessly attributable actions, it can still serve as a valuable means to initiate a risk mitigation strategy during the early stages.

• Greater complexity also brings additional burdens. It can be overwhelming for less experienced validators to determine which service to share security with. Also the initial setup period may involve a higher risk of errors. Additionally, there should be mechanisms in place to allow "less tech-savvy" validators and stakers to benefit from higher yields, provided they are willing to accept relatively elevated risks, without being constrained by their operational capabilities.

  • Rio Network and Renzo are all working on effectively addressing this challenge for Eigenlayer by offering a structured approach to cautiously select sophisticated node operators and AVS services for potential restakes, elevating security levels and reducing entry barriers for participants.

Furthermore, as Eigenlayer gains wider adoption, it could potentially open up new horizons in the realm of the Financialization of Security. This could facilitate the valuation of shared security and the various applications built upon it.

• An limitation that is presented to EigenLayer is in its ability to scale capital allocation to its system by outcompeting yield opportunities in DeFi for the same assets that it supports (LSTs). EigenLayer commoditizes the value of security and this opens up the door for many primitives to underwrite this value and provide the ability for restakers to both restake and participate in the greater DeFi ecosystem.

  • Ion Protocol is a product attempting to do this in order to scale the reach that restaking can have. Ion is building a price-agnostic lending platform that is built to specifically support staked and restaked assets via using ZK-infrastructure to underwrite the lower-level slashing risk present in such assets (ZK state proof systems + ZKML). This could initiate the beginning of the birth of many novel DeFi primitives built upon the underlying value of security that EigenLayer commoditizes, further enabling the ability of restaking to scale across the entire ecosystem.

As we stand at the cusp of significant transformations, it is crucial to embrace the principles of security, interoperability, and cost-effectiveness. These pillars will not only guide the development of more scalable and efficient blockchain solutions but will also pave the way for a more interconnected and accessible digital world. Embracing these changes with foresight and adaptability will undoubtedly lead to groundbreaking advancements in the blockchain ecosystem.

Reference

• https://www.cryptofrens.info/p/settlement-layers-ethereum-rollups

• https://www.layern.com/blog/zkfp

• https://www.blog.eigenlayer.xyz/intro-to-eigenda-hyperscale-data-availability-for-rollups/

• https://mp.weixin.qq.com/s/1kqbfC0q8wdQdm-2UpdlPQ

• https://polymerlabs.medium.com/modular-ibc-for-the-modular-world-9fc021f6322e

• https://forum.babylonlabs.io/t/announcing-the-babylon-btc-staking-protocol-litepaper-v1/33

• https://forum.celestia.org/t/a-comparison-between-da-layers/899

• https://github.com/OffchainLabs/bold/blob/main/docs/research-specs/BOLDChallengeProtocol.pdf

• https://www.blog.eigenlayer.xyz/eigenlayer-universe-15-unicorn-ideas/

• https://figmentcapital.medium.com/decentralized-proving-proof-markets-and-zk-infrastructure-f4cce2c58596

• https://github.com/osmosis-labs/mesh-security/blob/main/docs/UseCases.md

Thanks to @msfew from Hyper Oracle for the feedback and discussions.

0. Introduction

In this post, we will look at the recently popular concepts of zkCoprocessor and zkOracle and compare their differences.

1. Definition of zkCoprocessor

When a term is coined, its true meaning is not defined by itself. We have seen this very much in the case of blockchain.

• ZK: While academics often refer to the privacy features of Zero Knowledge, in the blockchain context, ZK often just stands for the succinctness and validity of the technology. Even projects that are built on ZK Rollup call themselves zkSomething.

• DA: Until now, many people didn't really realize that Data Availability is not Data Storage, so in order to solve this terminology problem, some people have suggested replacing DA with DP (Data Publishing), or making the whole meaning of DA into Data Publishing + Data Storage (1, 2).

We see a similar phenomenon in the term zkCoprocessor. Everyone uses the term, but they don't necessarily refer to the same things.

So we wanted to express what the project itself thinks about zkCoprocessor, what the community understands about zkCoprocessor, and what zkCoprocessor really means and does from our perspective.

a) From Project Themselves

Definition 1 from Axiom: zkCoprocessor proves historical data onchain.

The concept of the zkCoprocessor was popularized by Axiom, which originally conceived it as a zkAttestor. From Axiom’s idea, zkCoprocessor represents the component that “proves historical data on-chain and trustlessly use that data in a smart contract”.

Note that the Brevis team said that this type of zkCoprocessors are essentially an API/DSL layer on top of the underlying zk circuit. So this is not programmable.

Definition 2 from RISC Zero: zkCoprocessor offloads computation from onchain to offchain.

RISC Zero also often refers to itself as a zkCoprocessor. From their perspective, they see zkCoprocessor as a broader concept, “a tool for using ZKPs to offload computation from on-chain to off-chain”.

b) From Community

Definition from Peteris (the same as 1): zkCoprocessor can access to historical onchain state.

Peteris from Aera Finance believes that zkCoprocessor acts very much like a state oracle, with the main function being access to historical data. At the same time, he and Rishabh from BananaHQ believes that the description of definition 2 is more like a zkVM than a subclass of zkCoprocessor.

Definition from Messari, Modular Media, and Kobi (the same as 2): zkCoprocessor offloads computation from onchain to offchain.

Messari has also given its own definition of zkCoprocessor. Sami, a researcher at Messari, believes that zkCoprocessor enables smart contract devs to easily offload complex logic offchain without new trust assumptions. Modular Media also gives the same concept. Kobi from Geometry compares rollup with a coprocessor, Brevis added that zkCoprocessor trades off the cost of maintaining a permanent state storage against hyper-boosted performance, Taiko came up with the design of Booster Rollup that further explored the idea of Rollup Coprocessor. These are the same definition as RISC Zero.

c) From Us

To summarize, we conclude that there are two types of zkCoprocessor in practice, and they are as follows:

• Data Access zkCoprocessor: mainly feeds certain data (historical block, historical state) to a certain place (smart contract).

• zkVM Compute zkCoprocessor: computes by the offchain zkVM, then feed result back to the onchain. Compresses O(n) compute into just O(1) verification.

2. Definition of zkOracle

a) Oracle

Hyper Oracle provides us with an explanation of Oracle in Defining zkOracle for Ethereum.

Oracle practically sums up the "infra" in any blockchain space, as a better definition than coprocessor.

If the input to the infra/oracle is off-chain data and the output is on-chain, then it is an input oracle (e.g. Chainlink Price Feed). Conversely, it is an output oracle (e.g. The Graph). If the output oracle is first, then the input oracle, then it is an I/O oracle (e.g. Gelato Network).

In short, oracle is very similar to the concept of coprocessor, but at the same time have the characteristics of data access and computation.

b) zkOracle

Taking Hyper Oracle as an example, what is the relationship between a zkOracle and a zkCoprocessor?

The zkOracle discussed in Defining zkOracle for Ethereum actually has the capabilities of both zkCoprocessors.

For example, a zkOracle such as Hyper Oracle:

• Access to historical data in trustless way.

• Can perform programmable computations.

3. zkCoprocessor and zkOracle

When we directly compare the two types of zkCoprocessor with zkOracle, we can see that zkOracle has all the features of zkCoprocessor at the same time:

• asynchronous processing for smart contract

• interoperating with and enhancing the capabilities of existing applications on L1 or L2

• off-loading computation in a resource-constrained system

By direct comparison, zkOracle is a more end-to-end solution can provide developers with a more complete technology stack.

The two zkCoprocessors expand on their respective verticals, e.g., the Data Access zkCoprocessor unlocks cross-chain scenarios, and the zkVM Compute zkCoprocessor represents a zkVM-based zk rollup.

4. Build with zkCoprocessor or zkOracle

Which one to choose when building?

In a step-by-step order, we can make some decisions about building an application.

a) DApp with Pure Smart Contract

First, a pure Solidity implementation of smart contracts is still a very good choice. While pure smart contracts do not provide some of the best novel features, they are still sufficient in certain scenarios. Also the current availability of Arbitrum Stylus has unlocked a lot of new applications with pure smart contract.

b) DApp that Needs Richer Data Access

In many cases, developers may want to use Data Access zkCoprocessor or zkOracle for smart contracts to access richer data sources.

In this scenario, if Data Access zkCoprocessor is used alone, the computation is still handled in the smart contract. The role of the zkCoprocessor is to reduce the complexity of obtaining data in the traditional way, but not to make the smart contract more computationally powerful.

In this scenario, we see a lot of small data-related projects, rather than full-fledged DApps in the traditional sense:

• Using zkCoprocessor Axiom: Better LP Management, MEVictim Rebate

• Using zkOracle Hyper Oracle: Uniswap v2 Price, OpenSea Trade Activity, USDT Volume Monitor

c) DApp that Needs Heavy Compute

Often, some complex algorithms cannot be computed directly on the chain, for games, the computational logic is very complex, such as etherquake and GameOfLife that costs $2k to run one step. Or ML-related complex algorithms. Or ML-related complex algorithms that are impossible to run on chain. Therefore, we need zkVM zkCoprocessor or zkOracle to run the computation on the offchain, and then submit it to the chain as ZKP.

In this example, we can see some of their unlimited computational potential:

• Using zkCoprocessor Bonsai: Zeth (Prove Reth in zkVM), zk-sentiment

• Using zkOracle Hyper Oracle or zkWASM: zkGo (Prove L2-Geth in zkVM, Type-0 zkEVM), zkAMM

d) Next-Gen DApp with Full Features (DeFi 3.0)

Finally, we talked about applications that can only be built with zkOracle. Taking the DeFi application as an example, a complete DeFi is very complex. The next generation of DeFi applications, or DeFi 3.0 DApps, will require:

• Richer Data Access

• Unlimited Compute

• Autonomous (for liquidation, parameter update, protocol governance)

We have already discussed how zkOracle shares the capabilities of both zkCoprocessors, while fulfilling the first two functional requirements. How does zkOracle fulfill the autonomous feature and how does zkCoprocessor not?

• zkOracle: No human intervention is required. Data access and compute tasks can be fully defined and executed in programmable and automated way. All onchain calls are always trustless and valid ones.

• zkCoprocessor: Requires onchain monitoring and filtering of computation response by relayer, or the addition of extra complex mechanisms.

So what does the absence of autonomous in the zkCoprocessor entail: the

• Inadequate decentralization

• Risk of potentially complex mechanisms

• Developer mental burden

Therefore, zkOracle is a perfect and sufficient choice for a complete application like DeFi.

It is worth noting that Hooks can also handle some of zkCoprocessor's missing functionality, but ONLY in scenarios like DeFi, and not universally.

Special thanks to Alex from Maru Network, Brad from Uniswap Labs, Dong Mo from CelerNetwork, Shumo from Manta Network, and Suning from Hyperoracle for their invaluable discussions, insights, and feedback on this article.

Introduction

Undoubtedly, Uniswap stands as one of the most widely used decentralized applications. It relentlessly pioneers innovative solutions that redefine the game. In this article, I will delve into Uniswap's awe-inspiring journey from the past to a limitless future. By examining the features of each version of Uniswap, I uncover how it effectively tackles new challenges and adapts to ever-changing demands. Additionally, I explore how the latest advancements in crypto are shaping the future of decentralized exchanges (DEX). Prepare for a journey from zero to infinity.

v0: Proof of Concept

Prior to Uniswap, EtherDelta was the only decentralized exchange (DEX) that gained traction. However, the user experience was poor.

The concept of an Automated Market Maker (AMM), proposed by many industry leaders (Alan Lu of Gnosis and Vitalik), offers an alternative approach to on-chain trading compared to traditional order-book models.

Features

Constant Product AMM

Uniswap utilizes a constant product formula (x * y = k) to calculate the price of an asset. In this formula, x represents the reserve of the base asset, while y represents the reserve of the quote asset. When a token is withdrawn (bought) from a pool, a proportional amount must be deposited (sold) to ensure that the constant k is maintained. The ratio of tokens in the pool determines the price of a token.

It's worth noting that other AMMs utilize different mathematical formulas to represent liquidity curves. Some examples include Curve's Stableswap and Balancer's weighted pools.

Problems

Uniswap v0 is essentially a Proof-of-Concept, meaning there are still many unresolved issues. The two major problems are as follows:

• It only worked for a single ETH/ERC20 pair.

• It only worked for a single liquidity provider (LP).

v1: Functional DEX

Features

Uniswap v1 launched on Ethereum mainnet on November 2, 2018. It supports multiple liquidity providers using an internal token to track fees and collateral. It utilizes a factory contract that enables anyone to add any token for trading against native ETH. This version provides a functional DEX. However, there are certain issues that need to be addressed.

Problems

Since all tokens are paired with ETH, users can easily swap any ERC20 token for another ERC20 token by using ETH as an intermediary in a single transaction. However, there is a drawback: when it comes to frequent stablecoin swaps like DAI/USDT, relying on ETH as an intermediary for every swap becomes inefficient. In this case, a direct token pair is preferred.

v2: Money Lego

In May 2020, Uniswap v2 was launched, introducing numerous upgrades to the Uniswap Protocol. These enhancements increased its flexibility and expanded the range of trades possible.

Features

ERC20/ERC20 Pairs

In Uniswap V2, there is a notable difference: instead of only pairing ERC20 tokens with ETH, now any ERC20 token can be directly pooled with any other ERC20 token. This feature offers great utility for liquidity providers as they can now maintain more diverse ERC20 token-denominated positions, including Stablecoin pairs.

Price Oracle

Uniswap v2 offers on-chain price feeds that can be utilized by numerous DeFi applications. These price feeds are resistant to manipulation, making them highly valuable. The mechanism involves adding the end-of-block price to a cumulative-price variable in the core contract, which is weighted by the duration of time that particular price existed. This variable essentially represents the sum of Uniswap prices for every second throughout the contract's entire history.

This variable can be utilized by external contracts to accurately track time-weighted average prices (TWAPs) over any given time interval. The process involves reading the cumulative price from an ERC20 token pair at the start and end of the interval. By calculating the difference in this cumulative price and dividing it by the length of the interval, a TWAP for that specific period can be generated.

Flash Swaps

Uniswap v2 also introduces Flash Swaps, which is a type of flash loan pioneered by the lending market AAVE. This feature allows any user to withdraw as much as ERC20 token in a pool with no upfront cost and do arbitrary actions, provided that by the end of the transaction execution equivalent values (plus fees) are returned.

Flash loan features get its bad reputation as it comes often in headline with various kinds of attacks on DeFi protocols. But flash loans are not the problem. The real issues are existing vulnerabilities in a protocol. The atomic nature of flash loans eliminates the need for upfront capital requirements typically associated with operations such as arbitraging between pools and obtaining margin leverage.

Problems

Although the AMM is innovative and beneficial for bootstrapping trading and liquidity in new markets, it still has some inefficiencies. For instance, when dealing with low-volatility tokens, liquidity is only necessary within a narrow price range. However, the current design distributes liquidity uniformly across all price ranges.

v3: Capital Efficiency

Uniswap v3 aims to be the most flexible and efficient AMM due to its groundbreaking concentrated liquidity design.

Features

Concentrated Liquidity (CL)

In Uniswap v2, liquidity is evenly distributed across an x*y=k price curve, encompassing all prices from 0 to infinity. However, in most pools, a significant portion of this liquidity remains unused.

In Uniswap v3, liquidity providers (LPs) have the ability to concentrate their capital within specific price ranges, allowing for increased liquidity at desired prices. This customization enables LPs to construct personalized price curves that align with their preferences. These individual positions are then aggregated into a single pool, creating a unified curve against which users can trade. The outcome is mutually beneficial for both traders and liquidity providers: Traders experience reduced slippage and LPs earn greater fees, thanks to the higher concentration of liquidity within a customized range.

CL is incredibly valuable for stable pairs, such as stablecoins and liquid staking derivative tokens. These assets tend to be traded within a narrow price range. However, for more volatile token pairs, CL requires more advanced liquidity management techniques. It may be challenging for average retail liquidity providers to consistently manage their positions effectively.

Range Orders

With CL, a novel order type called range orders is introduced to complement market orders. LPs have the ability to deposit a single token within a custom price range, either above or below the current price. If the market price falls within their specified range, they can sell one asset for another along a smooth curve, all while earning swap fees.

A range order functions similarly to a 'limit order,' but with the drawback of reversibility- when the price reverses, the order is also reversed. However, fees are still earned in the process. Barry Fried (@BarryFried1) offers a detailed example of how to use range orders in this thread.

Multiple Fee Tiers

Instead of having one fee tier, Uniswap v3 introduces three separate fee tiers per pair — 0.05%, 0.30%, and 1.00%, allowing LPs to be appropriately compensated for taking on varying degrees of risk

Advanced Oracles

Uniswap v3 offers significant improvements to the price oracle. Instead of storing only one price cumulative variable, v3 stores an array of variables, making it far easier and cheaper to create more advanced oracles that include simple-moving averages (SMA), exponential moving averages (EMA), outlier filtering, and more.

Problems

Lack of Flexibility

Although concentrated liquidity and fee tiers provided more flexibility for liquidity providers and enabled the implementation of new strategies, Uniswap v3 fell short in accommodating the evolving functionalities and innovations in AMMs and markets. To incorporate additional features such as TWAP orders, limit orders, advanced oracles, and dynamic fees, it would have been necessary to re-implement the core protocol.

Certain features, such as the price oracle initially introduced in Uniswap v2, enabled integrators to utilize decentralized on-chain pricing data. However, this came at the expense of increased gas costs for swappers and lacked customization options for integrators.

Complicated Liquidity Management

As mentioned earlier, concentrated liquidity is challenging to manage for unsophisticated liquidity providers, particularly for volatile pairs. While several automatic liquidity management protocols have emerged, they mostly consist of simple rebalancing strategies designed for pegged assets. Unfortunately, these strategies are ineffective when it comes to volatile pairs because of the limitations imposed by long block times, gas costs, hedging costs, etc.

Moreover, since the concentrated liquidity position varies for each liquidity provider, the LP token is inherently non-fungible. It can only be represented by a non-fungible token (NFT), which presents a challenge for other DeFi protocols looking to integrate with it.

Numerous outstanding projects are actively addressing this issue using a range of strategies, including rebalancing, dynamic hedging with money markets, perpetuals, and options. Atis Elsts (@atiselsts_eth) has written a series of excellent articles on LP strategies that are highly recommended.

Value Leakage

The issue of value leakage is the most significant problem of all. While this is not unique to Uniswap v3, it has gained attention due to the increased volumes and wider adoption of AMMs since the launch of v3. This value leaks out of the DEX system in forms of

• Traders end up paying higher slippage than necessary due to front running and sandwich attacks.

• Liquidity providers lose value through CEX-DEX arbitrages (a.k.a. Loss-versus-Rebalancing)

To address these problems, we need more customized and sophisticated designs that go beyond what Uniswap v3 currently offers. We require a DEX platform that is more expressive and powerful.

v4: Ultimate Platform

Uniswap v4 builds on top of AMM models introduced in previous generations and through the introduction of 'hooks', it aims to be the ultimate DEX platform with high efficiency and customization.

Efficiency

Singleton

In Uniswap v3, every pool is created as an individual contract through a factory contract. On the other hand, in v4, all pools coexist within a single smart contract (known as a singleton). This singleton model significantly decreases the expenses associated with creating pools and executing multi-hop trades.

Flash Accounting

In v4, the singleton utilizes flash accounting to optimize asset transfers. Unlike in v3, where assets are moved in and out of pools after every swap, the v4 system only transfers based on net balances. This significantly reduces the cost of accounting. Transient storage (proposed in EIP-1153) makes it cheaper to set and clear a storage slot which is necessary for flash accounting to operate effectively.

Native ETH

Uniswap v4 reintroduces support for trading against native ETH, which brings several benefits. Traders can enjoy reduced gas costs for cheaper transfers and the elimination of additional wrapping costs.

Customization

Hooks

Hook contracts (or hooks), are externally deployed contracts that execute pre-defined logic at specific points during the execution of a pool. This is what makes v4 so expressive and customizable. Hooks allow for the implementation of features that were previously built into the protocol, such as oracles, as well as new features that previously required independent protocol implementations.

Uniswap v4 currently supports eight such hook callbacks:

• beforeInitialize/afterInitialize

• beforeModifyPosition/afterModifyPosition

• beforeSwap/afterSwap

• beforeDonate/afterDonate

The figure below illustrates the logic flow for a Swap Hook. Before and after executing a swap, there is a dedicated boolean flag that enables the execution of custom code at these specific points. This serves as the basis for developing advanced features like oracles, dynamic fee structures, auctions, and advanced order types, etc. We will delve deeper into these concepts in the following sections.

Oracles

In previous versions, oracles were integrated into the core Uniswap protocol. While this made integration easier and cheaper for other protocols, it came at the expense of customization options and higher costs for swappers. However, with the introduction of hooks, the design possibilities for oracles have significantly expanded. This opens up the opportunity to create manipulation-resistant oracles such as winsorized price oracles as well as volatility oracles. Additionally, the choice of who bears the costs of updating oracles can now be customized. For example, instead of burdening the first swapper with the cost, which may not be sustainable, a hook contract with an ETH balance could be used for gas payment. Despite these advancements, designing oracles still presents challenges. For instance, TWAP oracles are sometimes susceptible to manipulation and tend to lag behind current prices.

Uniswap Labs has introduced another intriguing price oracle known as the truncated price oracle. This oracle calculates the geometric mean price of an asset in a liquidity pool and imposes a limit on price movement within a single block. By truncating the price, this on-chain oracle mitigates the price impact of significant trades, enhancing resistance to manipulation attempts.

Dynamic Fees

While Uniswap v3 introduced additional fee tiers for LPs to select from, these fee structures remain static and do not consider current market conditions. Consequently, LPs are not adequately compensated for their services.

Alex Nezlobin (@0x94305) proposed a simple yet effective dynamic fee structure that considers the price impact of the previous block and applies different fees for buyers and sellers. The following figure illustrates an example: when the CEX price moves to p*, which is higher than the current AMM price p_AMM, the fee for sales decreases by δ, while the fee for buys increases by δ. The value of δ is proportional to the change in AMM price. The purpose of this dynamic fee structure is to differentiate between arbitrageurs and uninformed flows. Arbitrage flows are more likely to be autocorrelated with price movements.

Designing a robust dynamic fee structure presents several challenges. One challenge is how to incorporate off-chain signals, such as CEX price, liquidity depth, and volatility. Additionally, various on-chain signals, including address, size, and execution time, can be unreliable for differentiating between informed and uninformed flow, making it difficult to assess their effectiveness. Furthermore, it is important to ensure that fees do not go below zero in order to limit losses for LPs.

Auctions

Hooks can also serve as a means to implement auctions, which can aid in redistributing value back to LPs. Depending on the timing, auctions can be categorized as either ex ante or ex post auctions.

Ex ante auctions take place prior to the block. This concept was initially introduced in a research article that discussed MEV capturing AMM (McAMM). In this approach, the right to make the first swap in an AMM before a block is auctioned off, and the bid values are subsequently redistributed. However, there are challenges associated with this bidding process, as it essentially involves pricing an option, which can be highly complex. Additionally, since block proposers hold the ultimate decision-making power regarding whether to accept a block containing the trades, issues of censorship may arise. Ensuring fair and efficient distribution of bid values can also prove to be a complicated task. Moreover, there is no guarantee that the winners of the bids would exercise their right to swap first, potentially resulting in a deteriorating experience for other traders.

Ex post auctions are conducted after volatility has been realized, meaning that CEX prices have already moved but before the subsequent block has been committed. The advantage of these auctions is their increased effectiveness. However, they pose challenges as they require off-chain infrastructure that relies either on trusted parties or trustless systems. If trusted parties are used, concerns arise regarding censorship and bid privacy. On the other hand, designing trustless systems is much more complex. Ex post auctions also face the issue of proposers potentially playing games with bidders, such as excluding arbitrage trades from being included in a timely manner. Additionally, there is the significant problem of extra latency involved in bidding, running consensus on winning bids, and distributing those bids to block builders—all of which need to be completed before the subsequent block is committed. Finally, it may be difficult to ensure sufficient competitive conditions in these auctions to capture enough value. Sorella Labs (@SorellaLabs) is leading the charge in addressing these challenges by leveraging its advanced infrastructure and mechanism designs.

Diamond Hook

The Dimond Protocol was initially designed as an LVR-minimizing AMM. In Diamond, block producers conduct auctions to capture any arbitrage opportunities between the external market price of a Diamond pool and the pool's own price. The proceeds from these auctions are shared between the Diamond pool and the block producer in a manner that remains incentive compatible for the block producer.

A variation of the Dimond Protocol, as discussed in this post, involves implementing a collateral pool to maintain the end-of-block price in accordance with the committed price by block producers. Swaps will only execute if there is sufficient collateral available to restore the price to the committed value. Arrakis (@ArrakisFinance) is currently collaborating with Conor McMenamin (@ConorMcMenamin9), one of the authors of the Diamond Protocol, to develop an implementation using v4 hooks.

Advanced Order Types

Hooks could also enable more advanced order types that greatly improve trader experience. Some common order types include limit orders, stop loss, take-profit, TWAP

Limit Orders

Traders have the option to submit on-chain limit orders to the hook contract. When the price reaches the specified tick price, the order is filled. However, these on-chain limit orders have a significant disadvantage compared to traditional finance (tradfi) limit orders. This is primarily because on-chain orders cannot be cancelled without incurring gas fees. As a result, this creates a problem known as a low order-to-trade ratio.

TWAMM

One possible solution for facilitating the execution of large orders over time is to implement a time-weighted average market maker (TWAMM). This approach involves breaking down large orders into smaller blocks and ensuring they are executed as the first trades, thus preventing sandwich attacks. Additionally, reducing TWAP order fees could be considered since they typically involve uninformed flow. However, a challenge arises with the high gas costs and determining who should bear these expenses.

Other Hooks

There are several other features that could be implemented using hooks. Here are some ideas:

• A yield-bearing hook that lends out-of-range liquidity into money markets to increase capital efficiency.

• A pool with both xy=k liquidity curve and concentrated liquidity.

• A pool with withdrawal fees for LPs to reduce Just-in-time liquidity attacks.

Suning (@msfew_eth) provides an awesome curation of hook ideas and implementations on Github. Aiden (@aiden0x4) also publishes a nice open directory for hooks.

zkAMM & zkHooks

ZK Coprocessors empower smart contracts with the ability to access data insights akin to those offered by Dune Analytics, all without compromising trust due to the application of Zero-Knowledge (ZK) Proof technologies. The utilization of zk Coprocessors in AMM designs has been an emerging and actively researched area. The introduction of hooks now allows for the seamless integration of zk proofs into Uniswap v4, ushering in a new era of zero knowledge AMMs, or zkAMMs.

HyperOracle (@HyperOracle) demonstrates an implementation of zkAMM based on Uniswap v2, where the addLiquidity computation is offloaded off-chain. When a user adds liquidity, the token amount, price, and LP token shares need to be computed. In this particular implementation, HyperOracle's zkGraph captures the addLiquidity event, performs the necessary computation, generates the proof, and publishes it. This implementation of zkAMM includes an integrated automation layer that verifies the proof and mints the LP token for the user.

Diego (@0xfuturistic) introduces an implementation of zkAMM (zkUniswap) based on Uniswap v3 where a portion of the AMM swap computation is offloaded to the Risc Zero (@RiscZero) zkVM. When a user performs a swap in a pool, several parameters need to be calculated to finalize the swap. These parameters include the amount to be swapped in and out, the fee, and the price after swapping. Originally, this computation was performed within the solidity contract through EVM. However, with the new implementation, the swap inputs are picked up by the relayer, and the computation is carried out off-chain. The relayer then posts the output and the proof. The AMM verifies the proof and settles the swap. zkUniswap implements a lock auction mechanism to ensure concurrency control. While its current performance is comparable to that of the EVM, the efficiency could be significantly enhanced through parallelization for batched swaps.

Proof of volume serves as another use case for AMMs. Brevis (@brevis_zk) provides an interesting example where a fee rebate hook could be designed based on users' historical trading volume, similar to trading-volume-based fee rebates on centralized exchanges (CEXes). VIP traders now have the ability to calculate their monthly trading volume off-chain and then submit a low-cost ZK proof to the blockchain for asynchronous verification of their VIP status. Once verified, subsequent trades can utilize post-swap hooks to access a "VIP fee-tier table" populated by the ZK Coprocessors. This allows for automatic application of the appropriate fee rebates. Maru Network (@marunetwork) is currently developing trustless volume oracles as their initial use case for their zk Coprocessor network. By implementing a trustless volume oracle, DEXes would be able to distribute rewards in a fair and transparent manner. This approach allows for proportional incentivization of liquidity and user activities, fostering a positive flywheel effect. The cost of proof verification can be reduced by using zero-knowledge proof aggregation services such as NEBRA (@nebrazkp) UPA (Universal Proof Aggregation), which aggregates proofs from different circuits and different parties to a single proof to achieve reduced amortized verification cost.

In summary, the concept of zkAMMs involves utilizing zkCoprocessors or zkOracles to shift computations away from the EVM and verify the proof of computation on-chain. These computations can be significantly more intricate than those related to swaps and liquidity adjustments. For instance, they could involve calculating dynamic fees based on recent market volatility, proving historic volumes of users in a given pool or implementing rebalancing strategies using complex machine learning algorithms. The possibilities are endless, as any computation ultimately results in an O(1) verification cost, no longer constrained by the computational resources of the EVM.

v4 Challenges

Uniswap v4 introduces efficiency and customization to the AMM design space, enabling the creation of pools with diverse features and functionalities. However, this advancement comes with a foreseeable trade-off: an increase in liquidity fragmentation due to the proliferation of pools. As a result, routing becomes significantly more challenging.

UniswapX

UniswapX aims to tackle the issue of liquidity fragmentation by outsourcing the complexity of routing to a open network of third-party fillers. These fillers compete with each other to execute swaps using on-chain liquidity like AMM pools or their own private inventory. This design is intent-centric, allowing users to simply state their desired outcomes and rely on professionals to fulfill them.

These fillers are sophisticated agents equipped with advanced routing algorithms, high computational power, and substantial financial capital. They compete with each other under a predetermined auction mechanism to offer users the best possible execution. Users may also receive price improvement, ensuring that their executions are at least as good as trades directly from Uniswap AMM pools.

The architecture of the UniswapX protocol is depicted below. Swappers submit their intent orders, which are structured as Permit2 executable off-chain signatures, through the Uniswap API. Permit2 is an elegant model that ensures the safe movement of tokens held by users. Fillers devise strategies to fulfill these orders using any liquidity venue available to them, whether on-chain or off-chain. Finally, Order Reactors settle UniswapX orders. They are responsible for validating orders of a specific type, resolving them into inputs and outputs, executing them based on the filler's strategy, and verifying successful fulfillment of the order.

In the current Uniswap Labs interface implementation of UniswapX, the protocol is broken into two phases. The first is the RFQ phase, where whitelisted 'quoters' respond with quotes to the orders. The winner of the quote then has an exclusivity period to fill the order. If they don't, then it enters the second phase, the Dutch Auction phase, which is a auction that any filler can take part in. The plan is to make the quoting system fully permissionless in the near future.

The intent-centric design offers several benefits.

• Aggregating liquidity sources for better prices.

• No requirement for gas tokens, even for cross-chain swaps.

• Internalization of MEV through price improvement.

• No cost incurred for failed transactions.

Challenges

• Making the quoting system permissionless, e.g. Using an effective reputation system.

• Attracting more fillers to ensure a competitive and permissionless auction environment.

The Future: An Infinite Game

The development of DEXes doesn't stop here. In order for crypto to achieve mass adoption, there are several other issues that need to be addressed. Markus Schmitt (@_haikane_) from PropellerHeads (@PropellerSwap), in collaboration with Frontier Research (@FrontierDotTech), has written an outstanding article that delves into what constitutes a good DEX and identifies the remaining problems that need to be solved. According to his perspective, a good exchange should offer:

1. Trust: Ensuring transparency and minimizing custody risks before, during, and after transactions.

2. Best Prices: Providing confidence in obtaining the best or competitive prices consistently, eliminating the need to search elsewhere.

3. Fairness: Preventing abuse of orders and ensuring equal treatment regarding pricing and fees for all users.

4. Speed and Availability: Offering fast transaction processing and maintaining high availability to avoid delays and inconveniences.

5. Information: Assisting users in making informed decisions by providing order monitoring, settlement price estimates, and helpful suggestions for limit orders and slippage.

6. Liquidity: Demonstrating a robust liquidity pool across various asset pairs, instilling confidence in obtaining favorable prices.

If the smart contracts of DEXes are considered safe, trust is already built-in. DEXes don't hold users' assets. The information available to traders is quite extensive now. The permissionless nature of AMMs enables users to create and trade any asset pairs. However, due to blockchain characteristics, ensuring the best prices, fairness, speed, and availability is not always guaranteed. Gas fees, stale prices, and fragmented liquidity all contribute to impacting users' experience.

To address these issues, we are witnessing a proliferation of L2s and DEXes on L2s. Additionally, we are seeing more sophisticated routing, order batching, and Request-for-quotes systems. To prevent front-running and ensure fair value distribution, more MEV-aware channels are being employed. Moreover, efforts are underway to develop order flow auction markets that minimize value leakage for DEX users. The introduction of hooks and zk Coprocessors also significantly expands the design possibilities for AMMs, allowing for more complex logic and heavy computations without compromising trust.

Additionally, there is a suite of protocols that build upon AMMs, effectively stacking the “money lego”. These protocols include liquidity managers that assist unsophisticated users in automating liquidity rebalancing or leveraging farming. There are also protocols that utilize Concentrated Liquidity (CL) positions to create margin trading, perpetuals, options, and structured products.

AMMs have witnessed exponential growth thanks to their permissionless listing, passive liquidity, and ease of trading. However, this convenience also brings along several issues that we have explored so far. Uniswap is consistently pushing the boundaries to address these issues and enhance the user experience. As Dan Robinson (@danrobinson) aptly puts it in his speech at SBC23, DEX design is an infinite game. With increased adoption of DEXes in the future, new challenges and problems are likely to arise, necessitating innovative solutions.

References

On Path Independence

https://vitalik.ca/general/2017/06/22/marketmakers.html

Reddit - Dive into anything

Building a Decentralized Exchange in Ethereum

Building a Decentralized Exchange in Ethereum

Building a Decentralized Exchange in Ethereum *Please note that this article was originally published on March 6, 2017. The original post has been archived and re-published here in attempt to ...

https://medium.com

Building a Decentralized Exchange in Ethereum

Building a Decentralized Exchange in Ethereum *Please note that this article was originally published on March 6, 2017. The original post has been archived and re-published here in attempt to ...

https://medium.com

A Short History of Uniswap

A Short History of Uniswap

Uniswap changed my life and today is its birthday. To celebrate this, I want to tell its story from the beginning.

https://blog.uniswap.org

A Short History of Uniswap

Uniswap changed my life and today is its birthday. To celebrate this, I want to tell its story from the beginning.

https://blog.uniswap.org

Uniswap v2 Overview

Uniswap v2 Overview

All about Uniswap V2. Price oracles, optimistic swaps and much much more. Launching April 2020.

https://blog.uniswap.org

Uniswap v2 Overview

All about Uniswap V2. Price oracles, optimistic swaps and much much more. Launching April 2020.

https://blog.uniswap.org

Introducing Uniswap v3

Introducing Uniswap v3

All about Uniswap V3.

https://blog.uniswap.org

Introducing Uniswap v3

All about Uniswap V3.

https://blog.uniswap.org

Our Vision for Uniswap v4

Our Vision for Uniswap v4

All about our vision for Uniswap v4 and why we're building in public.

https://blog.uniswap.org

Our Vision for Uniswap v4

All about our vision for Uniswap v4 and why we're building in public.

https://blog.uniswap.org

Uniswap v4 Truncated Oracle Hook

Uniswap v4 Truncated Oracle Hook

This example Uniswap v4 hook implements a Truncated Geometric Mean oracle. This oracle is more resistant to manipulation techniques by blunting the price impact of large trades.

https://blog.uniswap.org

Uniswap v4 Truncated Oracle Hook

This example Uniswap v4 hook implements a Truncated Geometric Mean oracle. This oracle is more resistant to manipulation techniques by blunting the price impact of large trades.

https://blog.uniswap.org

Introducing the UniswapX Protocol

Introducing the UniswapX Protocol

UniswapX is a permissionless protocol for aggregating all sources of liquidity for onchain trading with gas free swaps, MEV protection, more liquidity, and better prices.

https://blog.uniswap.org

Introducing the UniswapX Protocol

UniswapX is a permissionless protocol for aggregating all sources of liquidity for onchain trading with gas free swaps, MEV protection, more liquidity, and better prices.

https://blog.uniswap.org

Uniswap 101: What is Uniswap? | Uniswap Labs

Uniswap 101: What is Uniswap? | Uniswap Labs

Learn more about the Uniswap Protocol, an Ethereum based exchange, with our complete guide. Explore more ways to trade crypto on our decentralized protocol.

https://blog.uniswap.org

Uniswap 101: What is Uniswap? | Uniswap Labs

Learn more about the Uniswap Protocol, an Ethereum based exchange, with our complete guide. Explore more ways to trade crypto on our decentralized protocol.

https://blog.uniswap.org

Overview | Uniswap

Overview | Uniswap

UniswapX is a permissionless, open source, auction-based swapping protocol for trading across AMMs and other liquidity sources. It improves swapping in several ways:

https://docs.uniswap.org

Overview | Uniswap

UniswapX is a permissionless, open source, auction-based swapping protocol for trading across AMMs and other liquidity sources. It improves swapping in several ways:

https://docs.uniswap.org

DeFi Flashloans Explained: How to Borrow Millions in uncollateralized loans (UniswapV2 + Foundry Examples - SmartBuilds.io

DeFi Flashloans Explained: How to Borrow Millions in uncollateralized loans (UniswapV2 + Foundry Examples - SmartBuilds.io

Ah, Flashloans. With great power comes great responsibility. Imagine being able to borrow millions of dollars in tokens instantly, without ... Read more

https://smartbuilds.io

DeFi Flashloans Explained: How to Borrow Millions in uncollateralized loans (UniswapV2 + Foundry Examples - SmartBuilds.io

Ah, Flashloans. With great power comes great responsibility. Imagine being able to borrow millions of dollars in tokens instantly, without ... Read more

https://smartbuilds.io

What Are Flash Loans? » Meaning and Definition | Chainlink

What Are Flash Loans? " Meaning and Definition | Chainlink

A flash loan is a type of loan where a user borrows assets with no upfront collateral and returns the borrowed assets within the same blockchain transaction.

https://chain.link

What Are Flash Loans? " Meaning and Definition | Chainlink

A flash loan is a type of loan where a user borrows assets with no upfront collateral and returns the borrowed assets within the same blockchain transaction.

https://chain.link

The next steps in DEX design

The next steps in DEX design

DEXs are still behind CEXs volume - but in the next iteration that might be the past.

https://www.propellerheads.xyz

The next steps in DEX design

DEXs are still behind CEXs volume - but in the next iteration that might be the past.

https://www.propellerheads.xyz

Liquidity Provider Strategies for Uniswap v3: Table of Contents

Liquidity Provider Strategies for Uniswap v3: Table of Contents

Liquidity Provider Strategies for Uniswap v3: Table of Contents This is a series of articles written in 2023 with the support of the Uniswap Foundation. An Introduction to Uniswap LP Strategies and ...

https://atise.medium.com

Liquidity Provider Strategies for Uniswap v3: Table of Contents

Liquidity Provider Strategies for Uniswap v3: Table of Contents This is a series of articles written in 2023 with the support of the Uniswap Foundation. An Introduction to Uniswap LP Strategies and ...

https://atise.medium.com

GitHub - fewwwww/awesome-uniswap-hooks: A curated list of awesome Uniswap v4 hooks resources.

GitHub - fewwwww/awesome-uniswap-hooks: A curated list of awesome Uniswap v4 hooks resources.

A curated list of awesome Uniswap v4 hooks resources. - fewwwww/awesome-uniswap-hooks

https://github.com

GitHub - fewwwww/awesome-uniswap-hooks: A curated list of awesome Uniswap v4 hooks resources.

A curated list of awesome Uniswap v4 hooks resources. - fewwwww/awesome-uniswap-hooks

https://github.com

Uniswap v4 Hooks Directory | Curated Open Source Hooks

Играть в официальном онлайн казино Вавада

Найти новое зеркало, чтобы совершить вход и зарегистрироваться в Vavada casino. Играть в лучшие онлайн слоты без рисков - в демо. Совершить депозит и забрать бонус - до 1000$. Играть в Вавада через телефон, используя мобильную версию сайта.

https://xobarpdx.com

Играть в официальном онлайн казино Вавада

Найти новое зеркало, чтобы совершить вход и зарегистрироваться в Vavada casino. Играть в лучшие онлайн слоты без рисков - в демо. Совершить депозит и забрать бонус - до 1000$. Играть в Вавада через телефон, используя мобильную версию сайта.

https://xobarpdx.com

🪢 zkUniswap: a first-of-its-kind zkAMM

🪢 zkUniswap: a first-of-its-kind zkAMM

TL;DR: We introduce zkUniswap, a first-of-its-kind zkAMM that uses a zk co-processor to offload the computation of swaps. Thanks to Cairo, William X, Trace, jseam, Barnabé Monnot, Xyn, Filip Siroky, Colin Roberts, Thogard, Alex Nezlobin and others for the generous feedback. What is zkUniswap?

https://ethresear.ch

🪢 zkUniswap: a first-of-its-kind zkAMM

TL;DR: We introduce zkUniswap, a first-of-its-kind zkAMM that uses a zk co-processor to offload the computation of swaps. Thanks to Cairo, William X, Trace, jseam, Barnabé Monnot, Xyn, Filip Siroky, Colin Roberts, Thogard, Alex Nezlobin and others for the generous feedback. What is zkUniswap?

https://ethresear.ch

Latest Applications Building on Hyper Oracle (Bonus: Things You …

https://mirror.xyz/hyperoracleblog.eth/Tik3nBI9mw05Ql_aHKZqm4hNxfxaEQdDAKn7JKcx0xQ

https://mirror.xyz/hyperoracleblog.eth/Tik3nBI9mw05Ql_aHKZqm4hNxfxaEQdDAKn7JKcx0xQ

The inevitable adoption of coprocessors, driven by Uniswap v4 - HackMD

The inevitable adoption of coprocessors, driven by Uniswap v4 - HackMD

Uniswap v4 is just announced. Although it's not fully built out yet, we expect the community to widely explore the boundary of newly enabled possibilities. Since there might be a ton of articles explaining how powerful v4 is on the defi side of things, I would like to address how Uniswap v4 can inspire a new category of blockchain infrastructure: coprocessors.

https://hackmd.io

The inevitable adoption of coprocessors, driven by Uniswap v4 - HackMD

Uniswap v4 is just announced. Although it's not fully built out yet, we expect the community to widely explore the boundary of newly enabled possibilities. Since there might be a ton of articles explaining how powerful v4 is on the defi side of things, I would like to address how Uniswap v4 can inspire a new category of blockchain infrastructure: coprocessors.

https://hackmd.io

Diamond: A Uniswap V4 Hook minimizing LVR

https://mirror.xyz/0x916563f8476b988855af0b8b8A3D56072E1917FA/ZhTQJ6qiurBTHBNuF08GudWBQ8x9p3P-12vo-C9czKE

https://mirror.xyz/0x916563f8476b988855af0b8b8A3D56072E1917FA/ZhTQJ6qiurBTHBNuF08GudWBQ8x9p3P-12vo-C9czKE

Acknowledgment to Lukas, co-founder of Safe, Noam, engineering lead of Alchemy, Kurt and Konrad, co-founders of Rhinestone, Arnav, investor of HashKey Capital.

Content:

• Introduction, a brief review of AA

• The Dilemma of SCA Adoption

• Modular SCA to Solving Engineering Difficulties

• Modular Architecture

  • Safe Smart Account Stack

  • ERC2535 Diamond Stack

  • Comparison

• Module Sequence

  • ERC6900: Validation, Execution and Hooks

• Modules Discovery and Security

  • Safe{Core} protocol

  • Rhinestone

Introduction

The shift from Externally Owned Accounts (EOA) to Smart Contract Accounts(SCA) is gaining momentum and has been embraced by many enthusiasts, including Vitalik himself. Despite the excitement, the adoption of SCA is not as widespread as EOAs. The key among them are the challenges posed by bear markets, the concern of migration, signing issues, gas overheads, and most critically, engineering difficulties.

The most significant advantage of Account Abstractions (AA) is the ability to use code to customize functionality. However, one major engineering challenge is the non-interoperability of AA functionalities, and the fragmentation hampers integration and opens the door to vendor lock-in. Additionally, ensuring security while simultaneously upgrading and composing features can be intricate.

Enter Modular Account Abstraction, as a subset of the broader AA movement, this innovative approach can separate smart accounts from their custom functions. The goal is to create a modular structure to develop secure, seamlessly integrated wallets with diverse functionalities. In the future, it can realize a free “app store” for smart contract accounts that sets wallets and dApps free from building features but focuses on user experience.

After reading this article, readers will gain insights about:

1. What is Modular Account Abstraction

2. How does the account interact with modules

3. What’s the sequence of calling modules

4. How to find and verify modules in an open way

A Brief Review of AA

The traditional EOA introduces many challenges like seed phrase, gas, cross-chain, and multiple transactions. We never intended to introduce complexity, but in fact, blockchain is not an easy game for the masses.

Account Abstraction leverages the smart contract account allowing programmable validation and execution, where the user is able to approve a series of transactions in one go, rather than sign and broadcast each, and implement many more features. It introduces benefits to user experience(eg. gas abstraction, and session keys), cost(eg. batching transaction), and security(eg. social recovery, multi-sig). Currently, there are two ways to achieve account abstraction:

• Protocol layer: Some protocols themselves provide native account abstraction support, ZKsync transactions follow the same flow whether originating from EOA or SCA which uses a single mempool and transaction flow to support AA, and Starknet has removed EOA and all accounts are SCA, and they have native smart contract wallets like Argent.

• Contract layer: For Ethereum and its equivalent L2s, ERC4337 introduces a separate entry point for transactions to support AA without changing the consensus layer. Projects like Stackup, Alchemy, Etherspot, Biconomy, Candide, and Plimico are building bundler infrastructure, and many more like Safe, Zerodev, Etherspot, and Biconomy are building stack and SDK.

👉 If you’re not familiar with AA or ERC4337, check SevenX’s previous research here.

The Dilemma of SCA Adoption

The topic of Account Abstraction (AA) has been under discussion since 2015 and it was further propelled into the limelight by ERC4337 this year. However, the number of deployed smart contract accounts still pales in comparison to EOAs.

Let’s delve into this dilemma:

1. Bear Market Impact:

   Even though AA introduces benefits like seamless login and gas abstraction, the prevailing bear market primarily consists of educated EOA users rather than new ones, thus there’s no incentive for dApps and wallets. Even saying that, leading apps are still on the way to adopting AA, like Cyberconnect drove around 360,000 UserOps(transactions of AA) a month alone by introducing their AA system and gasless solution.

2. Migration Obstacles:

   For wallets and applications that have accumulated users and stored assets in EOAs, migrating assets securely and conveniently remains a challenge. Nevertheless, initiatives like EIP-7377 permit EOAs to initiate a one-time migration transaction.

3. Signing Issue:

   The smart contract itself naturally can’t sign messages, since it does not possess a private key like EOAs. Efforts like ERC1271 make it possible but the message signing won’t work until the first transaction, which proposes a challenge for wallets using counterfactual deployment. And ERC-6492 proposed by Ambire is a backwards-compatible successor to ERC-1271 that potentially solves the previous issue.

4. Gas Overheads:

   Deploying, simulating, and executing SCAs incur higher costs compared to standard EOAs. This becomes a deterrent for adoption. However, several tests have been conducted, such as decoupling account creation from user operations, and eliminating account salt and existence checks are being considered to reduce these costs.

5. Engineering Difficulties:

   The ERC-4337 team has set up the eth-infinitism repo to provide developers with a foundational implementation. Yet, as we branch out to more nuanced or specific functions for different use cases, integration and decoding become challenging.

In this article, we’ll dive into the #5 problem: the engineering difficulties.

Modular Smart Contract Account to Address Engineering Difficulties

To further elaborate on the engineering difficulties:

• Fragmentation:

  Features are now enabled in various ways, whether natively by specific SCAs or through independent plugin systems. Each follows its standard, forcing developers to decide which platforms to endorse. This leads to potential platform lock-ins or redundant efforts.

• Security:

  While the flexibility between accounts and features offers advantages, it may amplify security concerns. Features might be audited collectively, but the absence of independent assessments could heighten the risk of account vulnerabilities.

• Upgradability:

  As features evolve, it’s important to retain the capacity to add, replace, or remove functionalities. Redeploying existing features with each update introduces complexities.

To navigate these waters, we need upgradable contracts that ensure secure and efficient upgrades, reusable cores to enhance overall development efficiency, and standardized interfaces to ensure contract accounts can transition smoothly between different frontends.

These terms converge on a singular concept: Building a Modular Account Abstraction Architecture (Modular AA).

Modular AA is a niche within the broader AA movement which envisions modularizing smart accounts to customize them for users and empower developers to seamlessly enhance features with minimal restrictions.

Yet, in any industry, setting up and promoting a new standard is a big challenge. Initial phases may witness many different solutions before everyone settles on the main one. However, it’s encouraging to see those working on account abstraction, whether it’s the 4337 SDK, wallet developers, infrastructure teams, or protocol designers, all coming together to speed up the process.

Modular Structure: Main Account and Modules

How does the account call modules to realize functions

Delegate call and proxy contract

External call and delegate call:

While delegatecall is similar to call, but instead of executing the target contract in its own context, it executes it in the context of the calling contract's current state. This means that any state changes made by the target contract are made to the calling contract's storage.

To realize the composable and upgradable structure, there’s a fundamental knowledge needed called “Proxy contract”.

1. Proxy contract: A normal contract stores its logic and states, which is unable to update after deployment. A proxy contract using delegatecall to another contract. By referring to the implementation function, it executes it in the proxy contract's current state.

2. Use cases: While the proxy contract stays immutable, you can deploy a new implementation behind the proxy. Proxy contracts are used for upgradability and are cheaper to deploy and maintain on public blockchains.

Safe Architecture

What is Safe:

Safe is a leading Modular Smart Account Infrastructure designed to provide battle-tested security and flexibility, it empowers developers to create diverse applications and wallets. Notably, many teams are building on top of Safe or inspired by it. Biconomy launched its account by expanding Safe with native 4337 and 1/1 multi-signatures. Witnessing the deployment of over 164,000 contracts and locking in excess of 30.7 billion in value, Safe is no doubt the premier in space.

What’s Safe’s structure

1. Safe account contract: Main proxy contract (Stateful)

   Safe account is a proxy contract because it delegatecalls a singleton contract. Safe Account contains owners, threshold, and implementation addresses are all set as the variables of the proxy, hence defining its state.

2. Singleton contract: Integration Hub (Stateless)

   Singleton serves the Safe account and integrates and defines different integrations including Plugin, Hook, Function Handler, and Signature Validator.

3. Module contract: Custom logic and features:

   Modules are powerful. Plugins as a modular type can define different features like payment streamings, recovery mechanisms, and session keys, and can serve as a bridge between web2 and web3 by fetching off-chain data. Other modules like Hook as a safety guard, and Function Handler respond to any instructions.

What happens when we adopt Safe:

1. Upgradable Contracts:

   Deploying a new singleton is needed whenever new plugins are introduced. Users retain the autonomy to upgrade their Safe to the desired singleton version, aligning with their preferences and requirements.

2. Composable and Reusable Modules:

   The modular nature of plugins empowers developers to craft functionalities independently. They can then freely select and combine these plugins based on their use cases, fostering a highly customizable approach.

ERC-2535 Diamond Proxies

About ERC2535, Diamond Proxies:

The ERC2535 standardizes diamonds, a modular smart contract system that can be upgraded/extended after deployment and has virtually no size limit. Up till now, lots of teams have been inspired by it, like Zerodev’s Kernel, and Soul Wallet’s experiment.

What’s diamond structure:

1. Diamond contract: Main proxy contract (Stateful) A diamond is a proxy contract that calls functions from its implementations using delegatecall.

2. Module/ Plugin/ Facet contract: Custom logic and features (Stateless) A Module or so-called Facet is a stateless contract that could deploy its features to one or more diamonds. They are separate, independent contracts that can share internal functions, libraries, and state variables.

What happens when we adopt Diamond:

1. Upgradable Contracts: Diamond provides a systematic way to isolate different plugins and connect them together and share data between them, also directly adds/replaces/removes any plugin using the diamondCut function. There is no limit to the amount of plugins that can be added to diamonds over time.

2. Modular and reusable plugin: A deployed plugin can be used by any number of diamonds to tremendously reduce deployment costs.

Difference between Safe Smart Account and Diamond Approach:

Similarities abound between the Safe and Diamond architectures, both relying on proxy contracts at their core and referencing logic contracts to achieve upgradability and modularity.

Nonetheless, the primary distinction lies in the handling of logic contracts. Here’s a closer look:

1. Flexibility:

   In the context of enabling new plugins, Safe necessitates the redeployment of its Singleton contract to implement the change in its Proxy. In contrast, Diamond achieves this directly through the diamondCut function in its Proxy. This variance in approach translates to Safe retaining a higher degree of control, while Diamond introduces enhanced flexibility and modularity.

2. Security:

   delegatecall, utilized in both structures for now, can permit external code to manipulate the storage of the main contract. In the Safe architecture, delegatecall points to a single logic contract , whereas Diamond employs delegatecall across multiple module contracts- plugins. Consequently, a malicious plugin holds the potential to overwrite another plugin, thereby introducing the risk of storage collisions that could compromise the integrity of the Proxy.

   • Cost:

     The flexibility inherent in the Diamond approach comes hand in hand with amplified security concerns. This increases the cost factor, necessitating comprehensive audits with every addition of a new plugin. The key is to make sure that these plugins don’t interfere with each other’s functions, presenting a challenge, particularly for small or medium-sized enterprises striving to maintain robust security standards.

3. The “Safe Smart Account approach” and the “Diamond Approach” serve as examples of distinct structures involving proxies and modules. How to balance flexibility and security is crucial, and these two methods could potentially complement each other in the future.

   ---

   Modules Sequence: Validator, Executor, and Hook

   What’s the sequence of calling modules

   Let’s expand our discussion by introducing ERC6900, a standard proposed by the Alchemy team, inspired by Diamond, and tailored specifically for ERC-4337. It addresses the challenge of modularity in smart accounts by providing common interfaces and coordinates the efforts between plugin and wallet developers.

   When it comes to the transaction process of AA, there are three main processes: validation, execution, and hook. These steps can all be managed by using the proxy account to call modules, as we discussed earlier. While different projects may use different names, it’s important to grasp the similar underlying logic.

   

   • Validation: Ensures the authenticity and authority of the caller to the account.

   • Execution: Performs any custom logic that the account allows.

   • Hook: Acts as a module that runs before or after another function. It can modify the state or cause the entire call to undo.

   

   It’s crucial to separate modules based on a different logic. A standardized approach should dictate how validation, execution, and hook functions for smart contract accounts should be written. Whether it is Safe or ERC6900, the standardization helps reduce the need for unique development efforts specific to certain implementations or ecosystems and prevents vendor lock-in.

   ---

   Modules Discovery and Security

   How to find and verify modules in an open way

   A solution that’s gaining momentum involves the creation of a place that allows users to discover verifiable modules, which we can call “registry.” This registry functions akin to an “App Store” and aims to foster a simplified yet thriving modular marketplace.

   Safe{Core} Protocol

   

   Safe{Core} Protocol is an open-source, interoperable protocol for smart contract accounts, designed to enhance accessibility for various vendors and developers while maintaining robust security through well-defined standards and rules.

   • Account: In Safe{Core} Protocol, the concept of an account is flexible and not bound to a specific implementation. This enables diverse account service providers to participate.

   • Manager: The Manager serves as a coordinator among Accounts, Registries, and Modules. It’s also responsible for security as the permissioning layer.

   • Registries: Registries define security attributes and enforce standards like ERC6900 for Modules, aiming to create an open “app store” environment for both discoverability and safety.

   • Modules: Modules handle functionalities and come in various initial types, including Plugins, Hooks, Signature Validators, and Function Handlers. These are open for developers to contribute to, provided they meet the established standards.

   Rhinestone Design

   

   The process unfolds as follows:

   1. Creating a Schema Definition: A schema serves as a predefined data structure required for attestation. It can be customized by businesses to align with their specific use cases.

   2. Creating Modules Based on A Schema: Smart contracts are registered as modules, obtaining bytecode and choosing a schema ID. This data is then stored within the registry.

   3. Attaining Attestation for a Module: Attestors/ auditors can provide attestations for modules on a schema basis. These attestations can include a unique identifier (UID), and references to other attestations for chaining. They can propagate across chains and verify if specific thresholds are met on destination chains.

   4. Implementing Complex Logic with Resolvers: Resolvers, optionally set in the schema, come into play. They could be invoked during module creation, attestation establishment, and revocation. These resolvers allow developers to incorporate intricate and diverse logic while maintaining attestation structures.

   5. User-Friendly Query Access: Queries offer a means for users to access security information from the front-end. Find this EIP here.

   While this schema is in its early stages, it holds the potential to establish a standard in a decentralized and collaborative way. Their registry enables developers to register their modules, auditors to verify their security, and wallets to integrate and enables users to effortlessly locate modules and verify their attestation information. Several future uses might be:

   • Attestors: Trustworthy entities, like Safe, could collaborate with Rhinestone as attestors for in-house modules. Simultaneously, independent attestors could join in.

   • Module Developers: As an open market takes shape, module developers could potentially monetize their work through the registry.

   • Users: Engaging via user-friendly interfaces, such as wallets or dApps, users can examine module information and delegate trust to diverse attestors.

   The concept of “Module Registry” opens avenues for monetization for plugin and module developers. It could further pave the way for a “Module Marketplace.” Some aspects might be supervised by Safe’s team, while others could manifest as decentralized marketplaces, inviting contributions and transparent audit records for all. By incorporating this, we can avoid vendor lock-in and support the expansion of the EVM by adding an enhanced user experience that attracts a wider audience.

   While these approaches guarantee the safety of a single module, the broader security of smart contract accounts isn’t foolproof. Combining legitimate modules and proof they don’t have storage collisions can be a challenge, underscoring the importance of wallet or AA infrastructure in addressing such concerns.

   ---

   Closing thoughts

   By utilizing a modular smart contract account stack, wallet providers and dApps can be liberated from the complexities of tech maintenance. Meanwhile, external module developers have the opportunity to offer specialized services tailored to individual needs. However, challenges to address include striking a balance between flexibility and security, pushing modular standards forward, and implementing standardized interfaces that empower users to easily upgrade and modify their smart accounts.

   Yet, modular Smart Contract Accounts (SCA) represent just one piece of the adoption puzzle. To fully realize the potential of SCA, additional protocol-layer support is needed from Layer 2 solutions, so as robust bundler infrastructure and peer-to-peer mempool, more cost-effective and feasible SCA signing mechanism, cross-chain SCA synchronization and management, and develop user-friendly interfaces.

   Looking ahead, we envision a future where participation is widespread, sparking intriguing questions: Once SCA order flow becomes sufficiently profitable, how will traditional Miner Extractable Value (MEV) mechanisms enter the field to build bundlers and capture value? When the infrastructure matures, how can Account Abstractions (AA) serve as the foundational layer for “intent-based” transactions? Stay tuned; the landscape is evolving by the minute.

   ---

   Important pieces

   • Safe whitepaper: https://github.com/safe-global/safe-core-protocol-specs/blob/main/whitepaper.pdf

   • Rhinestone doc: https://docs.rhinestone.wtf/

   • Biconomy doc: https://www.biconomy.io/post/making-biconomy-smart-accounts-modular

   ---

   Reference

   • Part 1: Modular account abstraction for everyone else @Kurt Larsen

   • WTF is Modular Account Abstraction

   • You Could Have Invented Account Abstraction: Part 1 — @Alchemy

   • rhinestone architecture

   • Ivo from Ambire wallet twitter thread

   • Awesome Account Abstraction — 4337 Mafia

   • ERC6900

As many have said, Account Abstraction technology, especially ERC-4337, promises to revolutionize the user experience of self-custodial wallets and enable their scaling for mass adoption. However, as we approach May 2023, it is crucial to recognize that the standard is in its early stage, and opportunities and risks coexist.

*Please note that this content may soon be outdated as the upgrade is rapidly evolving, and the content presented is based on my personal opinion.

TL;DR

On ERC4337:

The AA standard is still in its early stages, but many innovative builders are working to further develop it. With the support of the ecosystem and adoption by large products such as MetaMask, we can expect the process of pushing AA to accelerate and yield exciting developments.

On L2s:

Adoption of Account Abstraction (AA) varies among L2 solutions. Larger L2s such as Optimism and Arbitrum do not natively support AA, while ZKSync and Starknet do.

On Bundler service:

• If we are bullish on Account Abstration(AA), and all the EVM equivalent L2s in line with Ethereum — does not support native AA, then bundler service is a must need to support AA in the network.

• Open source nature make Bundler service non-exclusive, making monetization difficult. Different bundler services will be adopted for network security and stability.

• Private bundler could be possible a way for monetization through customizing the privacy, secure and other features for specific demand.

On Paymaster service:

• Paymaster service is relatively centralized (compared to bundler service) with open-sourced contracts but closed backend.

• Paymaster service has profit-making model, and can enhance payment scenarios by combining with fiat deposit, swaps, bridges, auto-payments, sessions, sponsored fees, and other features to improve dApp usability.

About AA wallet and SDKs:

• AA wallets can be evaluated from a product perspective, including key management systems, social recovery, gas sponsorship, multi-chain account sync, and supported blockchains, etc.

• The advantage of AA is not only smooth login (as Web3 Auth can achieve using custodial way), but in complex and customized on-chain interactions where AA can provide more benefits to dApps.

• BD is the key in this fierce competition. Most of wallets are targeting Defi, GameFi and working on getting ecosystem supports, persuading big dApps, finding breaking points.

• Monetization models need to be further explored. A To Business(ToB) models may not make much profit or accumulate their own users, while a To Customer(ToC) models need to find high-value scenarios to profit based on volume. Integrating swap and bridging functions can be profitable, but finding a sustainable model is crucial.

👀 Understand crypto wallet

👋 Proposal of ERC4337: Problem of CA, 4337 Mechanism, Pros n Cons

🔗 L2s: Adoption, Insights

⚙️ AA Infrastructure: Bundler and paymaster service, Evaluation

🛠️ AA Wallet& SDKs: Modular services, Evaluation

👀 Understanding crypto wallet

Classification

There’re two types accounts existing in the Ethereum network: EOA- Externally Owned Account wallets like MetaMask, and CA -Contract Account like Safe.

The main difference between EOA wallets and contract wallets is the way in which they are controlled. EOA wallets are controlled by individual users through the private key, while contract wallets are controlled by smart contracts. While EOA wallets are simpler and used for managing personal cryptocurrency holdings, contract wallets can have more complex rules and are used for specific purposes.

Painpoints

Using an Externally Owned Account (EOA) wallet puts the burden of protecting private keys on users. Any mistake or error can lead to the loss of funds, making it a costly and high-risk option. Even experienced cryptocurrency users have lost control of their accounts due to a single wrong or careless move. The complexity of operations, the inability to skip or outsource gas payments, and the limited functionality of wallets are all sources of frustration for users.

Smart contract wallets offer a solution to some of these issues, but Ethereum currently requires all operations to be packaged in a transaction from an ECDSA-secured EOA. This adds an extra transaction fee and 21000 gas overhead, as well as the potential for centralization and complexity: Users need to either manage two accounts and have ETH in a separate EOA to pay for gas, or rely on centralized relay systems.

These pain points motivate the creation of ERC-4337, the new standard of Account Abstraction.

👋 Proposal of ERC4337:

Problem of CA

It is possible to do all of these things with contract wallets today, but the Ethereum itself requires everything to be packaged in a transaction originating from an ECDSA-secured externally-owned account (EOA), that will lead:

• Extra transaction fee: Every user operation needs to be wrapped in a transaction from an EOA, adding 21000 gas overhead.

• Complexity and centralization: The user needs to either have ETH in a separate EOA to pay for gas, and manage balances in two accounts, or rely on a relay system, which are typically centralized.

Over the years, there have been various attempts to implement account abstraction on Ethereum-based blockchains, such as EIP-86 and EIP-2938. However, these approaches were not feasible because they needed modifications to the consensus layer, which was difficult to achieve.

4337 Mechanism

ERC-4337 achieves account abstraction by introducing a higher-layer pseudo-transaction object called a UserOperation, which is similar to rollups in terms of bundling concepts. Fortunately, this standard allows us to build account abstraction without consensus layer changes.

The Modularization design of EIP 4337 divides the account abstraction function of the smart contract wallet into multiple ports:

Bundler :

• Bundler is an EOA, since all transactions must be triggered by the EOA, Bundlers enable users to trigger smart contract wallet transactions without creating and memorizing the private key of the EOA.

• Bundler’s duty: Verify the UserOperation, Package up a set of UserOperation objects into a single “bundle transaction”. Broadcasting the verified UserOperation to public or private mempool.

• Bundlers can also gain economic benefits by: Pocket the difference between the maximum priority fee and the actual gas cost after performing the UserOperation . Similar to the relayer of ordinary transactions, Bundler can obtain MEV by sorting the UserOperation in the bundle.

Entry Point :

• Entry point is a global contract that all bundlers need to call to perform a UserOperation. It acts as an intermediary between bundler and smart contract wallet.

• Use handleOp to verify and execute: handleOp function accepts UserOperation as an input parameter, first verifies the UserOperation on-chain, checking that is signed by specified smart contract wallet address and wallet has enough gas to compensate the bundler, if successful, executes according to the function signature and input parameters.

• Needs deposit token of a smart contract wallet to pay gas fees to bundlers: When Bundler triggers the handleOp function using EOA, a gas fee is incurred. The smart contract wallet can pay gas with its own balance or request a Pymaster to pay. Possible fail: Without enough gas fails the verification step; Even with enough gas, the UserOperation may fail in execution steps, such as a runtime error. Regardless of whether the execution is successful or not, the entry point contract will pay the gas fee to the Bundler to trigger the handleOp function. The entry point contract provides the smart contract wallet with the function to add or withdraw tokens as collateral.

Smart wallet itself:

• Smart contract wallet master contract separates the verification and execution steps of UserOperation. By separating it, a bundler can verify the UserOperation off-chain, so malicious transactions can be filtered out without paying gas.

• Verification steps are defined in the validateOp function: 1st time to call validateOp is bundler to simulate the verification off-chain, verify the signature in the UserOperation , and ensure that the smart contract wallet has sufficient gas balance; 2nd time to call validateOp is entry point contract to perform on-chain verification before executing the UserOperation

Paymaster :

• Paymaster defines the logic of the gas abstraction for smart contract wallets, including paying for Ethereum gas with ERC20 homogenized tokens and transactions without gas.

• Paymaster is a smart contract deployed by a dApp that can trigger Paymaster’s validatePaymasterOp .

Wallet Factory :

• Wallet Factory is a public contract that creates a smart contract wallet. When the initCode is populated with the wallet factory address and the parameters of the new smart contract wallet, Bundler will trigger the corresponding Wallet Factory to create a smart contract with the specified parameters. Popular Wallet Factory undergoes a comprehensive code audit, safer for user to create wallet.

• Wallet Factory need to stake ETH on Entry Point and consistently serve UserOperations well in order to get more traffic from Bundler.

• Users can request Bundler to create a CA wallet by submitting a UserOperation populated with initCode.

• Users can customize their CA wallet by selecting Wallet Factory with specific customization parameters.

Signature Aggregator :

• Signature Aggregator is used to aggregate the signatures of multiple transactions into bytes for faster verification and execution of transactions. Different smart contract wallets use different signature algorithms, it is necessary to aggregate the UserOperations using the same signature algorithm first.

• Save gas: since on-chain cryptographic calculations consume a lot of gas, aggregated signature schemes (such as BLS) can save gas during on-chain verification.

• Bundler uses multiple signature aggregator contracts to generate multiple aggregate signatures instead of validating one UserOperation at a time.

• Bundler passes the UserOperation array, aggregate signature and aggregator address to Entry Point, and each UserOperation group meeting calls the validateSignature function of its corresponding signature aggregator.

• After the verification is passed, Bundler will execute this set of UserOperations on the smart contract wallet.

• Aggregators also need to stake Ethereum on entry-point contracts and maintain a track record of serving UserOperations well.

Pros of AA

1. Gas abstraction:

   Gas abstraction contains gas-free transactions and pays for gas with any ERC20 token. The logic could be implemented in a Paymaster contract or through a Relayer. In terms of AA, Many smart contract wallets implement EIP 4337-compatible Paymaster contracts themselves and pledge tokens on entry point contracts to help users pay for gas.

2. Social Recovery:

   In the event a private key is lost or compromised, users can authorize a new key as a legitimate wallet owner. The logic of social login and social recovery is generally defined in the main contract of the wallet. Multiple apporoad could be used like email, muti-signature, MPC or SWIE(Sign in with Ethereum) and so on.

3. Batching transaction:

   Transaction batching is a feature unique to smart contract wallets that allows wallet users to execute multiple transactions in a single on-chain transaction.

4. Ramps and bridges integration:

   Many existing wallets, already integrate fiat deposit and withdrawal channels and cross-chain bridges into wallets by working with third-party providers. These deposit and withdrawal channels and cross-chain bridges can be further integrated with the payment contract (Paymaster) in gas abstraction.

5. Modular design:

   One of the biggest advantage of AA could be the modular service that each part of bundler, paymaster and many could be flexibly combine.

Cons of AA

1. Fees can be relative higher(maybe):

   Simple transfers with ERC-4337 is much more expensive than simple transfers with a traditional wallet (often called an EOA) because a contract call has to be made with ERC-4337.

   However, ERC4337 can be cheaper than EOAs on rollups because it can aggregating signatures together to reduce the amount of data on mainnet.

2. Standard now not yet finalized:

   Some challenges like increased attack vectors due to the extended transaction extensibility, potential for unforeseen bugs or security risks when migrating to the new standard, the need for a robust and secure global entry point contract to ensure the appropriate signing and validation of all transactions.

🔗 L2s

✅ and ❌ mean whether support native AA.

Optimism: ❌

Optimism’s first version had three OVM OpCodes to enable account abstraction for smart contract wallets. However, the second version removed them for consistency and security reasons, and there’s no official statement about supporting account abstraction.

Arbitrum: ❌

While there are currently a few smart contract wallets being built on Arbitrum, there is no official statement about supporting account abstraction.

Starknet: ✅

Starknet only has smart contract accounts with validate and execute functions, which must be implemented in all accounts to verify signatures and ensure gas fees. Starknet prohibits the validate function from calling external contract state to prevent unexecuted transactions. However, there are differences from Ethereum, such as the lack of UserOperations, Paymaster-like transaction fee abstraction protocol, and the need for accounts with token balances to create new contracts. Also, if a verified transaction fails, Starknet’s sequencer cannot draw gas fees, which Ethereum does not have.

zkSync: ✅

zkSync does not distinguish between EOA and contract accounts. Its account model is similar to EIP 4337 and includes separate validateTransaction and executeTransaction functions. The Paymaster interface also includes validateAndPayForPaymasterTransaction and postOp functions. However, there are differences, such as the ability to call deployed external contracts and external storage during verification. Paymaster can also call external storage during transaction verification.

⚙️ AA Infrastructure:

Now we see great projects like Stackup, Etherspot, Candide, Infinistism, Pimlico are trying to build infrastructure

Bundler Service:

Builders:

• Golang Implementation by Stackup

• Python Implementation by Candide

• TypeScript Implementation by Infinitism

• Skandha — Typescript Implementation by Etherspot

Some consensus:

• Public-good Service. The open source nature of the vast majority of Bundlers makes them non-exclusive and non-competitive. Any RPC endpoint can run a Bundler by copying the open source code.

• Difficult to monetize: Even if the RPC endpoint running Bundler collects service usage fees through API keys, Bundler services are also harder to monetize than other infrastructure such as the payment contract Paymaster, because the Paymaster can easily earn the fee difference by partnering with a third-party deposit and withdrawal or DeFi protocol aggregator provider.

• Important infrastructure: Verification and execution of UserOperation requires as many Bundlers as possible to better decentralize. Because Stackup and eth-infinitism are the only third-party Bundler service providers currently available, we definitely need more of these Bundler service providers

• Mechanism: Bundlers themselves pass messages and propagate user actions, similar to shared memory pools, without having to agree on specific things. Bundlers have an important function of filtering spam, and for their own economic reasons, they want to monitor as much as possible to make mempool safe.

Differences between bundler services:

• The Bundler service can be either general infrastructure or built specifically for a wallet. Wallet projects may prioritize building the most basic Bundler, while third-party providers must build modular Bundlers for various scenarios.

• The Bundler service is implemented in different languages, similar to Ethereum nodes, to prevent single points of failure and benefit the ecosystem.

• The Bundler service supports private and public mempools and offers customization options for private memory pools.

Paymaster Service:

• Paymaster service is relatively centralized (compared to bundler service) with open-sourced contracts but closed backend.

• Paymaster service has profit-making model, and can enhance payment scenarios by combining with fiat deposit, swaps, bridges, auto-payments, sessions, sponsored fees, and other features to improve dApp usability.

🛠️ AA Wallet& SDKs:

Product evaluation:

1. Key management system:

   -Multi-Sign Logic(Safe): Can only achieve multi-signature logic such as 2/3 and 3/5;

   -Simple Rights Management(Sequence): You can set the weight to the key, and then there is a threshold for the operation account.

   -RBAC, role-based rights management(Unipass): You can set weights and roles for key. Different roles can perform different operations. Each role also has a corresponding threshold. If you exceed this threshold, you can perform the permissions of the corresponding role.

2. Social recovery method

3. Gas sponsor: Self-built Relayer, or set up Bundler + PayMaster

4. Multi-chain account synchronization

5. Multi-chain address unification

6. Supported blockchain

Business

• Business model: Tob/ ToB+ToC / ToC

• Collaborate with dApps: Collaborate with giant infra-like dApps on each chain, like stablecoin or Defi

• Utility: Integrate NFT marketplace, launchpad, etc.

• External support: From Ethereum Foundation or other giant VCs

Shout-out to amazing builders that helped me with this article:

Reference:

• Ethereum Accounts

• The History and Future of Account Abstraction by Nethermind

• Primer on AA by Argent

• Managing Complexity Via AA — A Simplified Future by Etherspot

• A brief note on the future of accounts

• AA as Ethereum’s broadband moment, by @loaf

• Account Abstraction is NOT coming

• 

  Pete Kim

  @petejkim

  

  1/ Several smart contract wallet startups (@argentHQ and @AmbireWallet) created tweet threads "debunking" my claims about a smart contract wallet vs a MPC wallet. Well, are you finished? Oh.
  
  Allow me to retort.

  195

  6:25 PM • May 25, 2022

• 

  Pete Kim

  @petejkim

  

  1/ Earlier this week, @Coinbase launched a web3 browser built directly into its main mobile app, powered by MPC tech that eliminates the need for you to manage your own keys.
  
  Here's why it's a big deal.

  3,556

  3:45 PM • May 18, 2022

• Random thoughts on Account Abstraction

• The current state of Account Abstraction

• Starkware Discussion on AA P1

• Starkware Discussion on AA P2

• Why we need wide adoption of social recovery wallets

• Seedless Self-Custody: On MPC and Smart Contract Wallets

• Using EIP-2535 Diamonds to implement the smart contract wallet for EIP-4337

Special thanks to Liesl and Simon from Essential, George from Flashbots, Anna and Alex from Cow Swap and Josh from Astria for incredibly valuable discussions, insights and feedback of this article.

Introduction

As web3 technology moves towards mass adoption, it is essential to ensure that users are able to navigate the complexities of the web3 jungle on their own. Unlike the early days of blockchain, where users had to decipher intricate technicalities, the future lies in providing a user experience that guides and empowers users to interact seamlessly with decentralized systems. Taking cues from the evolution of web2, where user needs became increasingly expressive through search engines and chatbots like ChatGPT, web3 must deliver an easy-to-use yet powerful user experience.

Intent-driven interactions become the foundation of a user-friendly web3 experience. Although there are a bunch of definitions of intents, I prefer to break down intents into 3 pairs of keywords:

• Outcome instead of path： users only need to express what they want and don't care about how the outcome is achieved

• Conditional Authorization instead of Code Authorization: when a user signs a blockchain tx, they are authorizing codes within the transaction the ability to execute arbitrary computations, modifying the state of the blockchain. In contrast, when a user approves an intent, they are authorizing releasing their assets and tips after being guaranteed that their desired outcome has been achieved (kind of like cash on delivery in online shopping)

• Competitive solver landscape instead of trusted dapps: In a tx-dominated world, users interact with the dapps they choose, and the dapps would act as the service provider to return the desired result, which is usually long-run and mainstream dapps like Uniswap. In an intent-dominated world, well-known or unknown solvers from both offchain and onchain can compete to achieve the intent for the user and get the bonus. From the principle of economy, more competition leads to more efficiency.

To sum up, users can express their intent clearly and directly; platforms can leverage solvers and executors to find the best execution path to fulfill user goals. Just as in web2, where black boxes work behind the scenes to optimize results, web3 executors can utilize algorithms and automated processes to handle the complexities of execution, ensuring that users receive the desired outcomes efficiently and get paid.

By prioritizing user experience and focusing on expressive intent, web3 can usher in a new era where the power and potential of decentralized systems are accessible to all. The future of web3 lies in democratizing access, simplifying interactions, and delivering seamless user experiences that guide and extract users from the execution complexity through the decentralized landscape.

Exploring Different Types of Intent Implementation

Various types of intents can exist based on their generalization, as shown below;

In fact, different levels of intents are everywhere because blockchain codes = sort of automation = extract away some complexity and return the desired outcome to users. However, we want the most general intent in the future, as AA+ intent-specific apps are not enough because they aren't functional in cross-domain and don't scale as effectively as intents with a more permissionless nature.

To understand how intent works, we can look at the currently available solutions, starting from intent-specific apps to general intent infrastructure like Anoma and SUAVE. The analysis would break down into 5 main parts with different questions to bear in mind:

• Intent Expression and Authorization: How do users input their intents; what type of intents and what level of intents can users express; what authorization do users give?

• Solver Candidates: Is it permissioned or permissionless? Are there high barriers to becoming a solver? Are there different types of solvers focusing on other specific areas？

• Solving Process: What's the main path to solving the solution; What determines the completion of the intents?

• Solver Selection: what's the rule for selecting the winner from several solver candidates? Will the competition pattern be winner-takes-all or discrete?

• Validation and Settlement: How to check whether the solver has completed the task? How do the settlement between users and solvers?

Here's a comprehensive overview of the current solutions. For more detailed information, delve into the remaining section.

Cow Swap & 1inch fusion（limit order intent）

• Intent Expression and Authorization:

  • Traders on the Cow Swap and 1inch Fusion platforms express their intents by interacting with the platform's interfaces, providing clear instructions for the desired trades or limit orders.

  • In terms of authorization, traders sign off-chain messages or transactions to grant permission. They pay fees in the traded tokens instead of ETH for gas and have no cost if the trade doesn't execute.

• Solver Candidates:

  • In the case of 1inch Fusion, the solvers, known as resolvers, operate in a permissioned manner. They are required to register, undergo KYC processes, and maintain a sufficient balance to cover the order fee.

  • On the other hand, Cow Swap's solvers are either whitelisted by creating a bonding pool of 1M$ (USDC & COW) or being included in the CoW DAO bonding pool or the Gnosis DAO Bonding pool and being whitelisted by the Cow DAO based on the DAO's criteria.

• Solving Process:

  • Solvers evaluate the existing batch to identify any coincidence of wants (CoW) that can provide the best price for executing the trades or limit orders. They consider various factors such as liquidity, order book depth, and price slippage to ensure the best execution for the traders.

  • Additionally, solvers may explore other underlying on-chain automated market makers (AMMs) directly, such as Uniswap, or leverage DEX aggregators like 1inch to find the most favourable prices and routes.

• Solver Selection:

  • In Cow Swap, traders are executed at the best possible price determined by any external solvers using a batch auction, maximizing trader surplus. The solver providing the most optimal solution is selected.

  • In contrast, the resolver competition in 1inch Fusion is more restricted and related to the 1inch token staked using a Dutch auction.

• Validation and Settlement:

  • The validation and settlement process occurs after the solvers execute the trades or limit orders. Solvers can move tokens on behalf of the users, utilizing the ERC20 approvals granted to the settlement contract. The settlement contract verifies the signature of the user's intent and ensures that the execution aligns with the specified limit price and quantity (enabled by EIP-1271). This validation confirms the successful completion of the intended trades or limit orders.

  • Once validated, the settlement contract facilitates the appropriate allocation of funds to the solvers and users involved in the transactions.

Recently, Cow Swap just announced the launch of Cow Swap Hooks, which enable the execution of more generalized swap intents by enabling custom-coded DeFi actions that execute directly before and/or after trades. It's great to see Uniswap v4 and Cow Swap are pushing their boundaries to more generalized intent activities and bringing us a new world of defi intents!

UniswapX（Swap Intents)

UniswapX's new features can be divided into 2 main parts:

• Signed orders with a Dutch auction mechanism

• Cross-chain Swaps

The signed orders with the Dutch auction are similar to 1inch Fusion and Cow Swap's limit order intents with the following differences:

• Intent Expression and Authorization: users have more freedom (also might bring more complexity) to define parameters, including the decay function for the auction, the initial Dutch order price, etc.

• Solver Candidates: permissionless instead of permissioned (can also be set permissioned by users);

• Solver Selection:

  • Dutch order which executes at a price that depends on the time of its inclusion in a block. The order starts at a price estimated to be better for the swapper than the current market price — for example, if the current market price is 1,000 USDC per ETH, a sell order may start at 1,050 USDC per ETH. The order’s price then decays until it hits the worst price the swapper would accept (e.g. 995 USDC per ETH). Fillers are incentivized to fill an order as soon as it is profitable for them to do so. If they wait too long, they risk losing the order to another filler willing to take a smaller profit.

  • UniswapX also enables including RFQ (allows orders to specify a filler that receives the exclusive right to fill the order for a brief duration) for initial Dutch price setting, in which case the selection process would be almost the same as 1inch Fusion's auction method.

  • Cow Swap is more of a batch auction compared to UniswapX and 1inch's independent auction, which enables combining orders and matching CoWs.

• Solving Process and Validation and Settlement are similar to Cow Swap and 1inch (more details are shown in the chart)

Cross-chain swaps can be achieved through similar processes with main differences in validation and settlement to enable multi-domain swaps:

• Solvers need to deposit more bond assets on the original chain to ensure safety and enable optimistic cross-chain protocols

• Needs an additional settlement oracle to feed into the origin chain's validation contract

• Needs UniswapX to deploy corresponding settle and validation contracts on different domains

Account Abstraction (Wallet-level Intent)

• Intent Expression and Authorization:

  • The process of intent expression and authorization begins when a wallet owner wants to perform a specific action. They craft a userop, typically through a 4337 wallet interface, to express their intent.

  • Off-chain, the wallet owner requests a bundler to handle the userop on their behalf, authorizing limited control according to the intent. For example, the wallet owner may authorize the private key to be able to transact from your main account, but ONLY with Dapp XYZ's hub contract.

• Solver Candidates:

  • Bundler services are considered public goods in the AA framework. The majority of Bundlers are open-source, which makes them non-excludable and non-competitive. Any RPC endpoint can replicate the open-source code and operate as a Bundler. Even when a Bundler RPC endpoint charges fees for its services, it can do so through API keys while still maintaining the non-excludable nature of the Bundler as a public good.

  • Two main types of bundlers: Bundler services purpose-built for wallets, catering to their basic needs & third-party infrastructure providers aiming to construct permissionless and modular Bundlers

• Solving Process:

  • Bundlers simulate the wallet's validateOp method on the userop to determine whether to accept or reject it offchain. Then, they send the transactions to the entry point of the AA system to call the handleOp method. This process also involves bundling multiple userops together to optimize gas and extract MEV.

  • The entry points contract would push the operation on the chain, and the chain node would validate the operation and get it into consensus.

• Solver Selection:

  • The selection of solvers in AA depends on various factors. The wallet used by the account owner might provide bundler service or use third-party infra, and users might also switch rpc endpoint to select a favoured bundler, in which case the success rate and reputation of the bundlers may influence their selection.

• Validation and Settlement:

  • The AA system's entry point validates and settles the operation on-chain. It ensures that the userop meets the requirements and security checks before executing the desired action. Once the operation is successfully executed, the entry point refunds ETH to the bundler from the wallet's deposited funds. This refund mechanism compensates the bundler for their work and prepayment.

Essential（Intent-Centric Account Abstraction Standard)

*Note that the Essential is still in an early stage; part of the descriptions and designs might evolve over time. For more info, keep an eye on Essential's website: https://essential.builders/

Essential： In the short term, it would be an asset-based intent standard（similar to the erc-4337 model but enables more generalized intents) with a set of facilitated infra. In the long term, it would also provide a modular intent layer and a new constrained-based language that casts off the constraints of Ethereum architecture and provides better intent executions.

• Intent Expression and Authorization:

  • Dapps or wallets adapting Essential standards can provide related intent-enabled service to users and extract the underlying complexity away. Users only need to interact with the interface and make authorization.

  • Intents can be expressed in Essential's standard in ST and more generally in LT using its new constrained-based language.

  • Compatible with EVM chains and no need to bridge funds

• Solver Candidates:

  • Essential enables code-expressive intents; various kinds of solvers can join the Essential network to solve corresponding types of intents, such as Cow Swap solvers for swap intents or builders for monitoring and executing chain-state-related intents.

  • A network of solvers would monitor the intents and try to achieve them. Essentials is considering existing solvers/bundlers (e.g. from CoW Protocol or 4337), current MEV searchers and market makers.

• Solving Process:

  • Solvers figure out the constraint environment they're solving in and then use offchain and onchain venues to try to solve these constraint-based intents

• Solver Selection:

  • The selection process is more like a Dutch auction in which users specify the constraint while solvers decide when they jump in to satisfy the intent based on the value they can extract from the satisfaction. The first solver to jump in and solve the intent will be the selected solver and likely the best solution that the market can bear then.

• Validation and Settlement:

  • The validation and settlement both happened by solvers triggering the specific onchain smart contract to verify and split payments. There will be a core contract to which all solutions and all intents and solutions are submitted and extensible with the Essential standards.

Flashbots SUAVE (Multi-domain Block-related Intent)

*Note that SUAVE is still under development; part of the mechanism described below might change.

Compared to the settlement of intents through smart contracts as seen in the previous examples, SUAVE takes a specialized approach by utilizing a dedicated chain for settlement purposes, which also serves as a messaging layer.

In contrast to Account Abstraction (AA) and intent-specific applications, SUAVE introduces an additional step of bridging funds to the SUAVE chain. This step is primarily driven by SUAVE's multi-chain capability and the desire for more cost-efficient, privacy-enabled transactions.

SUAVE just announced the launch of MEVM, a powerful modification of the EVM with new precompiles for MEV use cases. With MEVM, SUAVE chain would first efficiently serve MEV-related players like searchers, builders and other domains that want to capture MEV.

• Intent Expression and Authorization:

  • SUAVE users express their intents in SUAVE by writing EVM codes. These codes outline the desired outcome and functionalities they wish to execute by defining a list of contracts allowing access to the user’s confidential data. There might be some usable templates for normie users.

  • With MEVM, developers can deploy different types of smart contracts for specific MEV applications (e.g. OFA, block building, etc.) or new types of DEXes on SUAVE to be called by other users.

  • Users bridge funds to the SUAVE chain and deposit tips.

• Solver Candidates:

  • The primary participants acting as solvers in SUAVE might be searchers and builders. Searchers and other solvers are responsible for exploring and discovering potential solutions to fulfil user intents, while builders focus on implementing these solutions. They work together to form a robust ecosystem that solves the intents expressed by users.

  • To fulfil different domains' block-related intents, many types of solvers skilled at different domains may exist to support different VMs.

• Solving Process:

  • Solvers do credible and private off-chain computations that can be used in smart contracts on SUAVE through special precompiles in TEE environments.

  • Solvers collectively work on building blocks that contain a bundle of intents. The purpose of block building is to aggregate and organize the intents into valuable blocks that can be proposed to the network.

• Solver Selection:

  • In SUAVE, the selection of solvers follows two main approaches. Firstly, solvers who complete the intended tasks first are typically chosen. This incentivizes efficiency and promptness in delivering solutions. Alternatively, an order flow auction mechanism can be implemented, where solvers bid back to users, returning part of the orderflow value to users.

• Validation and Settlement:

  • To ensure the validity of intents and settle transactions, SUAVE employs oracles and SUAVE validators. Oracles provide external data to validate the execution of intents, while SUAVE validators validate and settle the intents on the SUAVE chain.

Anoma (Generalized Intent for Anoma Protocols)

Anoma is a general architecture similar to Cosmos and is preparing to launch an IBC-enabled Layer 1 Proof-of-Stake (PoS) chain. It combines intent-centric design with a homogeneous protocol powered by the Anoma Virtual Machine (VM), while also offering heterogeneous security features (different Anoma protocols have different consensus mechanisms).

• Intent Expression and Authorization:

  • Users express their intents defining the final State or the properties it should have by interacting with Anoma DApps.

• Solver Candidates:

  • Anoma welcomes a diverse range of solvers, each specializing in different types of applications. These solvers monitor the mempools that align with their interests and objectives. Depending on their specific focus, they observe either all intents or a subset of intents.

• Solving Process:

  • The solvers run solver algorithms that utilize their expertise in areas such as fungible token (FT) trading or computing rollup states.

  • Matching intents is also taken care of by solvers. Solvers take intents and make partially or fully matched transactions. Solvers determine what/when to match, what to charge for partial solving, and how to deal with surplus.

  • Once a solver forms a fully balanced transaction, they submit it to a mempool node which is part of the Anoma ecosystem.

• Solver Selection:

  • The selection can be influenced by the solver's ability to complete tasks efficiently and promptly, following a first-come, first-served approach where the solver who completes the task first is chosen.

• Validation and Settlement:

  • Validators from different Anoma protocols run Anoma Vm to finish execution and verification of intents. The Anoma VM ensures the integrity and validity of intent execution by checking all relevant Validity Predicates (declarative smart contracts) are satisfied.

  • The distribution of funds and rewards to solvers is based on the execution and verification of intents by the Anoma VM.

How intent revolutionizes the orderflow pattern

In the current transaction orderflow state, users must navigate the execution paths themselves, resulting in a relatively simple tx orderflow (as shown in the picture)

However, envisioning a future where the web3 ecosystem embraces an intent-centric approach, the orderflow of intents could become more intricate. In this new paradigm, users would be free to express their intents and delegate the complexity to a new role called solvers.

Before diving in, I would like to summarize 2 trends in the intent world:

• Leading dapps focusing on specific types of intents like Uniswap and Cow Swap are expanding intent features by involving solvers themselves.

• For more generalized intents, we need relatively new architecture, including a new intent language, a new VM, etc. Essential, Flashbots and Anoma are working toward this direction.

In this scenario, different types of intents might be served by specific platforms or protocols. For instance, swap intents may be handled by UniswapX and Cow Swap; intents with single-domain and wallet-related features could be handled by Account Abstraction (AA) wallets or essential-compatible dapps and wallets; platforms like SUAVE and Anoma might address more generalized and multi-domain intents.

Within this new world, the orderflow of intents could follow a more complex path. Let's explore a possible orderflow:

1. User express intents, deposit funds and authorize

   1. Intents are very expressive; normie users might need help translating their intents into codes. This can be achieved by dapps/wallets extracting this part away by providing a user-friendly interface, or there might be an aggregator providing a universal interface for expressing any intents like Google search with the help of AI.

2. Intents sent to related intent mempool

   1. Note that Anoma can have several mempools serving different types of intents and trusted by different dapps or protocols.

3. Solvers simulate offchain and compete to solve the intents.

   1. In the SUAVE ecosystem, solvers possess both solving capabilities and block-building abilities. Some intents involve solving cross-chain tasks by building blocks, such as cross-chain MEV operations. Skilled block builders have an advantage in building valuable blocks and completing tasks faster. Other intents may primarily require algorithmic expertise, such as optimizing liquidity aggregation across multiple chains. These intents may rely on type-specific solvers rather than extensive block building capabilities.

   2. In the AA ecosystem, bundlers perform simulation and bundling tasks. The bundled intents are then either sent to the public mempool for searchers to unbundle and potentially front-run, or directly sent to trusted builders. In the early stages, small-volume bundles may be more efficient to be privately sent to trusted builders to avoid potential loss. As 4337 wallets and other players with sufficient orderflow volume enter the market, they can operate as bundlers like searchers.

4. Validate the completion of intents

   1. Presently, various validation methods exist, each with its own set of trade-offs. Using smart contracts for validation, while reliable, often lacks scalability as different intents require specific validation logic and codes.

   2. Relying on oracles for validation introduces risks associated with oracles, yet offers the advantage of seamless integration with multiple chains.

   3. Leveraging the Anoma VM requires intent applications to adopt the Anoma framework but provides the capability to validate a wide range of intents.

In summary, in an intent-centric world, the orderflow is different to the tx-centric world:

• Users sign and authorize tx vs Users have more options to express their intents.

• Single mempool vs Multiple mempools for different purposes exist.

• Dapps is responsible for the execution vs A new role called solvers opt-in and competes to solve the problems.

• Settle on different chains one by one vs Several chains involved can be settled together (new types of cross-domain executions)

The Ripple Effects of Intents on the Rest of the Web3 World

An intent-powered world involves a lot of web3 participants. Let's get a rough look at the intent-factory landscape.

Note that this is just a rough landscape. As intents evolve gradually, more parties can participate in this new world. For example, shared sequencers like Astria and Espresso can give users quicker preconfirmations in terms of multi-domain intent executions.

• Upstream

  • Chains

    • New chains like SUAVE can facilitate more frequent and cost-effective intent settlement.

    • Anoma-structured chains support new virtual machines that solve the intent validation problem efficiently and generally.

    • Layer 2 or more scalable chains are suitable for performing inexpensive computations related to intent logic expression, validation, and settlement, as intents tend to be computationally intensive due to their expressive nature.

  • Privacy

    • Privacy is crucial in the intent world to prevent malicious MEV problems such as frontrunning and enables more orderflow value being bid back to users/dapps. Additionally, incorporating privacy features can support intents that require enhanced privacy.

    • SUAVE adopts SGX as a short-term solution, while Anoma supports zero-knowledge proofs (zk) and Distributed Key Generation (DKG) encryption.

  • Oracle

    • Oracles now have additional functionality: assisting in validating the fulfilment status of intents.

  • Intent-related Standard

    • A general standard helps to reduce the fragmentation problems brought by different types of intents; Solvers can find it easier to integrate with different intent-enabled apps; Dapps and developers easier to expand to intent systems;

    • Avoid reinventing the wheels for common intent infrastructure.

• Midstream（Potential Solvers）

  • Type-specific solvers like routers for CoW Swap and 1inch, eg, Propeller Heads, and market makers have accumulated large liquidity networks and advanced routing algos, outperforming other solvers and possibly receiving part of exclusive orders directly from swappers.

  • Builders:

    • Builders play a significant role as solvers, especially in the final settlement process involving different chains. Experienced builders can easily fulfil this responsibility.

  • Searchers:

    • Searchers possess expertise in routing and advanced algorithms, making them valuable for solving intents related to finding optimal solutions or accessing liquidity.

• Downstream

  • Intents have a broad impact on various dapps:

    • Enhanced user-friendliness leads to mass adoption.

  • Increased involvement of multiple parties results in more off-chain components, improving efficiency and flexibility.

    • Dapps can expand to provide more functionalities and features by incorporating intent solvers to incorporate more complex functions.

    • For example, in DeFi, intents can emulate atomicity in a cross-chain environment by involving a third party, the solvers, to execute the intent. Solvers take on the risk of failure, enabling a new realm of cross-domain DeFi.

  • More interactions and user instructions lead to the development of complex dapps.

    • For example, in GameFi, users now have greater options for gameplay:

      • Customized game strategies: Intents allow players to define and execute custom game strategies. They can express their game objectives and actions in their way and have solvers execute these intents within the game. This provides players with more freedom and control.

      • Support for economic systems: Through intents, players can participate in in-game economic systems such as trading game assets, providing liquidity, or engaging in lending. By expressing their intents, they can perform financial operations similar to DeFi within the game and earn economic rewards.

Ending

As I conclude this article, I notice the striking similarity between the philosophy of intents and rollups: executing off-chain and final settlement and validation on-chain. With the explosive growth of the rollup ecosystem, we are also now witnessing the explosive growth of intents, with dapps becoming more and more expressive and many projects developing intent-specific language and standards.

However, I want to draw attention to the potential centralization issues that may arise with intents. Just as we've witnessed in the case of private mempools and private order flows, players capable of handling complex user intents and providing a more efficient and user-friendly experience may stand out and attract more private intent order flow, resulting in better execution and increased order flow.

In addition, how intent players could start to involve solvers to efficiently fulfil the intents for users is a practical problem. For example, with current low-volume AA transactions, bundlers or builders do not have enough motivation to spend additional energy and time to provide a new tranche of service. This problem could also exist for more expressive intents.

In conclusion, the world of intents holds immense potential and transformative power. We must navigate the path forward, balancing innovation, decentralization, and user empowerment. Let's embrace this exciting journey and work together to unlock the full potential of intents！

Reference

askyv

@askyveth

For a while, both public mempool for transactions and for UserOps will exist and work together. This means bundlers will work like searchers and cooperate with block builders until AA becomes mainstream.

1

9:45 AM • Jun 8, 2023

apriori

@apriori0x

Thank you to @annaRrose @cwgoes @awasunyin @taurnchitra @cangurel @vveilen for insights, suggestions, and conversation around the topics discussed in this article.

Below I’ll break down Taiga in a thread.

ZKV

@zkv_xyz

Amidst these constraints, @apriori0x took a look at the solutions emerging in the privacy design space by examining the data and function privacy architectures of @namada, @penumbrazone, Taiga (@anoma), ZEXE & SGX. 

mtr.cool/wfvtszqvoe

25

6:16 AM • May 23, 2023

Awesome Intent & Preference Resources - HackMD

Awesome Intent & Preference Resources **TL;DR** Intent is a hot topic and everyone is talking

https://hackmd.io

The MEVM, SUAVE Centauri, and Beyond | Flashbots Writings

In this post we preview the upcoming SUAVE Centauri release and introduce the MEVM.

https://writings.flashbots.net

Intent-Based Architectures and Their Risks

Paradigm is a research-driven crypto investment firm that funds companies and protocols from their earliest stages.

https://www.paradigm.xyz

https://dba.mirror.xyz/NTg5FSq1o_YiL_KJrKBOsOkyeiNUPobvZUrLBGceagg

Uniswap v4 is just announced. Although it’s not fully built out yet, we expect the community to widely explore the boundary of newly enabled possibilities. Since there might be a ton of articles explaining how powerful v4 is on the defi side of things, I would like to address how Uniswap v4 can inspire a new category of blockchain infrastructure: coprocessors.

Short intro on Uniswap v4

As described in the whitepaper, v4 contains mainly 4 improvements:

• Hooks: Hooks are externally deployed contracts that execute some developer-defined logic at a specified point in a pool’s execution. These hooks allow integrators to create a concentrated liquidity pool with flexible and customizable execution.

• Singleton: UNISWAP v4 uses a singleton design pattern where all pools are managed by a single contract, making pool deployment 99% cheaper.

• Flash Accounting: each operation updates an internal net balance, known as a delta, only making external transfers at the end of the lock. Flash accounting simplifies complex pool operations, such as atomic swapping and adding.

• Native ETH: allows for both WETH and ETH pairs.

Most gas savings will certainly come out of the last 3 improvements, but certainly, the most exciting new feature brought in is by the first mentioned new shiny bits: the hooks.

Hooks make liquidity pools more complicated and powerful

Uniswap v4’s primary enhancement revolves around the programmability unlocked by hooks. This feature introduces a new level of complexity and power to liquidity pools, allowing them to be more flexible and customizable than ever before. In comparison to v3’s concentrated liquidity, which was a net upgrade from v2, v4’s hooks provide a wider spectrum of possibilities for how liquidity pools can behave.

This can be viewed as a net upgrade from v3, but it might not be when it comes to actual implementation. Uniswap v3 pools will always be an upgrade compared to v2 pools, since the “worst” upgrade you can do in v3 is to “concentrate” the liquidity across the entire price range, which works the same as in v2. In v4, however, liquidity pools can be programmed to an extent that might not result in a good trading or liquidity-providing experience. Bugs can occur and there will be new attack vectors. Due to the wider array of changes to the liquidity pool’s behavior, developers wishing to utilize the hook feature must exercise caution. They need to thoroughly understand the implications of their design choices on the pool’s functionality and the potential risks for liquidity providers.

The introduction of hooks in Uniswap v4 represents a significant shift in how code is executed on the blockchain. Traditionally, blockchain code is executed in a predetermined sequential manner. However, hooks allow for a more flexible execution order, ensuring that certain code is executed before other code. This feature pushes complex computations towards the edge of the stack, rather than solving them in a monolithic stack.

In essence, hooks enable more complicated computations to be performed outside of the Uniswap native contract. While this was achievable in v2 and v3 through manual computation outside of Uniswap and triggering with external activators such as other smart contracts, v4 integrates hooks directly into the liquidity pool’s smart contract. This integration makes the process more transparent, verifiable, and trustless compared to the previously manual process.

Another side benefit hooks bring is extensibility. Uniswap now no longer have to rely on new smart contract (which requires liquidity migration) or forks to deploy innovations. Hooks can now make an old liquidity pool feels like a new one by directly implementing new features.

What happens to v4 liquidity pools will happen to other Dapps

I expect more and more dapps to push computation outside of their own smart contracts, just like Uniswap v4.

What v4 is doing here is by allowing the liquidity pool executions to be broken up at any step, arbitrary conditions can be inserted, and triggering computation that’s outside of the v4 contract. The only similar thing we have seen so far is flash loan, where the executions will be reverted if the loan is not given back within the same block. Just that the computation still happens in the flash loan contract.

Uniswap v4’s design brings multiple benefits that either weren’t possible or were suboptimal to implement in v3. For instance, the use of embedded oracles is now possible, reducing the reliance on external oracles that often introduce potential attack vectors. This embedded design enhances the security and reliability of price information, a critical factor in the operation of DeFi protocols.

Furthermore, automation, which previously had to be triggered externally, can now be embedded directly into the liquidity pool. This integration not only mitigates security concerns but also addresses reliability issues associated with external triggers. It provides a more seamless and efficient operation of the liquidity pool, enhancing its overall performance and user experience.

Finally, the introduction of hooks in Uniswap v4 opens up the possibility for more diverse security features to be implemented directly into the liquidity pool. In the past, the primary security measures available to liquidity pools were audits, bug bounties, and insurance purchases. With v4, developers can now design and implement a variety of fail-safes and low liquidity warnings directly into the pool’s smart contract. This development not only enhances the security of the pool but also provides greater transparency and control to liquidity providers.

Compared to traditional phones, smartphones win on programmability. Smart contracts have kind of lived under the shadow of “persistent scripts” for a long time. Now, thanks to the advant of v4, liquidity pool smart contracts get a new programmable upgrade, getting “smarter”. I can’t imagine why not all dapps want to upgrade in this direction given the opportunity to upgrade yourself from a Nokia into an iPhone. Since Nokia is more reliable than iPhone I can see plenty of reasons why some smart contracts would like to stay the way it is, but you get my point on where the future of Dapps is heading to.

Dapps wish to use their own “hooks” has a scaling problem

Imagine applying this to all the other dapps, where we can insert conditions to be triggered, and then insert arbitrary computations in between the original sequence of transactions.

This sounds like how MEV works, but MEV is not an open design space for the dapp developers. It’s more like an uncertain hike into a dark forest, the best they can do is to seek external MEV protection but can only hope for the best.

Assume that the flexibility of Uniswap v4 inspires a new generation of dapps (or upgrade from existing dapp) to adopt similar philosophy, where they allow their own execution sequence to be more programmable. Since these dapps are usually only deployed on one chain (an L1 or an L2), we expect most state changes to be run on this chain.

• The additional computation inserted within the state-changing process of a dapp might be too complicated and burdensome to run on this chain. We might go beyond the gas limit really quickly, or it’s just outright difficult to realize. It also introduces a host of challenges, particularly in the areas of security and composability.

• Not all computations are equal. As evidenced by dapps’ reliance on external protocols such as oracles and automation networks. However, this reliance introduces security risks, as highlighted in a recent tweet by Dan Elitzer in.

Summarizing the problem: incorporating all computations into state-altering smart contract execution on one single chain is far from optimal.

Hint on the solution: it’s been solved in the real world

To solve this problem brought by the new generation of dapps (which might be largely inspired by Uniswap v4), we have to look into the core of the problem: that 1 single chain. Blockchain works as distributed computer, with one single CPU processing everything. On a PC, modern CPU has come a long way to solve that exact problem.

Just as computers have transitioned from single-core monolithic CPUs to a modular design comprising multiple efficiency cores, performance cores, GPUs, and NPUs.

DApp computation can scale in a similar manner. By specializing processors and combining their efforts, outsourcing some computation outside of the main processor offers flexibility, optimality, security, scalability, and upgradability.

Actual solution

There are really only two categories of coprocessors:

• External coprocessors

• Embedded coprocessors

External coprocessors

External coprocessors are like a GPU on the cloud. It’s nice and powerful, but you have additional internet lag between the CPU and the GPU communication. Plus you are not ultimately controlling the GPU, so you have to trust that it’s doing the job right.

Using Uniswap v4 as an example, let’s say some ETH and USDC were to be added to a liquidity pool at the last 5 mins TWAP. If the TWAP calculation is done in Axiom, v4 is essentially using Ethereum as the main processor and Axiom as the coprocessor.

Axiom

Axiom is a ZK coprocessor for Ethereum which provides smart contracts trustless access to all on-chain data and arbitrary expressive computing over it.

Developers can make queries into Axiom and trustlessly use the ZK-verified results on-chain in their smart contracts. To fulfill queries, Axiom performs three steps:

1. Read: Axiom uses ZK proofs to trustlessly correct reads from block headers, states, transactions, and receipts in any historical Ethereum block. All Ethereum on-chain data is encoded in one of these forms, meaning that Axiom can access anything an archive node can.

2. Compute: Once data has been ingested, Axiom applies verified compute primitives on top. This includes diverse operations from basic analytics (sum, count, max, min) to cryptography (signature verification, key aggregation) and machine learning (decision trees, linear regression, neural network inference). The validity of each piece of computing is verified in a ZK proof.

3. Verify: Axiom accompanies the result of each query with a ZK validity proof that (1) the input data was correctly fetched from the chain and (2) the compute was correctly applied. This ZK proof is verified on-chain in the Axiom smart contract, and the final result is then trustlessly available for use by any downstream smart contract.

Warp Contract (by RedStone)

Warp Contracts is the most popular implementation of SmartWeave, which is an architecture aimed at creating a reliable, fast, and production-ready smart contracts platform/engine on Arweave. Essentially, SmartWeave is a sequenced array of Arweave transactions that benefit from the absence of a fee market for transaction block inclusion on Arweave. These unique properties allow for unlimited transaction data without additional fees beyond storage costs.

SmartWeave utilizes a unique approach known as “lazy evaluation,” which transfers the responsibility of executing smart contract code from network nodes to the users of the smart contract. In essence, this means that the computation of transaction validation is deferred until it is required, reducing the workload on network nodes and allowing for more efficient processing of transactions. This approach enables users to execute as much computation as needed without incurring additional fees, offering functionalities that are not feasible with other smart contract systems. Obviously attempting to evaluate contracts with thousands of interactions on a user’s CPU would be a futile exercise. To overcome this challenge, an abstraction layer, such as Warp’s DRE, has been developed. It comprises a network of distributed validators that handle contract computations, leading to significant improvements in response time and user experience.

Furthermore, the open-ended design of SmartWeave enables developers to write the logic in any programming language, offering a refreshing alternative to the often rigid Solidity codebase. By delegating certain high-cost or high-throughput actions to Warp, existing social graph protocols built on EVM chains can be enhanced with seamless SmartWeave integration, leveraging the strengths of both technologies.

Hyper Oracle

Hyper Oracle is a network of zkOracles designed for blockchains. At present, the zkOracle network operates solely for the Ethereum blockchain. It retrieves the data from every block of the blockchain as a data source with zkPoS and processes the data using programmable zkGraphs that run on zkWASM, all in a trustless and secure manner.

Developers can define custom off-chain computations with JavaScript, deploy those computations to Hyper Oracle network, and utilize Hyper Oracle Meta Apps to index and automate their smart contracts.

Hyper Oracle’s indexing and automation Meta Apps are fully customizable and flexible. Any computation can be defined and all computations will be secured by a generated zk proof, even machine learning computation.

• Ethereum blockchain serves as the original on-chain data source for zkOracles, but in the future, any network can be used.

• The Hyper Oracle zkOracle node consists of two main components: zkPoS and zkWASM.

  • zkPoS fetches the block header and data roots of the Ethereum blockchain by proving Ethereum’s consensus with zk. The zk proof generation process can be outsourced to a decentralized prover network. zkPoS works as the foreign circuit of zkWASM.

  • zkPoS feeds the block header and data roots to zkWASM. zkWASM takes this data as essential inputs for running zkGraphs.

  • zkWASM runs customized data mappings or any other computation defined by zkGraphs and generates zk proofs of those operations. The operator of the zkOracle node can choose the number of zkGraphs they wish to run (from one to all deployed zkGraphs). The zk proof generation process can be outsourced to a decentralized prover network.

• The output of a zkOracle is off-chain data that developers can use through Hyper Oracle Meta Apps (which are covered in the next sections). The data also comes with zk proofs that demonstrate the validity and computation of the data.

Other projects worth mentioning

There are also a few projects that can be used as an external coprocessor if you decided to use them in that way. It’s just that they have overlapping positioning in other verticals of the blockchain infra space that they don’t uniquely classify as a coprocessor.

• RiscZero: if a dapp use RiscZero to compute ML tasks of an onchain agent and feeds the result back to it’s game contract on StarkNet, it’s using the StarkNet as main processor and RiscZero as the coprocessor.

• Ironmill: if a dapp runs their zk circuit in Ironmill but have smart contracts deployed on Ethereum, it’s using Ethereum as the main processor and Ironmill as the coprocessor.

Potential use cases for external coprocessors

• Governance and Voting: Historical on-chain data can help DAOs record how much voting power each member holds, which is necessary for voting. Without this data, members may not be able to participate in the voting process, which could hinder governance.

• Underwriting: Historical on-chain data can assist asset managers in assessing the performance of their managers beyond profits. They can see the level of risk taken and the types of drawdowns experienced, which can help them make better decisions when it comes to compensation or potential reward reductions.

• Decentralized Exchanges: Historical price data on-chain can assist decentralized exchanges in trading based on past trends and patterns, potentially leading to higher profits for users. Additionally, access to historical trading data can help exchanges improve their algorithms and user experience.

• Insurance Products: Historical on-chain data can be used by insurance providers to assess risks and set premiums for different types of policies. For example, when setting premiums for a defi project, an insurance provider may look at past onchain data.

Note that all of the above is async use cases because the customer dapp needs to call external coprocessors’ smart contract when triggered in block N. When coprocessors return the computation result, the result has to be accepted or verified in some shapes or forms in at least the very next block, which is N+1. This makes the utilization of co-processing results happen at least on the next block of triggering. It’s really like a cloud-based GPU. It runs your ML model well, but you can’t really play fast-paced games on it very well due to the lag.

Embedded coprocessors

Embedded coprocessors is like a GPU on your PC’s motherboard, sitting right beside the CPU. It has very little lag communicating with the CPU. Also it’s right below your palm so you can be pretty certain it’s not tampered with. Just that it will get very expensive if you wish it can run ML as fast as a cloud GPU.

Still using Uniswap v4 as an example, assume that some ETH and USDC were to be added to a liquidity pool deployed on Artela at the last 5 mins TWAP. If the pool is deployed in the EVM on Artela, and the TWAP calculation is done in the WASM on Aretla, this pool is essentially using Artela’s EVM as the main processor and Artela’s WASM as the coprocessor.

Artela

Artela is an L1 built with Tendermint BFT. It provides a framework that supports the dynamic extensions for any execution layer to achieve on-chain customized functionalities. Every Artela full node runs two VMs at the same time.

• EVM, which is the main processor storing and updating smart contract states.

• WASM, which is the coprocessor storing and updating Aspect states.

Aspects represent arbitrary computation that a developer wishes to run without touching the state of the smart contracts. Think of it as a Rust script that provides dApps with customized functionality beyond native composability of smart contracts.

If this is a bit hard to comprehend, try from the two following perspectives.

• From the perspective of blockchain architecture

  • Aspect serves as a new execution layer.

  • In Artela, the blockchain operates two execution layers simultaneously—one for smart contracts and one for other computations.

  • This new execution layer does not introduce new trust assumptions, thus not affecting the security of the blockchain itself. Both VMs are secured by the same set of nodes running the same consensus.

• From the perspective of application runtime

  • Aspects act as programmable modules that work in synergy with smart contracts, enabling the addition of customized functionality and independent execution.

  • It offers advantages over sole smart contract in several ways:

    1. Non-invasiveness: It can intervene before and after contract execution without modifying the smart contract code.

    2. Synchronous execution: It enables hook logic throughout the entire transaction lifecycle, allowing for fine-grained customization.

    3. Direct access to the global state and base layer configuration, allowing for system-level functionality.

    4. Elastic block space: providing independent block space with protocol guarantees for dApps with high transaction throughput requirements.

    5. Compared to static precompiles, it enables dApps to achieve dynamic and modular upgrades at runtime, balancing stability and flexibility.

The really exciting breakthrough Artela makes by introducing this embedded coprocessor is that now, Aspects, the arbitrary extension module, can be executed THROUGHOUT THE SAME TRANSACTION as smart contracts. Developers can bind their smart contract to Aspects and have all transactions calling the smart contract processed by Aspects.

Also, like smart contracts, Aspects store data on-chain, which enables smart contracts and Aspects to read the global state of each other.

These two features unlock a new level of composability and interoperability between smart contracts and Aspects

Aspect Functionality:

Compared to smart contracts, Aspects provide capabilities primarily focused on pre and post-transaction execution. Aspects do not replace smart contracts but rather complement them. In comparison to smart contracts, aspects offer the following distinct capabilities for applications:

• Automatic insertion of reliable transactions into upended blocks (e.g., for scheduled tasks).

• Reversion of state data changes caused by transactions (only authorized contract transactions can be Reversion).

• Reading of static environmental variables.

• Passing temporary execution states downstream to other aspects.

• Reading of temporary execution states passed from upstream aspects.

• Dynamic and modular upgradability.

Differences between Aspects and Smart Contracts:

What sets Aspects apart from smart contracts is:

• Smart contracts are accounts with code, while Aspects are native extensions of the blockchain.

• Aspects can run at different points in the transaction and block lifecycle, while smart contracts are only executed at fixed points.

• Smart contracts have access to their own state and limited context of the block, whereas Aspects are allowed to interact with the global processing context and system-level API.

• The execution environment of Aspects is designed for near-native speeds.

Aspects are simply pieces of code logic and are not associated with an account. Therefore, they do not have the ability to:

• Write, modify, or delete contract state data.

• Create new contracts.

• Transfer, destroy, or hold native tokens.

These aspects make Artela a unique platform that extends the capabilities of smart contracts and offers a more comprehensive and customizable development environment.

*Note that the Aspects we mentioned above is strictly called “built-in” Aspects, which is embedded coprocessors run by the full nodes of the Artela Chain. Dapps can also deploy their own Heterogeneous Aspect, which is run by external coprocessors. These external coprocessors can be executed on external networks, or by a subset of nodes in another consensus. It’s more flexible as the dapp developer can literally do anything they want with it as long as they feel safe and reasonably doing so. It’s still under exploration, and specific details are yet to be announced.

Potential use cases for embedded coprocessors

1. Complex calculations involved in new DeFi projects (such as intricate game-theoretical mechanisms) may require more flexible and iterative on-the-fly computational capabilities from an embedded coprocessor.

2. More flexible access control mechanisms for all kinds of dapps. Currently, access control is typically limited to blacklisting or whitelisting based on smart contract permissions. Embedded coprocessors can unlock instant and granular levels of access control.

3. Certain complex features in Fully On-Chain Games (FOCG). FOCGs have been constrained by the EVM for a long time. It might be easier if the EVM is reserved for simpler functionalities like transferring NFTs and tokens, while other logic and state updates are computed by coprocessors.

4. Security mechanisms. Dapps can introduce their own proactive security monitoring and fail-safe mechanisms. For example, liquidity pools can prevent from more than 5% of withdrawing every 10 mins. If one of the withdrawals is detected by a coprocessor, the smart contract can halt and trigger some alarm mechanisms, such as injecting emergency liquidity at a certain dynamic price range.

Closing thoughts:

Dapps getting big, bloaty, and uncomfortably complicated is inevitable. Therefore the adoption of coprocessors is also inevitable. It’s only a matter of time and adoption curve.

Running external coprocessors allows dapps to stay in their comfort zone: whatever chain they previously lives on. However, for the new dapp developers out there looking for an execution environment to deploy to, embedded coprocessors is like a GPU for a PC. If it calls itself a high-performance PC, it has to have a decent GPU.

Unfortunately, none of the above-mentioned projects are on mainnet yet. We can’t really benchmark and show which one is better for which use case. However, one thing is certain. Technology progresses in an upward spiral. It looks like we are going around in circles, but remember when seen from aside, history shows that it’s up only.

Long live the scalability trilemma, long live coprocessors.

Reference

https://www.axiom.xyz/blog/intro https://docs.artela.network/main https://docs.axiom.xyz/using-axiom/what-does-axiom-do https://docs.hyperoracle.io/how-hyper-oracle-works https://github.com/Uniswap/v4-core/blob/main/whitepaper-v4-draft.pdf https://medium.com/logos-network/everything-you-know-about-the-scalability-trilemma-is-probably-wrong-bc4f4b7a7ef https://medium.com/@warp_contracts/smartweave-vs-evm-403487e88fc6 https://mirror.xyz/hyperoracleblog.eth/qbefsToFgFxBZBocwlkX-HXbpeUzZiv2UB5CmxcaFTM https://twitter.com/delitzer/status/1661755779240841222?s=20

The popularity of Chat GPT and GPT-4 has shown us the power of artificial intelligence. Behind artificial intelligence, in addition to algorithms, what is more, important is massive data. Around the data, we have built a large-scale complex system, the value of which mainly comes from business intelligence (Business Intelligence, BI) and artificial intelligence (Artificial Intelligence, AI). Due to the rapid growth of data volumes in the Internet age, data infrastructure work, and best practices are also evolving rapidly. In the past two years, the core system of the data infrastructure technology stack has been very stable, and the supporting tools and applications are also growing rapidly.

Web2 Data Infrastructure Architecture

Cloud data warehouses (e.g., Snowflake, etc.) are growing rapidly, primarily focusing on SQL user and business intelligence user scenarios. Adoption of other technologies is also accelerating, with data lakes (e.g., Databricks) experiencing unprecedented customer growth, and heterogeneity in the data technology stack will coexist.

Other core data systems, such as data acquisition and transformation, have proven equally durable. This is particularly evident in the modern data intelligence space, where combinations of Fivetran and dbt (or similar technologies) can be found almost everywhere. The combination of Databricks/Spark, Confluent/Kafka, and Astronomer/Airflow is also becoming the de facto standard.

• Data Sources end generates relevant business and operational data;

• Data Extraction and Transformation responsible for extracting data from business systems (E), transferring to storage, aligning formats between data sources and destinations (L), and sending analyzed data back to business systems as required;

• Data storage storing data in a format that can be queried and processed, requiring optimization toward low cost, high scalability, and analytical effort;

• Querying and processing translates high-level programming languages (typically in SQL, Python, or Java/Scala) into low-end data processing tasks. Executing queries and data models based on stored data using distributed computing, including historical analysis (describing past occurrences) and predictive analysis (describing future expected events);

• Transformation Converting data into analytically usable structures, managing processes and resources;

• Analysis and Output is an interface for analysts and data scientists to provide traceable insight and collaboration, present the results of data analysis to internal and external users, and embed data models into user-facing applications.

With the rapid development of the data ecosystem, the concept of “data platforms” has emerged. From an industry perspective, the defining characteristic of a platform is the technical and economic interdependence of an influential platform provider and many third-party developers. From a platform perspective, the data technology stack is divided into a “front-end” and a “back-end.”

“Back-end” roughly includes data extraction, storage, processing, and transformation, and has begun to consolidate around several cloud service providers. As a result, customer data is collected in a standard system, and vendors are investing heavily in making this data easily accessible to other developers. This is also a fundamental design principle of systems like Databricks and is implemented through systems like the SQL standard and custom compute APIs like Snowflake.

“Front-end” engineers leverage this single integration point to build various new applications. They rely on data that has been cleaned and integrated into a data warehouse/lake warehouse without worrying about how they were generated. A single customer can build and buy many applications on top of one core data system. We are even starting to see traditional enterprise systems, such as financial or product analytics, being refactored using warehouse-native architectures.

As the data technology stack gradually matures, the data applications on the data platform also surge. Due to standardization, adopting new data platforms has never been more important, and correspondingly maintaining platforms has become extremely important. At scale, platforms can be very valuable. There is intense competition among core data system vendors for current budgets and long-term platform positions. The astonishing valuations of data ingestion and transformation companies are easier to understand if you consider that data ingestion and transformation modules are a core part of emerging data platforms.

However, these technology stacks have been shaped by a data utilization approach dominated by large companies. As society’s understanding of data deepens, it is believed that data, like land, labor, capital, and technology, are all marketable factors of production. As one of the five factors of production, it is the asset value of data reflected behind it.

The current technology stack is inadequate to enable the allocation of data element markets. New data infrastructures are developing and evolving in Web3, which is closely integrated with blockchain technology. These infrastructures will be embedded in modern data infrastructure architectures to enable data ownership rights definition, circulation transactions, revenue distribution, and factor governance. These four areas are critical from a government regulatory perspective and require special attention.

Web3 Hybrid Data Infrastructure Architecture

Inspired by the a16z Unified Data Infrastructure Architecture (2.0) and incorporating our understanding of the Web3 infrastructure architecture, we propose the following Web3 Hybrid Data Infrastructure Architecture.

Orange is the technology stack unit that is unique to Web3. Since decentralization is still in its early stages of development, most applications in the Web3 space are still using this hybrid data infrastructure architecture. The vast majority of applications are not true “superstructures”. Hyperarchitecture is a non-stoppable, free, valuable, scalable, license-free, positive externality, and trustworthy neutrality. It exists as a public good for the digital world, public infrastructure for the “metaverse” world. This requires a completely decentralized underlying architecture to support it.

The traditional data infrastructure architecture evolved in response to the business development of the enterprise. a16z summarizes it in two systems (analytics and business systems) and three scenarios (modern business intelligence, multi-model data processing, artificial intelligence, and machine learning). This is a summary made from the perspective of the business - data for the growth of the business.

However, not only enterprises, society, and individuals should benefit from the productivity improvement from the data element. Countries worldwide have introduced policies and regulations one after another, hoping to regulate the use of data from the regulatory level and promote the circulation of data. This includes various data banks that are common in Japan, data exchanges that have recently emerged in China, and trading platforms that have been widely used in Europe and the United States, such as BDEX (USA), Streamr (Switzerland), DAWEX (France) and CARUSO, etc.

As data begins to be titled, traded in flow, distributed, and governed, their systems and scenarios go beyond empowering companies to make decisions and grow their businesses themselves. These systems and scenarios either need to leverage blockchain technology or strongly rely on policy regulation. web3 is a natural ground for data factor markets, which technically eliminates the possibility of cheating and can greatly reduce regulatory pressure, allowing data to exist as a true factor of production and be allocated in a market-based manner.

In the Web3 context, the new paradigm of data utilization includes market systems that host mobile data elements and public systems that manage public data elements. They cover three new data business scenarios: property data development integration, composable initial data layers, and public data mining.

Some of these scenarios are tightly integrated with traditional data infrastructures and belong to Web3 hybrid data infrastructure architectures. In contrast, others are detached from traditional architectures and are fully supported by new technologies native to Web3.

Web3 and the Data Economy

The data economy marketplace is the key to allocating data elements, including developing and integrating product data and the initial data layer market with composability. In an efficient and compliant data economy market, the following points are important:

1. Data ownership rights are key to securing rights and compliant use and should be disposed of in a structured allocation, while data use requires confirmation of authorization mechanisms. Each participant should have the relevant rights and interests.

2. Circulation transactions must be combined on and off-site as well as compliant and efficient. It should be based on four principles: data source can be confirmed, use scope can be defined, circulation process can be traced, and security risk can be prevented.

3. The revenue distribution system needs to be efficient and fair. According to the principle of “who inputs, who contributes, who benefits”, the government can guide and regulate the distribution of data elements.

4. Factor governance is secure, controllable, flexible, and inclusive. This requires an innovative government data governance mechanism, the establishment of a data elements market credit system, and encourage enterprises to actively participate in the construction of a data elements market around data sources, data ownership rights, data quality, data use, etc., the implementation of the data circulation transaction statement and commitment system for data vendors and third-party professional service organizations.

The above principles are the basic principles for regulators to consider the data economy. These principles can be used to think about three scenarios: property data development and integration, composable initial data layers, and public data mining. What kind of infrastructure do we need to support this? What kind of value can these infrastructures capture at what stages?

Scenario 1: Data ownership rights development and integration

In the process of property rights data development, it is necessary to establish a categorical and hierarchical rights confirmation and authorization mechanism to determine the ownership, use rights, and management rights of public, corporate, and personal data. According to the data source and generation characteristics, the property rights of data are defined through “data adaptation”. Among them, typical projects include Navigate, Streamr Network, KYVE, etc. These projects realize data quality standardization, data collection, and interface standardization through technical means, confirm the rights of off-chain data in some form, and carry out data classification and hierarchical authorization through smart contracts or internal logic systems.

The applicable data types in this scenario are non-public data, namely enterprise and personal data. The value of data elements should be activated by “common use and shared benefits” in a market-oriented manner.

• Enterprise data includes all kinds of market subjects in the production and business activities collected and processed data that do not involve personal information and public interests. Market subjects enjoy the rights and interests to hold, use and obtain benefits in accordance with the law, as well as the right to receive reasonable returns for their input labor and other factor contributions.

• Personal data requires data processors to collect, hold, host, and use data per the scope of individual authorization and the law. Use innovative technical means to promote the anonymization of personal information and to safeguard the security of information and personal privacy when using personal information data. Explore mechanisms for trustees to represent the interests of individuals and supervise the collection, processing, and use of personal information data by market entities. For special personal information data related to national security, the use of relevant units may be authorized in accordance with the law and regulations.

Scenario 2: Combinable initial data layers

Composable initial data layers are an important part of the data economy market. Unlike general property rights data, the most obvious feature of this part of data is that the standard data format needs to be defined through “data schema management”. Different from the quality, collection, and interface standardization of “data adaptation”, the emphasis here is on the standardization of data models, including standard data formats and standard data models. Ceramic and Lens are the pioneers in this field. They respectively guarantee the standard mode of off-chain (decentralized storage) and on-chain data, thus making data composable.

Built on top of these data schema management tools are composable initial data layers, often called “data layers”, such as Cyberconnect, KNN3, etc.

The combinable initial data layers are less involved in the Web2 technology stack, but ceramic-based hot data reading tools break this point, which will be a critical breakthrough. Many similar data do not need to be stored on the blockchain, and storing them on the blockchain is difficult. Still, they need to be stored on a decentralized network, such as high-frequency low-value density such as user posts, likes, and comments Data, and Ceramic provides a storage paradigm for this type of data.

Composable initial data is a key scenario for innovation in the new era, and it is also an important symbol of the end of data hegemony and monopoly. It can solve the cold start problem of start-ups in terms of data, combining mature data sets and new data sets so that Startups can build a data competitive advantage faster. At the same time, it lets start-ups focus on incremental data value and freshness to win continuous competitiveness for their innovative ideas. This way, large data will not become a moat for large companies.

Scenario 3: Public Data Mining

Common data mining is not a new use case but has received unprecedented prominence in the Web3 technology stack.

Traditional public data includes public data generated by party and government agencies, enterprises, and institutions performing their duties according to law or providing public services. Regulatory agencies encourage providing such data to society in the form of models, verification, and other products and services per the requirements of “original data not out of domain, data usable but not visible” on the premise of protecting personal privacy and ensuring public safety. They use traditional technology stacks (blue and some orange, orange represents the intersection of multiple types of technology stacks, the same as below).

In Web3, the transaction data and activity data on the blockchain are another type of public data, which is characterized by “available and visible”, so it lacks data privacy, data security, and data use confirmation authorization capabilities, which is a real ” Public goods” (Public Goods). They use a technology stack (yellow and partially orange) with blockchain and smart contracts at its core.

The data on decentralized storage is mostly Web3 application data other than transactions. At present, it is mainly file and object storage, and the corresponding technology stack is still immature (green and some orange). The production and mining of this type of public data utilize storage common issues, including hot and cold storage, indexing, state synchronization, rights management, computation, etc.

Many data applications have emerged in this scenario. They are not data infrastructure but more data tools, including Nansen, Dune, NFT Scan, 0x Scope, etc.

Case: Data Exchange

Data exchange is a platform where data is traded as a commodity. They can be categorized and compared based on transaction objects, pricing mechanisms, quality assurance, etc. Data Stream X, Dawex, and Ocean Protocol are typical market data exchanges.

Ocean Protocol (200 million market cap) is an open-source protocol that enables businesses and individuals to exchange and liquidate data and data-based services. The protocol is based on the Ethereum blockchain and uses “datatokens” to control access to datasets. A data token is a special ERC20 token representing ownership or access to a dataset or a data service. Users can purchase or earn data tokens to access the information they need.

The technical architecture of Ocean Protocol consists of the following main components:

• Providers: Providers providing data or data services can earn revenue by issuing and selling their own data passes through Ocean Protocol.

• Consumers: Demanders who buy and use data or data services can purchase or earn the data passes they need to gain access through Ocean Protocol.

• Marketplaces: An open, transparent, and fair marketplace for data transactions provided by Ocean Protocol or a third party that connects providers and consumers globally and offers data passes in multiple types and domains. Marketplaces can help organizations discover new business opportunities, increase revenue streams, optimize operational efficiency, and create value.

• Network: refers to a decentralized network layer provided by Ocean Protocol that supports data exchange of different types and sizes and ensures security, trust, and transparency in the data transaction process. The network layer is a set of smart contracts that are used to register data, record ownership information, facilitate secure data exchange, etc.

• Curator (Curator): refers to a role in an ecosystem responsible for screening, managing, and reviewing datasets. They are responsible for reviewing information about the dataset’s source, content, format, and license to ensure that the dataset meets standards and can be trusted and used by other users.

• Verifier: A role in an ecosystem that verifies, vets data transactions and data services, and reviews and validates transactions between data service providers and consumers to ensure the quality, availability, and accuracy of data services.

“Data services” data providers create include data, algorithms, computation, storage, analysis, and curation. These components are tied to the service’s execution agreements (service-level agreements), secure computing, access control, and permissions. Essentially, this controls access to a “cloud service suite” through smart contracts.

The advantage is that, - Open-source, flexible, and extensible protocols help organizations and individuals create unique data ecosystems. - The decentralized network layer based on blockchain technology can ensure the security, credibility, and transparency of the data transaction process while protecting providers’ and consumers’ privacy and rights. - An open, transparent, and fair data market that can connect providers and consumers worldwide and provide data certificates of various types and fields.

Ocean Protocol is a typical representative of hybrid architecture. Its data can be stored in different places, including traditional cloud storage services, decentralized storage networks, or the data provider’s own servers. The protocol uses data tokens (datatokens) and non-homogeneous tokens (data NFTs) to identify and manage data ownership and access rights. In addition, the protocol also provides the function of computing to data (compute-to-data), enabling data consumers to analyze and process data without exposing the original data.

Although Ocean Protocol is one of the complete data trading platforms on the market at this stage, it still faces many challenges:

• Establish an effective trust mechanism to increase the trust between data providers and demanders and reduce transaction risks. For example, establish a data element market credit system, and carry out certification and verification through the blockchain to identify untrustworthy behaviors in data transactions, incentives for keeping promises, punishment for untrustworthiness, credit restoration, objection handling, etc.

• Establish a reasonable pricing mechanism to reflect the true value of data products, incentivize data providers to provide high-quality data, and attract more demanders.

• Establish a unified standard specification to facilitate interoperability and compatibility between data of different formats, types, sources, and purposes.

Case: Data Model Marketplace

Ceramic mentions in their Data Universe the open data model marketplace they want to create because data needs interoperability and it can contribute significantly to productivity gains. Such a data model marketplace is achieved through an urgent consensus on the data model, similar to the ERC contract standard in Ether, from which developers can choose a functional template to have an application that conforms to all the data in that data model. At this stage, such a marketplace is not a trading marketplace.

Regarding the data model, a simple example is that in a decentralized social network, the data model can be reduced to 4 parameters, which are

1. PostList: stores the index of user posts

2. Post: stores individual posts

3. Profile: stores the user’s profile

4. FollowList: stores the user’s follow list

So how can data models be created, shared, and reused on Ceramic to enable cross-application data interoperability?

Ceramic provides a Data Models Registry (Data Models Registry), an open source, community-built, reusable application data model for the Ceramic repository. Here, developers can openly register, discover, and reuse existing data models—the foundation for customer operations applications built on shared data models. Currently, it is based on Github storage, and in the future, it will be decentralized on Ceramic.

All data models added to the registry are automatically published under the npm package of @datamodels. Any developer can install one or more data models using @datamodels/model - name. These models are available for storing or retrieving data at runtime using any IDX client, including DID Data Store or Self.ID.

In addition, Ceramic has built a Data Models forum based on GitHub. Each model in the data model registry has its own discussion thread on this forum. The community can comment and discuss it. At the same time, it is also a place for developers to publish their ideas on data models so as to obtain the opinions of the community before adding them to the registry. At present, everything is in the early stage. There are not many data models in the registry. The data models included in the registry should be evaluated by the community and called the CIP standard, just like the smart contract standard of Ethereum, which provides data composability.

Case: Decentralized Data Warehouse

Space and Time is the first decentralized data warehouse to connect on-chain and off-chain data to support a new generation of smart contract use cases. Space and Time (SxT) has the industry’s most mature blockchain indexing service. The SxT data warehouse also employs new cryptography called The SxT data warehouse also employs new cryptography called Proof of SQL™ to generate verifiable, tamper-proof results, allowing developers to join untrusted on-chain and off-chain-data-in a simple SQL format and load the results directly into smart contracts, supporting sub-second queries and enterprise-level analytics in a fully tamper-proof and blockchain-anchored manner.

Space and Time is a two-tier network with a validator layer and a data warehouse. the success of the SxT platform depends on the seamless interaction of the validator and the data warehouse to facilitate simple and secure querying of both on-chain and off-chain data.

A data warehouse consists of database networks and computes clusters controlled by and routed to them by space and time validators. Space and time adopt a flexible storage solution: HTAP (Hybrid transactional/analytic processing).

The validator monitors, commands, and validates the services provided by these clusters, then orchestrates the data flow and queries between end users and data warehouse clusters. Validators provide a means for data to enter a system (such as a blockchain index) and data to exit a system (such as a smart contract).

• Routing - supports transactional and query interactions with a network of decentralized data warehouses

• Streaming - acts as a sink for high-volume customer streaming (event-driven) workloads

• Consensus - Provides high-performance Byzantine Fault Tolerance for data entering and exiting the platform

• Query Proof - provide SQL proof to the platform

• Table Anchor - provide proof of storage to the platform by anchoring the table on the chain

• Oracle - supports Web3 interactions, including smart contract event listening and cross-chain messaging/relaying

• Security - preventing unauthenticated and unauthorized access to the platform

Space and Time as a platform is the world’s first decentralized data structure, opening up a powerful but underserved market: data sharing. Within the Space and Time platform, companies can share data freely and use smart contracts to trade the shared data. Additionally, datasets can be proof-of-sql monetized and aggregated without giving consumers access to the raw data. Data consumers can trust that aggregations are accurate without seeing the data itself, so data providers no longer have to be data consumers. It is for this reason that the combination of SQL proofs and data structure schemas has the potential to democratize data manipulation, as anyone can contribute to ingesting, transforming, and serving datasets.

Web3 Data Governance and Discovery

Currently, the Web3 data infrastructure lacks a practical and efficient data governance architecture. However, a practical and efficient data governance infrastructure is essential to configure the data elements of each participant’s relevant rights and interests.

• For the data source, it is necessary to have informed consent and the right to acquire, copy and transfer the data itself freely.

• For data processors, they need to have the right to self-control, use data and obtain benefits.

• For data derivatives, operating rights are required.

Currently, Web3 data governance capabilities are single, assets and data (including ceramics) can only be controlled by controlling private keys, and almost no hierarchical classification configuration capability exists. Recently, the innovative mechanisms of Tableland, FEVM, and Greenfield can achieve trustless data governance to a certain extent. Traditional data governance tools such as Collibra can only be used within the enterprise and only have platform-level trust. At the same time, non-decentralized technology also makes it impossible to prevent personal evil and single points of failure. Through data governance tools such as Tableland, the security technologies, standards, and solutions required for the data circulation process can be guaranteed.

Case: Tableland

Tableland Network is a decentralized web3 protocol for structured relational data, starting with Ethereum (EVM) and EVM-compatible L2. With Tableland, it is now possible to implement traditional web2 relational database functionality by leveraging the blockchain layer for access control. However, Tableland is not a new database - just web3 native relational tables. Tableland offers a new way to enable dapps to store relational data in web3-native networks without these tradeoffs.

Solutions

With Tableland, metadata can be mutated (with access controls if needed), queried (using familiar SQL), and composable (with other tables on Tableland) - all in a fully decentralized manner.

Tableland breaks down a traditional relational database into two main components: an on-chain registry with access control logic (ACL) and an off-chain (decentralized) table. Every table in Tableland is initially minted as an ERC721 token on a base EVM compatibility layer. Thus, on-chain table owners can set ACL permissions on the table, while off-chain, the Tableland network manages the creation and subsequent alteration of the table itself. Linking between on-chain and off-chain is handled at the contract level. It points to the Tableland network (using base URI + token URI, much like many existing ERC721 tokens that use IPFS gateways or escrow servers for metadata).

Only people with the appropriate on-chain privileges can write to a particular table. However, table reads do not have to be on-chain operations and can use the Tableland gateway; therefore, read queries are free and can come from simple front-end requests or other non-EVM blockchains. To use Tableland, a table must first be created (i.e., cast on the chain as ERC721). The deployment address is initially set to the table owner, and this owner can set permissions for any other users who try to interact with the table to make changes. For example, the owner can set rules for who can update/insert/delete values, what data they can change, and even decide if they want to transfer ownership of the other side of the table. In addition, more complex queries can join data from multiple tables (owned or unowned) to create a fully dynamic and composable relational data layer.

Consider the following diagram, which summarizes a new user’s interaction with a table that has been deployed to Tableland by some dapp:

Here is the overall flow of information:

1. A new user interacts with the dapp’s UI and tries to update some information stored in a Tableland table.

2. The dapp calls the Tableland registration smart contract to run this SQL statement, and this contract checks the dapp’s smart contract, which contains a custom ACL that defines the permissions of this new user.

There are a few points to note:

1. Custom ACLs in separate smart contracts for dapps are a completely optional but advanced use case; developers don’t need to implement custom ACLs and can use the default policy of the Tableland Registry smart contract (only the owner has full permissions).

2. You can also use Gateway to write queries instead of calling Tableland smart contracts directly. There will always be an option for dapps to call Tableland smart contracts directly, but any query can be sent through the Gateway, which will relay the query to the smart contract on a subsidized basis.

3. The Tableland smart contract takes the user’s SQL statements and permissions, incorporating these into emitted events that describe the SQL-based actions to be taken.

4. The Tableland Validator node listens to these events and then takes one of the following actions:

   1. If the user has the correct permissions to write to the table, the validator will run SQL commands accordingly (insert a new row into the table or update an existing value) and broadcast the confirmation data to the Tableland network.

   2. If the user does not have the correct permissions, the Validator will not perform any operations on the table.

   3. If the request is a simple read query, the corresponding data is returned;

   4. Tableland is a completely open relational data network where anyone can perform read-only queries on any table.

   5. Dapps can reflect any updates on the Tableland network via the Gateway.

(Usage scenario) What to avoid - Personally identifiable data - Tableland is an open network; anyone can read data from any table. Therefore, personal data should not be stored in Tableland.

• High-frequency, sub-second writes - e.g., high-frequency trading robots.

• Storing every user interaction in the application - keeping this data in Web3 tables, such as keystrokes or clicks, may not make sense. The frequency of writes can lead to high costs.

• Very large data sets - These should be avoided and are best handled through file storage, using solutions such as IPFS, Filecoin, or Arweave. However, pointers to these locations and associated metadata are actually a good use case for Tableland tables.

Thoughts on valuation capture

The different units have an irreplaceable role in the overall data infrastructure architecture, and the value of their value capture is mainly reflected in the market value/valuation and estimated earnings, which can be obtained from the following conclusions:

1. Data Source is the module with the largest value capture in the whole architecture

2. Data Replication, Transformation, Streaming, and Data Warehousing are the next most important

3. Analytics layer may have good cash flow, but there will be an upper limit to valuation

Companies/projects on the left of the structure chart tend to capture greater value.

Industry Concentration

According to incomplete statistical analysis from the companies in the graph above, industry concentration is judged as follows:

1. the highest industry concentration is data storage and data query and processing two modules

2. the industry concentration is medium is data extraction and conversion

3. the two modules with low industry concentration are data source, analysis, and output

The industry concentration of data sources, analysis, and output is low. The preliminary judgment is that different business scenarios lead to the emergence of vertical scenario leaders in each business scenario, such as Oracle in the database, Stripe in third-party services, Salesforce in enterprise services, Tableau in dashboard analysis, Sisense in embedded analysis, etc.

The reason for the moderate industry concentration of data extraction and conversion modules is initially judged to be due to the technology-oriented nature of the business attributes. The modular form of middleware also makes the switching cost relatively low.

The data storage and query and processing modules with the highest industry concentration are initially judged to be due to the single business scenario, high technical content, high start-up cost, and high cost of subsequent switching, which gives the company/project a strong first-mover advantage and network effect.

Thoughts on business models and exit paths for data protocols

Judging from the time of establishment and listing

• Most companies/projects established before 2010 are data source companies/projects. The mobile Internet has not yet emerged, and the data is not very large. There are also some data storage and analysis output projects, mainly dashboards.

• From 2010 to 2014, on the eve of the rise of the mobile Internet, data storage and query projects such as Snowflake and Databricks were born, data extraction and conversion projects also began to appear, and a set of mature big data management technology solutions gradually improved. There are a large number of analysis output projects, mainly dashboards.

• From 2015 to 2020, query and processing projects have sprung up, and many data extraction and conversion projects are also emerging so that people can better exert the power of big data.

• From 2020 onwards, newer real-time analytics databases and data lake solutions emerged, such as Clickhouse and Tabular.

Infrastructure improvement is the premise of the so-called “mass adoption”. During the period of large-scale application, there are still new opportunities. Still, these opportunities are almost only “middleware”, and the underlying data warehouse, data source, and other solutions are almost a winner-takes-all situation unless there is a technically substantial Sexual breakthrough. Otherwise, it will be difficult to grow up.

And analysis output projects are opportunities for entrepreneurial projects no matter what period they are in. But it is also constantly iteratively innovating and doing new things based on new scenarios. Tableau, which appeared before 2010, occupied most of the desktop dashboard analysis tools. New scenarios emerged later, such as more professional-oriented DS/ML tools, A more comprehensive data workstation, and a more SaaS-oriented embedded analysis.

Looking at the current data protocol of Web3 from this perspective:

• The data source and storage projects are not yet determined, but the leaders are emerging. The state storage on the chain is led by Ethereum (220 billion market value), while the decentralized storage is led by Filecoin (2.3 billion market value) and Arweave (280 million market value). There may be a sudden emergence of Greenfield. ——Highest value capture

• There is still room for innovation in data extraction and conversion projects. The data oracle machine Chainlink (market value of 3.8 billion) is just the beginning. Event streaming and stream processing infrastructure Ceramic and more projects will appear, but there is not much space. - Moderate value capture

• For query and processing projects, the Graph (with a market value of 1.2 billion) has been able to meet most of the needs, and the type and quantity of projects have not yet reached the explosive period. - Moderate value capture

• Data analysis projects, mainly Nansen and Dune (valued at 1 billion), need new scenarios for new opportunities. NFT Scan and NFT Go are similar to new scenarios, but they are only content updates, not analysis logic New requirements at the /paradigm level. — Moderate value capture, strong cash flow.

But Web3 is not a copy of Web2 nor a complete evolution of Web2. Web3 has very original missions and scenarios, thus giving birth to business scenarios that are completely different from before (the first three scenarios are all abstractions that can be made now).

Special thanks to Brian Retford, Sun Yi, Jason Morton, Shumo, Feng Boyuan, Daniel, Aaron Greenblatt, Nick Matthew, Baz, Marcin, and Brent for incredibly valuable insights, feedback, and review of this post.

For us fellow degens living under a crypto rock, AI has been on fire for a while. Fun story, no one wants to see an AI go rogue. Blockchain was invented to prevent the US dollar from going rogue, so we might just give it a shot. Also, we now have a new technology called ZK, which is used to ensure stuff doesn't go wrong. I just assumed that the average degen knows a slight bit about what blockchain and ZK are. However, to tame the AI beast, we must understand how AI works.

Part I: An Understandable Intro to ML

AI has gone through a few names from “expert systems“ to “neural nets“, then “graphical models“, and eventually “machine learning“. All of these are subsets of “AI“ which people give different names and we learn more about AI. Let's take a small dive into ML and unveil the mysteries of machine learning.

Note: most machine learning models today are neutral networks for their excellent performance for many tasks. We mainly refer to machine learning as neural networks machine learning.

How does machine learning work?

First, let's join me on a quick journey through the inner workings of machine learning：

1. Input data Preprocessing:

   The input data needs to be processed into a format that can be used as input to the model. This often involves preprocessing and feature engineering to extract useful information and transform the data into a suitable form, such as an input matrix or tensor (high dimensional matrix). This is the expert systems approach. As deep learning comes across, layers appear to take care of the pre-processing automatically.

2. Set up initial model parameters:

   Initial model parameters include several layers, activation functions, initial weights, biases, learning rate, etc. Some can be adjusted during training to improve the model's accuracy using optimization algorithms.

3. Train the data：

   1. The input is fed into the neural network, typically starting with one or more layers of feature extraction and relationship modeling, such as convolutional layers（CNN）, recurrent layers（RNN）, or self-attention layers. These layers learn to extract relevant features from the input data and model the relationships between these features.

   2. The output from these layers is then passed through one or more additional layers, which perform different computations and transformations on the input data. These layers typically mainly involve matrix multiplication with learnable weight matrices and the application of non-linear activation functions, but they could also include other operations, such as convolutions and pooling in convolutional neural networks or iterations in recurrent neural networks. The output of these layers serves as input to the next layer in the model or as the final output for prediction.

4. Get the output of the model:

   The output of neural network computation is usually a vector or matrix that represents the probability of image classification, sentiment analysis score, or other results, depending on the application of the network. There is usually another error evaluation & parameter update module which allows automatic updates to parameters depending on the purpose of the model.

If the above explanation seems too obscure, you can look at the following example of using a CNN model to recognize images of apples.

1. An image is loaded into the model as a matrix of pixel values. The matrix can be represented as a 3D tensor with dimensions (height, width, channels).

2. The initial parameters of the CNN model are set.

3. The input image is fed through multiple hidden layers in the CNN, where each layer applies convolutional filters to extract increasingly complex features from the image. The output of each layer is passed through a non-linear activation function and then pooled to reduce the dimensionality of the feature maps. The final layer is typically a fully connected layer that produces the output prediction based on the extracted features.

4. The final output of the CNN is the class with the highest probability. This is the predicted label for the input image.

The ML trust framework

We can then summarize the above into an ML trust framework, which includes four essential layers of ML that need to be trustworthy for the entire ML process to be reliable:

• Input: Raw data needs to be preprocessed, and sometimes it needs to be private.

  • Integrity: input data is not tampered with, contaminated with adversarial input, and is correctly pre-processed.

  • Privacy: input data is not leaked if desired.

• Output: Needs to be accurately generated and transmitted

  • Integrity: output is correctly generated.

  • Privacy: output is not leaked if desired.

• Model Type/Algo: models should be correctly computed

  • Integrity: model is executed correctly.

  • Privacy: the model itself or the computation is not leaked if desired.

  • Different neural network models have different algos and layers, catering to different use cases and inputs.

  • CNNs are commonly used for tasks involving grid-like data, such as images, where local patterns and features can be captured by applying convolutional operations to small input regions.

  • RNNs, on the other hand, are well-suited for sequential data such as time series or natural language, where the hidden states can capture information from previous time steps and model temporal dependencies.

  • Self-attention layers are useful for capturing relationships between elements in the input sequence, making them effective for tasks such as machine translation or summarization, where long-range dependencies are essential.

  • Several other model types exist, including Multilayer Perceptron (MLP), etc.

• Model parameters: parameters should be transparent or democratically generated in some cases but not easy to tamper with in all cases.

  • Integrity: parameters are generated, maintained, and governed in the correct manner.

  • Privacy: Model owners often keep machine learning model parameters confidential to protect the intellectual property and competitive advantage of the organization that developed the model. This is only prevalent until transformer models become crazy expensive to train but nonetheless a major issue for the industry.

Part II: The Trust Issue with ML

With the explosive growth of Machine Learning (ML) applications (with CAGR exceeding 20%) and their increasing integration into daily life, such as the recent popularity of ChatGPT, issues of trust in ML are becoming increasingly critical and cannot be ignored. Therefore, it is crucial to discover and address these trust issues to ensure the responsible use of AI and prevent its potential misuse. However, what exactly are these issues? Let's dive in.

Lack of transparency or provability

Trust issues have long plagued machine learning for two main reasons:

• Privacy nature: As mentioned above, the model parameters are usually private, and in some cases, the model input also needs to be private, which naturally brings some trust problems between the model owner and the model users.

• Blackbox of the algorithm: Machine learning models are sometimes referred to as 'black boxes' because they involve many automated steps in the computational process that can be difficult to understand or interpret. These steps involve complex algorithms and large amounts of data which bring indeterministic and sometimes random output, putting the algorithm in a place to be blamed for biases and even discrimination.

Before going more profound, the bigger assumption in this article is that the model is already "ready to use, " meaning it's well-trained and fit for purpose. The model might not fit all cases, and models improve at a staggering rate that the normal shelf life of an ML model is anywhere from 2 to 18 months, depending on application scenarios.

A detailed breakdown of ML trust issues

There are trust issues that underlie the model training process, and Gensyn is currently working on generating effective proof to contribute to this process. However, this article will mainly focus on the model inference process. Now let's use the four building blocks of ML to discover underlying trust issues:

• Input:

  • The data source is tamper-proof

  • Private input data is not stolen by the model operators (privacy problem)

• Model:

  • The model itself is accurate as advertised.

  • The computation process is done correctly.

• Parameters:

  • The parameters of the model are not changed or are as advertised.

  • Model parameters which are valuable assets to the model owner are not leaked during the process (privacy problem)

• Output:

  • Output is provably correct (may improve with all elements mentioned above)

How ZK applies to the ML trust framework

Some of the above trust issues can be solved simply by going on-chain; uploading the inputs and ML parameters on-chain and calculating the model on-chain can ensure the correctness of input, parameters and model computations. But this method might sacrifice scalability and privacy. Giza is doing this on Starknet, but due to the cost problem, it only supports simple machine learning models like regression and does not support neural networks. ZK technology can solve the above trust issues more efficiently. Currently, ZKML's ZK usually refers to zkSNARK. First, let's quickly go over some basics of zkSNARKs:

A zkSNARK proof is proof that I know some secret inputs w such that the result of this computation f is OUT is true without telling you what the w is. The proof generation process can be summed up into several steps:

1. Formulating a statement that needs to be proved: f(x,w)=true

   "I have correctly classified this image x using an ML model f with private parameters w."

2. Converting the statement into a circuit（Arithmetization）: different circuit construction approaches include the R1CS, QAP, Plonkish, etc.

   ZKML needs an additional step called quantization compared to other use cases. Neural network inference is typically done in floating-point arithmetic, which is extremely expensive to emulate in the prime field of arithmetic circuits. Different quantization methods are a trade-off between accuracy and equipment requirements.

   Some circuit construction methods like R1CS are not efficient for neural networks. This part can be adjusted to improve performance.

3. Generating a proving key and a verification key

4. Creating a witness: when w=w*,f(x,w)=true

5. Creating a hash commitment: The witness w* is committed to using a cryptographic hash function to generate a hash value. This hash value can then be made public.

   It helps ensure the private inputs or model parameters have not been tampered with or modified during the computation process. This step is essential as even minor modifications can have significant impacts on the model's behavior and output.

6. Generating proof: Different proof systems use different proof generation algorithms.

   1. Special zero-knowledge rules need to be designed for machine learning operations, such as matrix multiplication and convolutional layers, allowing for efficient protocols with sub-linear time for these computations.

      1. The general zkSNARK systems like groth16 might not be able to process neural networks efficiently as the calculation workload is too much.

      2. Since 2020, many new ZK proof systems have emerged to optimize the ZK proof for the model inference process, including vCNN, ZEN, ZKCNN, and pvCNN. However, most of them are optimized for CNN models. They can only be applied to some primary datasets, such as MNIST or CIFAR-10.

      3. In 2022, Daniel Kang Tatsunori Hashimoto, Ion Stoica, and Yi Sun(founder of Axiom) proposed a new proof scheme based on Halo2, which first achieved ZK proof generation for the ImageNet dataset. Their optimization mainly falls on the arithmetization part, with novel lookup arguments for non-linearities and the reuse of sub-circuits across layers.

      4. Modulus Labs is benchmarking different proof systems for on-chain inference and finds that in terms of proof time, ZKCNN and plonky2 perform the best; in terms of peak prover memory usage, ZKCNN and halo2 perform well; while plonky, although performing well, sacrifices memory consumption and ZKCNN is only usable for CNN models. It is also developing a new zkSNARK system, especially for ZKML with a new virtual machine.

7. Verifying the proof: The verifier uses the verification key to verify without requiring the knowledge of the witness.

We can therefore demonstrate that applying zero-knowledge techniques to a machine-learning model can solve a lot of its trust issues. Similar techniques that use interactive verification can achieve similar results but will require more resources on the verifier side and potentially face more privacy issues. It is worth noting that depending on the exact model, generating proofs for them can be time and resource-consuming, so there will be compromises in various aspects when this technique is eventually implemented in real-world use cases.

Current Landscape of the solutions

Next, what's on the table? Bear in mind that there are many reasons why model providers may not want to generate a ZKML proof. For those who are brave enough to try ZKML and when the solutions make sense to be implemented, they can choose from a few different solutions according to where their models and inputs lie:

• If the input data is on-chain, Axiom could be considered as a solution:

  Axiom is building a ZK co-processor for Ethereum to improve user access to blockchain data and provide more sophisticated views of on-chain data. It is feasible to do reliable machine learning computation on on-chain data:

  • First, Axiom imports on-chain data by storing Merkle roots of Ethereum block hashes in its smart contract AxiomV0, which are trustlessly verified through a ZK-SNARK verification process. The AxiomV0StoragePf contract then allows for batch verification of arbitrary historic Ethereum storage proofs against the root of trust given by block hashes cached in AxiomV0.

  • Next, the ML input data can be pulled from imported historical data.

  • Then Axiom can apply verified machine learning operations on top; The validity of each piece of computing is verified using optimized halo2 as the backend.

  • Finally, Axiom accompanies the result of each query with the zk proof, and the Axiom smart contract would verify the zk proof. Any related parties who want the proof can access it from the smart contract.

• If the model is put on-chain, RISC Zero could be considered as a solution:

  The RISC Zero ZKVM is a RISC-V virtual machine that produces zero-knowledge proofs of the code it executes. Using the ZKVM, a cryptographic receipt is produced, which anyone can verify was produced by the ZKVM's guest code. No additional information about code execution (such as, for example, the inputs provided) is revealed by publishing the receipt.

  • By running a machine learning model in RISC Zero's ZKVM, it can be proven that the exact computation involved in the model is carried out. The computation and the verification process can be done either off-chain in a user's preferred environment, or in the Bonsai Network, which is a universal roll-up.

    • First, the model's source code would need to be compiled into a RISC-V binary file. When this binary file is executed inside the ZKVM, the output is paired with a computational receipt that contains a cryptographic seal. This seal serves as a zero-knowledge argument of computational integrity and links the cryptographic imageID (which identifies the RISC-V binary file that was executed) to the asserted code output in a way that third parties can quickly verify.

    • When the model executes inside the ZKVM, the computation about state change is done entirely within the VM. It does not leak any information about the model's internal state to external parties.

    • Once the model has finished executing, the resultant seal serves as a zero-knowledge proof of computational integrity.

  • The exact process of generating the ZK proof involves an interactive protocol with a random oracle as the verifier. The seal on a RISC Zero receipt is essentially the transcript of this interactive protocol.

• If you want to directly import a model from commonly-used ML software like Tensorflow or Pytorch, ezkl could be considered as a solution:

  • Ezkl is a library and command-line tool for making inferences for deep learning models and other computational graphs in a zkSNARK.

    • First, export the final model as a .onnx file and some sample inputs to a .json file.

    • Then, point ezkl to the .onnx and .json files to generate a ZK-SNARK circuit with which you can prove ZKML statements.

  • Looks simple, right? It is ezkl's goal to offer an abstraction layer that allows for higher-level operations to be called and laid out in a Halo 2 circuit. Ezkl abstracts away a lot of complexity while staying incredibly flexible. Their quantization model has a scale factor for automated quantization. They support a flexible change to other proving systems as new solutions emerge. They also support several types of virtual machines, including EVM and WASM.

  • Regarding the proving system, ezkl customs halo2 circuits through aggregation proofs (transform hard-to-verify into easy-to-verify through an intermediary) and recursion(can solve the memory problem, but hard to fit in halo2). Ezkl also optimizes the whole process using fusion and abstraction (can reduce overhead through high-level proof) 

• It is also worth noticing that, compared to other general-purpose zkml projects, Accessor Labs focuses on providing zkml tooling specially designed for fully onchain gaming, and might involve AI NPC, automatic updates of gameplay, game interface involving natural language, etc.

Part III: Where is the Use Case?

Solving ML's trust issues with ZK technology means it can now be applied to many more "high-stakes" and "highly deterministic" use cases rather than just keeping up with people's conversations or telling the pictures of cats apart from pictures of dogs. Web3 is already exploring a lot of these use cases. This is no coincidence as most Web3 applications run or intend to run on a blockchain because of the specific nature of a blockchain that can run securely, is difficult to tamper with, and has deterministic computations. A verifiably well-behaving AI should be an AI capable of conducting activities in a trustless and decentralized environment, right?

Web3 use cases where ZK+ML works

Many Web3 applications have been sacrificing user experiences for the sake of security & decentralization since that's obviously their priority, and limitations of infrastructures also exist. AI/ML has the potential to enrich user experiences, which will certainly help but previously seemed impossible without compromise. Now, thanks to ZK, we can comfortably see the marriage of AI/ML with Web3 applications without too much sacrifice on security & decentralization.

Essentially, it would be a Web3 application (which may or may not exist when writing this article) implementing ML/AI in a trustless way. By a trustless manner, we mean whether it's operated in a trustless environment/platform or its operation is provably verifiable. Note that not all ML/AI use cases (even in Web3) are needed or preferred to run in a trustless way. We will analyze each part of the ML functionality used in various Web3 fields. Then, we will identify the parts that require ZKML, usually the high-value parts that people are willing to pay extra money for proof.

Most use cases/applications mentioned below are still in the experimental research phase. Hence, they are still far from actual adoption. We will discuss why later.

Defi

Defi is one of the few demonstrated product market fit for blockchain protocols and Web3 applications. Being able to create, store and manage wealth & capital in a permissionless way is unprecedented in human history. We have identified many use cases where an AI/ML model would need to operate permissionless to ensure security & decentralization.

• Risk Assessment: Modern finance requires AI/ML models for all kinds of risk assessment, from preventing fraud and money laundering to giving out uncollateralized loans. Ensuring this AI/ML model functions in a verifiable way means we can prevent them from being manipulated into censorship, which hinders the permissionless nature of using Defi products.

• Asset Management: automated trading strategy has not been new for Tradfi and Defi. Efforts to apply AI/ML-generated trading strategies have been made, but only a few decentralized ones have been successful. Current typical applications of the defi sector include the rocky bot experimented with by Modulus Labs.

  • The Rocky Bot： Modulus Labs created a trading bot on StarkNet using AI for decision-making.

    • An L1 contract that holds funds and exchanges WEth / USDC on Uniswap.

      • This fits into the "output" part of the ML trust framework. The output is generated on an L2, transmitted to an L1, and used for execution. During the process, it's not to be tampered with.

    • An L2 contract implements a simple (but flexible) 3-layer neural network to predict future WEth prices. The contract uses historical WETH price information as input.

      • This fits into the "input" and the "model" parts. The historical price information input comes from the blockchain. The model's execution is calculated in CairoVM, a ZKVM whose execution trace will generate a ZK proof for verification.

    • A simple frontend for visualization and PyTorch code for training both regressors and classifiers.

• Automated MM and liquidity provision: this is in essence a combination of similar efforts conducted in risk assessment and asset management, just done in a different manner when it comes to volume, timeline, and asset types. There have been numerous papers on how ML can be used in market making in the equities market. It might only be a matter of time before some of them are applicable to Defi products.

  • For example, Lyra Finance is working with Modulus Labs to enhance their AMM with intelligent features to become more capital efficient.

• Honorable mentions:

  • Warp.cc team has developed a tutorial project on how to deploy a smart contract that runs a trained neural network to predict Bitcoin's price. This falls into the "input" and "model" parts of our framework as the input is fed with the RedStone Oracles feed, and the model is executed as a Warp smart contract on Arweave.

  • It’s first iteration and ZK is involved so it falls into our honorable mention, but in the future Warp team considers implementing a ZK piece.

Gaming

Gaming intersects with machine learning a lot:

The grey area in the figure represents our initial assessment of whether the ML functionality in the gaming section needs to be paired with corresponding ZKML proofs. Leela Chess Zero is a very interesting example of applying ZKML to gaming:

• AI Agents

  • Leela Chess Zero (LC0): a fully on-chain AI chess player built by Modulus Labs, playing against a collective of human players from the community.

    • LC0 and the human collective take turns playing (as it should be in chess).

    • LC0's move is calculated using a simplified, circuit-friendly LC0 model.

  • LC0's move has a Halo2 snark proof generated to ensure no human mastermind intervention. Only the simplified LC0 model is there to make the decision.

  • This fits into the "model" part. The model's execution has a ZK-proof to verify that the computation is not tampered with.

• Data Analysis & Prediction: this has been a common use of AI/ML in the Web2 gaming world. However, we find there to be very few reasons to implement ZK into this ML process. For the sake of not having too much value directly involved in the process, it might not be worth the effort. However, if some analytics & predictions are used to determine rewards for users, ZK might be implemented to ensure the results are correct.

• Honorable Mentions:

  • AI Arena is an Ethereum native game where players all over the world can design, train and battle NFT characters powered by artificial neural networks. Talented researchers from around the world compete to create the best Machine Learning (ML) model to battle in the game. AI Arena focuses on feedforward neural networks. In general, they have lower computational overhead than convolutional neural networks (CNNs) or recurrent neural networks (RNNs). Though, for now, models are only uploaded to the platform after it's trained, hence it's honorably mentioned.

  • GiroGiro.AI is building an AI toolkit that enables the masses to create artificial intelligence for personal or commercial use. Users can create various sorts of AI systems based on an intuitive and automated AI workflow platform. Only with a few inputs of data and a choice of algorithms (or models for refinement), users will generate and harness the AI model in mind. Though the project is in a very early stage, we are excitedly anticipating to see what GiroGiro can bring onto the table due to its focus on the gamefi and metaverse-focused products, hence its presence in honorable mentions.

DID and Social

In the realm of DID&social, web3 and ml’s intersection currently mainly lies in the proof of humanity and proof of credentials field; other parts might evolve but will take longer.

• Proof of humanity

  • Worldcoin uses a device called the Orb to determine if someone is an actual living person that is not attempting to defraud the verification. It does this with a variety of camera sensors and machine-learning models that analyze facial and iris features. Once that determination is made, the Orb takes a set of pictures of the person’s irises and uses several machine learning models and other computer vision techniques to create an iris code, which is a numerical representation of the most important features of an individual’s iris pattern. Specific steps to sign up are as follows:

    • The user generates a Semaphore keypair on her phone and presents the hashed public key (via QR code) to the Orb.

    • The Orb scans the user’s irises and locally computes the user’s IrisHash. It then sends a signed message containing the hashed public key and the IrisHash to the sign-up sequencer node.

    • The sequencer node verifies the Orb’s signature, then checks if the IrisHash does not match any already in the database. If the uniqueness checks passes, the IrisHash and the public key are saved.

  • Worldcoin uses the open-source Semaphore zero-knowledge proof system to transfer the uniqueness of IrisHashes to the uniqueness of user accounts, without ever linking them. This ensures a newly signed-up user can successfully claim his/her WorldCoins. The steps are as follows:

    • The user’s app locally generates a wallet address.

    • The app uses Semaphore to prove that it owns the private counterpart to one public key registered previously. Because it’s zero-knowledge proof, it does not reveal which public key.

    • The proof is sent again to the sequencer, which verifies it and initiates the deposit of tokens to the provided wallet address. A so-called nullifier is sent along with the proof and ensures the user cannot claim the reward twice.

• WorldCoin uses ZK technology to ensure the output of its ML models does not reveal the user's personal data, by not having them associated with each other. In this case, it falls in the "output" section of our trust framework as it's ensured that the output is transmitted and used in the desired way, in this case privately.

• Proof of action

  • Astraly is a reputation-based token distribution platform built on StarkNet for finding and supporting the latest and greatest StarkNet projects. Measuring reputation is a challenging task, as it is an abstract concept that cannot be easily quantified with simple metrics. When dealing with complex metrics, it is often the case that more comprehensive and diverse inputs yield better results. That’s why Astraly is seeking the help of modulus labs to use ML model to provide more accurate reputation ratings.

• Personalized recommendations and content filtering

  • Twitter recently open-sourced their algorithm for the "For You" timeline, but users cannot verify if the algorithm is running correctly as the weights for the ML models used to rank tweets are kept private. This has resulted in concerns about bias and censorship.

  • However, Daniel Kang, Edward Gan, Ion Stoica, and Yi Sun offer a solution using ezkl to help balance privacy and transparency by producing proof that the Twitter algorithm ran honestly without revealing the model weights. By using the ZKML framework, Twitter can commit to a specific version of its ranking model and publish proof that it produces the specific final output ranking for a given user and tweet. This solution enables users to verify that the computations were performed correctly without requiring trust in the system. While there is still work to be done to make ZKML more practical, this is a positive step toward improving transparency in social media. Thus this falls into our ML trust framework's "model" part.

Revisiting the ML trust framework from a use case perspective

It can be seen that the potential use cases for ZKML in web3 are still in their infancy but can not be ignored; in the future, as the use of ZKML continues to expand, there may be a demand for ZKML providers, forming the closed loop in the following figure:

ZKML service providers largely focus on the "model" and "parameter" part of the ML trust framework. Though most we see nowadays are more "model" related than "parameters". Note that the "input" and "output" parts are more addressed by blockchain-based solutions, either used as data sources or data destinations. ZK or blockchain alone might not achieve full trustworthiness, but probably united they do.

How far is mass adoption?

Finally, we can look at the current feasibility status of ZKML and how far we are from a large-scale application of ZKML.

Modulus Labs's paper has given us some data and insights into the feasibility of ZKML applications by testing Worldcoin (with strict precision and memory requirements) and AI Arena (with cost-effectiveness and time requirements):

If Worldcon uses ZKML, the prover's memory consumption will overwhelm any commercially available mobile hardware. If AI Arena's tournament uses ZKML, using ZKCNNs would increase the time and cost to 100x (0.6s vs original 0.008s). So sadly, both are not feasible for directly applying ZKML technology to prove time and prover memory usage.

What about the proof size and verification time? We can refer to Daniel Kang, Tatsunori Hashimoto, Ion Stoica, and Yi Sun's paper. As shown below, their DNN inference solution can achieve up to 79% accuracy on ImageNet（ model type: DCNN,16 layers,3.4 million parameters ） while simultaneously taking as few as 10s and 5952 bytes to verify. Furthermore, the zkSNARKs can be scaled down to take as few as 0.7s to verify at 59% accuracy. These results show the feasibility of zkSNARKing ImageNet-scale models in terms of proof size and verification time.

The main technical bottleneck now lies in proof of time and memory consumption. It is still not technically feasible to apply ZKML in web3 cases. Can ZKML potentially catch the development of AI? We can compare several empirical data:

• The development speed of ML models: the GPT-1 model released in 2019 had 150 million parameters, whereas the most recent GPT-3 model released in 2020 has 175 billion parameters, representing a 1,166x increase in the number of parameters in just two years.

• The optimization speed of the ZK system: The performance growth of ZK systems basically follows "Moore's Law"-like paces. New ZK systems come out almost yearly, and we expect prover performance's rocket growth to continue for a while.

The outlook is not very optimistic when comparing the improvement rate of cutting-edge ML to ZK. However, with the continuous improvement of rollup performance, ZK hardware, and the tailor-made ZK proving systems based on highly structured neural network operations, hopefully, the development of ZKML can satisfy web3 needs and start by providing some old-fashioned machine learning functions first.

Though we might have a hard time using blockchain + ZK to verify that the information ChatGPT feeds me is trustworthy, we might be able to fit some smaller and older ML models into ZK circuits.

Part IV: Conclusion

"Power tends to corrupt, and absolute power corrupts absolutely". With the incredible power of AI & ML, there is no surefire way to put it under governance for now. The governments have repeatedly proven to provide either late interference for aftermaths or early outright ban. Blockchain + ZK delivers one of the few solutions that can tame the beast in a provable and verifiable way.

We look forward to seeing more product innovation in the field of ZKML, where ZK and blockchain provide a secure and trustworthy environment for AI/ML to run. We also expect brand new business models to arise from those product innovations as we are not constrained by the go-to SaaS commercialization model here in the permissionless world of crypto. We look forward to supporting more builders to come and build out their exciting ideas in this fascinating overlap of the "wild west anarchies" and "ivory tower elites".

We are still early, but we might save the world on the way.

Reference

• https://a16zcrypto.com/content/article/checks-and-balances-machine-learning-and-zero-knowledge-proofs/

• https://arxiv.org/abs/2210.08674

• https://axiom.xyz/blog/intro

• https://cloud.google.com/learn/artificial-intelligence-vs-machine-learning#:\~:text=Differences%20between%20AI%20and%20ML,-Now%20that%20you&text=While%20artificial%20intelligence%20encompasses%20the,accurate%20results%20by%20identifying%20patterns.

• https://docs.aiarena.io/research-competition/neural-networks

• https://docs.google.com/presentation/d/1LwfJUD5osS2lAtnnZVuKn7OuiRlIxsiTpfsb2rUjyp4/edit#slide=id.g164009b3422_0_383

• https://docs.google.com/presentation/d/1LwfJUD5osS2lAtnnZVuKn7OuiRlIxsiTpfsb2rUjyp4/edit#slide=id.g164009b3422_0_383 https://docs.warp.cc/tutorials/ml/prologue

• https://drive.google.com/file/d/1tylpowpaqcOhKQtYolPlqvx6R2Gv4IzE/view

• https://eprint.iacr.org/2021/673

• https://girogiro-ai.gitbook.io/litepaper/technology/gamma-engine-intro

• https://github.com/Modulus-Labs/RockyBot

• https://github.com/zkonduit/ezkl

• https://medium.com/@danieldkang/trustless-verification-of-machine-learning-6f648fd8ba88