Web3 algorithms are still nascent, but we’re close to a tipping point due to a combination of:

1. Bigger Blockchains: The blockchain data behemoth is growing exponentially due to transaction capacity scaling and the number of layer 2 chains launched.

2. Modular Evolution: Web3 protocols are being optimized/simplified while actually emphasizing modular composability. That means anything that was the protocol team’s choice is now up to individual communities - everything from pricing curves, token and voting permissions, and customizable and conditional token marketplaces.

3. Hyperbundling: With increased composability comes heavy aggregation - across defi, NFTs, and DAOs. It should be no surprise that Uniswap and Opensea are already heavily focused here.

4. Maturing Data Infra: Everyone and their grandma knows how to do auto-generated decoded tables already. Data providers like Dune are maturing to compete at the aggregated tables level instead, across many chains.

All this means more data, more cross-app and cross-domain intricacies, and more powerful ways to index and analyze data. It’s inevitable that we’ll see this data leveraged to create curated and tiered experiences for communities and users alike.

Currently, web3 algorithms focus on calculating scores targeting different facets of a user’s profile (using aggregate activity across multiple chains/apps). Protocol teams and communities can then decide how to use those scores to enhance their product experience, governance processes, or tokenomics.

In this article, I’ll cover a framework for comparing scores and understanding the interlinked data stacks behind them.

⚠️ What I’m covering is separate from token-based reputation score systems. While they are related, I’m not interested in exploring tokenomics in this article. Most of the scores here don’t directly translate to some balance or distribution of tokens. I’ll have a follow on article at some point discussing how to possibly model tokenomics with various kinds of categorical scores, but I believe trying too hard to attach them at the start will lead to perverse incentives and difficulty growing organically.

Modeling the Web3 Score Landscape

Let’s start by defining some of the current main categories for user scores:

• Creators/Curators: Scoring someone’s ability to create or curate content

• DeFi/Credit Management: Scoring someone’s ability to deploy and manage capital

• Contributors: Scoring someone’s contributions to a protocol/DAO and their skill set

• Rule-based: Scoring someone based on a defined set of actions taken

• Sybil (Identity): Scoring someone based on how likely they are to be human

We’ll likely see Sybil scores that combine with credit scores for enhanced access in DeFi, or contributor scores that combine with rule-based scores for prioritized onboarding to DAOs. However, scores can differ a lot even within the same category, so let’s compare them on two axes:

• Contextual <> Generalized: how generalizable is a score across the communities its used in? The more contextual the score is, the more it doesn’t translate well across communities or applications. Generalized scores are good for comparisons across communities and protocols, while contextual scores are better for analyzing health over time granularly.

• Elo Scores <> Levels: is the score constantly adjusted and normalized, or is it a number that only goes up? Elo scores mean your score is more dependent/ranked against the actions of others in the ecosystem. For the most part, levels will be more beginner friendly and better understood than elo scores.

Here are some examples of scores from the real world, placed in our four quadrants:

• Contextual + Elo Score: Your competitive ranking in a game like Chess or League of Legends

• Contextual + Level: Your level in a rewards program, like Citibike Bike Angels

• Generalized + Elo Score: Your socioeconomic status (combination of education, income, health)

• Generalized + Level: Your years of education/degrees earned

Here’s where some of the top web3 scoring systems/protocols sit along the axes:

I’ll briefly describe what each score provider does, hopefully explaining their placement. Note that I placed each provider based on how I interpreted their main offerings (as of September 1st, 2022):

• Creators/Curators:

  • yup.io: Action adapters that are combined in a weighted score.

  • $WRITE airdrop: Betweenness centrality algorithm to give you a weighted score of your Mirror activity.

  • jokedao: Round-based voting on best jokes.

• DeFi/Credit:

  • MACRO: Creates a market conditions time-weighted score of your defi actions.

  • ARCx: A gamified and clearly defined three-part defi scoring system.

  • Degenscore: Ranks how much of a degen you are compared to other protocol users with a score and percentile format

• Contributors:

  • layer3xyz: Experience earned by fulfilling bounties and winning contests.

  • 0xStation: DAO passports where defined actions become a part of a searchable system of records (slide 10).

  • metropolis: Building generalized identity around gnosis safes to build more context-rich data around on-chain group interactions.

  • Coordinape: Your fellow contributors vote on your level of contribution per epoch (interval of time).

• Rule-based:

  • Rabbithole: Custom skill trees with specific completable actions.

  • Metagame: Activity logs of various categories of (decoded) transaction types.

• Sybil:

  • BrightID: A vote-in system where you get a fluctuating health score based on the SybilRank of the network closest to you

  • PoH: Also a vote-in system, but with deposit minimums. You’re either in the network or you’re out (challenged/voted out). Union kind of builds off of this.

  • Worldcoin: Scans your eye to give you a private key. Don’t lose either one!

Right now we’re mostly seeing scores used in tiered protocol access and leaderboards/ranked feeds, but more powerful curation algorithms are right around the corner. To understand how they’ll emerge, we need to take a look at both the score data stack and score aggregators.

From data provider to score provider - then back

Below is my simplified interpretation of the data stack used by score providers - I’ll walk through it from the bottom to the top.

Sourcing Data:

While you can still build your own events/transactions indexer off of node providers like Alchemy, a few web3 product teams are open-sourcing their decoding indexers (zora, gallery). This trend will continue for most standard contract types, such that direct JSON RPC calls get abstracted more and more.

For nicely aggregated data, you’d likely use the Graph (for mapped graphQL subscriptions) or Dune Analytics (for cross-protocol/chain abstractions).

Feature Engineering:

Unlike data mappers such as Dune that capture every on-chain data point since genesis, score providers usually ingest just a subset of data depending on the protocols they’re interested in. From there, we have three steps of feature engineering:

1. Splitting actions into binary and continuous components, such as “borrowed USDC” versus “health factor of loan over time”.

2. Generalize variables (usually complex structs) across protocols of the same type; for example, loan health factors across DeFi protocols must be generalized based on respective risk parameters.

3. Wallet normalization means choosing a score or level distribution you want wallets to fit into. Binary variables can be used to boost or decay scores for more custom distributions.

There are more steps if you consider including external datasets that are already preprocessed, but I’ve left that out of the diagram.

Model:

While inputs/outputs are much more transparent in web3, some models are still privately developed right now (either for IP or Sybil/gamification reasons). I don’t have much to say on the models themselves since we’re still using the same web2 academic algorithms and libraries, however, I do believe that open-source models will win in the long run.

At the end of the day, everyone needs to be able to trust and interpret these scores - especially if there’s any sort of tokenomics/value involved. You need both builder and consumer communities to buy into using your scores - there’s no way for you to just force it upon them (yet, and hopefully never). Once you generate that trust, then you’ll have an extremely strong flywheel of more applications, more data, more builders, and more users.

Most people already understand that it’s not the technology that’s special or proprietary in web3 - it’s the community. It doesn’t matter how strong your team is, you won’t win by keeping things private.

Service:

On top of their basic score offerings, score providers also act as special data providers. All their engineered features can be re-used as a tiered data service similar to hugging face NLP libraries.

There are three main data consumers:

• Apps: Some products/protocols/communities will integrate directly with base scores, most likely the contextual scores (especially contributor scores). Features within the model will likely be useful for various UI elements and forums/discords.

• Aggregators: Improving the developer experience for integrations is a no-brainer. Some examples out currently are:

  • Chainlink flux aggregator: for on-chain score deployment, you might want to take a weighted Sybil score from multiple sources.

  • Gitcoin passport: for social apps, you’ll likely want easy access to multiple types of scores to enhance user profiles. Ceramic fits here too.

  • Orange Protocol: for complex protocols (i.e. defi) you might need an ensemble model approach where you have complete control of inputs, models, and output variability.

  My guess is that aggregators will probably end up having larger and larger says in the governance and tech stack of score providers over time.

• Analytics Platforms: Wallet labeling has already proven extremely powerful for various analytics use cases. Having scores as joinable tables would give us completely new ways of measuring adoption/market share, user segments, and product-community-fit across protocols.

I believe that the more generalizable the score, the more valuable the final outputs are (compared to the features). Vice versa, the more contextual the score, the more valuable the engineered features are. This should be intuitive given that a community would likely have more use for contextual features in their forums and membership processes than something general like who voted for who in a Sybil network.

Concluding thoughts on the future of scores

There’s a ton we can extrapolate about where scores are heading and what they’ll be used for. I’m personally excited about how score providers will enhance web3 native profile search and contributor relationship management systems (web3 CRM).

Transaction hashes and addresses give us strong anchors for finding relationships between users, but individual activity isn’t easily searchable. For example, you can see an address has some number of tokens and transactions, but you can’t search for their first ENS registered, first liquidation, or when they fulfilled their first bounty. These require the data to be more semantically layered, which is what score providers do when they engineer features into binary/continuous variables most relevant to their models. A transaction that’s just a USDC transfer would now have standardized metadata around what grant proposal it was tied to, and a transaction showing an ETH deposit into Compound is now classified as “avoiding liquidation”. It might not seem like much on its own, but combining all the features across score providers gives us the ability to do a top-down search starting from some “category” to get a “proof of action” - rather than trying to filter bottoms-up manually for specific function calls or transactions.

Combing scores allow us to filter through large clusters of users and create flexible metrics for measuring community health. Each segment within a web3 community will have a sliding scale for understanding underlying users and their activity. Younger communities would likely check generalized + level scores more often to see if overall health is going up compared to others, while mature communities would rely more on contextual + elo scores to track the pulse and retention of top members. You could prioritize your onboarding funnel for voters by layering Sybil scores on top of historical governance activity (proposals, delegations, votes), or for builders by layering contribution levels on top of their historical compensation distributions (payment rate, token diversity). I’m sure there are many teams working on contributor relationship management systems, where contributor activity is layered on top of score-based filtering to build out a more accurate understanding of which contributors (old, new, or tangential) are most active and likely to be interested in taking on a new project.

Both activity search and score combinations go towards scaling membership - an article I have planned for later this year! As always, thanks for reading this far - if you enjoyed this piece, please collect and subscribe for more 🙂

My DMs are always open if you have questions or want to discuss anything I’ve written here (especially if you’re currently building or researching this topic).

*thanks to Serena for her feedback on this article! *😊

In these guides, I’ll cover protocols in three sections:

1. Protocol overview and high-level statistics

2. Main protocol interactions and how to derive the important data points from them

3. Some of the more complex queries and abstractions created

All data is sourced from Dune Analytics, a platform for building cross-chain queries and dashboards - most of which is open source.

I’ve also recorded a video walk through everything mentioned in this article:

If you collect this entry, you can get access to my web3 data degens telegram group by clicking here! We discuss the newest protocols, debug general query questions, and help analysts find jobs (freelance and fulltime).

collect://

A big shoutout to Jeff, Meghan, Rob, hildobby, and Danning for their help! 😊

Protocol Overview:

Sudoswap NFT AMM is a flexible liquidity pool solution for NFT (ERC721s) to token (ETH or ERC20) swaps using customizable bonding curves. This basically means you set discrete price changes per swap (i.e. 0.1 ETH price increase for each NFT sold), and you choose the side of liquidity you want to provide (NFTs to sell, or ETH for others to sell their NFTs into). Everyone’s pools are separate and only aggregated at a router level - kind of like Uniswap V3.

Over the last month, the protocol has generated a lot of momentum (some of which is wash trading, but consistently around 500 ETH a day of swaps is organic volume.

ilemi

@andrewhong5297

@sudoswap has passed 10k ETH in volume!

Some other impressive stats:
- 1k+ ETH daily volume
- 10k traders
- 66k NFTs exchanged
- 1.5k ETH and 63k NFTs are currently liquid in pairs, across 2.7k NFT collections
- 50 ETH in protocol fees, 300 ETH in pair owner fees

112

12:46 AM • Aug 18, 2022

They’re sure to take off more as ERC20 pairs and ERC1155 pairs are launched, especially once aggregators integrate their markets.

I’ve created two dashboards on Dune for Sudoswap:

1. A general overview of swap volumes, trader activity, fees, and trending collections.

2. A collection-specific view that dives deeper into liquidity depth, trading prices, and pair activity on top of the basic metrics.

You can read (or listen) about why this design is better than other NFT AMMs out there, or go directly to the docs. For those wanting to gain a deeper understanding of the Solidity behind the protocol, I recommend you read the Protocol Review piece here (coming soon!).

Data Sourcing Guide

Usually, there are a few elements that factor into the difficulty of data analysis:

• Complexity of contract patterns (standards like proxies, clones, diamonds)

• Complexity of mid-function states (if there’s a swap, how easy is it to calculate the price)

• Completeness of data across function parameters and event parameters

• Number of token interactions (one token in, one in one out, wrapping, bridging)

• How much math (interest rates, pricing curves, rebases, and anything that updates continuously even if no on-chain interactions occur)

• Documentation and code/developer accessibility (how quickly can I find answers to questions)

I won’t go into a complete scorecard in this guide, but will probably build up to one in the future. Usually, when I start my analysis, I figure out the main user interactions and then list out all the variables I can get about each one.

Below is my breakdown of interactions for Sudoswap. My DMs are always open for feedback on what could make this more clear/informative!

Some contextual callouts that don’t fit in the diagram:

Contract patterns:

While the basic contract pattern is simple (factory produces a pair as a proxy clone, only the pair creator has ownership of tokens in that pair), there are actually many different variations of the pair contract based on the type of ERC721 and ERC20/ETH token.

They all inherit from LSSVMPair.sol, and all the functions/swaps we care about are on that contract. To make our lives easier in Dune, we want all data to get decoded to the same contract table instead of seven different ones we then have to UNION over. I did this by compiling LSSVMPair.sol using hardhat, and pasting the ABI artifact in the contract decode request field under the name LSSVMPair_General.

Prices and parameters:

Spot price is set at the pair deployment, after each swap, and can also be set manually. The same SpotPriceUpdated() event is used for swaps and manual changes, but not emitted on deployment. This means we have to do an extra join when getting price histories, and also an extra join if we want to remove custom spot price changes from swaps data. It would be nice if there was a SpotPriceChanged() event for deploy and manual changes, so the data is cleaner. Same feedback for price delta.

Events, events, events:

My main gripe is that the router has 0 events for swaps, and the pair has empty events for swapping in and out. Including spotPriceOld, spotPriceNew, protocolFee, ownerFee, and tokenIds transferred in the swap events would have saved a lot of time and warehouse compute.

Thankfully Sudoswap has great docs and 0xmons was very responsive, so I was able to eventually piece together the data by manually going through internal traces for each transaction.

Internal traces are the function calls that happen within the main function call. In this case, the pair contract calls the NFT contract to transfer the NFT to the pair, then the pair contract makes an opcode call to transfer ETH from the contract to your wallet to complete the swap. Get price info and other read function calls also show up in internal txs as long as its a call across contracts (internal functions won’t show up in traces).

The best way to visualize this is using an explorer like Versatile blocksec, which visualizes transactions really well:

Dune v2 engine tables include the call_trace_address for all decoded functions, which means we can join with ethereum.traces and then filter for ETH calls (look at call value) and safeTransferFrom() call (0x42842e0e funcsig) within that subtrace stack. Then add on some simple math, and we can back into protocol fees and owner fees paid per swap.

You can find the full query below, which has some extra logic to fit into the nft.trades schema.

You can get started querying these tables yourself using any of the sudo_amm_ethereum tables or the abstraction sudoswap_ethereum.events , both on the v2 Dune engine!

Creating the Spellbook Abstraction

Dune’s v1 engine used an abstractions repo to create tables such as dex.trades and nft.trades by putting together enhanced tables out of decoded contract tables. Spellbook is for the v2 engine, this time built off of dbt (data build tool).

It’s a powerful way to semantically manage relationships between tables, easily generate documentation, and run tests. You also get to leverage Jinja to have more programmability around your SQL code (if, while, for loops, etc).

If you’re looking to contribute you can start with the Spellbook readme here, I’ll cover an example for Sudoswap trades below.

Getting your local environment set up:

First, you’ll need to install python and pipenv. I use anaconda to install and manage my python packages. Make sure you add the relevant scripts folders to your environmental PATH variable to be able to call commands globally. Once you’re able to call pip --version without errors, then you’re good to go!

1. fork and create a branch of abstractions on GitHub; we’ll PR back to main later.

2. Run pip install pipenv and add it to your environmental PATH variable (you should be able to find the folder path by searching for pipenv.exe ).

3. Make sure you’re pointing to the spellbook folder path, and then run pipenv install .

4. Run dbt init, you can put n for all responses and put anything for credentials. Make sure you put wizard in for schema

5. Now you can run dbt docs generate and dbt docs serve to look through the models (tables):

Anytime you want to test your queries in the dune engine, you can run dbt compile and look in the target folder for the compiled SQL.

Sudoswap PRs:

Here are the two PRs I made for Sudoswap. You can look at the “files changed” tab to see all the code I added. Most of what I changed can be found within the Sudoswap/ethereum folders within models and tests.

1. First one with all the main logic

2. Second one with some lines changed to fix prices and explode swaps so each token id gets its own row

For simple models, you just need a few steps:

1. define a sources.yml with the decoded tables you want to use (base tables like ethereum.transactions are already defined elsewhere so you don’t need to specify them again.

2. create a some_model.sql file with the filename as the table name you want to create. Don’t worry too much about the config for now; just use the following:

   {{ config(
           alias = 'some_model',
           materialized = 'incremental',
           file_format = 'delta',
           incremental_strategy = 'merge',
           unique_key = ['some_unique_key']
           )
   }}
   

3. These tables usually update incrementally as defined in config, unless you’re just creating a static table like tokens_ethereum_erc20.sql . All you need to do is place the following filter wherever it makes the most sense (to transform all of the new rows). In my case we apply this on all swap functions since we only want to add the newest swaps to the table.

   {% if is_incremental() %}
   -- this filter will only be applied on an incremental run. We only want to update with new swaps.
   AND call_block_time >= (select max(block_time) from {{ this }})
   {% endif %}
   

4. Make sure to add your new model into a schema.sql file with all your final columns defined. The more description and documentation you can add, the better!

5. Add your folder models/schema into the dbt_project.yml file.

6. Lastly, you should make sure to have some tests. I recommend adding some not_null tests to key columns in your schema file, but you can also make more custom SQL tests.

After you compile your models and test them in the dune v2 engine interface, go create a PR and wait for reviews!

Simulating Liquidity Depth Across Pairs (i.e. Routing)

One of the more fun and complex queries I made for Sudoswap was simulating buying or selling X number of NFTs across all pairs to get total profits, costs, liquidity, and slippage.

You can observe the steepness of the price curve for any NFT collection that has pairs on Sudoswap, and see where there are inflection points once liquidity for pairs pricing at the same spot price, delta, and curve runs out (basically like moving between ticks on Uniswap v3, but not following a xy=k constant product curve).

The main simulation query can be found here. The full walkthrough and creation of the buy side of this query can be found on YouTube here. Sell side simulation was a little more complicated since we have to calculate maximum NFTs that can be sold into a pair based on ETH liquidity, spot price, and delta. For exponential curves, we can calculate it directly with geometric finite series summation formulas. For linear curves, because the pair stops processing trades if spotPrice is going negative, we calculate for the nth term where spotPrice = 0 and then truncate by ETH balance. The key logic is in the pairs_max_sell CTE, if you have questions or suggestions on how to calculate it more exactly/efficiently please let me know!

You’ll notice that Total ETH Liquidity is greater than the total profits you can make by selling NFTs (left side of cumulative area chart) - this is because some pairs have a spot price higher than the ETH held in the pair, so it’s technically illiquid.

There’s also one protocol technical nuance in this query is how the buy price and sell price are calculated. Pair bonding curves work by incrementing/decrementing the spot price by some delta along the curve. So if it’s linear, you do newPrice = spotPrice + delta, and if it’s exponential then you do newPrice = spotPrice^delta instead.

While the sell price is the current spotPrice, the buy price is the current spotPrice + one delta. 0xmons explained the rationale behind this decision as

“if spotPrice was both the price to buy and sell, lps could be arbed (e.g. buy at 1 eth, spot price goes up to 1.1 eth, then sell for 1.1 eth)” - 0xmons

You’ll see this logic used in the parts of the query where I calculate simulated_prices, slippage nominal and slippage_percentage.

That’s a wrap!

I’m excited for the protocol to release ERC1155 and ERC20 support, as well as more bonding curves (Gradual Dutch Auctions have been teased 👀). I’ll do a part 2 of this guide once those come out.

I hope you enjoyed this data guide - please collect, subscribe, and share if you did! I’m open to any feedback on how to make these guides better so don’t hesitate to DM me on Twitter.

Again, If you collect this entry, you can get access to my web3 data degens telegram group by clicking here! We discuss the newest protocols, debug general query questions, and help analysts find jobs (freelance and fulltime).

I believe each monetization method has its place, but we need to be more intentional about:

1. who in the community they’re targeting

2. how they’re composed together

In this piece, I want to propose some frameworks and implementation tools/examples for monetizing membership across clusters in a community.

I recommend reading my previous piece on the membership stack (defined as access → permissions → status) for some background. That piece is focused on what can be built on top of tokens, this one is focused on how the tokens get into the community.

Note that I use the words sale/monetization/distribution interchangeably throughout this piece.

Who is in your community?

There are tons of ways to visualize a token community’s social circles, usually shown in the way Brian has illustrated below as an “anatomy”:

brian flynn

@Flynnjamm

the anatomy of a DAO

224

6:51 PM • Sep 1, 2021

I think this is a good framework for approaching any community, but we can take it further. Let’s start with the building blocks of a web2 corporate structure:

1. Stock: “known as equity, is a security that represents the ownership of a fraction of the issuing corporation”

2. Governance: defined here as public-facing decisions, so mainly assigning board seats and other related shareholders proposals.

3. Service: the core business output(s) of the company

I’ll admit that this is a very US-centric/capitalist view that may not translate well to everyone reading this, but it should still make sense at a high level. We can see how various stakeholders are defined across stock, corporate governance, and the business itself.

There’s a ton of inefficiency and misalignment across all groups here due to resource, time, information, and permission constraints (if you want a short primer this book is a really good summary).

Now let’s redefine these circles for a web3 community:

1. Tokens: the building blocks of ownership; these can be community-specific or something more general. Usually a combination of ERC20 and ERC721 standards. Does not have to represent equity.

2. Governance: a contract or set of contracts that collect and manage incoming token revenue/contributions. Usually some combination of a multi-sig and a governor contract (propose, queue, vote, execute).

3. Service: tokens alone don’t create a service - normally the service is built separately and integrated (hopefully enhanced) by tokens + governance. Some examples:

   • Water & Music services include database access + special reports. Tokens are used and distributed in various ways, including airdrops and open sales.

   • NounsDAO (and affiliates) services are capital allocation (in the form of Prop House). Tokens are auctioned off periodically and used for governance.

Putting it together we get:

It might seem like it’s just wordsmithing and all these words represent the same web2 groups as earlier - and to an extent you’re right. However, Web2 corporates run on sluggish tech (lol banks), are mostly human-operated (excel and ppt input goes brrr), and have little capital raising flexibility (due to legal structures). Web3 communities move as fast as transactions get confirmed, can be augmented by smart contracts, and have flexible tokens (not tied to equity, and have programmable distribution + supply mechanics).

This all adds up to make the behaviors enabled of each web3 community segment really different from their web2 counterparts!

Monetizing the Membership Stack

Because of the malleability of the web3 infrastructure stack, it isn’t immediately clear what tokens should represent what and how they should be distributed to the community. In the last few years, we’ve seen a slew of attempts that independently use different monetization methods - from airdrops to auctions, some immediate and others time or round-based.

“Membership” has been the main utility lauded by tons of projects, alongside some opaque roadmap. Besides being a smart marketing tactic, using a membership-based perspective does provide clarity on how to distribute/leverage tokens across a community.

Let’s first map the membership stack on top of the community circles:

1. Tokens <> Access: You typically need to own core community tokens to access anything within a community. Other segments will still interact with these core tokens, but in a secondary/vertical manner instead.

2. Governance <> Permissions: There have been all sorts of explorations around on-chain governance that play with levels of permissioning, and use a combination of ownership and delegation.

3. Service <> Status: Status applies most closely to anyone who is using or building the service itself. Tiers and levels are what we want to capture here.

Now we layer different monetization methods across each segment:

Let’s start with the three groups that are more “benefactors” than “builders” of the community. These groups should have as simple a system as possible since they are unlikely to have continuous context.

• Supporters/Investors <> Edition Sales: This largely homogenous group has capital they want to allocate purely for investment or public good purposes.

  Fixed editions help to reach some one-time funding goal and quickly sell a set supply of NFTs (Mirror) without taking away too much time and attention from the community or the funders. If no funding goal is set, then you can do a time-limited supply as Pooly NFTs did.

• Power Users/DAO Partners <> Fractional NFTs: These are users who want to shape the service in some way that benefits them/their products. There’s a limited number of partners you can realistically take on (marginal benefit decreases at some point), but you still want to tier them so set supply/price edition sales don’t make sense.

  So for example, let’s take the NounsDAO prop house which operates weekly and has some level of topic segmentation. Assume there are a few large DAOs who want to integrate with Nouns in some way - instead of what we’ve typically seen with large treasury swaps or open market buying of tokens, these large DAOs could join a single party bid on a “seasonal partnership NFT”. As a start, holding these tokens could act as a simple multiplier when proposals go to vote.

  This works better than something like a Dutch auction because prices are decided competitively versus some monotonically decreasing - and all parties are guaranteed to get some allocation. You can set some threshold of # of tokens you need to qualify for a prop house proposal and anyone who doesn’t make it can carry on (swap) their tokens in the next party bid. If you add in baskets with Fractional.art, you could get some really fun combinations across houses.

  For power users of your service, you could imagine that what they’re bidding on is an extra “tier” of technical support for a season (3 months). Some simple composability would be using your fractional “tier” multiplied by your streaming rate as a method of “locking in a contract”. Typically longer-term contracts for a service come with a lump sum payment + monthly scaling extra costs and have an array of discounts. This web3 version combines a large bid w/ payments streams, and then could calculate scaling discounts as a factor of the two.

• Normie Users <> Token Streams: These users just want to use the core part of your service, so you want the experience for both them buying in and you managing payments to be as seamless as possible (so no auctions or bidding lol).

  Assuming you’re some X-as-a-service provider, I believe that streaming tokens (SuperfluidHQ) are the strongest access solution because you can control the rate required (tokens paid per second/block). Streams are also highly composable with each other because of their asynchronous nature (combining streams for group access or varied requirements) and can be enhanced by mapping to dynamic identity solutions across the space (collab.land). This is much better for managing large groups of people than lump-sum payments or static POAPs. It’s worth mentioning that checking the current rate, adjusting the rate, or adjusting the rate requirement are each only one action - super easy to manage and communicate!

  Obviously, this one varies a lot depending on the service, and wouldn’t apply as well to fee-based models.

These next four groups are all “builders”, focused on building and governing the service and community. These groups should have more context generally, so we can make their systems more complex.

• Treasury Managers + Grant Proposers <> Permissioned NFT Auctions: These are token holders who want an immediate and larger say in the community, so usually governance is the best next step for them.

  I’ve started becoming a bigger believer in gradually scaling governance (NounsDAO) versus airdrop + delegate kickoff governance (literally every other protocol). Doing an airdrop + delegate kickoff is like taking a ton of caffeine all at once - your community will be super energized but constantly clashing against each other for attention and funds for a long time to find equilibrium.

  Instead, if you “release” governance power over time you can allow the most interested and motivated parties to continuously buy in. Uninterested parties will just slowly unwind their tokens in secondary markets, which will become a smoother process as NFT AMMs (sudoswap) and aggregators (Genie) improve. We’ll take a deeper data dive into this in the next section.

  Even bribes/loans which are popular for ERC20 governance tokens are becoming possible to replicate with NFTs through innovations like the new renting module from Fractional.art.

• DAO Operators <> Delegated Permissioned NFTs: DAO operators are insanely important and they need better ways to manage all the chaos. Unlike the auctioned NFTs above where each one gives you the same weight/powers, these NFTs should have much more specific permissions which are granted.

  I think there are two approaches that make sense here:

  • Setting up governance pods (Orca) that can be adjusted. Delegating NFTs from the DAO treasury instead of just gifting them ensures representatives don’t become too comfortable or complacent.

  • DAO constitutions and manifestos have become increasingly important as a guiding framework for DAOs with growing complexity across tokens, governance, and services. This is a perfect use case for an on-chain tied language document (OpenLaw example). Adding editing permissions (through NFTs) to both on-chain parameters and accompanying off-chain language will make rules and operations much more flexible and easily understood (as opposed to forcing people to dig through old proposals and forums to track changes in both the parameters and the rationale).

• Niche Contributors <> SBT and Credit: For contributors who pop in and out of various communities and contribute where they can, building a credit system is a benefit for both the contributors and the community at large.

  We could expect it to be built with a simple scorecard system that’s level-based (Backed), using soulbound tokens. Once you have a basic score system, you could then utilize peer-to-peer undercollateralized lending. Union protocol is a great example of this, where contributors could gain delegated access to portions of the DAO’s treasury.

  The undercollateralized loan is “secured” by the symbiotic working relationship between the contributor and the DAO - assuming the contributor’s work is useful and the service continues to do well.

• Core Contributors <> Passport: These are your main “employees”. This group should get identity “passports” that are stronger (more complex) than the basic credit systems niche contributors get.

  The key difference between the passport and basic SBTs is their direct integration with all layers of the DAOs operating system. The deck in the previous link explains it much better than I can; I highly recommend taking a look!

And last but not least, our tokenomic favorites:

• everyone <> airdrops and staking: I believe these methods do apply to all community segments - but need to be experimented with and tiered accordingly.

  • Airdrops will likely become more round-based (like Optimism) and targeted towards either specific community segments (Water & Music $STREAM) or community structures (community reputation score $WRITE). Coinvise is a great tool for executing this.

  • Staking should be applied to something like reputation staking around key decisions, especially longer time-horizon ones (Koop).

Whew - that was dense! If you have any suggestions or arguments on stuff I’ve missed, please reach out because I’d love to discuss them.

There’s still one big elephant in the room - who is going to manage and implement all these methods? There are already some ongoing interesting experiments we can study to build monitoring metrics for “monetization overhead”.

On Scaling and Managing Membership

We now have a cleaner way of segmenting our communities and some preferred monetization methods to go along with each one. It’s worth mentioning that there are some all-in-one-token approaches that look promising for starter communities. Scaling membership and community in sync is a topic I’m very interested in, and will be keeping a close eye on for the next year or so.

And with scaling membership comes the increasing complexity of managing implementations. A good start would be creating monitoring metrics for “monetization overhead”. This is separate from “contract/token composability overhead”, which is a contract architecture issue that will just take many iterations to get right. Contract modules have already been implemented in evolving fashion to make it easier to add management capabilities to tokens, such as gnosis zodiac, default core OS, tributeDAO, and fractional V2 modules. Various interfaces are being created to make it easier and easier to compose and deploy contracts too.

Monetization overhead is focused on the social elements, such as noise, frequency, and energy within the context of token interactions. An example of bad overhead is a large about of ETH requested in governance proposals multiple times a month (high noise + frequency), where voter turnout is minimal (low energy). There’s likely some combination of metrics unique to each monetization method that would make it much easier to monitor over time. Layering these metrics upon something like Forefront Terminal could further enhance operator understanding of the community health.

These metrics would also eventually help tune on-chain parameters such as quorum and minimum proposal thresholds. Instead of static thresholds, I’d imagine we’d scale thresholds based on noise levels:

• You could make things independent and linear: For every additional 1% of treasury requested in just this proposal, 1 more permissioned NFT is required by the proposer.

• Or you could make it dependent and non-linear: For every additional 1% of treasury requested in total across the proposal queue, we could use n^2 - (n-1)^2 additional NFTs required where n = total % treasury requested in the queue . So if 5% of the treasury is already being requested and you’re requesting 1% in your proposal, then 11 more NFTs are required (36 - 25).

There are plenty of interesting experiments such as NounsDAO already happening for us to study, and I have more fun news coming next month regarding how I’d like to engage the data community as a whole to study this deeper.

But for now, that’s a wrap! Please challenge me and let me know where you think differently (or where I’ve missed something).

I really do appreciate each and every one of you who read this far. If you’d like to support more of this kind of writing then don’t forget to collect this entry, subscribe, and share! 😊

Most explanations of storage and state on Ethereum are diagram heavy and don’t get into the meat and bones of how to access or work with that data. I’m going to avoid the diagrams and theoretical explanations, but you should still have some understanding of data tries so skim this and this if you know absolutely nothing about how data gets into blocks.

Essentially, what you need to know is that every time you make a transaction with a contract involved, there will be state and storage changes saved to the block. State changes involve things like an addresses’ Ether balance or Nonce, storage changes involve storage slots on contracts such as your balance of ERC20 tokens in a contract.

To access these changes from an Ethereum node, you’ll need to use a replay VM - meaning you have to replay a transaction or block to get the raw diffs in state and storage. This is easily done using Infura or Alchemy, without running your own node. You can go to Alchemy’s API composer to play with the trace_replayTransaction endpoint (here). If you want to read more about how traces works, read the standards here - but I’ll run through one example.

Let’s use this ERC20 transfer, whose contract code includes the following:

function transfer(address to, uint256 value) external returns (bool) {
        _transfer(msg.sender, to, value);
        return true;
    }

function _transfer(
        address from,
        address to,
        uint256 value
    ) private {
        balanceOf[from] = balanceOf[from] - value;
        balanceOf[to] = balanceOf[to] + value;
        emit Transfer(from, to, value);
    }

You can see that there are two storage changes, the balance of the from address and the balance of the to address. Here’s your JSON rpc request:

{
    "jsonrpc":"2.0"
    "id":0
    "method":"trace_replayTransaction"
    "params":[
        0:"0x8e5cd4d97881b558b7c4a120c41193b944db0612bc45807ae056d5488b82700e"
        1:[
            0:"trace"
            1:"stateDiff"
        ]
    ]
}

And here’s the response you’d get from Alchemy, I’ll explain it briefly here:

• state diffs for 0x320d83769eb64096ea74b686eb586e197997f930 are balance changes in the mapping of balanceOf

• state diffs for 0x52bc44d5378309ee2abf1539bf71de1b7d7be3b5 are addition to balance of gas fees being paid to miner

• state diffs for 0xdad3fd6c9fb0c2b56228e58ae191b62bfb1bec83 are subtraction from balance of gas fees being paid by the sender for this transfer

• the first trace action is the actual transfer() call

• the second trace action is a delegatecall because a proxy pattern is used for this contract, and the logic is stored somewhere else (like forwarding an email).

{
  "jsonrpc": "2.0",
  "id": 0,
  "result": {
    "output": "0x0000000000000000000000000000000000000000000000000000000000000001",
    "stateDiff": {
      "0x320d83769eb64096ea74b686eb586e197997f930": {
        "balance": "=",
        "code": "=",
        "nonce": "=",
        "storage": {
          "0x108f03599a018b41696aad07e2100369d10122c70bf27d8e9fe3e425885a246b": {
            "*": {
              "from": "0x0000000000000000000000000000000000000000000000357bdcd5b903c454ce",
              "to": "0x0000000000000000000000000000000000000000000000000000000000000000"
            }
          },
          "0xdc7fafdc41998a74ecacb8f8bd877011aba1f1d03a3a0d37a2e7879a393b1d6a": {
            "*": {
              "from": "0x000000000000000000000000000000000000000000000215d691e79c0df07a6f",
              "to": "0x00000000000000000000000000000000000000000000024b526ebd5511b4cf3d"
            }
          }
        }
      },
      "0x52bc44d5378309ee2abf1539bf71de1b7d7be3b5": {
        "balance": {
          "*": {
            "from": "0xfdc72883cded567d93",
            "to": "0xfdc728a1a08db20db3"
          }
        },
        "code": "=",
        "nonce": "=",
        "storage": {}
      },
      "0xdad3fd6c9fb0c2b56228e58ae191b62bfb1bec83": {
        "balance": {
          "*": {
            "from": "0x59e417e4aa86a74",
            "to": "0x599f683b630a274"
          }
        },
        "code": "=",
        "nonce": {
          "*": {
            "from": "0x159",
            "to": "0x15a"
          }
        },
        "storage": {}
      }
    },
    "trace": [
      {
        "action": {
          "from": "0xdad3fd6c9fb0c2b56228e58ae191b62bfb1bec83",
          "callType": "call",
          "gas": "0x938c",
          "input": "0xa9059cbb000000000000000000000000000000000000000000000000000000000000dead0000000000000000000000000000000000000000000000357bdcd5b903c454ce",
          "to": "0x320d83769eb64096ea74b686eb586e197997f930",
          "value": "0x0"
        },
        "result": {
          "gasUsed": "0x45d0",
          "output": "0x0000000000000000000000000000000000000000000000000000000000000001"
        },
        "subtraces": 1,
        "traceAddress": [],
        "type": "call"
      },
      {
        "action": {
          "from": "0x320d83769eb64096ea74b686eb586e197997f930",
          "callType": "delegatecall",
          "gas": "0x7e24",
          "input": "0xa9059cbb000000000000000000000000000000000000000000000000000000000000dead0000000000000000000000000000000000000000000000357bdcd5b903c454ce",
          "to": "0x5b5746f6f5e2db8bf5e260829ca7a004c876b167",
          "value": "0x0"
        },
        "result": {
          "gasUsed": "0x323b",
          "output": "0x0000000000000000000000000000000000000000000000000000000000000001"
        },
        "subtraces": 0,
        "traceAddress": [
          0
        ],
        "type": "call"
      }
    ],
    "vmTrace": null
  }
}

vmTrace is a deeper trace, which tells you the order in which opcodes were executed by the EVM. I won’t cover that in this article (mainly since I am not familiar enough with those yet lol)

You can check the balanceOf changes using hex to num here, remember to div by 1e18. If you’re curious about how subtraces work, they are nested function calls:

• subtraces = number of sub-calls by function

• type = call (normal call), staticcall (doesn’t mod state), delegatecall (proxies calling logic), create (contract deploys)

  traceAddress = order of call execution, see example below from open Ethereum:

  A
    CALLs B
      CALLs G
    CALLs C
      CALLs G
  

  then it should look something like:

  [ {A: []}, {B: [0]}, {G: [0, 0]}, {C: [1]}, {G: [1, 0]} ]

A good tool to use is the Versatile block explorer, where you can check the “Invocation Flow” for a more detailed view of traces than Etherscan provides.

Don’t worry too much if that doesn’t all make sense, we’ll dive into the data here using a data warehouse instead of RPC calls from here on out.

Accessing State and Storage Data using Ethereum Data Warehouse (EDW)

EDW is events, calls, transactions, state, and storage data in a Snowflake shared database. You can get setup here with a custom app or just start querying on Flipside by changing the schema to tokenflow_eth. There are some example queries here. I’m assuming you’ve already used something like Dune Analytics or Flipside, so I’ll only cover the new tables.

• state_diffs: use this for tracking ETH balances. here are the main state fields and reasons they change:

  • I like to use this table for wallet balances analysis - much faster than trying to aggregate on ETH calls from traces.

• storage_diffs: tells you current and last value for different storage slots, with mappings nicely nested already in dict formats.

  • Contract storage layout comes from solc compiler. you can use a hardhat plugin to do this to get the following on each compile (or the raw JSON output):

• This table is useful for tracking variables on contracts, most commonly would be stuff like historical balances.

• storage_reads: this tells you when a certain slot of storage was read during a transaction.

  • useful for assessing contract interoperability/dependencies, when joined on call_id and tx_hash with traces (calls)

  • exploring what variables are read when a certain function is called, when joined on transactions with a selected funcsig in call_data.

  • sometimes it's easier to use calls table (traces) when the value you want is in a function output, instead of the storage variables.

Example Queries: Looking at On-chain Price Oracles

I used the data outputted from these queries in further analysis, you can read and play with the code in the repo here.

Let’s run through a harder example than just balances and transfers: looking at price oracles on-chain. If you have no familiarity with Oracles, check out this explainer and this repo of older oracles.

tldr; Chainlink pushes prices from the real world on-chain so smart contracts can access it, Uniswap V2 and V3 have methods for calculating the token price purely on-chain. There are likely differences in prices at any given moment for a variety of reasons.

All the queries I’m going through below are public here, you can run them once you have EDW set up - just remember to change EDW_SNOWFLAKE_SECURE_SHARE_1645732350028 to whatever your secure share instance name is! I also set up udfs on a new database in Snowflake, since you can’t create udfs in a shared db.

Chainlink Query

Let’s start with Chainlink data feeds! The contract is set up such that the proxy itself doesn’t store historical prices - it calls aggregator contracts instead. There have been five such aggregators used for the ETH/USDC price feed, which I queried for here:

SELECT * --use curr_value for aggregator addresses to use in the next query
FROM EDW_SNOWFLAKE_SECURE_SHARE_1645732350028.raw.storage_diffs 
WHERE contract = lower('0x5f4eC3Df9cbd43714FE2740f5E3616155c5b8419') --ETH/USDC proxy aggregator, ref https://data.chain.link/popular 
AND location LIKE '4[%].0'
ORDER BY timestamp ASC;

I queried for any address in slot 4 mapping. Mappings in EDW are stored as slot[value].keys where keys apply to structs in mappings. Here there is no struct, so we take .0 instead. The variable this tracks is:

"4": [
    {
      "from": 0,
      "name": "phaseAggregators",
      "to": 31,
      "type": "mapping(uint16, address)"
    }
  ],

Then, I plugged the five addresses manually into a query over the calls table, so I could get the values from each time the proxy contract called an aggregator.

SELECT 
    distinct 
    timestamp, 
    block,
    udf.public.hextoint(return_value)/1e8 as eth_price
FROM EDW_SNOWFLAKE_SECURE_SHARE_1645732350028.raw.calls 
WHERE from_address= lower('0x5f4eC3Df9cbd43714FE2740f5E3616155c5b8419') --ETH/USDC proxy aggregator
AND to_address IN ('0x37bc7498f4ff12c19678ee8fe19d713b87f6a9e6',    '0xd3fcd40153e56110e6eeae13e12530e26c9cb4fd', '0xb103ede8acd6f0c106b7a5772e9d24e34f5ebc2c', '0xf79d6afbb6da890132f9d7c355e3015f15f3406f', '0xc7a37b03690fb9f41b5c5af8131735c7275446') --historical phaseaggregators for ETH/USDC as of 3/1/2022
AND substring(call_data,1,10) = '0x50d25bcd' --funcsig for getting latestRoundData
AND block >= 12381343
AND status = 1
ORDER BY 1 ASC;

Chainlink oracle usage queries are just an aggregation change, so I’m not including it here.

Uniswap V2 Query

Uniswap V2 pairs track token prices separately, in variable price1CumulativeLast and price0CumulativeLast for token1 and token0 respectively (and in storage slots 10 and 9 respectively)

For this pair, I queried storage at slot 10 and parsed curr_value into an integer:

SELECT 
    block, 
    timestamp, 
    -- location, 
    -- tx_hash, 
    udf.public.hextoint(curr_value) as cumulative_last
FROM EDW_SNOWFLAKE_SECURE_SHARE_1645732350028.raw.storage_diffs 
WHERE contract = lower('0xB4e16d0168e52d35CaCD2c6185b44281Ec28C9Dc')
AND location = '10' -- token1 price slot, which is WETH
AND block >= 12381343
AND status = 1
ORDER BY block, timestamp, order_index;

The rest of the data processing to get price was done in Python (see repo from top of section). V2 Oracle usage queries are basically the same as in v3, so I won’t go over them here.

Uniswap V3 Query

Uniswap V3 is much more complex, because of two reasons:

• prices are tracked in ticks now

• historical prices are stored in the contract now to make accessing them easier, but this means dealing with a struct array. This is the observations[] array, created using the Oracle.sol library and implemented on the pair contract.

Because the array is the last slot initialized in the contract (the 9th slot), we know the index of the array is slot - 8. Structs are stored in reverse and tightly packed within each slot, such that for the struct:

struct Observation {
        uint32 blockTimestamp;
        int56 tickCumulative;
        uint160 secondsPerLiquidityCumulativeX128;
        bool initialized;
    }

The slot can be decoded into: first two bytes have initialized then 40 for secondsPerLiquidityCumulativeX128, 14 for tickCumulative and 8 for blockTimestamp. The query I used was:

SELECT timestamp, 
       location::integer - 8 as index, 
       -- substr(curr_value, 3, 2) = '0x01' as initialized,
       -- hextoint(substr(curr_value, 5, 40)) as secondsPerLiquidityCumulativeX128, 
       hextoint(substr(curr_value, 45, 14)) as tickCumulative
       -- hextoint(substr(curr_value, 59, 8)) as blockTimestamp
FROM EDW_SNOWFLAKE_SECURE_SHARE_1645732350028.raw.storage_diffs 
WHERE contract = lower('0x8ad599c3A0ff1De082011EFDDc58f1908eb6e6D8')
      and location not like '_[%' and location::integer > 8
      AND block >= 12381343 
      AND status = 1
ORDER BY block asc, order_index asc;

The rest of the data price processing was done in Python.

I also made a query to get daily reads/unique contracts reliant on the Uniswap V3 oracle, found here. First, I had to filter out transactions where there was a call to mint, burn, swap, or flash in the pool. Then, I joined the storage_reads table with the calls table on call_id and tx_hash to figure out which contracts were reading the tickCumulative data values, presuming they did so to get ETH prices.

WITH 
    --ignore calls that affect observation state as that doesn't count as oracle usage
    ignore_interactions as (
        SELECT 
            *
            --distinct tx_hash
        FROM EDW_SNOWFLAKE_SECURE_SHARE_1645732350028.raw.calls
        WHERE to_address = lower('0xB4e16d0168e52d35CaCD2c6185b44281Ec28C9Dc')
        AND substring(call_data,1,10) IN ('0x6a627842','0x89afcb44','0x022c0d9f') --mint, burn, swap
    )

-- query top contracts using calls instead of txs, we count distinct tx_hash since otherwise reads might be cheating with multiple reads per oracle use. 
SELECT 
    call.from_address , 
    COUNT(distinct call.tx_hash) as total_times_queried
FROM EDW_SNOWFLAKE_SECURE_SHARE_1645732350028.raw.storage_reads read 
LEFT JOIN EDW_SNOWFLAKE_SECURE_SHARE_1645732350028.raw.calls call 
    ON call.tx_hash = read.tx_hash AND call.call_id = read.call_id
WHERE contract = lower('0xB4e16d0168e52d35CaCD2c6185b44281Ec28C9Dc')
    AND location = '10'
    AND read.status = 1
    AND read.block >= 12381343
    AND NOT EXISTS (SELECT 1 FROM ignore_interactions ii --ignore non-oracle interactions
               WHERE ii.tx_hash = read.tx_hash)
GROUP BY 1
ORDER BY total_times_queried DESC
LIMIT 100;

*I highly encourage you to check out the longer deep dive into oracle prices here to get a sense of how this data is then applied in real analysis.*

Concluding Thoughts

I believe working with storage and state data is harder than events/function call data at the start because you need more familiarity with Solidity. Many variables are encoding in weird ways to make manipulation easy between smart contracts - so they really aren’t human interpretable at all. There are also a bunch of different storage patterns that people use, which take time to get used to.

However, this data does provide two primary benefits:

• allow you to avoid a ton of expensive event/call aggregations, which will become increasingly important for wallet analysis in the future.

• it captures cross-contract interoperability and dependencies. I think this enables us to do much deeper ecosystem analysis and risk analysis, as well as more accurate simulations.

I can’t wait for the production release of this product, huge kudos to the TokenFlow insights team (and thanks a bunch for answering all my questions! 🙂 )

(here’s the edition I mentioned at the start!)

edition://0xDF5b5ee15CC96ba7d0CB6BD9b2c0fc4417ab6445?editionId=4053

For this inaugural guide, I’m going to try to keep things very concise and highlight my thoughts across these three pillars:

1. Thinking: why an open data pipeline re-shapes how data work gets done

2. Tools: an overview of tools along the web3 data stack, and how to leverage them

3. Teams: basic considerations and skills to have in a web3 data team

This is probably a good time to say that this is all just my view of the space, and I’ve showcased only a few facets of the tools/communities mentioned below.

Let’s get into it!

On Data Thinking

Let’s start by summarizing how data is built, queried, and accessed in web2 (i.e. accessing Twitter’s API). We have four steps for a simplified data pipeline:

1. events API triggered (some tweet sent out)

2. ingestion to the database (connection to existing user models/state changes)

3. transformations of data for specific product/analytics use cases (addition of replies/engagement metrics over time)

4. model training and deployment (for curating your Twitter feed)

The only step when the data is sometimes open-sourced is after transformations are done. Communities like Kaggle (1000’s of data science/feature engineering competitions) and Hugging Face (26,000 top-notch NLP models) use some subset of exposed data to help corporates build better models. There are some domain-specific cases like open street maps that open up data in the earlier three steps - but these still have limits around write permissions.

I do want to clarify that I’m only talking about the data here, I’m not saying web2 doesn’t have any open-source at all. Like most other engineering roles, web2 data has tons of open-source tools for building their pipelines (dbt, anything apache, TensorFlow). We still use all these tools in web3. In summary, their tooling is open but their data is closed.

Web3 open-sources the data as well - this means that it's no longer just data scientists working in the open but analytics engineers and data engineers as well! Instead of a mostly black-box data cycle, everyone gets involved in a more continuous workflow.

The shape of work has gone from web2 data dams to web3 data rivers, deltas, and oceans. It’s also important to note that this new cycle affects all products/protocols in the ecosystem at once.

Let’s look at an example of how web3 analysts work together. There are dozens of DEX’s out there which use different exchange mechanisms and fees for allowing you to swap token A for token B. If these were typical exchanges like Nasdaq, each exchange would report their own data in a 10k or some API, and then some other service like capIQ would do the work of putting all exchange data together and charge $$$ for you to access their API. Maybe once in a while, they’ll run an innovation competition so that they can have an extra data/chart feature to charge for in the future.

With web3 exchanges, we have this data flow instead:

1. dex.trades is a table on Dune (put together by many community analytics engineers over time) where all DEX swap data is aggregated - so you can very easily search for something like a single token’s swap volume across all exchanges.

2. A data analyst comes along and creates a dashboard using a bunch of community open-sourced queries, and now we have a public overview of the entire DEX industry.

   1. even if all the queries look like they’re written by one person, you can bet that there was tons of discussion in a discord somewhere to piece it together accurately.

3. A DAO scientist takes a look at the dashboard and starts segmenting the data in their own queries, looking at specific pairs such as stablecoins. They take a look at user behaviors and business models, and start building a hypothesis.

4. Since the scientist can see which DEX is taking a larger share of trading volume, they’ll then come up with a new model and propose changes to governance parameters to get voted on and executed on-chain (s/o Alex Kroeger for the proposal example here).

5. Afterwards, we can check the public queries/dashboards at anytime to see how the proposal created a more competitive product

6. In the future, if another DEX comes out (or upgrades to a new version), the process will repeat. Someone will create the insert query to update this table. This will in turn reflect in all the dashboards and models (without anyone having to go back and manually fix/change anything). Any other analyst/scientist can just build upon the work Alex has already done.

Discussion, collaboration, and learning happen in a much tighter feedback loop due to the shared ecosystem. I will admit this gets very overwhelming at times, and the analysts I know basically all rotate data burnout. However, as long as one of us keeps pushing the data forward (i.e. someone creates that insert DEX query) then else everyone benefits.

It doesn’t always have to be complicated abstracted views either, sometimes its just utility functions like making it easy to search up an ENS reverse resolver or improvements in tooling like auto-generating most of the graphQL mapping with a single CLI command! All of it is reusable by everyone, and can be adapted for API consumption in some product frontend or your own personal trading models.

While the possibilities unlocked here are amazing, I do acknowledge that the wheel isn’t running that smoothly yet. The ecosystem is still really immature on the data analyst/science side compared to data engineering. I think there are a few reasons for this:

• Data engineering was the core focus of web3 for years, from client RPC API improvements to basic SQL/graphQL aggregations. Work on products like theGraph and Dune really exemplifies the effort they’ve put into this.

• For analysts, there’s a tough ramp-up to understand the unique cross-protocol relational tables of web3. For example, analysts can understand how to analyze just Uniswap but then struggle with adding in aggregators, other DEXs, and different token types into the mix. On top of that, the tools to do all this weren’t really there until last year.

• Data scientists are used to basically going to a raw data dump and working through all of it solo (building their own pipelines). I don’t think they’re used to working so closely and openly with analysts and engineers earlier on in the pipeline. This took a while to click for me personally.

On top of learning to work together, the web3 data community is also still learning how to work across this new data stack. You don’t get to control the infrastructure or slowly build up from excel to data lake or data warehouse anymore - as soon as your product is live then your data is live everywhere. Your team is basically thrown into the deep end of data infrastructure.

On Data Tools

Here’s what most of you came here for:

*These tools are not comprehensive of the entire space - they’re just the ones that I’ve found myself or others consistently using and referencing in the Ethereum ecosystem (some of them cover other chains as well).

*The “decentralized” tag means there’s either an infrastructure network or guideline framework to stop changes from happening unilaterally. I like to think of it as decoupled infra versus cloud infra, but that will need to be its own article.

Let’s walk through when you would need to use each layer/category:

• Interaction + Data Source: This is mostly used in frontends, wallets, and data ingestion to lower layers.

  • Clients: while the underlying implementation of Ethereum is the same (with Geth as the canonical repo), each client has different extra features. For example, Erigon is heavily optimized for data storage/sync, and Quorum supports spinning up privacy chains.

  • Nodes-as-a-Service: You don’t get to choose which client they run, but using these services will save you the headache of maintaining nodes and API uptime yourself. Nodes have a lot of complexity depending on how much data you want to capture (light → full → archive).

• Query + Data Mapping: The data in this layer is either referenced in the contract as a URI or comes from using the contract ABI to map transaction data from bytes to table schemas. A contract ABI tells us what functions and events are contained in the contract since otherwise, all we can see is the deployed bytecode (you can’t reverse engineer/decode contract transactions without this ABI).

  • Transaction Data: These are the most popularly used, mostly for dashboards and reports. theGraph and Flipside APIs are used in frontends as well. Some tables are 1:1 mappings of contracts, and some allow extra transformations in the schema.

  • Metadata “Protocols”: These aren’t really data products, but exist for storage of DIDs or file storage. Most NFTs will use one or more of these, and I think we’ll start using these data sources more and more to enhance our queries this year.

  • Specialty Providers: Some of these are very robust data streaming products, Blocknative for mempool data and Parsec for on-chain trading data. Others aggregate both on-chain and off-chain data, like DAO governance or treasury data (I’m pretty confident Boardroom and DeepDAO will have more expansive APIs/query-ability in the future).

• High Touch Data Providers: You can’t query/transform their data, but they’ve done all the heavy lifting for you.

  • “Enterprise” Services: You’ll use these a lot if you’re a VC, investigator, or reporter. You can go deep into wallets and wallet relationships using Nansen’s wallet profiler or Chainanalysis’s KYT. Or use Tolken Terminal for beautiful out-of-the-box charts on tons of projects/chains.

  • Enhanced APIs: These are products that bundle what would be many queries into just one, such as all token balances of an ERC20 token or pulling the contract ABI for a given address.

It wouldn’t be web3 without strong, standout communities to go alongside these tools! I’ve put some of the top communities next to each layer:

• Flashbots: Working on everything MEV, providing everything from a custom RPC for protecting transactions to professional whitehat services. MEV refers mostly to the issue of frontrunning, when someone pays more gas than you (but directly to the miners) so that they can get their transaction mined first. A great story to learn about this is written here by Samczsun.

  • how to get involved: read through the docs to figure out which vertical interests you and jump in the discord

  • contact: Bert Miller

• Dune Wizards: Handpicked wizards who have contributed something significant to the Dune data ecosystem.

  • how to get involved: start off in the discord and tag @DuneAnalytics on Twitter with your work.

  • contact: Boxer

• Flipside Gunslingers: Handpicked gunslingers who have contributed something significant to the web3 data ecosystem.

  • how to get involved: start off in the discord and create cool custom reports.

  • contact: Eric or GJ

• MetricsDAO: Working across the ecosystem to tackle various data bounties on multiple chains.

  • how to get involved: read through their intro and jump in the discord

  • contact: Danner or GJ

• DiamondDAO: Stellar data science work on governance, treasury, and token management in DAOs.

  • how to get involved: they literally have an organized “join” page here.

  • contact: Christian Lemp

• IndexCoop: Lots of token and domain specific analysis to create the best indexes in crypto.

  • how to get involved: jump into the discord, and read some of their index proposals like the metaverse index (MVI).

  • contact: BigSky or JD Cook

• OurNetwork: Consistent data coverage of protocols and other verticals across web3 each week.

  • how to get involved: subscribe to the newsletter, if you do stellar analysis you’ll end up in our telegram eventually 😉

  • contact: Spencer Noon

(yes, we need to do better on diversity. Go follow/reach out to Danning and Kofi too, they’re amazing!)

Every one of these communities has done immense work to better the web3 ecosystem. It almost goes without saying that the products with a community around them grow at 100x the speed. This is still a heavily underrated competitive edge, one I think people don’t get unless they’ve built something within one of these communities.

On Data Teams

It should also go without saying that you want to look within these communities for the people to hire onto your teams. Let’s go further and break down the important web3 data skills and experiences, so you actually know what you’re searching for. And if you’re looking to be hired, view this as the skills and experiences to go after!

If you’re new and want to dive in, start with the free recordings of my 30-day data course which is purely focused on the first pillar. I’ll hopefully have educational content on everything here and be able to run cohorts with it one day!

At a minimum, an analyst should be an Etherscan detective and know how to read Dune dashboards. This takes maybe 1 month to ramp up to leisurely, and 2 weeks if you’re really booking it and binge studying.

There’s a little more context you should have in your mind as well, specifically on time allocations and skill transferability.

1. On Time: In web3, about 30-40% of a data analyst's time is going to be spent keeping up with other analysts and protocols across the ecosystem. Please make sure you aren’t suffocating them, otherwise, it’ll become a long-term detriment to everyone. Learning, contributing, and building with the larger data community are absolutely essential.

2. On Transferability: Skills and domain are both highly transferrable in this space. If I went to a different protocol, likely the ramp-up time would be minimal since the table schemas for on-chain data are all the same. If anything, I’ve probably already worked with that protocol’s data before even joining!

Remember, it’s less about knowing how to use the tools - every analyst should more-or-less be able to write SQL or create dashboards. It’s all about knowing how to contribute and work with the communities. If the person you’re interviewing isn’t a part of any of web3 data communities (and don’t seem to express any interest to start doing so), you may want to ask yourself if that’s a red flag.

And a final tip for hiring: pitching the analyst using your data will work much better than pitching them with a role. Graeme had originally approached me with a really fun project and set of data, and after working through it I was fairly easily convinced to join the Mirror team.

Disclaimer: I would not recommend taking my word as definitive criteria for everyone, but at least it’s the frame of mind I’ll be using to hire data friends at Mirror (sometime soon 🙂).

I’d also have loved to talk about team structure, but to my knowledge, most product data teams have been 0 and 2 analysts full-time. I’m sure I’ll have better-informed thoughts on this in a year or two.

Thanks for reading!

It’s honestly quite amazing how far data in web3 has come over the last year, and I’m very excited to see where the ecosystem grows to by 2023. I’ll try and make this a yearly thing - if you want to support me and this kind of work, feel free to collect an edition of this entry (at the top) or one of the landscape NFTs below:

edition://0xDF5b5ee15CC96ba7d0CB6BD9b2c0fc4417ab6445?editionId=4053

If you have ideas/questions on web3 data topics you’d like to learn more about, just dm me on Twitter. I’m always looking for educational partners as well!

Special thanks to Elena as always for reviewing and making great suggestions!

Hopefully, you're familiar with the Web3 product stack, and why it's considered composable. If not, then here's a quick refresher that explains the following layers:

• Frontend Layer: Webpages and/or wallets that allow us to interact abstractly with everything web3

• Smart Contract Layer: The protocols that hold interaction logic and storage, like Uniswap

• Blockchain Layer: Layer ones like Ethereum, or L2's like Optimism

Another precursor for this article that will be helpful is how web3 communities create tokens that become the membership structure that platforms are built off of, whereas web2 communities are forced into membership structures of existing platforms.

Communities in web3 issue tokens which are primitives used for defining membership - and if platforms want communities to use them, then they must support flexible membership structures.

Now if you follow all the concepts above, you'll start to realize that web3 communities already have two sources of contextual complexity:

1. They live on many different frontends, many different smart contracts, and many different chains. It isn't clear where the community lives or where to start.

2. While they might only have one ERC20 token right now, we're entering a phase where they might have seasonal sets of NFTs as well (or multiple ERC20 tokens). It isn't clear what the tokens are for/value they hold.

At least that's a lot of contexts (data and storytelling) to play with! So what do community entry points look like? Well...

...that doesn't really help me understand the community or token much at all. It doesn't really feel fun either - but that isn't FWB's or Uniswap's fault. After all, how is FWB supposed to know where I'm going to enter from and how is Uniswap supposed to know (or care) which ERC20 tokens represent what community (or the tokens' significance)?

That brings me to the point of this article:

How do we comb through the complexity of web3 communities such that we can provide helpful context about what people are "buying" into when they go to app.uniswap.org or opensea.io or any other marketplace/exchange?

I'm not using the term "DAO" here, but you can mentally swap that with "web3 community" if you'd like.

What is the token context?

Web2 communities are mostly stadiums, where everyone is fighting to appear on one surface (Twitter feeds, subreddits, GitHub repo PRs/issues, Medium publications, etc.). What happens on that one surface gives us most of the community interaction, with only one layer beneath it for discussion (sometimes it's a comment section, sometimes a separate Discord server).

The discovery surface is fairly shallow, but community managers/leaders have full admin control of these main surfaces on the platform. This "control" is a bit of a farce since they don't really have control of the rest of the community stack. Community leaders have limited say in the stability or features of tooling, and have no way to change payment infrastructure or support/incentive structures. As a result, the ability to cohesively bring together those different community subsets is rarely well supported. Even though the underlying relationship graph across communities is probably a fairly distributed web, that network pattern is only really leveraged by the algorithms.

In Web3, there are many different community contexts (surfaces) - all linked by the token(s). Communities pool around different facets of the token, becoming more uniform as you move down the token context stack.

My working assumption is that token context looks like this:

Anyone can create a new application using "X" token, and it will be fully composable with the existing systems around "X" token. But that new application is now tied to that token ecosystem, where changes at the tokenomics/token/utility levels can affect the applications at the surface. Here are some examples:

• The team that manages or deployed the token has little say about what can or can't be built. For Mirror, anyone can build discovery or interaction layers on top of our entries or economic blocks. This has led to projects like askmirror.xyz, mirror-feed, and backdrop which use different curation methods. You may say "this is just like Facebook showing content from New York Times," but the difference is Mirror can reward and work with them through our tokens/treasury, as well as build their membership status into our utility tools and discussion platforms.

• Sometimes what's built takes advantage of the existing system, such as pooling user deposits together in Yearn before depositing in PoolTogether, leading to a higher chance of payout and higher yield for Yearn users. But even this is symbiotic, leading to PoolTogether building a better and more fair system from the bottom upwards. This is a bit of an optimistic take, but users will earn Pool tokens even while abusing strategies like Yearns, which should incentivize them to take a more long-term health viewpoint on the PoolTogether ecosystem.

• Some applications are still team-led, for example, FWB partnering with SquiggleDAO and TileDAO to combine token applications for a live event in NYC. BrightMoments partnering with leading artists from ArtBlocks like Tyler Hobbs and Jeff Davis is another great example that also showcases how building off of NFT based systems works just as well as ERC20 tokens.

• Some tokens start with a fixed subset of communities. TheGraph uses one token (GRT) to bond together Subgraph Developers (mapping smart contract events into data schemas), Indexers (running the nodes that run subgraphs), and Curators (signalers who stake GRT on subgraphs). Each community continues to build and improve on its own tools but is incentivized to work with each other to maintain data composability. Compare this to the less cohesive effort across Ethereum clients (Geth), infrastructure providers (Infura), and data mappers (Dune Analytics) who aren't tied together by a token (ETH doesn't really count).

Bottom-up management in token context is a key difference from the top-down management of traditional web2 communities. The team has control over treasury/token distribution economics, as well as the base membership definitions (token utility) that govern the community. However, while the team's decisions have a lot of power, they are inherently low discovery. With the discovery stack so deep, how do we choose and represent what to show?

How do we show token context?

The difficulty here is not just in capturing and displaying the elements in token context but in doing so concisely. Forefront DAO profiles do a great job of showing stuff like access benefits, historical events, and general token stats. But I think we also need to capture something more dynamic.

Let's say I'm deciding whether or not to move into a new city. Pretty pictures, testimonials from friends, and statistics on job opportunities/economics may convince me to visit - but they won't be the reason that I choose to stay. What convinces me to stay is that first-week impression of the "pulse" of the city. You hear people say all the time, "the air is electrifying!" or that "you can just feel that the pace, culture, and emotion" in one city is different from another. That "pulse" is the dynamic bit we want to try and capture and show about tokens.

Continuing with the city model, I'll suggest four themes to work with as research jump-off points:

Bridges/Doors: If I'm buying a token, I obviously am curious about thresholds. But how many times have they been changed and applied? For what purposes? At what pace do I need to accumulate?

People/Politics: I want to know about the history and momentum of proposals, participation, and execution. What tools were used and token/delegation structures?

Public Parks/Spaces: What does the community composition look like and where do they live? What is the relationship graph of interactions across different surfaces?

Energy Consumption/Production: Which tokens are moving and which aren't? Are any of them interlinked/dependent on each other? What about the token's relationships with other communities tokens?

Ultimately, these questions should get us closer to the pulse across the layers of a token's context, in a more concise way than just listing everything or joining a flooded discord. With this data, we get community activation effects alongside more meaningful token trading volume:

• Prospective members: give those on the sidelines an active and energized interest in some layer of context at the same time as they buy the token.

• Inactive members: convince passive token holders to accumulate tokens and find a new interest. Or alternatively, give them a reason to dump so that new blood is more likely to come in and contribute.

• Prospective partners: convince builders/communities to build their applications into your token ecosystem (either symbiotically or parasitically) to drive long-term community health up.

Some of this sounds harsh, but I believe the worst is when both the community and the token holders are stagnant. Web3 communities can't really bankrupt like a typical company, but they can definitely still become zombies - no pulse at all. And if your token is trading actively but has no pulse/context - well that’s a clear warning sign too. Pulse and token context can help both initiate and revitalize communities!

On a more technical level, I think this means being able to map and track token activity (across all context layers) - using some sort of changepoint or momentum algorithm. Maybe token context can be represented in pulse as a couple values/models, maybe it can't 🤷‍♂️

Creating a data guild for tackling the "pulse and token context" quest

Now, I'd like to start a data guild for studying all this within Mirror DAO. Note that you don't need to agree with my viewpoints in order to join, in fact, I would see that as a plus!

There will be roughly three main goals, using a few case-study DAOs as our base research data:

• Create a clean way to store a community membership registry over time

  • Probably following some epochs/seasons format

• Find and define data points for token context (likely split into groups based on earlier themes).

  • Standardize data points into inputs/outputs or a list of tools.

  • Find a way to push them onto an open data source like Ceramic API.

• Work with these data points to represent the "pulse" of a community's token(s)

  • Make a cohesive story behind these

  • Design mockups

On top of providing context to newer or inactive community members, having this data will also help us unlock more comprehensive community health dashboards and better context around decentralized digital identities.

Please fill out this form if you are interested in being a core contributor. Looking for anyone with experience in DAOs from a data, storytelling, design, or tool liaison background!

Even if you don't make it into the core group, we'll still find ways for everyone who wants to be involved to contribute or lurk.

The form will stay open until November 26th, 6 pm EST.

For what it's worth, I think this is a strong step towards providing better context to decentralized digital identities as well.

What's in your wallet?

If you've ever made a transaction on Ethereum (or any smart contract enabled blockchain), then you've probably looked it up on a block explorer and seen this heap of information:

Learning to read the details of a transaction will be the foundation for all your Ethereum data analysis and knowledge, so let's cover all the pieces and how to work with them in SQL. I'll be using Dune Analytics to run my queries, but there are many other tools you can use to query the chain such as Big Query and Flipside Crypto.

If you're completely new to SQL and Ethereum I recommend starting with my full beginners' overview first.

Guide Overview

We're going to cover transactions in four layers:

1. Transaction basics

2. Function calls and state

3. Internal Transactions (Traces)

4. Logs (Events Emitted)

As the base for our transaction examples, we'll be using the Mirror Crowdfund contract. Put simply, this is a smart contract that allows you to get ERC20 (fungible) or ERC721 (NFTs) tokens in exchange for donating ETH to the contract. The creator of the contract can then withdraw those funds by closing the crowdfund. This is by no means a simple contract, but the point I want to make here is that you don't need to understand all the solidity code to start your analysis - you just need to know how to navigate the four layers above.

The three transactions we'll study are:

1. Creation/deployment of the crowdfund contract

2. Contributions of ETH to the contract

3. Closing and withdrawing funds from the contract

Side note, we also just opened up crowdfunds for anyone to use, so if you're curious or want to create a crowdfund head to mirror.xyz/dashboard to get started. Hop into our discord while you're at it!

Creation/Deployment of the crowdfund contract

First Transaction: 0x5e5ef5dd9d147028f9bc21127e3de774a80c56a2e510d95f41984e6b7af1b8db

Let's start with the transaction basics.

1. Each transaction has a unique keccak256 transaction hash of a few different variables

2. There's a blocknumber associated based on when the transaction was mined, typically a new block is created every 15 seconds.

3. From is the one who signed the transaction, To is the contract address that was interacted with

4. Value is the ETH value that was transferred from the signer's wallet. Even if that value is 0 that doesn't mean that no ETH was transferred during the transaction.

5. Gas is a bit complicated (especially with EIP-1559), but just keep this formula in mind:

   Gas Price * Gas Used by Transaction = Transaction Fee

Now for the meat and bones, the input data of a transaction:

This is just bytecode for any function call and the parameters passed in. The first 8 characters (4 bytes) are the function signature 0x849a3aa3, essentially a hash of the function name and parameter types. And no, these are not always unique which can lead to hacks/security issues. In this case, this function calls the factory contract to create the crowdfund contract (it's a proxy, but we won't get into that).

createCrowdfund((uint256,uint256,bytes32)[], (address,uint256), string, string, address, address, uint256, uint256, uint256)

This shows up if you click "decode input data", and you can see the various variables values set as well. Every subsequent 64 characters (32 bytes) is a different input variable. The crowdfund comes with three tiers of editions. In this crowdfund for BLVKHVND they used quantities of 1000, 250, and 50 with prices of 0.1, 0.3, and 1 ETH respectively.

Notice that the price actually shows up as 100000000000000000 , which is because the first 18 zeroes represent decimals. We'll have to do the conversions by dividing by 10^18 in our data.

That was a lot, let's get to querying. Dune has a table called ethereum.transactions which has all the variables we've talked about above for every transaction since the first block. We can query this table for the appearance of 0x849a3aa3 in the last few months:

SELECT * FROM ethereum.transactions 
WHERE "block_time" > now() - interval '3 months'
AND "data" is not null
AND SUBSTRING ( encode("data", 'hex'), 1, 8 ) = '849a3aa3'

ethereum.transactions is a very large table, so if you query without filters the query is going to timeout (taking more than 30 minutes). Filtering by block_time is usually most useful, and in this case we're taking all the rows that have occurred within 3 months. Also, many transactions are just ETH transfers without any data attached so we'll filter that out by only keeping data is not null. Now for checking for the function signature, we need to encode the data into a string from hexadecimal, then take only the characters from position 1 to position 8 using SUBSTRING.

Now the complicated parts, internal transactions and events emitted. For this, it'll be easier to look at the code. If you go to the contract tab on etherscan and do a ctrl+f on file 1 of 10 you'll find the following code (I've edited out some bits to make this more readable).

function createCrowdfund(
        ...variables...
    ) external returns (address crowdfundProxy) {
        ...some variable prep code...

        crowdfundProxy = address(
            new CrowdfundWithPodiumEditionsProxy{
                salt: keccak256(abi.encode(symbol_, operator_))
            }(treasuryConfig, operator_)
        );

        emit CrowdfundDeployed(crowdfundProxy, name_, symbol_, operator_);

        ...register to treasury code...
    }

The first key line here is crowdfundProxy = address(contract_to_be_created), which is what deploys the new contract and creates an internal transaction of type CREATE 0. Transferring ETH also creates an internal transaction of type CALL , which we'll see in the next transaction we study.

We can query for all the crowdfund contracts created with:

SELECT tx."block_time", tx."from", tr."type", tr."code"
FROM ethereum.transactions tx 
LEFT JOIN ethereum.traces tr ON tx."hash"=tr."tx_hash" --tracks internal transactions
WHERE tx."to" = '\x15312b97389a1dc3bcaba7ae58ebbd552e606ed2' -- crowdfund podiums edition
AND tr."type" = 'create' 

We need ethereum.transactions because we want to filter for traces (internal transactions) only related to transactions on the factory contract. We need this since an internal transaction will not always have the same to as that of the overall transaction. We can JOIN the tables on the transaction hash, and then filter for only internal transactions of the create type.

The second key line here is emit CrowdfundDeployed, which creates a log that is stored in the node but not in the block. If you look at the logs, you'll notice that EditionCreated events are also emitted, but this is from another contract that actually creates the ERC721 tokens (hence a different address).

Similar to a function signature, events have a unique hash as well that sits in Topic 0. So in the events above, 0x5133bb164b64ffa4461bc0c782a5c0e71cdc9d6c6ef5aa9af84f7fd2cd966d8e is the hash for CrowdfundDeployed and 0xbaf1f6ab5aa5406df2735e70c52585e630f9744f4ecdedd8b619e983e927f0b6 is the hash for EditionCreated.

We can query the ethereum.logs table in dune to see all crowdfunds created as well:

SELECT * FROM ethereum.logs
WHERE "topic1"='\x5133bb164b64ffa4461bc0c782a5c0e71cdc9d6c6ef5aa9af84f7fd2cd966d8e'::bytea

topic2 and topic3 typically hold the data for ETH transfers, otherwise, event data will show up in the data column. We'll get more into how to work with this later.

Logs are very helpful, as they can be used to emit state variables instead of just the function call values (TheGraph uses logs to model subgraphs for GraphQL queries). Next, we'll utilize everything we've covered to study the contributions of ETH to our newly created crowdfund contract (sitting at the address 0x320d83769eb64096ea74b686eb586e197997f930 ).

If you've made it this far, then you're already through all the tough concepts. Give yourself a pat on the back! We'll really be getting into the details in the next two sections, so take a breather if you need to.

Contributions of ETH to the contract

Second Transaction: 0xd4ce80a5ee62190c5f5d5a5a7e95ba7751c8f3ef63ea0e4b65a1abfdbbb9d1ef

This one is fairly simple to read. Jesse paid 1 ETH to mint an edition of tokenId 167 from the BLVKHVND crowdfund. He also got 1000 HVND, the ERC20 token the crowdfund gives out based on the size of the donation.

But what if we wanted to see how much ETH has been contributed over time, or how many editions have been sold? Sometimes contracts will have a view function in Read Contract on etherscan where you can get total balances. But in this case, the contract doesn't have that.

Remember that function calls change the state data, which we'll need to piece together the overall state data by aggregating over transaction history. Sometimes the overall state of a contract can be emitted in events, such as with Compound V2's AccrueInterest event.

In our case, we'll need to do two things in one query to get to total ETH contributed:

1. get the transactions that have the "contribute" method called

2. sum the total ETH transferred by filtering for internal transactions which have the type CALL

Remember, I can get the method function signature by decoding the input data on etherscan.

SELECT SUM(tr."value"/1e18) as contribution FROM ethereum.transactions tx 
LEFT JOIN ethereum.traces tr ON tx."hash" = tr."tx_hash"
--transactions filtering 
WHERE tx."to" = '\x320d83769eb64096ea74b686eb586e197997f930'::bytea
AND tx."data" is not null
AND SUBSTRING ( encode(tx."data", 'hex'), 1, 8 ) IN ('a08f793c', 'ce4661bb')
--traces filtering 
AND tr."success"
AND tr."value" > 0
AND tr."call_type" = 'call'

There was technically another method called contributeForPodium, which is why we check for two function signatures above. The CALL type actually has subtypes as well at the opcode level, so we need the specific base call_type of call (if you're familiar with a delegatecall, then you'll know that would give us a double count). We joined on transaction hash, and then divided by 10^18 to get the right decimals of ETH value.

Let's move on to the last transaction, where the data starts to get really tricky on us.

Closing and withdrawing funds from the contract

Third Transaction: 0xe9d5fefde77d4086d0f64dd1403f9b6e8e12aac74db238ebf11252740c3f65a8

Here, we can see that 337 ETH was transferred and 1,012,965 HVND tokens (the latter of which was decided by operatorPercent_ in the first transaction). After this function is called, the contract just operates the way any normal ERC20 would.

In the case that a crowdfund was already closed, we could have gotten the total raised from the data in this transaction - such as value transferred in an internal transaction of CALL type. It's better to tie this to an event though, in case there are some transfer behaviors that we don't know about. But wait, why are the logs not readable?

Well, this is where we start to get into some pretty confusing patterns. Earlier I mentioned that this crowdfund is deployed as a proxy - that means it’s just like an empty USB that plugs into a computer that actually holds the logic. It's much cheaper to create USBs than computers - and that logic holds for on-chain too (except the cost is in gas). If you want to read about proxy patterns, I'd check out this great article by the OpenZeppelin team.

The computer in this case is known as the logic and is only deployed once. The proxy is deployed many times, and it doesn't have the logic functions or events in the contract code. Therefore, etherscan isn't equipped to show the decoded data in logs. So then how do we piece this together? We could take the keccak256 hash of the event, just like we did for function signatures. But here's where reading the code will help save you some time. If you go to Read Contract on the factory contract, you'll see the address of the logic contract:

From there, we can look for the closeFunding() function in the code:

function closeFunding() external onlyOperator nonReentrant {
        ...code...

        _mint(operator, operatorTokens);
        // Announce that funding has been closed.
        emit FundingClosed(address(this).balance, operatorTokens);

        ...ETH value transfers...
    }

ETH value transfers don't emit events since they are just internal transactions. And if you are familiar with how the ERC20 standard works, you'll know that _mint actually creates a Transfer event (meaning that covers our first event). That means that FundingClosed must be the second log, with the topic of 0x352ce94da8e3109dc06c05ed84e8a0aaf9ce2c4329dfd10ad1190cf620048972. Can you figure out why else it couldn't be the third log (hint: what's a key difference between the first two logs and the third log)?

With that knowledge, we can query this just like any other event, with some fancy data decoding (remember parameters are every 64 characters (32 bytes). We have to turn it into a string to slice it, and then we change it into a number and divide by 10^18 to get rid of decimals.

SELECT "contract_address", 
        bytea2numeric( decode ( SUBSTRING ( encode("data", 'hex') , 1, 64 ), 'hex'))/1e18 as eth_raised, 
        bytea2numeric ( decode ( SUBSTRING ( encode("data", 'hex') , 65 , 64 ), 'hex'))/1e18 as tokens_allocated_owned
FROM ethereum.logs
WHERE "topic1"='\x352ce94da8e3109dc06c05ed84e8a0aaf9ce2c4329dfd10ad1190cf620048972'::bytea
AND "contract_address"='\x320d83769eb64096ea74b686eb586e197997f930'::bytea

Congrats, you now know your way around ethereum.transactions, ethereum.traces, and ethereum.logs. They can always be joined by transaction hash, and then the rest is just knowing how to manipulate the data with encode/decode, substring, and some bytea operators. Woohoo!

We could have done this exercise for the contribute method in the last transaction too. Since this is all happening on the proxy contract.

Putting it all together

Now, if we had to go and keep track of function signatures and event topics - as well as decoding all the variables in each query - I think we would have all quit data analysis by now. Luckily, most data services have some variation of contract decoding, meaning I can give a contract address and the ABI and Dune will take care of the decoding for me. That way, events/functions become their own tables and I can easily make the same "total contributions" query from earlier with this:

WITH 
    union_sum as (
        SELECT SUM("amount")/1e18 as raised FROM mirror."CrowdfundWithPodiumEditionsLogic_evt_Contribution"
        WHERE "contract_address"='\x320d83769eb64096ea74b686eb586e197997f930'
        
        UNION ALL 
        
        SELECT SUM("amount")/1e18 as raised FROM mirror."CrowdfundWithPodiumEditionsLogic_evt_ContributionForEdition"
        WHERE "contract_address"='\x320d83769eb64096ea74b686eb586e197997f930'
    )
    
SELECT SUM("raised") FROM union_sum

https://dune.xyz/queries/192571

Thankfully this query is much more readable and easier to write. They even take care of proxy/factory logic patterns - thanks team! Without this abstraction, I guarantee that data analysis would be ten times messier to write and one hundred times worse to debug. Dune has plenty of other useful tables as well, such as prices.usd for daily token prices and dex.trades for all token trades across all main exchanges (and event nft.trades for OpenSea NFT actions).

While most of the time you'll be playing with decoded data, knowing what really sits underneath it all will help you level up a lot faster in Web3! Plus, you're now etherscan fluent - which I promise will be a part of every crypto job description in the future. I hope you found this helpful, and as always feel free to reach out if you need some help getting started.

1. How access, permissions, and status are the primary components of membership (and the role of tokens)

2. Membership in Web2 versus Web3 platforms, and why composability matters here

3. Designing a membership system to generate social capital

Access, Permissions, and Status

Our data and actions give us identity and relationships, but having a social graph does not give you community. Put in a Web3 context, a group of people holding or trading the same token does not mean you are a community - at best, it gives you clusters. **An actual community has fairly clear boundaries for growth (in more than a financial sense). You've probably seen some variation on the chart below...

... where you can identify your core contributors, and try to reward or incentivize them accordingly. Pet3rpan has put this concentric community chart nicely in a Web3 context, showcasing the funnel of activities a participant might engage in during their lifecycle in a community:

But just because someone has participated in any of the above events, doesn't mean they feel like they're part of the community. Even giving them tokens for their participation doesn't really move the needle.

Tokens are open primitives that membership should be built upon, and just issuing more POAPs or tokens based on events and then moving on is like distributing the raw materials for a house and expecting the people to build up a town. Without a blueprint, people will just stash the materials and leave (or sell them to someone else). In crypto, we’re supply rich, but guidance poor.

Some communities at least take the first step to have some sort of access dependent on your token activities, usually a Discord server or channel. But access is only the first step to membership. Permissions are too closely tied to access right now, and status is left as too abstract a concept.

So let's improve on the current model. Access is discovery, permissions are responsibility, status is weight. This membership funnel is shaped different from the activity funnel above, as the customizable design area gets larger the deeper you go:

These are basic examples, but they help lay the groundwork for how each layer could be tied to different types (and combinations) of tokens. Let's look at some token design examples for each level:

• Access: For the most part, ERC20's are best for access because requirements are fluid and the tokens are more liquid. Today I might require 50 tokens to join, but in the future, I may drop it to 25 or increase it to 75 based on community needs (going wide versus deep). NFTs are cultural assets, so if you use that as the gate then it'll be tough to say you need two or more to have access now. Even if you implement project derivatives, liquidity becomes fragmented and confusing. This is project-dependent though, and Creator Cabins did a great overview of some of the tradeoffs.

• Permissions: I personally believe permissions should be tightly coupled to a stake. To be given the responsibility for some set of actions, there should be risk involved. Unlike access, permissions should evolve and can be revoked. If I'm making proposals, allowing staking will show my commitment and also give others the opportunity of joining my stake. This line of design can help a community better balance abuse of power and opportunities for discovery (of great new contributors).

• Status: This layer has the largest area because it is tied to attributes and metadata. Status in a community has the highest amount of variance, as it will likely be event-dependent. Taking the treasury "weighted voting power" example, if you give status based on what is being voted on that highlights specific community members while also ensuring that it isn't always the participant with the most tokens has the most sway. NFTs are a great place to play with these attributes, especially through derivative and forge-based projects. To clarify, this is not the same as reputation since reputation needs to be less fluid as to not confuse people. We'll get more into that later.

Different combinations of these layers could be bundled into "roles" for easier management and assignment. I'd love to dive right into a membership system, but first we need to clear out the current mental heuristics around how Web2 "membership" works.

Composable Membership and the Role of Platforms (Web2 vs Web3)

In Web2, membership forms around services. In Web3, services from around membership. Put simply, being a YouTube subscriber does not affect your privileges when engaging in a Twitch chat or a subreddit. This kind of membership interoperability is certainly possible in Web2, and I think Patreon was the closest to getting a hierarchical membership structure in place. We could have seen a world that looks like this:

But obvious arguments would ensue over who owns the master API, and economic flow incentives also act as a blocker since the platform at the top of the hierarchy likely has the highest money-in/money-out flow and take-rate. So in the end we have like 7 or more different memberships to manage! No one actually wants that besides the platforms themselves.

In Web3, no one is fighting to be at the top of the membership hierarchy because the primitives (tokens) aren't owned by any one platform. Smart contracts are our APIs and we can code revenue splits right into them. If that isn't enough, we can devise governance and attribution systems over the treasury to ensure the services and tooling built around membership continuously benefits communities that rely on them. In the end, the above Web2 platforms will naturally fall lower on the hierarchy. They will be forced to fit around the composable membership systems that we all enable and create. In other words, the community is the platform:

There's an argument to be made that in the end, apps will just be a bunch of composable components where each piece is owned by a different platform. But in the interim, it'll be Web3 communities plugging into the currently better interfaces of Web2. When planning a membership system, we'll need to think about membership as bundles of access, permission, and status across these Web2 services. If I'm a multisig member, what roles will I need across all these services and how can that be managed in a single token/membership model? To figure this out, we'd need to experiment. And to experiment, we need social capital.

A Membership System for Generating Social Capital

We've defined the components of membership and situated them within the Web2 and Web3 context. Now, let's have fun designing a system that could actually go into implementation. We'll define social capital as building an allowance for trust, experimentation, and flexibility within a community. This means having a membership system that improves ownership of decisions and prevents a social fork. For now, my guess is there are three main steps to execute membership and generate this new capital:

1. Define Roles: Roles are bundles of the different layers of membership and are tied to specific participant personas. There are likely hundreds of combinations (especially if you layer token types on top), but some immediate roles that come to mind are:

   • Low Access (ERC20) + Low Permissions (ERC20 Stake) = new member

   • High Permissions (ERC20 Stake) + High Status (NFT) = long time contributor

   • High Access (NFT) + High Status (NFT) = subject matter expert

   • High Access (ERC20) + High Permissions (ERC20 Stake) + High Status (NFT) = founding member

   Doing this well opens up a pathway for growth and also clarity for all members. This starts to give us history too, as I can clearly see how a community has matured by its role definitions and distribution (much more so than if I just look at the token price).

2. Specify Incentivizes: Pet3rpan mentioned in the article linked earlier that trust and engagement are key to participation. Financial incentives are a great starting point - if you airdropped me $1000 of your community's tokens, I'd definitely come to take a look and see what I can help with. But if these drops/bounties are filtered by role or membership history, we can be much more flexible with asks and rewards and I'd likely feel more fulfilled. This is the difference between me tackling a data bounty as a nobody versus as a Dune Wizard.

   This could also look like a hackathon where teams are built based on role/membership history, such that newer members are matched with older members. Or imagine if hackathon bounties rewarded community roles instead of just $$$, I think we'd end up seeing an even larger diversity of projects and teams in hackathons than we do today.

   Also, I cannot stress enough the importance of history here. Membership + Identity + Relationships will probably be the best proxy for a reputation score, which opens up the path to more creatively weighted incentives.

3. Assign Roles: This step is more technically difficult than the others, given the challenge of plugging membership systems into existing platforms. I think this is similar to the integration problems that any wallet aggregator already faces, for example, Zapper's protocol requests. Integrating into existing Web2 platforms shouldn't be difficult, but as more Web3 platforms are built the composability will be difficult to manage.

   But as long as we have an identity solution (connecting addresses to user metadata across platforms) and a membership solution (managing roles and tokens intuitively), I think we'll be able to tackle the platform integration problem elegantly. Besides, building the first two will take the ecosystem the next six months anyway.

Hopefully, this system helps show how much we can bring a community together while experimenting by using membership built off of tokens. I know for some people this is all still too fluffy, so I plan on doing data-driven case studies to better inform this kind of membership system in the future (and to try to be more quantitative about social capital). Some communities I'm interested in studying under this lens are Yearn, Blitmaps, and Superfluid - but if you have other case studies that come to mind, please do let me know.

Concluding Thoughts

Composable membership shapes more than just communities, it will shape ecosystems and platforms as well. In crypto, we're all building platforms that give communities the tooling for implementation, visibility, collaboration, and much more. Ecosystems are forming around metadata already, and this will force a reorganization of Web2 platforms to provide a much more open surface to be built upon.

Mirror is one platform trying to improve the tooling for communities to implement a rewarding membership system:

Seed Club is a good example of the model above, where they deploy ERC20 tokens for specific communities and causes, and once members are more defined they move on to specialized token distribution methods such as the token race for engaging the community and adding new cohorts. I think there is a lot more we all can do to standardize the membership implementation process laid out in this article, using Mirror and ecosystem tooling.

But none of what I've written about can sustainably happen unless communities take their thinking around membership more seriously (and that goes for me too). If any of this interests you, please reach out so that we can build out this future together. My twitter DMs are always open! 🙂

Life Before Crypto:

Before I entered crypto, I worked at a large investment bank in their rotational grad program. Over two years, I worked with different teams on everything from managing the bank's liquidity to our commercial real estate portfolio.

When I started there in the Summer of 2019, I saw there was a lot of unstructured data both public and internal that could be leveraged with the right tools. I quickly decided I needed to upskill from just Excel, and immediately began learning Python to do some time series forecasting for liquidity risk. I would read every day about autoregressive and moving average models, try to implement them for some basic dataset, and then do a presentation to my few coworkers to get their feedback. That feedback would lead to lessons learned on how I could better apply and communicate my findings, but also would usually end with "here's this other person who you can learn from about this". This would create multiple avenues of learning to then go down again, repeating the process. I eventually called this a learn-build-teach cycle, and it helped me stay motivated and keep momentum.

After a while, I realized that I did most of my learning through reading other peoples' presentations on what they had already built (mostly on Towards Data Science). Around May of 2020, I wanted to try and give back to the data science community and write about the skills and projects I'd developed over time. This was a great way to learn in the open where I would get comments or messages giving me feedback, and also led to many new connections on LinkedIn who were farther out from my usual network.

I was heavily reliant on developer communities when learning various Python packages, and could clearly tell the difference between a product that was loved by its community versus one that didn't really connect as well with newer developers. Seeing the stark differences, I decided to go deep down the community software-as-a-service rabbit hole to find out what tools were being built to help dev tool product teams with building their communities. This research led me to conversations with builders like John Yen (Yeniverse), and Patrick Woods (orbit.love). They inspired me to try and improve the developer onboarding experience for dev tooling communities.

This started with building an NLP analysis product that could help with three things:

1. identify the most commonly asked questions to enhance or auto-generate an FAQ page.

2. match questions and answers to create an auto-response discord bot. The idea here was just to try and save the team time.

3. identity the questions developers ask over time in the discord to map out when users try and develop with different features. This could help with user research and also product road mapping.

While that endeavor of mine didn't get too far, this had planted a seed in me to continue observing and thinking about community building. Fall of 2020 came along, and I began to tap into one of the most well-connected and vibrant communities out there: Ethereum. Right away I could see that everything was open, from the data to the code to the people. This was a large departure from traditional finance, where it often takes months just to get a conversation or data partnership going within the same bank.

How I Got Started:

It took many many tries over the years to start "getting into" Ethereum. In July of 2020 I took a more serious stab at it; I tried reading and re-reading the Ethereum and Uniswap whitepapers, digesting my transactions on Etherscan, and listening to various podcast interviews. Bankless helped me understand the people and ideas to follow, but things didn't fully click until I started learning Solidity. Some helpful soul linked me to a tweet from Linda Xie mentioning an Ethereum developer onboarding session happening over the weekend, and that one-hour session led by Linda, Trent, and Austin probably contributed a lot to me changing my career path.

I barely knew what "full-stack development" meant, since most of my experience up to then was purely analytics of financial statements (writing scripts that didn't even remotely resemble building an app). But in that short session, we were all walked through the smart contract dapp development process and tools, and then given a clear guide to start with. The exact path I took during that September to November period looked like this:

1. Watch all of eth.build to understand wallets, transactions, and proof of work/authority consensus mechanisms in an interactive way

2. Go through all six lessons of Cryptozombies.io (the web3.js chapter may be a bit dated, so skipping it and learning to use ethers.js could be better instead)

3. Go through Austin's Bowtie Friday videos to learn scaffold-eth basics.

4. Go through all of solidity-by-example to understand common use cases like transferring ETH and working with ERC20 and ERC721 tokens. This is also a good intro to some common security risks in smart contracts such as reentrancy.

5. Participate in ETHOnline 2020 Hackathon (by the way ETHOnline 2021 is kicking off soon), creating a full-stack dapp in under 30 days while also pitching the final product to a panel of judges.

   1. I asked tons of questions in the hardhat discord to help me learn test suites and ethers.js concepts (thank you bunches @alcuadrado and @fvictorio)

   2. the hackathon sponsor list is helpful for both using bounties to define the project and also to motivate studying the various products and tooling available (I remember setting up this page at the start)

   3. after winning the Gnosis prize, my partner Diego and I ended up taking this to a UNICEF MVP over the following months.

   4. I wrote about my first Ethereum hackathon experience to cap it all off

After this experience, I was convinced that crypto was indeed the future and I needed to go much deeper into the ecosystem. This was very exhausting though, with about 6-7 hours of learning and coding every day after work during that period.

As a side note, if I was to go back in time I would have started by looking at Dune dashboards before diving into solidity development since I think looking at data from events and function calls is a much easier starting point for learning Ethereum than reading contracts on Etherscan or trying to understand the protocol level elements right away.

The Compounding Power of Learning and Contribution

I continued with on-off momentum through the rest of November and December, until I joined Kernel Block 2 in January. @KERNEL0x has an element of referrals/applications - but getting noticed happens largely through contributing publicly. @sm_judge reached out to me because I'd started participating more in hackathons with @ethglobal:

Shortly after KB2 started, Sishir found me in the KB2 slack channel and asked if I wanted to join the Spectral team to do solidity research and architecture - on-chain credit scoring sounded pretty interesting and close to my finance knowledge domain, so I joined them part-time. Around the same time, I took up a business analyst role at Consensys both to learn product management and to see how enterprises really thought about blockchain. By early March I was fairly comfortable with the basics of Ethereum/Solidity and was getting more familiar with the teams of many larger projects in the space.

This career shift from an investment bank to crypto/tech kicked off a three-month full ecosystem mapping period, which was formative for helping me figure out what problems there were to solve and which communities I vibed with the most.

There were three main reasons I needed to spend the time doing this:

1. It isn't clear how things got to where they are today. For example, what are the different parts of the crypto wallet, and which were built first and why? How did certain communities form around or leave behind different problems?

2. It isn't clear where things are going in the short or medium term. There are so many different solutions proposed and attempted every day that it's hard for a generalist to understand things at a glance. This was especially true for me when I was studying interchain bridges.

3. I had no idea what I wanted to do within crypto, but I knew it likely wasn't going to be in defi. Spectral was much closer to the identity side of defi, which is what had initially piqued my interest.

The big idea I kept hearing about since day one was "composability", so I decided to dive in and see just how composable every part of the Ethereum ecosystem was. I think things were already loosely clear to me, but researching and writing it out helped me really understand which elements of the product stack were composable, how open communities means fractals form elegantly around problem spaces, and the incredible importance of infrastructure composability when it comes to both wallets and interchain bridges. All this exploration culminated in my research on the composability of identity, where I connected my thinking across all of the previous dimensions.

Along the way, I had many helpful conversations across a dozen communities and leaders, who gave me insight into how they thought about problems (or how they didn't). As I watched these communities grow, I started to notice a strong trend with their hires. Oftentimes, these new hires were already strong contributors within the community. This went against my notion that you should build up your resume first and then apply to a company/job. This is a very important frame of mind that anyone trying to break into crypto needs to quickly build up: contribute, don't apply.

One good example is @artblocks, which has one of the strongest and most vibrant communities in crypto. Their hires of @ponyo and @druid are perfect examples of contributors who were always present.

Another obvious shill for me is @DuneAnalytics. Becoming a @DuneAnalytics wizard 🧙🏼‍♂️ requires consistently showing up in the discord, making helpful dashboards or queries, and otherwise contributing to the community in a meaningful way. Over time, you'll be exposed to many bounty opportunities, get reached to by other companies looking for data/crypto experts, and maybe get tapped for hire by the Dune team. You can tell that the community and team are closely aligned and working together, purely based on the fact that outsiders have trouble separating the two:

My recent collaboration and constant engagement with the Mirror community led directly to my eventual hire. It all started with a single tweet analyzing the $WRITE race.

ilemi

@andrewhong5297

Each week 10 new writers gain access to the platform, meaning a potential 10,000 more votes to spread around. It's currently round 20, so we should have 200,000 available votes from winning writers alone. Here's the distribution of how many votes you needed to get to top ten:

17

4:06 PM • Jul 13, 2021

I want to stress that contributing can be literally anything: engineering product 🏗️, data analytics 📊, shipping memes 🍍, coordinating the masses ☎️, etc. Don't overcomplicate the task, but do take the time to find the missing links within or between communities 🔗. Once this contribution lightbulb clicks, you'll start to see opportunities everywhere and the whole crypto space will start to open up to you. From finding your way into a community to eventually joining the team, it's all about finding a way to contribute. Contributions may sound a little too transactional, so I like to think about it as community building.

Community, Community, Community

Earlier I called learn-build-teach a cycle, but it looks more like a branching tree because the work is spread over a wide breadth of topics and audiences (most of whom you never really interact with repeatedly). However, once you understand the connection between contribution and community you can build a learning flywheel that brings you closer to the community and its mission. In a great 0xSTATION article, this product flywheel came up in relation to a person's lifecycle in a community:

The new community-contribution driven flywheel I leverage is a linear and actionable version of the contribution <> consumption portion in the flow above:

One example of this cycle is the story behind how I became a Dune Wizard. When I first started learning about Dune and hopping around the Discord in April, I noticed that most people were struggling with learning SQL, Ethereum, or both. Pointing to guides that taught only one of the two domains led to a large learning friction. So, I decided to learn Dune with some simple example queries, and then use those to write articles on how to learn SQL and Ethereum at the same time on Dune. I started with a beginner and then an advanced guide, which together has now garnered over 20k views on Medium (expert guide coming later this year). I've continued using and improving this teaching-as-contribution methodology over the last few months, doing deep dives with my friend 0xkowloon on interesting new protocols like barnbridge and fractional_art.

The larger point here I want to make about contributions is that what you create doesn't have to be crazy innovative or even tackle the core product/problem of a community. Building an intrapreneur skillset and perspective will serve you much better 90% of the time instead of focusing on an entrepreneurial approach when trying to contribute to a community for the first time. Since in crypto we see innovations every day, I think this creates a pressure that makes it hard for most people (me included) to balance the two ways of thinking. If this feels abstract, then think about it as building something for the ease of use or building of a product rather than just extending the use of the product itself. Intrapreneurs are usually born from environments where there are lots of silos and collective improvements that can be made. This is typically found at a large enterprise company, where a lot of your efforts are around making collaboration and knowledge sharing more accessible by creating a mix of solutions such as clearer standards, working groups, or dashboards/applications. Open communities in Ethereum make intrapreneurship a lot easier to learn as there is usually less bureaucracy around budgets and more political will to chase and implement changes, even though the communities are large in size.

Looking Forward and Joining Mirror

Joining Mirror means that for the first time in my professional life, I'll be devoting all my efforts to one thing instead of multitasking many projects at once. It's a big decision for me, one that's driven by my interest to understand exactly what community building looks like over time and how the process can be made more fluid or fun.

I've thought for almost a year now about the kinds of issues communities face while growing, starting from developer communities and now more recently on DAOs. Over time, my goal has become more refined: I want to build tools that enable and incentivize contributions within a community, and also make it easier to see the full context of and leverage a community's social graph. These are key elements to strong community building, and I believe will be core to Mirror's mission. Having the opportunity to build up a strong community around this tooling is a huge motivator for my move as well.

Those are some pretty dense sentences, so let me break down my decision to join Mirror further:

• The Product:

  • So far I've only worked on/with various defi protocols, and seeing/using the creator block contracts produced by the team (such as edition crowdfund and splits) felt like such a breath of fresh air. I'm excited to see further iterations leverage what has been built in defi already.

  • There's a lot of creative space in NFTs and transactions that build connections to people instead of pools. This includes variations on the NFTs themselves, auction mechanisms, ownership structures, and much more.

• The Team:

  • I've spoken with Patrick, Graeme, Jon-Kyle, and Denis over the past few months, and I found myself constantly inspired by our conversations. They have a lot more experience than me in building and thinking about products (especially in consumer), so I greatly look forward to learning more from them and the rest of the team.

  • The team has a rare balance I haven't seen that often, which is a willingness to think big but the ability to still execute quickly. I experienced this firsthand while working with them on the $WRITE token airdrop.

• The Community:

  • Sybil resistance methods can be useful for a lot more than just unique identities, and I think the $WRITE race is a perfect example of this. When what you're signaling is much more than just "I think this is a real person," the implications on the strength of a community are tenfold. I still remember those who gave me even just one vote in the race, and I'm sure others feel the same. It's also probably one of the only "governance" functions that have an active voting participation of greater than 30% (without using delegation).

  • While right now I think the community is already very strong on Twitter and Discord, I'm very excited by how the physical world will eventually play a role in this too. I'd imagine one day we'd have a coworking space, and be able to work together with DAOs and creators who want to sit in for a day or two. I see Mirror as being a natural fit for this, given the product and community ethos.

• The Data:

  • It goes without saying that the possibilities within the data played a huge factor in my decision. First and foremost that lies in the potential applications of graph data science, essentially analyzing our interaction networks on various levels for insights that further collaboration and contribution. The fact that the data could easily drive new products or economic distributions is extremely exciting to me.

  • Crypto-native UX research is also a fairly unexplored field, I think there are some very creative data and user group applications we can make here. For one, user behaviors can be put together from across various platforms (instead of just studying behaviors on my platform like you would in Web2). I can understand auction behaviors using Artblocks, Mirror, and Opensea data to get even stronger user personas to drive product development. I do hope to collaborate with other teams across crypto to collectively improve the UX of Web3.

  • Lastly, the creative space within data-driven design of tokenomics (with a focus on continual distributions) cannot be understated. Closely studying the repeated game theory and micro-economies that will come from Mirror should lead to some innovative token mechanisms and tools. I'm keen to explore how protocol token distributions can inform different approaches for creator/DAO tokens on the platform (and vice versa).

Now I want to emphasize that there is no single path through crypto, but I hope that sharing my experiences and how my thinking/methods of navigating the space led me to my dream job will be helpful to someone out there. I want to give a huge thank you to everyone who has supported me and come along on this journey thus far!

Thank you to Elena for proofreading and providing very helpful comments 🙂

“Everything has a personality; everything sends an emotional signal. Even where this was not the intention of the designer, the people who view the website infer personalities and experience emotions. Bad websites have horrible personalities and instill horrid emotional states in their users, usually unwittingly. We need to design things — products, websites, services — to convey whatever personality and emotions are desired.”

Ethereum's personality is someone who is extremely inscrutable and easily misunderstood. To make matters worse, most users don't even think about it as interacting with Ethereum when they're using your interface or a wallet. If you're ever in the live chat of an Artblock's auction, you'll notice as soon as an auction ends there are at least a dozen people complaining that it is Metamask's fault that they didn't get a mint. I think in the last year the UX of many dapps on Ethereum has improved greatly in both product interaction and explainability of transactions. For the most part, dapps don't just leave you with a loading spinner after you sign the transaction anymore.

Even with the design of dapps is generally improving, I'm not sure how deep UX research has gone yet. When I see data analytics or research on various protocols, users are mostly treated as homogenous. This is somewhat shifting with some of the analysis I've seen of Uniswap V3 liquidity providers and Rabbithole Questers, but even those are still heavily focused on just confirmed transactions on-chain. From my own experience, most of the emotion evoked and behavioral quirks happen when I'm submitting, waiting, speeding up, or canceling my transaction. For some applications, the user might leave and go do something else after submitting a transaction. But for a product like Artblock's auctions, they're going to stay around until the confirmation has happened, likely checking anything they can for updates and with compounding anxiety.

I think we can do a much better job of understanding user behaviors and frictions by starting to leverage the mempool more. The mempool is where unconfirmed transactions are temporarily stored by a node. This means if you submit, speed up, or cancel your transaction then those actions will show up in the mempool first. It's important to note that the data from the mempool is not stored in the node, so you can't query historical data the same way you could on confirmed transactions. For example, you could observe that someone submitted a few transactions, sped up the transactions quite a few times at the same gas price, and saw confirmations 20 blocks later. I believe studying their behavior throughout the transaction lifecycle is a pretty good proxy for user experience and likely emotions they were feeling throughout the whole process. If we understand how different groups of users behave in this cycle, we can figure out how to supplement their decision-making or ease their anxiety. To my knowledge, pretty much only the Ethereum Foundation, All Core Devs, and some wallet teams leverage the mempool data for UX reasons.

UX Research Thesis: By looking at a user's behavior through auctions over time and also their wallet history, we can start to give behavioral identities to different groups of users. From here, we can identify the main issues to try and alleviate. We'll do this by taking a month of Artblocks Auctions data using Blocknative, and layering in the history of these addresses using Dune queries.

This article will be more technical than some of my previous ones since I believe the work can and should be generalized fairly easily. I want to emphasize that my background is not in UX research, I'm purely experimenting with what I think crypto-native UX research could look like.

Data Sourcing and Preprocessing All Auction Data

If you have no interest in the technical bits, skip to the next section on Feature Engineering

Blocknative and Mempool Data Streaming

Using Blocknative's Mempool explorer, you can filter for transactions submitted to specific contracts or from specific wallets. In my case, I wanted to listen to the whitelisted minter contracts for Artblock's NFT contract. You can find the stream that I used here, and save it down if you want to use the exact same setup.

You can find the whitelisted minter addresses with the following query in their subgraph:

{
  contracts(first: 2) {
    id
    mintWhitelisted
  }
}

There were three steps to get to a subscription filter for all purchases:

1. Add the new address with the "create new subscription" button

2. Add the ABIs by clicking the "ABI" button next to the address. In my case I just needed the "purchase" function.

{
    "inputs": [
      {
        "internalType": "uint256",
        "name": "_projectId",
        "type": "uint256"
      }
    ],
    "name": "purchase",
    "outputs": [
      {
        "internalType": "uint256",
        "name": "_tokenId",
        "type": "uint256"
      }
    ],
    "stateMutability": "payable",
    "type": "function"
  }

1. Add filters for the methodName matches purchase (make sure you don't do a global filter)

In the end, your setup should look like this:

To store this data down, I created a ngrok/express endpoint to store in an SQLite database, run locally. I've created a GitHub template with steps to replicate this setup. Probably the most important point to remember here is that you need to include the POST endpoint as part of the ngrok URL when adding it as a webhook in the Blocknative account page.

Key Preprocessing Functions

Multiple Transaction Hashes

When you speed up or cancel a transaction, the original transaction hash is replaced with the new transaction. This means if you want to track a users' transaction through its full lifecycle, you need to reconcile the new transaction hashes with the original one. Assuming you speed up a transaction five times, you'll have six hashes total (the original hash + five new hashes). I reconciled this by getting a dictionary mapping of tx_hash to the new replaceHash, and then replaced recursively.

replaceHashKeys = dict(zip(auctions["replaceHash"],auctions["tx_hash"])) #assign tx_hash based on replacements, just to keep consistency. 
replaceHashKeys.pop("none") #remove none key

def recursive_tx_search(key):
    if key in replaceHashKeys:
        return recursive_tx_search(replaceHashKeys[key])
    else:
        return key

auctions["tx_hash"] = auctions["tx_hash"].apply(lambda x: recursive_tx_search(x))

Blocknumber Issues

Dropped transactions have a blocknumber of 0, so to deal with this I sorted my dataframe by timestamp in ascending order, and then did a backward fill so the 0 would be replaced by the correct blocknumber it was dropped in. This is important fix for feature engineering.

auctions = auctions.sort_values(by="timestamp",ascending=True)
auctions["blocknumber"] = auctions["blocknumber"].replace(to_replace=0, method='bfill') #deal with dropped txs that show as blocknumber 0

Dealing with Mints Outside of Main Auction Period

For most projects, the artist will mint a few pieces before the auction opens up to the public. Some projects don't sell out right away, so you will get mints still occurring a few days after the auction has opened. My analysis is focused on the key auction period, mostly the first 30 minutes. To get rid of the two mint cases above, I removed outliers based on blocknumber.

to_remove_indicies = []
for project in list(set(auctions["projectId"])):
    auction_spec = auctions[auctions["projectId"]==project]
    all_times = pd.Series(list(set(auction_spec.blocknumber)))
    to_remove_blocktimes = all_times[(np.abs(stats.zscore(all_times)) > 2.5)]
    if len(to_remove_blocktimes)==0:
        break
    to_remove_indicies.extend(auction_spec.index[auction_spec['blocknumber'].isin(to_remove_blocktimes)].tolist())
auctions.drop(index=to_remove_indicies, inplace=True)

Adding on Dutch Auction Prices

For all projects in the dataset besides project 118, a Dutch auction price format was used. I took the mint price data using a dune query, and then merged it onto the dataset. I had to use a forward and backward fill for the blocks that had mempool actions but no confirmations during the auction.

auction_prices = pd.read_csv(r'artblock_auctions_analytics/datasets/dune_auction_prices.csv', index_col=0)
auctions = pd.merge(auctions,auction_prices, how="left", left_on=["projectId","blocknumber"],right_on=["projectId","blocknumber"])
auctions.sort_values(by=["projectId","blocknumber"], ascending=True, inplace=True)
auctions["price_eth"].fillna(method="ffill", inplace=True)
auctions["price_eth"].fillna(method="bfill", inplace=True)

Feature Engineering For Each Auction

If you have no interest in the technical bits, just read the parts in bold and skip the rest.

In data science, a feature is a variable that is calculated from the larger dataset to be used as an input in some sort of model or algorithm. All features are calculated in the preprocess_auction function and are calculated per auction rather than combining all the auctions into a feature set.

The first set of features are the totals for transaction states, and was a simple pivot_table function:

• number_submitted : total number of transactions submitted

• cancel : count of transactions that ended in canceled

• failed: count of transactions that ended in failed

• dropped: count of transactions that ended in dropped

• confirmed: count of transactions that ended in confirmed

I mentioned earlier some data was not captured for auctions due to various issues, these transactions were dropped from the dataset.

The next set of features included their gas behavior. The key concept here was capturing how far away their transaction gas was from the average confirmed gas per block (shifted by 1 block). Then we can create features for the average, median, and standard deviation of gas price distance over the whole auction. There are a bunch of transposes and index resets to get the blocknumber columns in the right order, but the important function is fill_pending_values_gas which forward fills the gas price between actions captured. This means that if I put in a transaction at blocknumber 1000 with a gas of 0.05 ETH and my next action wasn't until blocknumber 1005 where I sped up to 0.1 ETH gas, then this function will fill in the blocks between 1000-1005 with 0.05 ETH.

def fill_pending_values_gas(x):
    first = x.first_valid_index()
    last = x.last_valid_index()
    x.loc[first:last] = x.loc[first:last].fillna(method="ffill")
    return x

The third set of features were calculating the total number and frequency of actions taken in the auction. Here we start with a pivot of total actions (speed ups) per block, with some special calculations for getting the first instance of pending for each transaction:

get_first_pending = df[df["status"]=="pending"] #first submitted 
get_first_pending = get_first_pending.drop_duplicates(subset=["tx_hash","status"], keep="first")
auctions_time_data = pd.concat([get_first_pending,df[df["status"]=="speedup"]], axis=0)
time_action = auctions_time_data.pivot_table(index=["sender","tx_hash"], columns="blocknumber",values="status",aggfunc="count") \
                    .reindex(set(df["blocknumber"]), axis=1, fill_value=np.nan)

From here we get to average_action_delay in three steps:

1. we take the number of actions per block (yes some people sped up transactions multiple times in the same block)

2. we drop the blocks with no actions, and then take the difference between the blocknumbers that remain. We add a 0 for each additional action taken per block.

3. Taking the mean across the differences and added zeroes gives us the average_action_delay

def get_actions_diff(row):
    row = row.dropna().reset_index()
    actions_diff_nominal =list(row["blocknumber"].diff(1).fillna(0))
 
    #take the blocks with muliple actions and subtract one, then sum up. 
    zeros_to_add = sum([ actions - 1 if actions > 1 else 0 for actions in row[row.columns[1]]])
    actions_diff_nominal.extend(list(np.zeros(int(zeros_to_add))))
    actions_diff = np.mean(actions_diff_nominal)
    if (actions_diff==0) and (zeros_to_add==0):
        return 2000 #meaning they never took another action
    else:
        return actions_diff

total_actions is much simpler, as it is just the sum of actions across the pivot.

time_action["total_actions"] = time_action.iloc[:,:-1].sum(axis=1)

The last time-dependent feature is block_entry, which is an important one due to the introduction of Dutch auctions. Essentially this tracks what block the transaction was submitted on since the start.

get_first_pending["block_entry"] =   get_first_pending["blocknumber"] - get_first_pending["blocknumber"].min()

entry_pivot = get_first_pending.pivot_table(index="sender",values="block_entry",aggfunc="min")

price_eth is added as a feature as well, which is tied to the block_entry point.

The last set of features were based on a Dune query, specifically the days since the first transaction, total gas used in transactions, and the total number of transactions. To get the address array in the right format I used the following line of code after reading in the SQL data:

all_users = list(set(auctions["sender"].apply(lambda x: x.replace('0x','\\x'))))
all_users_string = "('" + "'),('".join(all_users) + "')"

The dune query for this is fairly simple. I pasted the addresses string under VALUES, and made some CTEs to get the features I wanted. In the final SELECT I tried to add each address's ens as well. You can find the query here: https://dune.xyz/queries/96523

Lastly, we just merge in the per wallet data on days active, total gas used, and the total number of transactions.

auctions_all_df = pd.merge(auctions_all_df,wh,on="sender",how="left")
auctions_all_df.set_index(["sender","ens"],inplace=True)

With all this done, we're finally ready to run some fun unsupervised learning algorithms and try and validate our hypothesis on user groups.

Clustering and Visualizing User Groups

Before I started this project, I had expected to see the following user groups pop out of the data:

• Set and Forget: There should be two groups here, those who set a transaction with really high gas and an average/low gas and then don't touch it for the rest of the auction.

• Speed Up: There should be two groups here as well, those who are speeding up often and updating the transaction directly as a factor of gas prices and those who are speeding up often but with basically no change in gas price.

I was very interested in validating these groups, seeing how large each group was, and seeing if any users moved between groups over the course of many auctions. The easiest way to do this was to use unsupervised machine learning to identify clusters of user groups based on the variability across all features. Essentially this is like looking at the distribution of income in a state and then splitting it into sub-distributions for different income concentrations, geographic coordinates, and age. Note that this isn't binning, where the distribution is split into equal ranges - it is instead calculated based on the density of observations within the whole range. The approach we will take is called "unsupervised" because our dataset doesn't have any existing labels, rather than something like a regression where there is a value being predicted that can be verified as right or wrong.

The algorithm I decided to use is called k-means, where k stands for the number of clusters you are expecting to identify. Each cluster has a "centroid", which is like the center of a circle. There are various methods for figuring out how many clusters are optimal, and the two I used were elbow points and silhouette scores. Those are both fancy ways of asking,

"does each additional cluster help increase the density of the cluster (calculated as the average distance of points in a cluster from the centroid) and maintain adequate separation between clusters (no overlap between two clusters)?"

I found that 3 clusters were optimal in terms of most inertia improvement while keeping a high silhouette score (around 0.55).

With clusters chosen, we want to be able to visualize and verify their existence. There are over 15 variables, so we need to reduce the number of dimensions in order to plot it. Dimension reduction typically relies on either PCA or t-SNE algorithms, in our case I went with t-SNE. Don't worry too much about understanding this part, these algorithms essentially capture the variance across all features to give us X and Y components that maximize the spread of points from each other.

Let's look at project 118, LeWitt Generator Generator, from August 4th:

After taking a look at the sub-distributions for each variable and some data examples, I was able to categorize the clusters. The Orange cluster is the group that is speeding up the most while also submitting slightly lower gas transactions on average. Blue and Green clusters exhibit similar behavior to each other, but addresses in Blue typically have less history than addresses in the Green cluster.

Looking at the overall picture, it seems the original hypothesis on "speed-up" and "set high and set low" producing two groups each was wrong. Instead, we have a single "speed-up" group (Orange) and a single "set-and-forget" group (Blue and Green are behaviorally the same). I think the new (Blue) versus old wallets (Green) in the "set and forget" group probably carries a lot of overlap in actual users, where users just created new wallets to bid on more mints. Based on their impatience in average_action_delay and lower-than-average average_gas_behavior, the "speed-up" group reads to me as less experienced and more anxious than other users. What did surprise me is that the speed-up group makes up a smaller proportion (30%) of total bidders, as I had expected that group to make up 60-70% of bidders instead.

Now, where this user behavior study gets really interesting is when comparing project 118 (the set price of 0.75 ETH) to project 140 (Dutch auction with the price decreasing from 1.559 to .159 ETH).

Here's the clustering for project 140, Good Vibrations from August 21st:

We can see that now most of the clustering variability comes from block_entry, price_eth, and all of the gas_behavior features. This is a big departure from the main variables for project 118. In 118, the set price meant that people entered the auction in a fairly uniform distribution (quantity left didn't seem to matter), and the "speed-up" group submitted actions up fairly endlessly.

In project 140, we don't see the same difference in actions within average_action_delay or total_actions , instead, we see a new group (Orange) entering at a very late stage block and setting far below-average gas prices as seen in average_gas_behavior. If I was to try and map this to the clusters in 118, I believe the "speed-up" group has now become the "greedy" group (Orange) that is entering late and bidding a low gas. The Green cluster probably represents users with more experience than the Orange cluster, but who are still transitioning between Orange and Blue in terms of their behavior. The "set-and-forget" group maps pretty well to the "early-grab" group (Green and Blue) since they all exhibit pretty good patience and an adequate safety net on gas bids.

I call the Orange group "greedy" not just because of their behavior, but also because of their rate of failed transactions.

For project 118, fail rates are of the "speed-up" versus "set-and-forget" groups are within 10-15%.

For project 140, the fail rate of the "greedy" cluster is around 69% versus the "early-grab" group at around 5-15%.

Overall, my read of this is that the group's bad habits and emotions were amplified - it feels to me like we made a tradeoff in anxiety → greed. This may have made the auction less stressful, but ultimately led to more users being upset (due to failed mints).

I'm sure there's a more granular analysis that can be done to segment the auctions further based on factory/curated/playground or by artists themselves too. This will only get more interesting and complex as the community continues to grow, and emotions play a larger factor in both a single auction and on if they return for future auctions.

This study of multiple auctions helped us validate our assumptions, understand the proportions of user groups, and see how users' good or bad behaviors shift over time (and other parameters). Now we need to plug it into the rest of the product cycle process.

Where We Go From Here:

The reason that I chose just Artblocks auctions for this and not a mix of platforms is because I wanted to look for a place where the variability in terms of interface and project type is mostly controlled. This should have given us fairly consistent users and behavior types.

This is just the start of a UX research cycle, so ideally we could continue in the following steps:

1. Use an unsupervised machine learning algorithm to identify the user groups (clusters) and see how many people are making "mistakes" when entering an auction. This is the step we covered today.

2. Create a new user interface, such as a histogram view on the bidding screen, or showing historical data on when most people usually enter/crowd the auction and at what prices. Anything to give both current and historical context to the user, especially those from the speed cluster.

3. With each auction, run the mempool/wallet data through the created algorithm to see if the user groups have shifted and if specific users have "learned" to participate in the auction differently (i.e. did they move between user groups). I think that the most value can be found in this step if done well. Using ENS or other identifiers to help supplement this grouping would be exponentially helpful too

4. Based on the results, continue to iterate on the user interface and design. You could also run more informed A/B testing since you could even figure out what screens to show by making an educated guess based on the users' last cluster (or use label propagation for new users).

The Dutch auction-style change is also an example of step #2, and we were able to see a clear shift in user behaviors. While typically this sort of research and testing is focused on increasing engagement or conversions, here we are optimizing for the user's ability to learn and improve instead. This may become even more robust if this was iterated in a multiplatform context, so that we can study how someone is learning at an ecosystem level (maybe even supplement with Rabbithole data and user profiles). Since my Artblocks user research is all based on publicly sourced data, it can be replicated and supplemented by any other auction/sale platform. Crypto could be the first industry that would have a synchronized and transparent set of user groups and UX research, to be applied in products and academia. Nansen wallet labeling is already a step towards this, but it's different when teams from different products build this up from various facets and approaches.

What I would eventually envision is using data to build on the following user personas (with subgroups/levels within them):

• I want to buy a Fidenza, so I can either buy one through a private sale, bid on one in an auction myself, bid on one in a prtyDAO bid auction, or buy a fraction of one with fractional.art

• I like Fidenzas in general, so I'll just buy the NFTX Fidenza index token or an NFT basket of Artblocks Curated on fractional.art

• I'm a collector already, so I want to swap or bid on a Fidenza using a curated set of NFTs and ERC20s I already hold (using genie-xyz swap).

• I like the rush of acquiring through initial mint versus secondary market, and heavily participate in auctions like Artblocks live mints.

I hope you found this project interesting and/or helpful, I had a ton of fun with it. Thanks to the folks at Blocknative for setting me up, and the community at Artblocks for answering my many auction questions. As always, feel free to reach out with any questions or ideas you may have!

You can find the GitHub repo with all the data and script here. The script may be a bit hard to read since I’m still refactoring and cleaning it up. The script and some of the analysis here may get updated as I analyze the last few auctions of August for new patterns.

Airdrops have come a long way in the last year, most notably starting from Uniswap's 400 UNI token airdrop in September of 2020. From there, we've grown into airdrops that are calculated based on the frequency and volume of interactions with different facets of a protocol. Below is one such example:

banteg

@bantg

You've been photographed. Expect a gift.

243

7:32 AM • Jul 15, 2021

We've also seen NFT drops recently that rely on more standardized tools built for quickly setting rules/requirements and rewards/distribution. None of these methodologies are bad, but I do believe there's still a lot of room for improvement. This could come from the distribution model, drop frequency, or even the token rules themselves. In this proposal, I'll be focusing on a new distribution model, one that combines data from user interactions in the Mirror $WRITE race, on Twitter, and in Mirror-related Ethereum transactions.

Rewards, Incentives, and Governance:

The Mirror $WRITE race is a vote-based competition that occurs once a week, where the top ten writers get 1 $WRITE token. This gives them access to the platform as well as 1000 votes for the future rounds of the race. The structure is not dissimilar from most Sybil-resistant approaches where a validated party votes in a new party, however, the propagation of rounds is a lot slower as it only happens once a week. This means it takes a lot longer to really diversify away from the friends/communities of the genesis writers of the $WRITE race. For new participants who aren't very close to that initial circle, it becomes harder and harder to win as the top 10 vote floor continues to rise:

ilemi

@andrewhong5297

Each week 10 new writers gain access to the platform, meaning a potential 10,000 more votes to spread around. It's currently round 20, so we should have 200,000 available votes from winning writers alone. Here's the distribution of how many votes you needed to get to top ten:

17

4:06 PM • Jul 13, 2021

While increasing the # of winners and base vote allocation or gamifying the prompt system could incrementally improve the race, an airdrop of $WRITE tokens would offer a much faster method of reducing the influence of the starter group and increase the diversity of writers (winners) on the platform.

A key first decision here is that airdropped tokens should not affect the current vote allocation for each participant in the $WRITE race. But, they can still be redeemed for publications**, at 1 $WRITE token per publication.

My first thoughts when structuring this airdrop model were around "what should an airdrop really be targeting?" The three pillars that came to mind were:

1. Rewards: How can we fairly compensate early users for their support and risks that were taken? They chose to publish on Mirror rather than their existing publishing platform, carrying both new funnel and smart contract risks.

2. Incentives: Knowing that this model will set a precedent for rewards distribution criteria, how will that affect voter/creator/contributor behaviors going forward?

3. Governance: How can the balance of power be shifted in a way that diversifies the writer community much faster?

From there, I set three goals for the token formula:

1. Rewards: Creators should be rewarded for their usage of web3 platform features, and contributors should be rewarded for the value (ETH) they've provided to creator projects

2. Incentives: We should also incentivize contributors to support a larger array of creators, rather than just the same two or three creators (or a single community).

3. Governance: We should try to amplify anyone who bridges different communities by giving them more tokens.

We know that for Mirror to thrive long-term, there needs to be diversity in creators with an equally diverse set of contributors. However, with over 7000 users it can be hard to choose who gets what allocation off of usage alone. This is where the social graphs enter the picture, as we can use that data to identify who might be best from a community perspective to allocate greater or fewer airdrop tokens to.

Token Reward Formula:

For all of the following, $W$ stands for a weight between 0-1 and $T$ stands for $WRITE token.

1. First, let's take care of the community diversification weighting based on centrality:

• Betweenness centrality is a measure that highlights the nodes that bridge the most communities. This serves to incentivize everyone to support and collaborate with more diverse communities in the future. See this post for details on this metric and the methodology. We multiply it by 50 to give it more weight.

• While the linked post above does not account for the difference in edge/relationship data type, so a mention and a 1 ETH contribution would have the same non-scaled weight), for this airdrop, we converted the multi-edge graph into a weighted directional graph. This means normalizing each edge by total mentions/contributions/votes a certain user has given, and then weighting mentions, contributions, and votes by 15x, 40x, 40x respectively. Another important change from the original post is the use of a directional graph.

2. We still want to make sure creators and contributors are getting fairly rewarded. Let's define the T_baseReward formula as:

• V_votesAllocated represents the current allocation of votes someone has in the $WRITE race. This is divided by 1000 to represent the $WRITE token balance. This is weighted by W_voted which represents the total votes used out of votes allocated each week (cumulatively).

• B_hasCreated represents a Boolean (1 or 0 value) for if someone has created at least one entry or block (edition, split, crowdfund, auction).

• V^n_contributed is the value of ETH contributed to creators (across n different block types). This is weighted by the unique # of creators someone has contributed to, max_min scaled into W_contributed. This means if the maximum number of unique creators someone has contributed to is 10, and I've also contributed to 10 creators, then I would have a W_contributed of 1. Depending on the distribution of contributors, if I've only contributed to 1 creator, I would have a W_contributed of something like 0.0833. Thus, we are rewarding those who both contribute to creators and support many creators, in an effort to incentivize that behavior.

• The constants of 0.65 and 0.15 were chosen based on how many total tokens we wanted to distribute.

3. Putting it all together, we get our total $WRITE token reward formula:

Airdrop Results:

The results of the airdrop can be found in the following CSV file.

mirror_finalAirdrop_cleaned_w.csv

The data dictionary for this file can be found below:

Mirror $WRITE Airdrop Data Dictionary

What a Reputation-based Airdrop Enables for the Community:

I believe this airdrop has iterated upon everything we built in web2 and web3, mainly due to its combined use of multiple social graphs alongside tokenized ownership. As I've covered before, betweenness centrality is a measure that helps us target a more resilient and diverse network. Tokens are a great way to reward those who have fulfilled that behavior historically, and also as an incentive mechanism to further that diversity and resilience (support) in the future.

This proposal indirectly tackles the issue of long-term ownership of tokens, where many receivers (other than core investors and the product team) sell the tokens soon after the drop. My hope is that if the tokens are distributed to an aligned subset of the community, we won't see this issue appear as strongly. I think the Ohmies are a great example of this done well.

While this airdrop is at a protocol level, there's no reason why we couldn't go more micro in our approach and provide creators and DAOs betweenness scores for their specific social graph of contributors, voters, and Twitter interactors. They could leverage that data to then airdrop their own tokens as well. Ultimately, I think that what has been done here can be further expanded and productized to help all of us gain better visibility into and reward/incentivize each step of the community flywheel created by the team at 0xSTATION:

Sources and Methods:

• All data (besides identity mappings) were publicly sourced. Please see the data dictionary for details on sources.

• Data analysis and visualization were performed in Python, using a mix of Pandas, Numpy, Scikit-learn, Networkx, Matplotlib, Seaborn, Plotly, Cdlib, Gephi, and Neo4j. The Neo4j Dashboard, Bloom, and Graph Data Science (GDS) packages were used as well for ancillary analysis. Desmos was used for early exploration and tuning of the reward formula.

Typically in Web2, users are "rewarded" by an algorithm which will highlight them based on the engagement and attention they bring to the platform. Thus, their reputation score is just the number of likes or followers they have - regardless of who those vanity metrics come from. In Web3, we typically reward with tokens representing the value of the protocol or product. These tokens also carry a lot of power in the form of voting and other privileges. As such, a score that signals not only influence but also how supportive or aligned a user is with the rest of the community will become increasingly important over time.

For this post, we will be focusing on creating a reputation score for Mirror users (voters, writers, contributors) based on where each user sits on an interaction-based social graph across Ethereum, Twitter, and Mirror data. I chose Mirror for three reasons:

1. I'm already very familiar with their community and product

2. They have verifiable consolidation of identity on their platform (Ethereum <> Twitter), allowing me to layer social graphs together.

3. Their product and governance are heavily user-to-user interaction focused, something most products on Ethereum don't currently have (we mostly interact with pools or marketplace protocols).

Social graphs themselves are not new, but selectively layering them will open up quite a few new doors of usability and meaningfulness. There are two main reasons for this:

1. Enabling new applications: We can use this graph data to create a context-dependent community-specific reputation score, which is then applicable in many situations.

2. What the data represents: The creation of a representational network built off of scarcity creates a social graph that proxies how users support one another.

Enabling new applications: The way that we understand the layering affects what we do with the data. The way I see it, Ethereum is a base layer social graph that everything else is built upon. Platforms like Mirror and Twitter are contexts that sit on top of this base layer and shift how we see the connection of users across the space.

Because I wanted to analyze only users of Mirror, I took just a subset of available Ethereum and Twitter data. Those who have a high community-specific reputation score could get a larger token airdrop, be delegated for roles (or more votes) in treasury management, or get preferential access to new protocol features. There are also many more such contexts like Mirror built on top of Web2 + Web3 platforms, such as NFT communities (Cryptopunks, BAYC, Blitmaps) and gaming communities (Axie Infinity, Darkforest).

As DAOs start working together more and the metaverse becomes more interconnected, we'll see more communities (and contexts) overlap. I imagine that studying how people and communities interact within and across different contexts of the social graph could produce mixed-community reputation scores which could be applied in quite a few different situations. For example, this score could be used for multi-token or collaborative NFT airdrops, as well as choosing leaders for partnerships and programs like rabbithole's pathfinder initiative.

What the data represents: The kinds of data collected and used affect the social graph as well. Using data from $WRITE votes, funding contributions (across editions, splits, crowdfunds, and auctions), and Twitter mentions, I want to represent three buckets of scarcity (respectively): belief, capital, and attention.

I believe that analyzing how interactions connect different people across these three buckets gives us a proxy for how much they support each other. I also believe that networks of support give us a more accurate representation of communities within social graphs. These two assumptions gave me confidence in using a concept called "betweenness centrality" as a primitive for a reputation score.

The selection of data representation and contexts was key to the form the social graph ultimately took. If I wanted to proxy just willingness to contribute, then I would probably create a completely create node schemas based on different kinds of products/categories of creators rather than a pure user-to-user graph. This would likely shift the shape of the social graph completely.

Using Graph Data Science to Create a Community Reputation Score: Betweenness Centrality

Let's try to target and quantify the strong connection points of the social graph of race participants, writers, and contributors. I've chosen to use a concept called "betweenness centrality" to represent the score of each node. Betweenness centrality is calculated using an algorithm that calculates the shortest path between all nodes for all nodes).

Community clusters in a social graph usually look something like this:

For Mirror, the base social graph of who has voted for who looks like this:

However, this doesn't have an outside-of-race social context yet. Let's layer in Ethereum transaction-level data - this is limited to Mirror-related transactions between participants, such as sending a split, funding a crowdfund, and buying an edition or reserve auction.

Now we'll add Twitter data too, which will link nodes based on who has mentioned another participant in their last 2000 tweets.

Showing the interactions (edges) in a cleaner fashion, the mix of interactions across the social graph looks like this:

Now getting back to the original point, the expectation is that some of these nodes between large clusters would get a higher weighting than others since they've connected the paths across the most other nodes.

This is called "betweenness" and is a factor I believe to be very important when trying to proactively grow a community. The base idea here is that the higher the "betweenness" factor a person has, the more likely they will lead to creating connections and branches that build up a more diverse community. There have been a few research papers that highlight the beneficial effect that nodes with high betweenness have on the diffusion of a community as well as building the resilience of a network.

Some of you might be wondering why I chose betweenness centrality over something like closeness or degree centrality. Those latter two metrics highlight pure influence, and I don't think reputation in a community should be based on just those who already have that level of influence. The Graph Algorithms textbook by Neo4j puts the concept behind betweenness very well:

"Sometimes the most important cog in the system is not the one with the most overt power or the highest status. Sometimes it’s the middlemen that connect groups or the brokers who have the most control over resources or the flow of information. Betweenness Centrality is a way of detecting the amount of influence a node has over the flow of information or resources in a graph. It is typically used to find nodes that serve as a bridge from one part of a graph to another."

A lot of Web2 has been about concentrated influence and echo chambers, I believe Web3 should try to enable and incentivize creating bridges as much as possible instead.

For anyone who wants to see the scores, check out the sheet here. Note that these values have not been weighted based on data/edge type, so some users may have higher betweenness due to Twitter interactions rather than Mirror/Ethereum interactions.

Why is this Methodology Important to the Score?

We've already discussed some of the applications of this score, but the solution I used may still appear unnecessarily complex or overengineered. I think this methodology is required to provide composability while also measuring reputation in a way that is much tougher to game.

Composability: As mentioned before, this graph is built up layer-by-layer. The data elements are all publicly available to collect, the model follows a search algorithm and is flexible to whatever nodes or edges (users or interactions) are chosen, and once the pipeline is built once it can be reused or forked for any set of tweaks. Hopefully in the future, all the available data components just sit in a user interface and this model becomes drag and drop. From there I imagine you could export scores or connect directly to something like disperse.app or galaxy drops.

Durability: The problem with many scores and measures is that once they have been publicly used once, then people can start to figure out how to game the system. This is especially true for anything that is just user <> protocol interaction based. Creating a model that is user <> user <> user dependent is harder to game because the users in the current community won't necessarily reciprocate interactions with a bad actor. Actions are also compounding, so having just one interaction (or one type of interaction) will not be good enough to get a higher score). Even if they do find some way to game it, then congratulations you now have another active participant who is contributing to the community.

These two elements allow for an overall stronger reputation score mechanism, which I believe justifies the efforts it takes to get there.

In Conclusion:

I would like to continue iterating this research and possibly use this kind of score for specific use cases from protocols and communities (please reach out if interested). If you've made it this far and want to contribute to this analysis with ideas or technical expertise, I do invite you to join ImagDAO for further work on social graph data and decentralized identity).

The next steps include more data collection, setting up a Neo4j dashboard for others to play with, and of course applying more data science algorithms. Machine learning and community detection will likely take on a bigger role in the future as we try to predict how the social graph will grow or try to target specific sub-communities.

I mentioned before that this analysis was only possible because of the verifiable consolidation of identity by Mirror's platform (Ethereum <> Twitter). Continuation of this work heavily depends on an identity consolidation API as well, so anyone working in this context please connect with me as well (looking at you Ceramic and Gitcoin 👀).

The data and scripts may be shared publicly later once I refactor and figure out how to protect the privacy of users in this social graph.

Special thanks to Ben Schecter for all his help and ideas in reviewing this post

If you want to support more of this research, consider bidding on the auction for "Decentralized Community" or contributing to the split below:

Token #4194

Social Graph Searchers

Yet all-in-all, your digital worth is measured by the engagement (and advertising revenue) garnered. Sometimes you get a share of that revenue, but for 99% of us, the value of our digital identity is locked within the platform. This doesn't just apply to Facebook, Twitter, or Reddit - the "platform" can be our workplace as well.

Let's say you've worked at a company for a few years, and you've decided to move on. When you leave the company, your reputation still stands with individuals you've worked with and in the many programs/emails you've created, but otherwise, the company's reputation is your reputation. This dynamic has slowly changed with the rise of the internet - hence the rise of the personal brand. However, your personal brand is still built off of your own word (which may or may not have much value when first starting out).

Our digital identity exists in fragments, it isn't flexible past the platform it was built on, and it has little reusability other than cross-platform authentication. We don't own our digital identity, and it's not composable at all. Now, what if you could own all of the pieces of your digital identity permanently, while also controlling who you reveal that data to and how it's represented? This would enable us to tell more compelling stories and have a user-controlled value function for identity. I think we're (finally) not so far off from a future where all of that is possible.

But the phrase "digital identity" is really complex and abstract. Let's start by sorting it into a treemap of identity type -> platform type -> transaction type -> action type.

Hierarchy of Digital Identity

For identity types, I've split it into physical and digital identity (physical is hidden for this article). The next layer is platform types, which I've just split into Web2 and Web3 for now. In the future, it may not be so simple to separate which platform different types of transactions are coming from. Now for the meat and potatoes of this breakdown, transaction and action types:

Social Transactions: These are transactions where you have a set individual or group of individuals you know you want to interact with.    - Individual: This is just like Venmo when you pay or interact with someone for any number of reasons    - Community: These are actions that go towards some group cause, and can be anything from funding a treasury to governing actions for spending from that treasury.    - Gaming: This is another way of saying actions where the result/end party of the action isn't always known. In many games, this is common, like going on raids or adding to/interacting with some city on your journey.

Protocol Transactions: These are transactions where you interact with a product for its specific benefits or utility.   - Selling/Acquiring: These are marketplace actions, where the protocol is used to conduct a peer-to-peer interaction.   - Staking: This is an action where you are staking your belief in a protocol, acting as a core member of the product community. The actual use of the stake may vary, but the signaling is the same.   - Use: I'm defining "use" here as the primary use case action of the protocol. Sometimes that may be "selling/acquiring" or "staking."

Conversations: These are transactions where you are trying to work with (or against) others to directly push towards some goal or understanding (in a friendly and sustainable way). In the context of this article, these conversations are focused on the growth and governance of communities/products/protocols.    - Proposals: These are like Ethereum Improvement Proposals, but can be much more general too.    - Moderation: Some examples are Reddit mods and Twitch chat mods, where they enforce rules set by proposals.    - Feedback: This is usually in direct response to proposal or moderation actions. A lot of this is providing a larger context and bridging the conversation between old and new users.

Contributions: These are transactions where you either create something or engage (without conversation) with something someone created.    - Consumption: Actions here include liking or saving an article, or running npm install on someone's node js package.    - Creation: This includes creating new ideas, products, integrations, and much more.    - Sharing: These actions contribute directly to network effects, and typically act as bridges across platforms and communities.

Our web of actions ultimately forms our digital identity. I know that even this is still pretty abstract, so let's look at how this could be realistically represented and used.

Representing Digital Identity in Practice

Here's the structure I've been working with on what digital identity on a more technical level:

The first layer is comprised of transaction types (the four we talked about) and social graphs. I didn't include social graphs as part of the identity breakdown since the graph's nodes are each their own digital identity, connected by lines that represent transactions. I've labeled this whole layer as "not composable," meaning I can't detach them from an identifier or really move them around from platform to platform to form a single identity. While transaction data is immutable, it is typically exportable and can be represented in tokens. Social graphs are more difficult to manage in a portable way, first and foremost because current platforms limit the ability to export that data. This has been a heated topic of discussion for years now, so there's a large lack of transparency on what they actually look like and how to break them down. However, new products and approaches allow you to build up and represent the nodes closest to you in the social graph - which can then also be stored into tokens. That representation as tokens brings us to the second layer of the chart.

The second layer (and onwards) highlights categories and products that allow us to represent that transaction data and/or social graph as tokens. Since tokens have the qualities of existence, flexibility, and reusability - then by the transitive property - our digital identity now does as well. I can move around these tokens at will to different accounts and in different combinations. If we add on the permission/connection rules wallets already have to the tokens' data, we're coming very close to the picture I painted at the start.

Let's talk about what's behind the tokens of this composable digital identity layer =>

Tokens built off of transaction data will likely rely on different models and algorithms that may start centralized and then transition to community-governed. These models will take different combinations of transaction and action types depending on what they want to represent. SourceCred allows custom-set rules for measuring "contributions". Spectral.finance creates machine learning credit score models that may take on a Numerai style many-model architecture in the future. I'd expect Rabbithole.gg to have some token(s) representing levels of different skills in the future, where the levels are calculated with different models. How we tokenize web2 account transaction data probably won't be that different in structure data => model => token. They may rely on existing web2 aggregators like orbit.love and their model for calculating reputation by community.

The fungible versus non-fungible identity token approach here reflects what kind of identity economies will be generated. Fungible tokens act as a standard and stable reputation coin that holds regardless of the type of community or contribution. These may help facilitate creative cross-community collaboration and creative talent acquisition, leading to new treasury management strategies and considerations. Non-fungible tokens are identity bonds that can be staked or lent out, which growing in value over time as an individual's actions continue to evolve. Identity forges (a protocol where you and I place our creditworthiness NFT to create a new representative token) will increase the utility and complexity of this token. Multiply this across all proof tokens and we have the beginnings of an "identity marketplace."

Earlier I said that social graph tokens would likely be represented by your closest nodes instead of the full graph. This comes from the two popular approaches I've seen today:

1. Sybil Resistant: The theory here is that if I am verified by enough other "people" who are real, then I am real. BrightID's health score reflects this, and it seems to be working well.

2. Tiered Entry: Let's start with 100 people who we know are real and trusted, and then bring in more people based on the selection/voting of those 100 people

With both of these approaches, your social graph is the people who verified or voted for you. Whether or not these decentralized identities (DIDs) aggregate under addresses or the other way around largely depends on how we interface with the digital world - I believe it will be the former for the same reasons we don't "login" to pages with our ENS. The tokenization of these graph shards could take many forms and will likely be layered upon by proof tokens.

All of these identity tokens represent the earliest primitives, and I'm sure they will be built upon with additional complexity as our mental and technical understanding of "trust" and "digital identity" evolve. With that as the expectation, our digital identity becomes more like a portfolio, requiring a new set of tools. We've seen community SaaS tools grow over the last few years that help product teams increase engagement with their community of users across platforms, but what I'm pushing for here is an identity management tool for the users themselves.

Building an Identity Management tool

While we have asset management tools like Zerion and Zapper, there is no "identity management tool" yet. This isn't too surprising as proof tokens are largely still under development for the Ethereum transaction data side, and most protocols still link web2 accounts independently (like mirror.xyz does when you sign up for the $WRITE race). For now, I view the product stack as follows:

The data aggregation layer should be managed by other protocols, where each protocol will have decentralized governance of standards and usage. An identity management tool would play with everything in blue, starting as a specialized SDK between all our identity tokens interactions/metadata for wallets to use. While there are many features required for managing identity, I want to start with the actions of verification and permissioning.

Let's focus on connecting unique web2 identifiers (i.e., Github and Twitter accounts) to Ethereum address(es). There is a growing list of tools (Ceramic IDX and Serto/Veramo Agents) for representing identifiers and action data in standard ways (DIF, W3). We can build a verification process users have to pass once and then issue a verifiable credential using existing tools onto a decentralized database. Then we build the plugin that fits into a wallet to allow users to manage data sharing with apps. I recently built a proof of concept for this:

tech demo

You also may be wondering what a "snap" is - this is like a plugin/API built directly into the Metamask wallet. I believe that an identity management tool should be controlled by the user in the most decentralized and private way possible, so building it into a wallet makes the most sense.

Other added benefits of this approach are:

1. Developers can stop reinventing the wheel and save on AWS overhead/privacy concerns, requiring just one provider request to the wallet instead (no API key or sign-up process required).

2. Users can stop going through the input account -> sign -> post sig -> refresh flow once every few weeks when a new social dapp launches.

3. Users can start to manage who has access to their identity branches/data since it isn't on a centralized/controlled database.

This third point is critical, as it opens up a whole new world of applications in terms of verifiable storytelling and data ownership/economies. I think that's the ultimate goal - anyone should be able to curate an identity for the story they want to tell, and if they don't have the pieces then they can build (through transactions) or borrow (through a marketplace) up to it.

If you like the vision and want to see this built out more, then pick up some $IMAG to bootstrap the project (I'll also create a group for further discussion about this with supporters - maybe make this a DAO if there's enough interest):

Identity Curators

While I've written about composability of dapps and wallets before, I don't believe I've explicitly defined the word "composable." Generally, I think composability is comprised of existence, flexibility, and reusability. As a pre-condition, everything should be accessible in a permissionless manner (i.e. sufficiently decentralized).

Existence

I believe that right now, most people think of bridges as "how can I get token A transferred from blockchain X to blockchain Y," or slightly fancier as "how can I swap token A on blockchain X for token B on blockchain Y."

In other words, right now the focus is on asset existence - meaning the same asset should exist on every chain. For example, USDC should exist on both Ethereum and Polygon and remain fungible between chains (almost as if they were the same ERC20 contract). Another example would be getting Bitcoin from its chain to Ethereum through the Ren wrapper as renBTC.

A second facet is continuous versus ubiquitous existence. Continuous is the earlier example where my tokens are either on chain A or chain B. However, some tokens may need to exist on multiple chains at the same time, hence, ubiquitous. This can apply to something like a proof token (NFT). Let's say I have an NFT representing my credit score which I can use to borrow with special terms inside some defi dapp. I wouldn't want to transfer this NFT across chains, since I'm likely borrowing from multiple chains at the same time. Hence, the NFT needs to exist on both chain A and chain B at the same time - and remain in sync on mints, transfers, and URI updates. This can be extended to DAOs as well, though this one is harder to think about since DAOs don't have a set contract standard.

A more technical way of framing asset existence is as the data on a blockchain. Usually, we either access data stored in a contract or we mutate the state by interacting with the contract. As far as I know, we don't have a good way of accessing or mutating data between chains, besides what is under research like optics or sharding. This means that currently, smart contracts (the logic and assets) are portable from chain to chain but are non-interactive between them. Instead of the vertical legos we're used to, we really only have horizontal integration right now. Each chain is built up as a walled city, with resources traded around between them all.

Flexibility and Reusability

I think that the scope of asset transfer for understanding bridges is a little narrow. The truth is, we've actually all been using bridges for years now! Let's redefine a bridge to be "the transport, authentication, and ordering of information between two or more parties."

This is very similar to how the internet works today, where data is split and transferred in packets between a web of nodes before reaching their desired destination (IP) and being put together in a specific order to represent the original data. Our extra layers of complexity with bridges involving blockchains come from security and verification:

• As chain B, can you trust the message being sent across the bridge is true, like when using wrappers like renBTC?

• Conversely as chain A, can you trust that the message you sent through was committed into a transaction (i.e. calling a smart contract on chain B)? I won't go deep into this today but you can learn more about the technical difficulties here and here.

Now, let's think about what common party-to-party bridges exist in the our ecosystem:

• database <> database (bridged by ACID methodologies and microservices)

• database <> blockchain (oracle networks that bridge data to a blockchain)

• blockchain <> database (event reader networks that bridge blockchain to a database)

• blockchain <> blockchain (the bridges we've been talking about for the internet of blockchains fit here)

• dapp <> dapp (everything in between is one large, flexible bridge)

I've observed that bridges are first built for a specific purpose of supporting data existence from point A to point B, but then become more and more flexible as they can be built upon by anyone).

Generally with any bridge there are three roles:

• Node Runner (manage the bridge infrastructure)

• Creator (developers who create a marketplace of possible interactions/queries to load onto nodes)

• User/Query Runner (consumers of data, who can use anything from the marketplace built by creators to port data from point A to point B through node runners)

Once these three roles have been set up nicely, you obtain the properties of flexibility and reusability. Let's look at some concrete examples:

• Chainlink: On Chainlink, you have a party that is specifically responsible for running a node that takes in requests from an on-chain oracle contract and returns the value from the endpoint queried. Creators are developers who can create external adapters that allow for API keys and complex logic (i.e. the data returned is a multi-key JSON object, and I can use an adapter to concatenate that into a bytes32 object to easily parse in solidity).

• theGraph: This service has the three roles most clearly defined, with indexers (node runners that read emitted events from on-chain), curators (creators of the subgraphs/API endpoints), and consumers (the ones running each query).

• Numerai: While their user use case may not be as flexible (given they are not purposed to output data on-chain past staking purposes), they allow creators to upload machine learning models onto hosted infrastructure. Given the prevalence of ML/AI in our world, I imagine that one day someone will provide a similar service/structure for running models with inputs/outputs on-chain.

Now for some bridges that I think will reach the same structure:

• Teller: While right now their node network supports only the transfer of your bank account data on chain, it is possible that they become a network that allows any fintech data provider to onboard and create a data service. This is similar to chainlink, but their infrastructure has better privacy management/implications.

• Mina: Speaking of privacy, Mina is actually a super interesting project because they enable use cases like Teller with their technology. Their tooling allows for a zk/privacy overlay end-to-end, which in my mind feels very similar to how we have gone from TCP/IP to VPN in the traditional internet world. I'm assuming they'll be like any node network where these privacy use cases can be built on top of it and used by anyone on another blockchain.

• Cosmos: With CosmWasm/IBC/Tendermint/Ethermint, you can actually start building logic into bridges that are similar to smart contracts (like DEXs). Some examples that have come out of their SDK are proof of stake style chains like Thorchain.

• Connext: With state channels/ILP, you get a level of user flexibility that is hard to manage with IBC. Their custom transfer definitions and routers may in the future allow for special use cases for DAOs, NFTs, and other self-managed multi-chain needs.

Building freely upon bridges gives us flexibility, and the shared infrastructure and split roles gives us reusability. Many of our bridges already have these two properties, and the bridges between blockchains are getting there as well. One day these bridges will enable full (veritcal/horizontal) existence as well, giving us composability across the ecosystem.

A Token from the Haert

We're all out here building bridges right now, trying out best to move in sync. Walls are being torn down, and the groundwork is being laid for the many future generations to build upon. We must exist together, with flexibility of mind, and a shared heart. There will be no walls in the metaverse. I wanted to represent this ethos in an algorithmically generated art piece from Ethereum data:

Token #3347

The identicons are generated from 100 unique Ethereum addresses. Through 100 rounds, each "block" moves towards the center following the Ethereum PoW Poisson distribution. The "heartbeat" is generated by the amplitudes of the sine transform of Ethereum transaction history.