The more one writes about identity, the more the word becomes a term for something that is as unfathomable as it is all-pervasive.

So said Erik Erikson, the psychologist who coined the term "identity crisis.” It often feels true: identity is a nebulous concept, with many connotations and its meaning highly context-dependent. This is no different in Web3.

In this post I’ll try to alleviate this: a framework for thinking about identity on the web as primarily a tool to store, manage and retrieve information.

It won’t clear up every use and misuse of the term, but I hope it lends clarity in thinking about how identity can shape Web3, how applications can build for this, and what these choices will mean for our experience of the web.

What’s in a name? Three meanings of ‘identity’

When people talk about identity they usually mean one of three related but quite different scopes: (a) a unique identifier, (b) a holistic view of an entity, or (c) a specific piece of context about an entity.

Unique identifiers are critical in any social setting. Names are an adequate ‘identifier’ amongst friends, family, or small tribes (under Dunbar’s 150 person threshold) where familiarity can be assumed. Beyond that, more rigid identifiers help make actors ‘legible’ in a broader system. States implemented IDs to manage taxes, conscription and social programs. Web applications have userIDs in a user table to track, manage and serve their customers.

A holistic view refers to all the possible information about a user or other actor. Attempts to attach lots of data to unique identifiers can create rich information sets about people or entities. Pursuit of this is seen in the user databases of Facebook & Google, India’s Aadhaar and China’s universal reputation system, and customer data platforms like Segment and LiveRamp.

A specific piece of context can refer to any of the many subsets of the holistic view. KYC or identity verification - a many-billion-dollar industry - is about verifying that someone is the unique identifier they are claiming to be within a state system. Authentication, anti-fraud, anti-spam, and reputation algorithms are, similarly, specific services focused on a subset of information within the holistic view.

Do I contradict myself? Very well then I contradict myself, I am large, I contain multitudes. -Walt Whitman

Identity: attaching information to an identifier

Unique identifiers are necessary, but on their own pretty useless. They are nearly always used to route to some information. That might be a name and address in state records, a document in a file system, a password in an application database, or a token balance or transaction history on a blockchain. In every case, the identifier is useful because it conveys related information.

Many situations call for retrieving or verifying a specific piece of context linked to an identifier. For example, Gitcoin needs an ‘identity system’ that prevents attacks on its Grants platform. In practice, they need to map evidence of personhood (KYC verification, twitter account) to a unique identifier. The more more information they have on how likely that actor is unique or fraudulent, the better their platform can function.

Holistic views of identity are always incomplete — just as we can never perfectly describe our ‘true selves’ in meatspace, our digital selves will never be fully consistent or comprehensive. But the more data is gathered around a single (or set of linked) identifiers, the more information we have to draw on for any given context.

The common thread is: identity systems create the ability to reliably associate information with a unique identifier. An identity system is more reliable, and therefore useful, the more it is:

1. Dependable: available, fault-tolerant, tamper-resistant

2. Flexible: can handle more types of information

3. Accessible: can be used across more contexts, unifying rather than fragmenting information

Different contexts call for different considerations for privacy and security; for trust via 3rd parties or auditability or decentralization; for consistency vs. availability. But at the most primitive layer, an identity system is stronger if it has the potential to make more information more legible more consistently.

Digital identity as the 🔑 to the web.

The web, to massively oversimplify, runs on hardware, code, and data. Every website you go to has logic and rules written in code, and nearly all are populated with information encoded in data. That data - whether today’s news or your friends’ tweets or your last email drafts - has to be retrieved accurately and reliably when you arrive at the site. That’s done through identifiers.

Just as unique identifiers are not useful without attached data, data on the web isn’t very useful if it can’t be retrieved at the right time. Unique identifiers, and routing tables and logic built around them, are used throughout the web to organize the data that populates it. Who is creating those identifiers? And who is organizing data around them?

Today, it’s nearly every site you visit, product you use, or company you encounter. Identifiers are listed in a database they have created, mostly private and siloed from every other company’s. Data is put there, and linked there, as well. Often this organized around a user table: each row represents a user, each column a type of data, and the table stores or points to each users’ record of that type of data.

How does this identity system hold up to our criteria above?

Dependable: 👍🏽 pretty dependably available, but 👎🏽👎🏽 with no auditability, highly susceptible to hacks and errors Flexible: 👍🏽 database types can be linked to handle all kinds of information, though it can be a bit of a mess Accessible: 👎🏽👎🏽👎🏽 with every app needing their own identifier, information (and managing it) is incredibly fragmented, redundant and inefficient

From a macro perspective, this is a pretty bad identity system for the web - because it’s not an identity system, it’s many different ones. It fragments information, limiting its value and use for every participant. (It also creates terrible incentives to hoard and abuse user data that are beyond the scope of this article).

From a more micro perspective, in a users experience with any given application, it is the application who is responsible for the user’s identity - for their unique identifier, the data associated to it, and the reliable links between them. This arose simply because there was, until recently, no other option. This is intuitively wrong.

Decentralized identity: how Web3 overtakes Web2

https://twitter.com/dazuck/status/1459131116678438916?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed&ref_url=notion%3A%2F%2Fwww.notion.so%2Fthreebox%2FIdentity-article-d8ec96ab757f40d8bb57f2d80371b47e

Blockchains are a form of distributed ledger technology (DLT), which are basically shared databases. A shared database seems like a great place to put a unified user table, and get rid of the archaic need for every application to create their own identity system.

This is the promise of decentralized identity and a core pillar of the Web3 vision: every user and builder is in control of their own data, value, relationships, and information. In this vision, each user becomes the unified discovery point for their own data, creating reuse and composability across applications. This creates shared network effects, native interoperability, and compounding experiences that siloed, centralized applications can’t compete with.

A primitive version of this envisions a single unified registry of users (on a single DLT) and a standard way to add information to that registry for all apps. Users are given control their own cryptographic, sovereign address (or identifier) with which they sign all data to create the trust needed for data in an open context. We get every app to use the same registry (blockchain) and issue data using a standard format (NFTs) and we’re theoretically in identity nirvana - a web in which we bring our social graphs across apps, engage with audiences and communities seamlessly across platforms, and move easily between new products and services as soon as they become available because they all natively interoperate.

However, this vision of decentralized identity - most recently relying on addresses and NFTs, quickly breaks down in practice. It is too rigid and doesn’t perform well as an identity system to manage and route to data at scale. On our rubric:

Dependable: 👎🏽 blockchains today, designed for consensus on scarce financial assets, cannot possibly scale to meet the scale of abundant data; nor can they handle off-chain (or partitioned) updates Flexible: 👍🏽👎🏽 most on-chain ledgers enable new data structures and standards, but within fairly defined limits bound by the consensus system. This restricts the use cases and applications of this system Accessible: 👎🏽 a single registry limits users and apps to a single DLT or blockchain, when inevitably different chains and networks will be used

We can learn from the shortfalls of primitive cryptographic identity systems to see what’s needed in a more tenable decentralized identity system. It’s clear a single registry (index), identifier standard, or data structure standard will always be too rigid.

It must work with a variety of identifiers. It must be open to a flexible, extensible set of data models and structures. It must work across contexts and networks on the web. It should be designed with the principle that identity is about managing and discovering information, and so it should put data first.

How Web3 will do user tables

To manage data, we need a protocol that makes it easy to store, discover, and route information about an identifier. For Web3 to live up to its promise, that routing table should (a) be unified rather than siloed by application or any other boundary, and (b) be sovereign, give control of data directly to each identifier.

This suggests a simple design: every identifier maintains a table of its own data. Unified, these identity-centric user tables form a distributed user table for the internet. This distributed user table is not an actual table but a virtual one that arises out of a few component parts that correspond to parts of a traditional user table:

Identifier: rather than an entry in an application database, a decentralized identifier should be provably unique and cryptographically controlled. Accessibility requires accepting multiple forms of identifiers across a variety of networks - a la the DID standard for decentralized identifiers.

Data Structures: similar to how application developers define their own data structures, a decentralized data layer needs to enable developers to define custom data models while ensuring that those models are reusable and stored publicly.

Index: users bring their identifier while applications define the data models. Standard indexes can combine these elements into a user table (or application table) so that when a user interacts with an application – creating data – that information is catalogued appropriately for future routing. This creates an easily-discoverable record of a user’s data — mapped to the data model and cryptographically associated to the identifier.

For this distributed user table, from the Ceramic blog: “Each user has full sovereign control over their row(s) and can bring that data with them to each app they visit. If an app wants to know what data is available and how to use it, they can reference the data model which will contain a name, description, and other metadata.”

Building with a distributed user table

How does this identity system, based on DIDs and data models and a distributed user table, hold up on our criteria?

Dependable: 👍🏽👍🏽 runs on a collection of public networks that anybody can participate in, including partitioned or local ones Flexible: 👍🏽👍🏽 works with any data structure that a developer can define Accessible: 👍🏽👍🏽 works with any open network and unique identifier

This system also has a number of additional properties that make for a highly flexible and reliable identity system. It is:

• Pseudonymous-first: there is no account creation or verification to get started, a user (or other entity) simply brings a cryptographic keypair and can begin accumulating information around it

• Generative: information accumulates over time, creating an emergent holistic identity

• Composable: information can be discovered and shared across contexts without pre-defined integrations or portability standards

• Separable and selective: information sets can be encrypted or obfuscated, or separated across multiple identifiers, or otherwise divisible at the controllers preference

If “identity systems create the ability to reliably associate information with a unique identifier” as described earlier, we want an internet identity system that establishes the bare minimum protocol for managing and routing to trustworthy data, and everything else to be left to the ingenuity and diversity of application developers.

We want to avoid siloed systems - including specific applications, registries or blockchains - and maximize the flexibility of data types. We want an easy-to-use system that lets us build applications with rich forms of data, have that data associated to the appropriate identifier, and get maximum utility out of our identities and collective information.

This technology and model is newer, but growing rapidly. Thousands of Web3 developers are already building with this identity system using tools like Spruce and infrastructure like Ceramic. Not only will this model for identity and data help developers build more powerful and scalable Web3 apps much more quickly than ever, but it’s how Web3 can collectively build a composable dataverse that Web2 platforms can’t possibly compete with.

Get started building with the DID datastore library or join the conversation in chat.ceramic.network.

Thanks to Mason, Jad, David and Kevin for the reviews, edits and inspirations for this post.

split://0x1e114D5Aa63b24e88114A120Ba7BDccD91210451?network=mainnet

But on a complex issue more in the spotlight than ever, most commentary is missing a simple framing: internet-based platforms are natural monopolies. If our prescriptions miss this diagnosis and treat the symptoms rather than the underlying cause, we’re doomed to recycle the ailment. If we recognize the situation, we can design the right governance for online networks to keep private and public interests (sufficiently) aligned.

Tech networks are Natural Monopolies

A natural monopoly is an economic term for a monopoly in an industry in which high infrastructural costs and other barriers to entry relative to the size of the market give the largest supplier in an industry, often the first supplier in a market, an overwhelming advantage over a potential competitor (Wikipedia).

This has historically applied mostly to large-scale physical infrastructure projects with high upfront costs, returns to scale, and network effects: public utilities, railroads, ISPs. But natural monopolies aren't limited to physical infrastructure networks. Setting up an online platform may have much lower upfront costs than a railroad. But network effects online can grow faster and more global - and so achieve dominance in a similar way.

The exponentially climbing value of networks (powered by Reed's law as well as Metcalfe's) means there is a natural tendency towards a winner-take-all natural monopoly in online networks. Winners are protected from competition by their strong, ever-growing network effects.

Specifically, online networks run on both code (which define the services, algorithms, products, etc) and on data - the information that populates the apps and websites we use every day. It's the value of this information - our social graphs, user data, information and interests, browsing history, etc. - that gives large players such a huge advantage over any competition and effectively locks users into their products and services.

Mixing private and public interests

In the analysis of big-tech platforms, a common mantra of libertarians, technologists, and home-office-chair economists has been “if you don't like it, go elsewhere." This line of reasoning argues that these are private companies in a free market; if you don’t like how Twitter operates, go use another social network. This stance doesn’t hold in natural monopolies. As with delivering tap water, the natural (economically efficient) number of service providers for any given network online is 1. There is no ‘elsewhere’ to turn to in monopolistic markets because the market conditions ensure there won't be viable competitors.

Our largest tech companies aren't natural monopolies in random vertical markets. The web is our digital society, and these platforms operate as our public square (Twitter and Facebook), our public utilities (Google, Apple), and our public infrastructure (AWS). Anyone excluded from these platforms will be unable to participate on the web. They'll have no voice and little chance to participate, compete and thrive. Right now, big tech companies have the power to disenfranchise any participant or perspective from our digital society. That makes them the government of our digital society. And it is a dictatorship.

Economists and policymakers going back to John Stuart Mill have known that natural monopolies can't be left alone in a free market because they have too much power, no competition, and the ability to extract from the public without checks. Some intervention or public-interest governance is necessary to prevent any network owner from abusing their uncompetitive position.

Throughout most of history, the only realistic way to apply this public-interest governance to private companies was through regulation or privatization by state governments. This nearly always leads to inefficiency, often to capture or corruption, and usually isolation from the valuable advances and innovations that competitive markets provide. Government action, when necessary, comes with a high cost.

Another option is to ‘break up big tech’ to re-establish a competitive market. But in markets driven by strong network effects, the tendency towards monopoly will simply lead to reconsolidation and recycling this pattern with ever-greater inefficiency and confusion. Others argue for forced interoperability, similar to regulating telcos - forcing big tech companies to open access to their underlying networks (databases). This is possible but would require more technical nuance and foresight than most regulation can supply.

We now have a new alternative: distributed ownership and governance of high-value networks -- powered by cryptography, blockchains and public data systems

Web3: cooperate on the network, compete to apply it

At the heart of a better model is a simple idea: separate the network layer from the application layer. Today, platforms bundle both their product & service with their database and network. In the future, applications can be proprietary and fragmented, but the data that populate them (social graphs, user accounts, information) can be shared and natively interoperable across them.

A physical-world analogy makes it pretty obvious that this is how things should work.

https://twitter.com/MaciekLaskus/status/1347902379249852417

Today's "Web2" is held back by a few companies' control over these critically valuable assets. Another example: ‘if building houses were like building software, you'd tell your contractor you want the wall blue, then they'd tell you they’ll check whether there's enough demand to paint every homes’ walls blue.’ We should be able to have many versions of software, modified at the app layer with our own needs and preferences. But because all data is gatekept in a rigid, siloed, company-controlled way, we have very little flexibility in our online experiences.

We've tolerated the platform-owned model until now because it's been the only viable option. The data, social connections, and user accounts have to be stored somewhere. Most users can't do it themselves. So companies ran servers and built databases to do this. Then big companies started offering their infrastructure, software tools, and some starter data to smaller ones (e.g, FB Connect). In return, they asked for access to the small company's data. Suddenly a few big companies had all the data about everything happening across the web. And all the network effects that come with that.

New technologies have made an alternative viable: we can store the data on shared, open, distributed networks. We've had some of the core peer-to-peer storage building blocks for a while. But with many authors and participants writing and reading data to the same data network, challenging problems emerge. Among them: how can you trust data on the network and know that it hasn't been tampered with? New technologies are addressing this:

• Cryptographic keys (and IDs) let anyone sign, transact, and participate in these networks, enabling direct management of information (I control my data directly) and the ability for network participants to trust (because they can audit) the network

• Distributed, content-addressed storage offers reliable storage and retrieval based on the content rather than its physical location (URL)

• Tokens, or crypto assets, let network designers build incentives directly into the network, helping ensure it stays secure and aligned to the interests of the participants while rewarding contributors to the network (and its value) economically

These technologies enable a "Web3" that is far more interactive and fluid than today's web because many apps can operate on the same core data. Instead of a company having to choose Notion or Evernote for their productivity suite, each person can choose their own tools and still collaborate on the documents and data. You could use Twitter, I could use Chirper - because I prefer their feed algorithm and moderation policies - but we could still like, RT, and comment on the same messages threads.

https://twitter.com/jack/status/1349510780043988992

When the common layer is the protocol, not the platform, innovation isn’t throttled by a single corporate gatekeeper that decides what can happen on the platform or not.

This model's magic is that it gives us more diversity and more network effects at the same time. The app layer can diverge — many different filters, views, interfaces, and services — while the underlying data and network for all are shared. Network effects grow faster than ever. Apps and services work together seamlessly because they don't require complex integrations - they are operating on the same data layer directly.

In the presence of dominant shared network effects, incentives change too. "Composing" on existing products is far easier than ever because of the shared data - whether assets on a blockchain like Ethereum, or other data on a network like Ceramic - so entrepreneurs can create useful products and services far faster. And for users, switching costs between products are fall dramatically because data comes with them.

This means there will be more economic value in creating useful products and services than trying to build a huge, proprietary user base for its network effects. The quality of the products and services we use will take off. Networks will be far more powerful but now shared by all. Apps will be individually far less powerful than today’s platforms, but collectively far more powerful.

In Web3 we won't wait for "10x better” products — we'll have constant marginal improvements that compound way beyond 10x. With public networks, the web will be competitive again.

Composable data is a paradigm shift for how the web works because it not only changes how applications are built but what an application is. This post aims to make this shift clear and concrete.

Data: the foundation of most apps

Applications are basically interfaces + some business logic + databases. That's true of the vast majority of products online. The interface is what you see and interact with. The business logic defines and delivers the core functionality. Databases store the information presented, the events that occur, and the record of everything that has happened that might be needed in the future.

In many products, especially highly successful ones, much of the value is in the data. The aggregation of everything that has happened before makes the product increasingly valuable and sticky over time and usage. Interface changes are frequent. New logic and services are constant. However, the data are persistent and ever-growing. Facebook, Twitter, or Airbnb all change their UIs all the time, tweak their feed algorithms, and update their services (Marketplace, Spaces, Experiences). It’s the friends, personalization, and network of content (posts, tweets, rentals) that compounds.

This applies in the vast majority of cases. Some exceptions seem like pure services. Zapier, for example, is mostly logic that connects others' databases through APIs. But even here, your built-up Zaps are stored in their database and make it less likely you'll switch to IFTTT. Plaid connects financial databases, but the authorizations are stored on their server so you don’t have to reconnect every time.

Databases today are painfully siloed

Today, databases are basically siloed — every application has its own. This has many bad implications: redundant infrastructure, honeypots of data with poor security, fragmented data. It also means that every application must have its own database to feed its logic and its interface. The 3 layers — interface, logic, and database — have to be bundled by every application.

This basic stack is so accepted (and frankly so much easier now with cloud services) that we rarely question it. But it's ridiculously inefficient. Why should every potential business need to build all 3 parts of this stack when their core innovation or value add comes primarily from one or two? If an entrepreneur has a vision for an improved service or interface, she can't just build that. She has to build the whole stack - from scratch - and compete on all of it.

One direct consequence of this is far fewer things get built or used. Because data is the foundation for so much value over time, and because it's so much more valuable when networked with other data (network effects, big data, etc.), there's far less aggregate value when data is spread across more fragmented databases and applications.

This creates a natural limit to competition and natural suppression of innovation. Any given database is proprietary, controlled by a specific app and siloed off from any other use cases. Only the company that controls that database can build new services or interfaces with it or permit others to do the same. And only in exceptional and intentional cases  - one-off integrations or pre-defined APIs -  are data shared between products.

For example, in its early days Twitter allowed 3rd parties to build interfaces and apps like TweetDeck and others to give users a different experience of the same ‘tweets’ and ‘followers.’ Then they shut this access down, squeezing those apps out. If you want a different feed algorithm or interface now, you’re out of luck.

Censorship is the removal of things already created and triggers massive uproar. But the hidden and the much larger impact of siloed control is the gatekeeping on innovation: the suppression of things that could never be created in the first place.

Composable data: a new paradigm

When database functionality is not siloed but open, this all changes. Any app can build on the same data. No app is a gatekeeper to it. And not every app needs to build an entire siloed stack.

This enables 'permissionless innovation' - anyone can build any new service (logic) or any new interface (app) on the same data layer. An improved product or feature can come from anyone anywhere (not just the original company), and it just adds to all the existing services and interfaces that can be used. Any developer can build a new interface to see tweets or interact with followers.

Today’s web browsers are relatively composable - you can add new functionality and features with extensions. Imagine if web browsers were locked down, and you had to choose between the core feature set alone: Chrome with built-in casting, Firefox with Pocket, or Brave with a crypto wallet. Thankfully you don't have to choose and be left so wanting because plugins let independent developers add functionality. This is enabled, in part, by the local database built into browsers that new plugins and apps can all leverage. Most apps are the opposite - closed to outside innovation or extension. They are like browsers with locked functionality.

A composable data paradigm makes apps extendible by making the data layer shared across all the apps using it. This means the value of aggregated data and its network effects builds even though the services, logic, and interfaces are more varied. You get the benefits of diversity without the costs of fragmentation. Composable apps are far more likely to be complementary - delivering new services and interfaces to fill an unfilled niche— and far less likely to try to outcompete incumbents on a brand new (heavy, expensive) proprietary stack.

Some simple examples:

• If Medium were built on open data, Substack wouldn't have to build an editor, interface, and CMS from scratch. They'd build a subscription module that operates on top of the Medium editor and content. You could continue writing in Medium (or elsewhere) and publishing there while using Substack's subscription feature to build an audience and deliver directly to them. (This is what we’re beginning to see with Mirror’s ecosystem with apps for curation, discovery and more.)

• If Notion were built on open data, anybody could be adding features that end up in your Notion board. Currently, there's no good, quick, 'to-do list' in notion. Someone could build a super-fast lightweight standalone to-do list app and have additions sync into the right place you want in Notion. Somebody else could build a tool that generates these notes from a calendar invite or email. This is starting to happen on composable data.

• If Lever or another recruiting platform (or SaaS products generally) were built like this, you wouldn't be locked into their single feature set. We use Lever as it's got an easy-to-use interface, tracks candidates well, and will scale. But I dislike their interviewing and scorecard template. If it were built on open data, we'd build our own scorecard on the same underlying candidate database and be able to add the features we need anytime.

This model's real power is in how fast innovation can happen when new development doesn't require an entirely new stack and data model and when emergent benefits arise from many apps building on the same underlying structure.

Ceramic is building infrastructure for composable data. Within weeks after it became available several developers in the community built two apps with zero knowledge of the other, but instant ability to use and edit the same data.

https://twitter.com/ceramicnetwork/status/1364631929262235648

The power of compounding

Sun Microsystems founder Bill Joy said that "no matter who you are, most of the smartest people work for someone else.” With permissionless innovation, all these people can work together and drive a dramatically faster pace of innovation.

The need for "10x" improvements on products falls away because there's so little switching cost between applications - the underlying data stays the same. Every experience can be constantly improving, with iterations coming from anyone rather than only the original creator or company. The web becomes more composable, with more builders.

The net effect is a flywheel driving apps and experiences online to far greater heights. More apps building on the same data leads to more (and more valuable data), which enables better experiences, which attracts more developers to build more apps, and so on.

Applications no longer need to build an entire stack and compete for the best underlying data. Instead, anyone with an idea for improving the features, services, or interfaces of a use case can plug into the existing ecosystem and its data and start offering their improvement. Builders can build faster, users get more choice, and the Web as a whole accelerates through rapid, permissionless innovation.