Originally Published - December 23, 2020

NFTs lack an efficient price discovery mechanism, so we built an NFT appraisal product powered by peer prediction.

It’s no secret that NFTs are notoriously difficult assets to price. NFTs define an asset class that is both non-fungible and low-velocity (they don’t change hands very often). This makes it very difficult to find the price of NFTs. If an NFT only changes hands once every six months, how do you know how much you should pay for it? How do you know how much you should sell it for? Having a means to efficiently price NFTs would tear down existing barriers to adoption and enable a wave of powerful, new financial products built on top of this burgeoning asset class (NFT indices, underwriting debt against NFTs, etc). So how does one efficiently price NFTs?

Existing Approaches

Existing approaches to improving NFT price discovery involve using auctions or creating fungible versions of NFTs. While steps in the right direction, there are still shortcomings to these approaches.

While auctions are powerful price discovery mechanisms, the asset still has to change hands for said price to be found. Given the (low) velocity of these assets, that means prices won’t be updated very often and will lose relevance and thus usefulness over time.

Creating fungible versions of NFTs is an interesting approach we’re seeing more experimentation with as of late. There are several ways to do this, but our favorite implementation comes courtesy of NFTX. Their general idea is to create pools of similar NFTs (e.g. CryptoPunks); each pool represented by its own ERC20 backed 1:1 with the NFTs in the pool. Users can deposit an acceptable NFT into the pool and get an ERC20 in return representing their contribution. At any point, these ERC20s can be sent back to the pool and the sender will receive a random NFT from the pool in return. This is cool because it lets the open market price the pool’s NFTs in a similar way to how fungible assets are priced. The problem is that it only creates a lower-bound on the price of the NFTs. If I own an expensive CryptoPunk, I won’t pool it with other, less-valuable CryptoPunks and risk getting a less valuable one in return.

We’ve drawn inspiration from elsewhere in our attempt to solve this problem.

Appraisals

NFTs aren’t the only type of non-fungible, low-velocity assets that exist. Consider physical art and real estate. To solve their own issues with inefficient price discovery, both physical art and real estate rely heavily on appraisals.

Appraisals let us query experts’ opinions regarding an asset’s worth. Estimating an asset’s value this way does not require the asset to change hands or for there to be sufficient levels of market maturity (often absent from the NFT space). However, appraisals rely heavily on trust, the appraiser’s reputation, and their desire to have a job tomorrow. In a decentralized setting, with varying levels of pseudonymity, we cannot rely on these conditions. We, instead, need additional incentives to ensure quality appraisals.

Luckily, peer prediction solves this exact problem. It considers the structure of information to identify and incentivize honest answers to subjective questions.

UpshotOne Alpha

UpshotOne Alpha is the first product of its kind. It leverages a mechanism on the frontier of peer prediction research, the DMI-Mechanism, to accurately aggregate appraisals of NFTs from a decentralized, pseudonymous set of appraisers.

The DMI-Mechanism has some desirable properties in that it is both dominantly-truthful and informed-truthful. This means that there is no way of answering that pays an appraiser more than if they only provided informed, honest responses. It is these properties that forge incentives necessary to elicit quality, subjective appraisals.

From these appraisals, any NFT can have an efficiently, honestly estimated price. UpshotOne Alpha enables a sort of “Zestimate” for NFTs. With accurately estimated prices, we expect a number of gains for the broader NFT space: UX will improve across the board, new users will come into the space, and new, powerful NFT products will be created.

How UpshotOne Alpha Works

UpshotOne is designed to maximize the rate at which unanswered questions become answered while mitigating risks to answers’ integrity.

Questions are grouped and asked to appraisers, eliciting their opinions about what an NFT is worth. Appraisers, who a group deems eligible to answer, provide answers to these questions. On a regular basis (initially, daily), scoring rounds are run. It is within these rounds that appraisers are rewarded and resolutions to questions created.

At the beginning of a scoring round, a small committee of appraisers is randomly selected from each group to have their answers considered in resolving a set of questions (or, associating an honest answer to a question). Each appraiser’s answers are compared to those of other appraisers in the committee and scored by the DMI-Mechanism. Scores determine both how much the appraiser is rewarded and how much their answers influence resolutions to questions.

Groups

Almost everything in UpshotOne Alpha happens within specific groups. Groups primarily specify two things: the types of questions that will be asked (in this case the NFTs being appraised) and how influential different appraisers’ are. Initially, the product will launch with groups appraising digital art from leading digital art marketplaces as well as CryptoPunks, CryptoKitties, Axies, and $MEME NFTs.

Identifying Expertise

For appraisals to be reliable, they need to come from people who know what they’re talking about — they need to come from experts. It’s beneficial that the barrier to becoming an appraiser is not controlled by a single party so we need to weigh an appraiser’s influence based on a metric that is accessible publicly. What better place to look for this than on-chain metrics?

Initially, one’s influence is determined by reading their balance of specific NFTs. If someone is appraising digital art and holds NFTs from leading digital art marketplaces, then we know that they have some understanding of digital art prices. Such people have provable experience valuing NFTs given by the fact that they’ve purchased them in the past.

Types of Questions

At first, there will be two types of questions: comparison and multiple choice questions.

Comparison questions provide the user with two NFTs side-by-side and ask them “Which one is more valuable?” They repeat this for different pairings of NFTs. This creates a very accessible user-experience, which is important when appraising the value of such nebulous assets — the cognitive overhead may already be taxing enough.

In comparison questions, we only learn how NFTs rank in price relative to each other, which leaves open the task of obtaining an absolute price for an NFT. We remedy this by always only asking comparison questions involving NFTs from two distinct sets: a corpus set and a non-corpus set. The corpus set is made up of NFTs that have had a higher turnover on the open market. This means that we can place more trust in the prices the market quotes for these NFTs. The non-corpus set is made up of lower-volume NFTs — the NFTs whose open market prices aren’t as informative, if they even exist. Every comparison is made of a corpus and non-corpus item. Each comparison question, then, when resolved, places a bound on the price of the non-corpus NFT in question. As more comparison questions involving the same non-corpus NFT are resolved, we achieve an increasingly narrower range of possible answers for this NFT, and this narrow range determines its price.

Multiple choice questions are a bit more straightforward. Users are shown an NFT with four possible price ranges. They are tasked with picking the range they believe the NFT falls within. After each scoring round, the possible ranges for an NFT adjust based on the responses from that round. Over time, the ranges displayed will converge on sets of ranges that enable more granular appraising (discussed in a later post).

What this all means

We’re excited to bring UpshotOne Alpha into the world for several reasons.

For one, it provides price insights for an asset class where there currently are none. So many of the obstacles halting a mass-embrace of NFTs stem from the lack of reliable NFT price mechanisms. Improving how they’re priced will open the floodgates to a new wave of NFT users as well as new NFT products and protocols.

UpshotOne Alpha is also important for how it lets experts capture value in ways they couldn’t before. With UpshotOne Alpha, NFT experts will be able to get paid for providing their honest insights. As the platform expands, anyone with valuable insights will be able to get paid for them. This product is, in a way, an experiment in enabling people to get paid for their data and expertise.

Lastly, it is one of the first real-world experiments of peer prediction. Peer prediction is an extremely powerful field of mechanism design that has not been practical for real-world use until very recently. And for an industry (the crypto space) so dedicated to turning esoteric, academic ideas into useful products, finding an entire field that has been left completely untapped is a rarity. We believe peer prediction is a field of mechanism design that will underpin a generation of new products that rewards people for being honest.

What’s Next?

As the name of this first product suggests, UpshotOne Alpha is still in its alpha stage. This means that features will continue to be introduced over time, that there are experiences to improve, and insights made more robust. We’ll be opening access to this initial version to a small set of appraisers in the new year. This is so we can identify and fix any important, last-minute tweaks before opening UpshotOne to the public. The platform will be available publicly shortly following this initial testing phase.

During and after this time, we will be improving how answers are turned into prices, progressively decentralizing the stack, expanding the types of NFT analytics available to users, and configuring incentives to involve real money. Expanding the platform to assess assets and events beyond NFTs is currently in the works as well. We’ll release more specifics around this shortly.

UpshotOne Alpha is the first step in a long journey towards creating a platform for crowdsourced insights — a home for honest information sourced from a network of properly incentivized, decentralized experts.

We will be releasing much more information regarding this initial product, how the different parts of it work, and where it is going from here in the coming weeks. If you would like to get involved in any way, don’t hesitate to reach out on Twitter.

Acknowledgements to my co-founder Kenny Peluso for collaboration on this post as well as the entire team for the work they’ve put into this product.

Originally Published - December 23, 2020

NFTs lack an efficient price discovery mechanism, so we built an NFT appraisal product powered by peer prediction.

It’s no secret that NFTs are notoriously difficult assets to price. NFTs define an asset class that is both non-fungible and low-velocity (they don’t change hands very often). This makes it very difficult to find the price of NFTs. If an NFT only changes hands once every six months, how do you know how much you should pay for it? How do you know how much you should sell it for? Having a means to efficiently price NFTs would tear down existing barriers to adoption and enable a wave of powerful, new financial products built on top of this burgeoning asset class (NFT indices, underwriting debt against NFTs, etc). So how does one efficiently price NFTs?

Existing Approaches

Existing approaches to improving NFT price discovery involve using auctions or creating fungible versions of NFTs. While steps in the right direction, there are still shortcomings to these approaches.

While auctions are powerful price discovery mechanisms, the asset still has to change hands for said price to be found. Given the (low) velocity of these assets, that means prices won’t be updated very often and will lose relevance and thus usefulness over time.

Creating fungible versions of NFTs is an interesting approach we’re seeing more experimentation with as of late. There are several ways to do this, but our favorite implementation comes courtesy of NFTX. Their general idea is to create pools of similar NFTs (e.g. CryptoPunks); each pool represented by its own ERC20 backed 1:1 with the NFTs in the pool. Users can deposit an acceptable NFT into the pool and get an ERC20 in return representing their contribution. At any point, these ERC20s can be sent back to the pool and the sender will receive a random NFT from the pool in return. This is cool because it lets the open market price the pool’s NFTs in a similar way to how fungible assets are priced. The problem is that it only creates a lower-bound on the price of the NFTs. If I own an expensive CryptoPunk, I won’t pool it with other, less-valuable CryptoPunks and risk getting a less valuable one in return.

We’ve drawn inspiration from elsewhere in our attempt to solve this problem.

Appraisals

NFTs aren’t the only type of non-fungible, low-velocity assets that exist. Consider physical art and real estate. To solve their own issues with inefficient price discovery, both physical art and real estate rely heavily on appraisals.

Appraisals let us query experts’ opinions regarding an asset’s worth. Estimating an asset’s value this way does not require the asset to change hands or for there to be sufficient levels of market maturity (often absent from the NFT space). However, appraisals rely heavily on trust, the appraiser’s reputation, and their desire to have a job tomorrow. In a decentralized setting, with varying levels of pseudonymity, we cannot rely on these conditions. We, instead, need additional incentives to ensure quality appraisals.

Luckily, peer prediction solves this exact problem. It considers the structure of information to identify and incentivize honest answers to subjective questions.

UpshotOne Alpha

UpshotOne Alpha is the first product of its kind. It leverages a mechanism on the frontier of peer prediction research, the DMI-Mechanism, to accurately aggregate appraisals of NFTs from a decentralized, pseudonymous set of appraisers.

The DMI-Mechanism has some desirable properties in that it is both dominantly-truthful and informed-truthful. This means that there is no way of answering that pays an appraiser more than if they only provided informed, honest responses. It is these properties that forge incentives necessary to elicit quality, subjective appraisals.

From these appraisals, any NFT can have an efficiently, honestly estimated price. UpshotOne Alpha enables a sort of “Zestimate” for NFTs. With accurately estimated prices, we expect a number of gains for the broader NFT space: UX will improve across the board, new users will come into the space, and new, powerful NFT products will be created.

How UpshotOne Alpha Works

UpshotOne is designed to maximize the rate at which unanswered questions become answered while mitigating risks to answers’ integrity.

Questions are grouped and asked to appraisers, eliciting their opinions about what an NFT is worth. Appraisers, who a group deems eligible to answer, provide answers to these questions. On a regular basis (initially, daily), scoring rounds are run. It is within these rounds that appraisers are rewarded and resolutions to questions created.

At the beginning of a scoring round, a small committee of appraisers is randomly selected from each group to have their answers considered in resolving a set of questions (or, associating an honest answer to a question). Each appraiser’s answers are compared to those of other appraisers in the committee and scored by the DMI-Mechanism. Scores determine both how much the appraiser is rewarded and how much their answers influence resolutions to questions.

Groups

Almost everything in UpshotOne Alpha happens within specific groups. Groups primarily specify two things: the types of questions that will be asked (in this case the NFTs being appraised) and how influential different appraisers’ are. Initially, the product will launch with groups appraising digital art from leading digital art marketplaces as well as CryptoPunks, CryptoKitties, Axies, and $MEME NFTs.

Identifying Expertise

For appraisals to be reliable, they need to come from people who know what they’re talking about — they need to come from experts. It’s beneficial that the barrier to becoming an appraiser is not controlled by a single party so we need to weigh an appraiser’s influence based on a metric that is accessible publicly. What better place to look for this than on-chain metrics?

Initially, one’s influence is determined by reading their balance of specific NFTs. If someone is appraising digital art and holds NFTs from leading digital art marketplaces, then we know that they have some understanding of digital art prices. Such people have provable experience valuing NFTs given by the fact that they’ve purchased them in the past.

Types of Questions

At first, there will be two types of questions: comparison and multiple choice questions.

Comparison questions provide the user with two NFTs side-by-side and ask them “Which one is more valuable?” They repeat this for different pairings of NFTs. This creates a very accessible user-experience, which is important when appraising the value of such nebulous assets — the cognitive overhead may already be taxing enough.

In comparison questions, we only learn how NFTs rank in price relative to each other, which leaves open the task of obtaining an absolute price for an NFT. We remedy this by always only asking comparison questions involving NFTs from two distinct sets: a corpus set and a non-corpus set. The corpus set is made up of NFTs that have had a higher turnover on the open market. This means that we can place more trust in the prices the market quotes for these NFTs. The non-corpus set is made up of lower-volume NFTs — the NFTs whose open market prices aren’t as informative, if they even exist. Every comparison is made of a corpus and non-corpus item. Each comparison question, then, when resolved, places a bound on the price of the non-corpus NFT in question. As more comparison questions involving the same non-corpus NFT are resolved, we achieve an increasingly narrower range of possible answers for this NFT, and this narrow range determines its price.

Multiple choice questions are a bit more straightforward. Users are shown an NFT with four possible price ranges. They are tasked with picking the range they believe the NFT falls within. After each scoring round, the possible ranges for an NFT adjust based on the responses from that round. Over time, the ranges displayed will converge on sets of ranges that enable more granular appraising (discussed in a later post).

What this all means

We’re excited to bring UpshotOne Alpha into the world for several reasons.

For one, it provides price insights for an asset class where there currently are none. So many of the obstacles halting a mass-embrace of NFTs stem from the lack of reliable NFT price mechanisms. Improving how they’re priced will open the floodgates to a new wave of NFT users as well as new NFT products and protocols.

UpshotOne Alpha is also important for how it lets experts capture value in ways they couldn’t before. With UpshotOne Alpha, NFT experts will be able to get paid for providing their honest insights. As the platform expands, anyone with valuable insights will be able to get paid for them. This product is, in a way, an experiment in enabling people to get paid for their data and expertise.

Lastly, it is one of the first real-world experiments of peer prediction. Peer prediction is an extremely powerful field of mechanism design that has not been practical for real-world use until very recently. And for an industry (the crypto space) so dedicated to turning esoteric, academic ideas into useful products, finding an entire field that has been left completely untapped is a rarity. We believe peer prediction is a field of mechanism design that will underpin a generation of new products that rewards people for being honest.

What’s Next?

As the name of this first product suggests, UpshotOne Alpha is still in its alpha stage. This means that features will continue to be introduced over time, that there are experiences to improve, and insights made more robust. We’ll be opening access to this initial version to a small set of appraisers in the new year. This is so we can identify and fix any important, last-minute tweaks before opening UpshotOne to the public. The platform will be available publicly shortly following this initial testing phase.

During and after this time, we will be improving how answers are turned into prices, progressively decentralizing the stack, expanding the types of NFT analytics available to users, and configuring incentives to involve real money. Expanding the platform to assess assets and events beyond NFTs is currently in the works as well. We’ll release more specifics around this shortly.

UpshotOne Alpha is the first step in a long journey towards creating a platform for crowdsourced insights — a home for honest information sourced from a network of properly incentivized, decentralized experts.

We will be releasing much more information regarding this initial product, how the different parts of it work, and where it is going from here in the coming weeks. If you would like to get involved in any way, don’t hesitate to reach out on Twitter.

Acknowledgements to my co-founder Kenny Peluso for collaboration on this post as well as the entire team for the work they’ve put into this product.