The concept of crowd wisdom had a moment in the spotlight in 2024. With the popularization of prediction markets like Polymarket, there's a heightened realization in the power of markets to aggregate information.

As amazing as markets are in aggregating information from disparate sources, are there ways in which we can better identify expert opinions among the crowd? The surprisingly popular algorithm is an intriguing attempt at just that.

The Surprisingly Popular Algorithm (SPA) comes from the 2017 paper published in Nature titled "A Solution to the single-question crowd wisdom problem" by Drazen Prelec, H. Sebastian Seung, and John McCoy.

SPA is a novel method to extract the truth from a crowd even when the majority opinion is wrong. The key insight comes from the fact that the expert often possesses an additional signal beyond just the right answer: they will know not just the correct answer but also what the layperson will believe. In the author's own words: "the genius is you let a more knowledgeable minority reveal itself through predictions that the majority of people will disagree with them."

The paper showcases SPA's performance across diverse domains and a range of question types. But the easiest way to grok the SPA is to consider the world of a Yes/No question. The leading example in the paper is the following question: "Is Philadelphia the capital of Pennsylvania?" It is a question that more often than not, people will get wrong. They incorrectly believe Philly is the capital (it is Harrisburg). In other words, a democratic vote will most likely yield the wrong answer of "Yes". But, by applying the SPA, we can expect to get the right answer of "No" under certain conditions.

Below, I've simulated the SPA for this verifiable binary question (Y/N) under certain assumptions of the probability distributions of respondents' beliefs. We will see when SPA works, and when it can break down.

https://www.kylejwaters.com/writings/spa_simulation.html#Define-the-SPA-Algorithm

The concept of crowd wisdom had a moment in the spotlight in 2024. With the popularization of prediction markets like Polymarket, there's a heightened realization in the power of markets to aggregate information.

As amazing as markets are in aggregating information from disparate sources, are there ways in which we can better identify expert opinions among the crowd? The surprisingly popular algorithm is an intriguing attempt at just that.

The Surprisingly Popular Algorithm (SPA) comes from the 2017 paper published in Nature titled "A Solution to the single-question crowd wisdom problem" by Drazen Prelec, H. Sebastian Seung, and John McCoy.

SPA is a novel method to extract the truth from a crowd even when the majority opinion is wrong. The key insight comes from the fact that the expert often possesses an additional signal beyond just the right answer: they will know not just the correct answer but also what the layperson will believe. In the author's own words: "the genius is you let a more knowledgeable minority reveal itself through predictions that the majority of people will disagree with them."

The paper showcases SPA's performance across diverse domains and a range of question types. But the easiest way to grok the SPA is to consider the world of a Yes/No question. The leading example in the paper is the following question: "Is Philadelphia the capital of Pennsylvania?" It is a question that more often than not, people will get wrong. They incorrectly believe Philly is the capital (it is Harrisburg). In other words, a democratic vote will most likely yield the wrong answer of "Yes". But, by applying the SPA, we can expect to get the right answer of "No" under certain conditions.

Below, I've simulated the SPA for this verifiable binary question (Y/N) under certain assumptions of the probability distributions of respondents' beliefs. We will see when SPA works, and when it can break down.

https://www.kylejwaters.com/writings/spa_simulation.html#Define-the-SPA-Algorithm