GovXSMetrics-Based Voting - How Quantitative Metrics Shape Better Decisions
AbstractMetrics-based Voting is an emerging class of governance mechanisms that integrates quantitative data with voter preferences to support collective decision-making. In this article, we explore the design space of such systems, with a focus on how metric scores—whether retrospectively measured or predictively generated—can be combined with expressed preferences to allocate resources. Drawing on recent experiments in the crypto sector, we examine key design choices, including the selectio...
GovXSMeasuring Simplicity for Voters at Retro Funding
Voter participation is key to establishing a successful, trusted retrofunding system. Simplicity for voters plays a crucial role in securing participation. This is part 2 of GoXS’ evaluation of Retro Funding Voting Design Tradeoffs, where we show how we measure if voters can easily understand how to achieve their best results. The GovXS framework is designed to evaluate any retrofunding voting system. As an open-source tool, it allows for analysis of previous voting rounds and helps communiti...
GovXSSecuring Retro Funding Against Malicious Behavior
GovXS’s Evaluation Framework is designed to evaluate retro funding voting systems. It enables the analysis of voting designs and helps communities achieve their prioritized design goals by identifying optimal voting rules. The framework applies both axiomatic analysis and agent-based simulations to ensure comprehensive evaluation. GovXS’s resources provided for retro funding voting design evaluation include:Part 1: Making Optimism Retro Funding StrategyproofPart 2: Measuring Simplicity for Vo...
Token-based Governance & Social Choice. We support DAO communities in building secure & trusted decision-making systems.
GovXSMetrics-Based Voting - How Quantitative Metrics Shape Better Decisions
AbstractMetrics-based Voting is an emerging class of governance mechanisms that integrates quantitative data with voter preferences to support collective decision-making. In this article, we explore the design space of such systems, with a focus on how metric scores—whether retrospectively measured or predictively generated—can be combined with expressed preferences to allocate resources. Drawing on recent experiments in the crypto sector, we examine key design choices, including the selectio...
GovXSMeasuring Simplicity for Voters at Retro Funding
Voter participation is key to establishing a successful, trusted retrofunding system. Simplicity for voters plays a crucial role in securing participation. This is part 2 of GoXS’ evaluation of Retro Funding Voting Design Tradeoffs, where we show how we measure if voters can easily understand how to achieve their best results. The GovXS framework is designed to evaluate any retrofunding voting system. As an open-source tool, it allows for analysis of previous voting rounds and helps communiti...
GovXSSecuring Retro Funding Against Malicious Behavior
GovXS’s Evaluation Framework is designed to evaluate retro funding voting systems. It enables the analysis of voting designs and helps communities achieve their prioritized design goals by identifying optimal voting rules. The framework applies both axiomatic analysis and agent-based simulations to ensure comprehensive evaluation. GovXS’s resources provided for retro funding voting design evaluation include:Part 1: Making Optimism Retro Funding StrategyproofPart 2: Measuring Simplicity for Vo...
Token-based Governance & Social Choice. We support DAO communities in building secure & trusted decision-making systems.
Share Dialog
Share Dialog
Subscribe to GovXS
Subscribe to GovXS
Shaping Optimism Retro Funding so voters can achieve their best outcome by voting according to their true preferences, part 1 of GoXS’ evaluation of Retro Funding Voting Design Tradeoffs.
Are you serious about shaping the future of Optimism governance? Watch the GovXS team’s in-depth presentation on their findings and recommendations here!
Optimism Retro Funding is a funding mechanism that rewards individuals and projects for past contributions to the Optimism Network. Instead of funding based on future potential, it retroactively compensates work already proven valuable. Invented at Optimism in 2021, more crypto ecosystems, like Filecoin and Arbitrum, started implementing Retro Funding.
Optimism Retro Funding is governed by community voting on projects’ grant applications. In July 2024, GovXS was selected by the Optimism Foundation to evaluate the voting design tradeoffs for Retro Funding in Mission Request #188.
Applying a new, comprehensive framework for DAO Voting Evaluation
Shaping Optimism Retro Funding so voters can achieve their best outcome by voting according to their true preferences, part 1 of GoXS’ evaluation of Retro Funding Voting Design Tradeoffs.
Are you serious about shaping the future of Optimism governance? Watch the GovXS team’s in-depth presentation on their findings and recommendations here!
Optimism Retro Funding is a funding mechanism that rewards individuals and projects for past contributions to the Optimism Network. Instead of funding based on future potential, it retroactively compensates work already proven valuable. Invented at Optimism in 2021, more crypto ecosystems, like Filecoin and Arbitrum, started implementing Retro Funding.
Optimism Retro Funding is governed by community voting on projects’ grant applications. In July 2024, GovXS was selected by the Optimism Foundation to evaluate the voting design tradeoffs for Retro Funding in Mission Request #188.
Applying a new, comprehensive framework for DAO Voting Evaluation
Optimism Retro Funding
So far, Optimism has conducted 4 complete Retro Funding rounds, distributing more than 60M OP since 2022. Round 3, the largest round to date, saw 145 votes and 643 applications, resulting in individual projects receiving 600K OP and more. The round size varies from 1M USD in Round 1 to 30M OP in Round 3. So far, the amount to be distributed (round size) has been pre-defined and all funds must be distributed through Retro Funding. In the latest iterations, Round 5 and 6, Optimism plans to implement a new approach, a flexible round size, where the total amount of funding is determined by voters as well.
Over the past four Retro Funding rounds, Optimism tested a number of different voting rules to sharpen a round’s scope and react to insights gained round by round. In the first round, Optimism implemented Quadratic Voting (R1 Quadratic Voting). The second round utilized the mean rule, where the funding allocation was determined by averaging the votes across all participants (R2 Mean Rule). For the third round, Optimism introduced a minimum number of votes for a project to be considered, and then a Normalized Median rule was used to reduce the impact of outlier votes (R3 Quorum Median). Finally, Optimism adopted the Impact Metric Score in the fourth round, which let voters define the relevance of KPIs rather than a funding allocation. Here, projects were asked to provide KPI values to prove their achievements. Based on the voters’ KPI preferences, the Impact Metric Score, a project’s funding allocation, was determined using a Normalized Median, including capping and redistributing overflow funds (R4 Capped Median Rule).
Our evaluation found that none of the voting rules applied in Rounds 1–4 are strategyproof (see Table 2: Properties of Voting Rules (Part 2), page 10).
GovXS Evaluation Process
Retro Funding is built on the core philosophy that “Impact = Profit”, so that positive ecosystem contributions (impact) are aligned with financial rewards (profit). The idea is that in decentralized ecosystems like Optimism, creating value for the community—such as building apps and creating demand for OP blockspace—should also lead to a reliable stream of funds that allows teams to sustain and continue adding value to the ecosystem.
To achieve optimal funding allocations following this vision, Optimism must ensure that voters report their preferences truthfully and can not improve their results by voting strategically.
In Social Choice, the scientific field studying voting theory and incentives, this property is defined as strategyproofness. We can describe strategyproofness formally as follows:
The table below illustrates it. We consider a voting profile with two projects (P1-2), and two voters (V1-2). Let’s focus on the preferences of voter 1 (V1). On the left, we see the true preferences of V1. On the right, V1 misreports their preferences to achieve a result that is closer to their true preference. With a voting rule that is not strategyproof, like Mean, V1 can achieve this. We can measure it using the L1 distance. The L1 distance sums up the absolute differences between the voting outcome and the voter’s preference. The smaller the L1, in other words, the distance between a voter’s true preference and the result, the better the outcome for the voter.
In contrast, with a strategyproof voting rule, V1 can not improve the results, the L1 distance for strategic voting is not smaller. Strategyproof voting rules prevent voters from benefiting by not voting according to their true preferences.
A Lack of Strategyproofness Hurts Retro Funding
A voting rule that is not strategyproof poses a risk for Retro Funding in several ways. Most obviously, it encourages voters not to vote truthfully, an issue in any voting system.
Second, Optimism Retro Funding relies on learnings from analyzing voting data and voter preferences. If this data is skewed, wrong conclusions will be taken. The current voting for Round 5 rolls out a new experiment, asking if expert voters create better funding allocations in Retro Funding. Expert voters are selected guest voters with proven expertise in building the OP Stack, the key scope of Round 5. Optimism assumes that individuals who have contributed work and impact to the Collective will demonstrate relevant competencies and understanding of the Optimism ecosystem. Round 5’s setup includes selecting and verifying these expert voters, providing special voting infrastructure, and analyzing the results by surveying the community. If expert voters misreport their preferences, this entire setup to gain insights is doomed to fail.
Finally, “Impact = Profit” aims to find the objective truth in funding allocation. There is no instrument or algorithm to measure positive impact and the profit it deserves, yet. That’s why Optimism Citizens are asked to report their assessment when voting in Retro Funding, a concept we know well in Social Choice Theory/Epistemic Democracy.
The task at hand includes finding the voting rule and aggregating all votes to an outcome supporting finding the objective truth. If the voting rule provides voters with wrong incentives, the results are distorted, and Optimism Retro Funding can’t get any closer to the objective truth in “Impact = Profit”.
Using the Median without normalization
There are two good options to make Retro Funding strategyproof. First, we can ensure strategyproofness by removing the normalization step to calculate the funding allocation and applying the plain Median rule. Note that in this case, voters determine not only the funding allocation per project but also, indirectly, the total funds allocation (round size) since it is a function of the Median. The example below illustrates it. Important to know: the total funding allocation can be below or above the maximum amount a voter can distribute.
The non-normalized Median ensures strategyproofness. Below, on the left, we see voters’ true preferences in a Median rule voting. On the right, we see the effects of V1 voting strategically. With the Median rule, they cannot create a better outcome, and we find a larger L1 distance (in red). In grey, the outcome using the Normalized Median for comparison.
Majoritarian Phantoms
Majoritarian Phantoms provide an alternative, strategyproof solution. Majoritarian Phantoms is a voting mechanism developed by Rupert Freeman, David M. Pennock, Dominik Peters, and Jennifer Wortman Vaughan in “Truthful Aggregation of Budget Proposals”. This mechanism belongs to a wider class of mechanisms addressing a particular category of problems in social choice called divisible participatory budgeting (also referred to as portioning). Here, a certain, pre-defined amount of funding should be distributed to n number of projects.
With this voting rule, we employ artificial, algorithmic voting agents to compute the funding allocations. The algorithmic agents behave so that they “balance” finding the median, eliminating the need for the normalization step without distorting the real voter’s choice. Here's how it works: At first, none of the algorithmic agents allocate any tokens, so the median for all projects is 0. Then, gradually, each agent starts assigning tokens evenly to all projects. This continues until the total median reaches 1, at which point we stop. The mathematical proof is available here (Theorem 4.4, page 9).
Let’s look at an example to gain intuition. We assume a voting profile with three projects (P1–3) and 3 voters (V1–3). We focus on V1 again. On the left, we find the true preference of V1. On the right is an edited, strategic vote of V1 (while maintaining the same true preference). The example below stops when two algorithmic agents (“Phantoms”) have voted with a value of 1, one with 0.28, and the last one with 0. V1 can not improve the result by voting strategically, resulting in a larger L1. This effect applies to any constellation and voter profile with more than two projects in voting.
Next, we evaluate how the funding allocations differ using the above-mentioned voting rules. The goal of any Median voting rule is to mitigate the effects of extreme votes and reduce vulnerability to malicious voters, e.g., 100% votes or zero votes. In the verification below, we take in real votes from Optimism Retro Funding Round 4, run in July/August 2024, and compute the results employing the following mechanisms:
The Normalized Median, and for comparison,
Majoritarian Phantoms, and
the Median rule (without normalization).
Note that these results are not equal to the actual funding outcome of Optimism Retro Funding Round 4. Here, the funding allocations were capped at 500K OP tokens at maximum and 1K OP tokens at minimum, and overflow tokens were redistributed.
In the next step, we compare the funds allocation across the three voting rules and all projects:
First, with the Normalized Median Allocation method, the lowest allocation was 0.21. The median allocation for this method was 11,345.38, meaning that half of the projects had allocations below this value. The highest normalized allocation recorded was 1,457,783.
For the Majoritarian Phantoms Allocation voting rule, the smallest allocation recorded was 0.27. The median was 11,909.46, indicating that half of the projects had an allocation less than this amount. The maximum allocation recorded for Majoritarian Phantoms Allocation was 1,394,006.
The smallest allocation for the Median Allocation method, which does not apply normalization, was 0.19. The median allocation for this method was 10,384.22, meaning that half of the projects had allocations less than this amount. The maximum allocation recorded for the Median Allocation method was 1,334,281.
In both the Normalized Median and the Majoritarian Phantoms, the total allocation sums up to the pre-defined funding amount of 10M OP tokens - as intended. In contrast, the total allocation of the non-normalized Median deviates. In our example, using the R4 votes, we land at 9,152,812.79 OP tokens - even though a voter could allocate up to 10M tokens.
Depending on the voter profile, using the Median can lead to lower or even higher total funding allocations than the maximum amount a voter can allocate to a single project.
GovXS evaluated the voting design of Optimism Retro Funding across Rounds 1–4. We found that all voting rules applied so far are not incentive-compatible. We propose two voting rule variants to remove this vulnerability: non-normalized Median and Majoritarian Phantoms. We have proven that Majoritarian Phantoms and non-normalized Median are strategyproof, showing how the funding allocation differs from Normalized Median. Currently, we support Optimism Retro Funding in implementing these new voting rules in Retro Funding Round 5 and future rounds - to ensure that Retro Funding voters can only achieve their best outcome by voting according to their true preferences.
A Social Choice Analysis of Retroactive Funding (PDF, formal description the of GovXS Voting Design Evaluation Framework)
GovXS Simulator (Open-source Github repository)
GovXS is a research initiative under Token Engineering Academy. Team members include Nimrod Talmon, PhD, Angela Kreitenweis, Eyal Briman, and Muhammad Idrees. GovXS is a member of the Token Engineering Academy Applied Research Network. Register now to join GovXS’ deep dive into findings above happening on Monday, September 23, at 17:00 UTC, and sign up to receive further updates related to GovXS.
Optimism Retro Funding
So far, Optimism has conducted 4 complete Retro Funding rounds, distributing more than 60M OP since 2022. Round 3, the largest round to date, saw 145 votes and 643 applications, resulting in individual projects receiving 600K OP and more. The round size varies from 1M USD in Round 1 to 30M OP in Round 3. So far, the amount to be distributed (round size) has been pre-defined and all funds must be distributed through Retro Funding. In the latest iterations, Round 5 and 6, Optimism plans to implement a new approach, a flexible round size, where the total amount of funding is determined by voters as well.
Over the past four Retro Funding rounds, Optimism tested a number of different voting rules to sharpen a round’s scope and react to insights gained round by round. In the first round, Optimism implemented Quadratic Voting (R1 Quadratic Voting). The second round utilized the mean rule, where the funding allocation was determined by averaging the votes across all participants (R2 Mean Rule). For the third round, Optimism introduced a minimum number of votes for a project to be considered, and then a Normalized Median rule was used to reduce the impact of outlier votes (R3 Quorum Median). Finally, Optimism adopted the Impact Metric Score in the fourth round, which let voters define the relevance of KPIs rather than a funding allocation. Here, projects were asked to provide KPI values to prove their achievements. Based on the voters’ KPI preferences, the Impact Metric Score, a project’s funding allocation, was determined using a Normalized Median, including capping and redistributing overflow funds (R4 Capped Median Rule).
Our evaluation found that none of the voting rules applied in Rounds 1–4 are strategyproof (see Table 2: Properties of Voting Rules (Part 2), page 10).
GovXS Evaluation Process
Retro Funding is built on the core philosophy that “Impact = Profit”, so that positive ecosystem contributions (impact) are aligned with financial rewards (profit). The idea is that in decentralized ecosystems like Optimism, creating value for the community—such as building apps and creating demand for OP blockspace—should also lead to a reliable stream of funds that allows teams to sustain and continue adding value to the ecosystem.
To achieve optimal funding allocations following this vision, Optimism must ensure that voters report their preferences truthfully and can not improve their results by voting strategically.
In Social Choice, the scientific field studying voting theory and incentives, this property is defined as strategyproofness. We can describe strategyproofness formally as follows:
The table below illustrates it. We consider a voting profile with two projects (P1-2), and two voters (V1-2). Let’s focus on the preferences of voter 1 (V1). On the left, we see the true preferences of V1. On the right, V1 misreports their preferences to achieve a result that is closer to their true preference. With a voting rule that is not strategyproof, like Mean, V1 can achieve this. We can measure it using the L1 distance. The L1 distance sums up the absolute differences between the voting outcome and the voter’s preference. The smaller the L1, in other words, the distance between a voter’s true preference and the result, the better the outcome for the voter.
In contrast, with a strategyproof voting rule, V1 can not improve the results, the L1 distance for strategic voting is not smaller. Strategyproof voting rules prevent voters from benefiting by not voting according to their true preferences.
A Lack of Strategyproofness Hurts Retro Funding
A voting rule that is not strategyproof poses a risk for Retro Funding in several ways. Most obviously, it encourages voters not to vote truthfully, an issue in any voting system.
Second, Optimism Retro Funding relies on learnings from analyzing voting data and voter preferences. If this data is skewed, wrong conclusions will be taken. The current voting for Round 5 rolls out a new experiment, asking if expert voters create better funding allocations in Retro Funding. Expert voters are selected guest voters with proven expertise in building the OP Stack, the key scope of Round 5. Optimism assumes that individuals who have contributed work and impact to the Collective will demonstrate relevant competencies and understanding of the Optimism ecosystem. Round 5’s setup includes selecting and verifying these expert voters, providing special voting infrastructure, and analyzing the results by surveying the community. If expert voters misreport their preferences, this entire setup to gain insights is doomed to fail.
Finally, “Impact = Profit” aims to find the objective truth in funding allocation. There is no instrument or algorithm to measure positive impact and the profit it deserves, yet. That’s why Optimism Citizens are asked to report their assessment when voting in Retro Funding, a concept we know well in Social Choice Theory/Epistemic Democracy.
The task at hand includes finding the voting rule and aggregating all votes to an outcome supporting finding the objective truth. If the voting rule provides voters with wrong incentives, the results are distorted, and Optimism Retro Funding can’t get any closer to the objective truth in “Impact = Profit”.
Using the Median without normalization
There are two good options to make Retro Funding strategyproof. First, we can ensure strategyproofness by removing the normalization step to calculate the funding allocation and applying the plain Median rule. Note that in this case, voters determine not only the funding allocation per project but also, indirectly, the total funds allocation (round size) since it is a function of the Median. The example below illustrates it. Important to know: the total funding allocation can be below or above the maximum amount a voter can distribute.
The non-normalized Median ensures strategyproofness. Below, on the left, we see voters’ true preferences in a Median rule voting. On the right, we see the effects of V1 voting strategically. With the Median rule, they cannot create a better outcome, and we find a larger L1 distance (in red). In grey, the outcome using the Normalized Median for comparison.
Majoritarian Phantoms
Majoritarian Phantoms provide an alternative, strategyproof solution. Majoritarian Phantoms is a voting mechanism developed by Rupert Freeman, David M. Pennock, Dominik Peters, and Jennifer Wortman Vaughan in “Truthful Aggregation of Budget Proposals”. This mechanism belongs to a wider class of mechanisms addressing a particular category of problems in social choice called divisible participatory budgeting (also referred to as portioning). Here, a certain, pre-defined amount of funding should be distributed to n number of projects.
With this voting rule, we employ artificial, algorithmic voting agents to compute the funding allocations. The algorithmic agents behave so that they “balance” finding the median, eliminating the need for the normalization step without distorting the real voter’s choice. Here's how it works: At first, none of the algorithmic agents allocate any tokens, so the median for all projects is 0. Then, gradually, each agent starts assigning tokens evenly to all projects. This continues until the total median reaches 1, at which point we stop. The mathematical proof is available here (Theorem 4.4, page 9).
Let’s look at an example to gain intuition. We assume a voting profile with three projects (P1–3) and 3 voters (V1–3). We focus on V1 again. On the left, we find the true preference of V1. On the right is an edited, strategic vote of V1 (while maintaining the same true preference). The example below stops when two algorithmic agents (“Phantoms”) have voted with a value of 1, one with 0.28, and the last one with 0. V1 can not improve the result by voting strategically, resulting in a larger L1. This effect applies to any constellation and voter profile with more than two projects in voting.
Next, we evaluate how the funding allocations differ using the above-mentioned voting rules. The goal of any Median voting rule is to mitigate the effects of extreme votes and reduce vulnerability to malicious voters, e.g., 100% votes or zero votes. In the verification below, we take in real votes from Optimism Retro Funding Round 4, run in July/August 2024, and compute the results employing the following mechanisms:
The Normalized Median, and for comparison,
Majoritarian Phantoms, and
the Median rule (without normalization).
Note that these results are not equal to the actual funding outcome of Optimism Retro Funding Round 4. Here, the funding allocations were capped at 500K OP tokens at maximum and 1K OP tokens at minimum, and overflow tokens were redistributed.
In the next step, we compare the funds allocation across the three voting rules and all projects:
First, with the Normalized Median Allocation method, the lowest allocation was 0.21. The median allocation for this method was 11,345.38, meaning that half of the projects had allocations below this value. The highest normalized allocation recorded was 1,457,783.
For the Majoritarian Phantoms Allocation voting rule, the smallest allocation recorded was 0.27. The median was 11,909.46, indicating that half of the projects had an allocation less than this amount. The maximum allocation recorded for Majoritarian Phantoms Allocation was 1,394,006.
The smallest allocation for the Median Allocation method, which does not apply normalization, was 0.19. The median allocation for this method was 10,384.22, meaning that half of the projects had allocations less than this amount. The maximum allocation recorded for the Median Allocation method was 1,334,281.
In both the Normalized Median and the Majoritarian Phantoms, the total allocation sums up to the pre-defined funding amount of 10M OP tokens - as intended. In contrast, the total allocation of the non-normalized Median deviates. In our example, using the R4 votes, we land at 9,152,812.79 OP tokens - even though a voter could allocate up to 10M tokens.
Depending on the voter profile, using the Median can lead to lower or even higher total funding allocations than the maximum amount a voter can allocate to a single project.
GovXS evaluated the voting design of Optimism Retro Funding across Rounds 1–4. We found that all voting rules applied so far are not incentive-compatible. We propose two voting rule variants to remove this vulnerability: non-normalized Median and Majoritarian Phantoms. We have proven that Majoritarian Phantoms and non-normalized Median are strategyproof, showing how the funding allocation differs from Normalized Median. Currently, we support Optimism Retro Funding in implementing these new voting rules in Retro Funding Round 5 and future rounds - to ensure that Retro Funding voters can only achieve their best outcome by voting according to their true preferences.
A Social Choice Analysis of Retroactive Funding (PDF, formal description the of GovXS Voting Design Evaluation Framework)
GovXS Simulator (Open-source Github repository)
GovXS is a research initiative under Token Engineering Academy. Team members include Nimrod Talmon, PhD, Angela Kreitenweis, Eyal Briman, and Muhammad Idrees. GovXS is a member of the Token Engineering Academy Applied Research Network. Register now to join GovXS’ deep dive into findings above happening on Monday, September 23, at 17:00 UTC, and sign up to receive further updates related to GovXS.
<100 subscribers
<100 subscribers
No activity yet