For the Optimism RetroPGF Round 3, we applied with the Superscan, the unified block explorer for the Superchain. I can’t say what’s the outcome yet, because we need to wait for the final results to be delivered in January, but I want to take some time to think about what was good, what went wrong and how it could be improved. I really like the way the Optimism Collective is trying to self-govern itself by establishing processes and iterating through them in the open air, transparently. What I’m writing does not need to be considered a critique, but a constructive criticism that will hopefully lead to better outcomes in the future.

At the moment of writing, the Superscan has collected 45 votes and we’re 2 hours away from the voting deadline.

Even if the outcome seems good, it may not be: it is true that 45 badgeholders voted for the Superscan, but it is also true that badgeholders may have voted with an associated allocation of 0 OP. This is doable, and it’s what makes the outcome uncertain until the final results come in in January.

I’m not mad of the outcome of this Round, whatever it may be - I really enjoyed the ride, to be honest: applying, looking at the community’s reaction, then watching the badgeholders’ discussions on Discord, and back on the RetroPGF Tracker by Growthepie.

All really as transparent as possible.

But there are some aspects that can be greatly improved, starting from the issue I mentioned above about the possibility for badgeholders to allocate 0 to a project. I’m going to dive in this first, and then I’m going to analyze what I think has impacted worse outcome that we imagined for our application (could be or not, we don’t know yet). And finally, I’ll have some additional thoughts on improvements of the overall process.

As I said, we don’t know what’s the outcome, because some badgeholders may have voted for us, but just allocated 0 OP, because they didn’t have any clue of what our impact was (remember, RetroPGF aims to render the Impact = Profit equation true), or for whatever other reason. I’m saying this because that’s what I saw in the #badgeholders-discussion channel in the OP Discord server. Pasting a message that tells the story, I think.

There are two issues in this message: the first one is about the value of 0, and the second one is about the value of any other number you want to assign to a project. It all starts when we consider that the final outcome of the allocation for every single project is the median of the badgeholders’ allocations, not the average. By definition, the median is the middle number in a sorted list of numbers.

• If there is an odd amount of numbers, the median value is the number that is in the middle, with the same amount of numbers below and above.

• If there is an even amount of numbers in the list, the middle pair must be determined, added together, and divided by two to find the median value (source)

The choice of the median, to me, is absolutely perfect, because it’s an arithmetical way of going to consensus on a given topic:

• The more people give a number that’s near to the other, the more the median is close to the ranges.

• If numbers input are way too far from each other, the median helps neutralize extremely biased inputs.

Let’s assume 10 badgeholders need to vote for a project, but just 7 decide to put it in their ballots. Let’s also assume these 7 badgeholders all input an allocation > 0. We can have this configuration (out of 10 OP of total allocation for each badgeholder):

1 | 3 | 3 | 5 | 5 | 6 | 10

The median, in this case, would be 5, that’s exactly in the middle. This is our Normal Allocation Scenario. Let’s now assume, in another scenario, that the same 7 badgeholders vote like this:

1 | 4 | 5 | 8 | 8 | 9 | 9

In this case, the median would be 8. This is the Highest Allocation Scenario. What changed? Well, the median has increased because the weight of the upper numbers has increased, so the median has been shifted by a few spots, up to 8. So, what happens if the 3 badgeholders that didn’t vote wanted to vote but didn’t have time to research the project or they don’t have any more OP to allocate, so they just allocate 0?

In both cases:

• Normal Allocation Scenario: 0 | 0 | 0 | 1 | 3 | 3 | 5 | 5 | 6 | 10 → Median: 3

• Highest Allocation Scenario: 0 | 0 | 0 | 1 | 4 | 5 | 8 | 8 | 9 | 9 → Median: 4.5

In the NAS, the 30% 0 allocation has brought down the median from 5 to 3 (-40%).

In the HAS, the 30% 0 allocation has brought down the median from 8 to 4.5 (-43.75%)

This is just the start of some research that could be done in the future. What this means is that higher allocations get more penalized by 0 allocations, but there could be a lower bound, meaning that 0 allocations could penalize lower allocations more in absolute terms. I took the 30% 0 allocations as a constant to simplify calculations for the sake of this article, but more calculations with different variables could be taken into account, like: how does to median change when 0 allocations change? Or, what’s the exact correlation between the amount of votes, the 0 allocations and the median? Can we find a specific value for the amount of votes that’s more fair for lower allocations than higher ones? And what if we virtually limit the amount of 0 allocations?

I hope I got the idea out: 0 allocations can be bad if no further research is done. Until new specific solutions are found, this is what I suggest: Do not allow badgeholders to give 0 allocations - either the vote is cast, and a > 0 allocation is assigned, or the vote is not cast at all. I also have other ideas, but I’m not comfortable sharing them right now as they’re confused and would create confusion to who reads as well.

As I saw in some articles by badgeholders that described their voting framework (no need to point out specific individuals, that’s not the point), VC funding was considered as a minus, hence some badgeholders applied a very low multiplier that would actually lower the score / amount of OP allocated (like 0.1x instead of 2x for some other aspect, for example). This is really good thought, I think, but there was no indication that we would have to report our previous funding in any other community outside OP and Ethereum. We took an grant from Avalanche in 2020 and then we made a business out of the explorer product, so we didn’t put anything in the application.

Some badgeholders thought that we were VC funded, so they applied a bad multiplier to us for this. But I can’t think of any source anywhere on the internet that tells that we’re funded, because we’re not. But I think what happened is that these badgeholders simply thought that we were funded because we have some clients, and had little time to do proper research since they had to review 642 applications in addition to ours.

Yes, we do have clients, but outside the Superchain. So for other chains we get paid for a service, while on the Superchain our goal is to get paid to provide a public good to the community (and we do that for all the other chains as well of course, because an explorer has always been perceived as a public good. Is that a bad thing? I’m genuinely asking, I guess I’m biased on this topic.

A few improvements can be done, that require little effort to execute:

• In the application, include separate fields that applicants use to report about: Ethereum-based grants | Superchain-based grants | Other source of revenue outside the Superchain | Sources of revenue in the Superchain

• To reduce the amount of work every single badgeholder needs to do to review every project, ask badgeholders which categories they want to review, so that each badgeholder only reviews projects in a specific vertical and needs to do less context-switch that damages the review process.

Some thoughts on what I saw during the last 15 days of constant monitoring:

Many applicants were afraid of not reaching quorum (us too) because 7 days were left and only 5% of projects were eligible. We would expect that badgeholders, who were elected to do this a few weeks earlier, would take time to review and the majority would vote in the first couple of weeks, out of the 4 weeks total to vote. For badgeholders, this is a vote, but for applicants this is money, revenue and validation: if a project is not eligible, it may decide to not renew its efforts in the community, because it’s not sustainable anymore. Or a project may wait for the RetroPGF to hope for renewed visibility in the community, as projects go up and down the ranks. It seems like there should be ways to keep badgeholders accountable over the voting process, like dividing badgeholders in 4 groups (maybe divided by categories they want to vote on), and force each group to vote in a specific week of the 4-week voting timeframe, putting closed deadlines.

I see there’s an applications review period in which selected badgeholders need to review all the applciations and discard the ones that are not fit for whatever reason. But since the total allocation will grow (hopefully) even more in the future, we need to think of some way to analyze some 100,000 projects in a few weeks. I do have some ideas in this sense:

• Think about the allocation as a per-year allocation, not per-round

• Organize more rounds over the course of a year, with the same shared allocation, divided equally

• Projects that applied to one round in a year cannot apply to another round in the same year

• Projects that applied but are discarded in the first review process, can only apply once more in the same year

• Extra: each round could be related to one of the 4 categories

This system should spread the applications over a year, instead of a few weeks, and this makes way to a rotating badgeholders list that’s less fatigued by the less-huge amount of projects in a 1-out-of-4 rounds in a year.