One of my favourite things about the crypto community is our deep love of hindsight bias. This is probably a consequence of working in a space with so much randomness. Humans need to find order in chaos, retrospective narratives help create a false sense of order out of this chaos. Lately, I’ve been working to make sense of everything that’s happened in the five years since the launch of the HAV token in March 2018. I keep returning to the same question: why didn’t we figure all of this out faster? Mainly because a bunch of jerks in Discord keep asking it.

Knowing what we know now, many of our current design decisions seem inevitable, and therefore the time spent on the random walk to this point seems to be a wasted effort. That is an easy conclusion to draw, of course, but I think the truth is more interesting. Each obvious thing we know was discovered through experimentation and iteration as we tried to solve the next hard problem. Combine this with the fact that there were often multiple simultaneous interrelated and dependent "next hard problems," and you can see how the fog of war rendered forward planning almost impossible.

One of my favourite examples of this was during the pivot from the Havven stablecoin to Synthetix, the multi-asset platform. I recall a discussion with one of the engineers working on the new system; he suggested we use pull-based oracles rather than push oracles. There were many considerations as we weighed up this fundamental design choice, but ultimately we decided to push oracles would provide better UX, as it would mean there was always a live price when a user wanted to convert, say, sBTC to sETH, rather than requiring the user to call an oracle on every exchange. One reason was efficiency; we assumed one Oracle update every 15 minutes would suffice for possibly hundreds of transactions. The other consideration was, iirc, a concern about the liveness of pull oracles. Essentially, we worried a user might pull a price and by the time the tx settled, it might be rejected as stale, creating UX issues. We also worried this would provide a free option, if you let people pull prices and wait until a deviation happened they could potentially scalp the oracle deterministically. There is a lot of irony here, considering how many tens of millions of dollars worth of oracle scalping happened against the push oracles.

In the end, we decided to push oracles would be simpler, and again, if I recall correctly, there were already a few examples of oracles designed this way, for example, Maker and some of the lending protocols worked like this. So we rolled out a proprietary push oracle. A year later we began discussions with Chainlink after realising what a nightmarish task operating an oracle for twenty different assets actually was. We pretty quickly ran into a problem: Chainlink was planning to deploy pull oracles 🤯

After significant review, Chainlink was willing to deploy oracles for us that matched our proprietary oracle; these push-based oracles went on to become the standard across DeFi over the next few years. There was just one minor issue: they were extremely susceptible to front-running when used by a trading platform while being super efficient for almost every other use case. Synthetix and Chainlink spent years trying to optimize the efficiency of these oracles, but even after porting to an L2 (Optimism), oracles were still the fundamental limiting factor in scaling volume on the exchange. Even after developing hybrid oracles that leveraged Chainlink and Uniswap TWAP, it was still possible to have an edge on the oracle consistency.

Synthetix had pushed (pardon my pun) for an oracle design that made Chainlink superior to other oracles. This contributed to Chainlink dominating DeFi, yet these same oracles didn't actually work at scale for Synthetix. Now, at some point, we probably should have stopped trying to optimize the existing oracles and accepted the architecture needed to be redesigned. But there were always more pressing issues at hand, and some red herrings, like atomic swaps, actually functioned well with push oracles.

Then Perps V2 launched. Perps V1, by the way, is another great example of path dependency and the consequences of resource constraints. But that is for another post. Suffice to say, that story is even crazier than this one. Perps V2 had some significant advantages over V1, namely being functional. But having a working system is a double-edged sword, as the Synthetix community knows very well. Because each optimized component puts pressure on the less optimized components. Perps V2 finally put the nail in the coffin on isolated push oracles, but the process by which this happened was a little weird. Pyth, a "Chainlink competitor," deployed a pull oracle on Optimism in late 2022. Now, to be honest, I thought the market had decided Chainlink competitors were a dead-end in 2020, but apparently, someone forgot to send the memo to the Solana ecosystem.

They had apparently seen the spec for the Perps V2 hybrid Oracles and proposed a similar solution to the Synthetix engineers. This pull-based oracle was not perfect, though, for a number of reasons, both technical and operational. From an operational standpoint, it was almost impossible to conceive of switching to another Oracle provider after 4+ years using Chainlink oracles. And I said as much on Twitter at the time.

My reasoning was two-fold: the primary reason was that Chainlink has battle-hardened its infrastructure over the years, and I’m extremely confident nothing comes close to it. Much of this hardening was done in close collaboration with Synthetix CCs and myself. Absent a similar relationship, I can't see how an Oracle provider would be able to deliver the same level of confidence. Quick aside, this is the same reason Synthetix picked Optimism; we had a seat at the table to help ensure the network was optimized for complex smart contracts in production and not for some esoteric or theoretical goals that didn't exist in the real world. My personal view is this relationship improved both projects significantly, in both cases.

Synthetix governance was faced with a very challenging decision. How this process was handled can teach us a lot about the differences between DAOs and Corporations. We know the DAO decided to implement the Pyth Oracle. This was always going to be a controversial decision, regardless of which way it went. But the fact that it was almost unanimous speaks to the level of frustration about front-running and oracle efficiency. We obviously can't know how this decision would have played out if Synthetix was a corporation, but I personally have some insight since, if it were, I would likely be the CEO (thankfully we consigned corporations to the rubbish bin of history a while ago). I can say definitively I would not have deployed Pyth oracles. My reasoning would have been that the risk of a Pyth oracle failure outweighed the time-to-market advantage vs waiting on Chainlink to deploy a pull oracle. The issue is no one knew how long it would take for a Chainlink pull oracle to be deployed. Chainlink was fairly adamant they could get something working in a few months. We will never know whether they could have moved faster with Synthetix than they have with GMX, but we can certainly say the decision to push ahead with Pyth put a lot of strain on the relationship.

B2B relationships are already a complex area to navigate, but now we have B2D (business to DAO) and D2D relationships to contend with. In this case, a long-standing four-year mutually beneficial relationship between Synthetix and Chainlink was severely strained because a choice was made to move quickly rather than wait an unknown amount of time for Chainlink to implement pull oracles. Again, I will say definitively I would have made a different call, but then I am a human and I have longstanding relationships with people at Chainlink. We have worked closely for years to try to optimize these systems and push DeFi forward, not just for Synthetix and Chainlink but for the entire ecosystem. I would not have put that at risk to buy a few months. And yet today, I saw a post that quantified the likely cost my decision would have cost stakers, something on the order of $10m USD in fees from perps, and all of the commensurate momentum gained from having a working product. The value of that momentum is huge and probably far outweighs the $10m in fees to stakers, given how competitive the perps landscape is in 2023.

The question is, why did a DAO make a different decision than our hypothetical CEO in this scenario? One answer is that while DAO’s are comprised of people, a DAO is not as biased in its decision-making as a hierarchical organization, in this case, because there is diffusion of responsibility when explaining to a long-standing partner that you went with a competitor. I’m not yet sure whether this is a feature or a bug, but I have said before that I think DAOs make fewer bad decisions than CEOs by being better at incorporating information and reducing the impact of biased individual humans. This is a great test case. But we also have the consequences to contend with. Chainlink clearly saw an opportunity to leverage this for their benefit by going to a competitor and offering to design a pull oracle for them on a different network. It can be argued that incentivizing a long-standing partner to court a competitor was winning the short-term game at the expense of the long-term game. Obviously, we will never know how the counterfactual would have played out, but this is definitely a process to watch as it impacts other aspects of Synthetix, with Chainlink's CCIP a critical component of the multi-chain Synthetix V3 vision.

It is tempting to look at Perps V2 and this new hybrid push/pull oracle and conclude everything we did from 2018 to 2022 was a waste of time and effort by a bunch of lazy green circles (the color of the core contributors in Discord). But even if we could go back in a time machine to late 2018, we would fail to implement Synthetix as it is now. So many of the dependencies were either not ready or not conceived at the time. Much of what we do in crypto is adapting to a shifting environment; in that respect, flexibility and openness are the most valuable traits in a community. Even if we knew the optimal path forward in 2018, we would have had to wait years to implement it; and very few aspects of 2023 were inevitable, let alone predictable back in 2018.

Even knowing the futility of planning in a chaotic environment, we must still make decisions and challenge current assumptions. We must look for inefficiency hiding in plain sight. This is the process I’m currently embarking on as I reflect on the state of Synthetix in 2023. Already I’ve identified many potential changes. Some of these are likely dead ends, but some may unlock new paths forward. One thing is certain: we are in a completely different environment today, given the existence of a working product on Perps V2, so we must adapt our focus accordingly. Ensuring the protocol can scale as user demand grows, and even more importantly, driving user demand in the first place.

I will expand on what this entails in a future post, hopefully in the near future.

If you feel rugged by the title of this post, please consider it a reasonably low-cost lesson in the value of using ChatGPT to summarize my posts in the future.

One of my favourite things about the crypto community is our deep love of hindsight bias. This is probably a consequence of working in a space with so much randomness. Humans need to find order in chaos, retrospective narratives help create a false sense of order out of this chaos. Lately, I’ve been working to make sense of everything that’s happened in the five years since the launch of the HAV token in March 2018. I keep returning to the same question: why didn’t we figure all of this out faster? Mainly because a bunch of jerks in Discord keep asking it.

Knowing what we know now, many of our current design decisions seem inevitable, and therefore the time spent on the random walk to this point seems to be a wasted effort. That is an easy conclusion to draw, of course, but I think the truth is more interesting. Each obvious thing we know was discovered through experimentation and iteration as we tried to solve the next hard problem. Combine this with the fact that there were often multiple simultaneous interrelated and dependent "next hard problems," and you can see how the fog of war rendered forward planning almost impossible.

One of my favourite examples of this was during the pivot from the Havven stablecoin to Synthetix, the multi-asset platform. I recall a discussion with one of the engineers working on the new system; he suggested we use pull-based oracles rather than push oracles. There were many considerations as we weighed up this fundamental design choice, but ultimately we decided to push oracles would provide better UX, as it would mean there was always a live price when a user wanted to convert, say, sBTC to sETH, rather than requiring the user to call an oracle on every exchange. One reason was efficiency; we assumed one Oracle update every 15 minutes would suffice for possibly hundreds of transactions. The other consideration was, iirc, a concern about the liveness of pull oracles. Essentially, we worried a user might pull a price and by the time the tx settled, it might be rejected as stale, creating UX issues. We also worried this would provide a free option, if you let people pull prices and wait until a deviation happened they could potentially scalp the oracle deterministically. There is a lot of irony here, considering how many tens of millions of dollars worth of oracle scalping happened against the push oracles.

In the end, we decided to push oracles would be simpler, and again, if I recall correctly, there were already a few examples of oracles designed this way, for example, Maker and some of the lending protocols worked like this. So we rolled out a proprietary push oracle. A year later we began discussions with Chainlink after realising what a nightmarish task operating an oracle for twenty different assets actually was. We pretty quickly ran into a problem: Chainlink was planning to deploy pull oracles 🤯

After significant review, Chainlink was willing to deploy oracles for us that matched our proprietary oracle; these push-based oracles went on to become the standard across DeFi over the next few years. There was just one minor issue: they were extremely susceptible to front-running when used by a trading platform while being super efficient for almost every other use case. Synthetix and Chainlink spent years trying to optimize the efficiency of these oracles, but even after porting to an L2 (Optimism), oracles were still the fundamental limiting factor in scaling volume on the exchange. Even after developing hybrid oracles that leveraged Chainlink and Uniswap TWAP, it was still possible to have an edge on the oracle consistency.

Synthetix had pushed (pardon my pun) for an oracle design that made Chainlink superior to other oracles. This contributed to Chainlink dominating DeFi, yet these same oracles didn't actually work at scale for Synthetix. Now, at some point, we probably should have stopped trying to optimize the existing oracles and accepted the architecture needed to be redesigned. But there were always more pressing issues at hand, and some red herrings, like atomic swaps, actually functioned well with push oracles.

Then Perps V2 launched. Perps V1, by the way, is another great example of path dependency and the consequences of resource constraints. But that is for another post. Suffice to say, that story is even crazier than this one. Perps V2 had some significant advantages over V1, namely being functional. But having a working system is a double-edged sword, as the Synthetix community knows very well. Because each optimized component puts pressure on the less optimized components. Perps V2 finally put the nail in the coffin on isolated push oracles, but the process by which this happened was a little weird. Pyth, a "Chainlink competitor," deployed a pull oracle on Optimism in late 2022. Now, to be honest, I thought the market had decided Chainlink competitors were a dead-end in 2020, but apparently, someone forgot to send the memo to the Solana ecosystem.

They had apparently seen the spec for the Perps V2 hybrid Oracles and proposed a similar solution to the Synthetix engineers. This pull-based oracle was not perfect, though, for a number of reasons, both technical and operational. From an operational standpoint, it was almost impossible to conceive of switching to another Oracle provider after 4+ years using Chainlink oracles. And I said as much on Twitter at the time.

My reasoning was two-fold: the primary reason was that Chainlink has battle-hardened its infrastructure over the years, and I’m extremely confident nothing comes close to it. Much of this hardening was done in close collaboration with Synthetix CCs and myself. Absent a similar relationship, I can't see how an Oracle provider would be able to deliver the same level of confidence. Quick aside, this is the same reason Synthetix picked Optimism; we had a seat at the table to help ensure the network was optimized for complex smart contracts in production and not for some esoteric or theoretical goals that didn't exist in the real world. My personal view is this relationship improved both projects significantly, in both cases.

Synthetix governance was faced with a very challenging decision. How this process was handled can teach us a lot about the differences between DAOs and Corporations. We know the DAO decided to implement the Pyth Oracle. This was always going to be a controversial decision, regardless of which way it went. But the fact that it was almost unanimous speaks to the level of frustration about front-running and oracle efficiency. We obviously can't know how this decision would have played out if Synthetix was a corporation, but I personally have some insight since, if it were, I would likely be the CEO (thankfully we consigned corporations to the rubbish bin of history a while ago). I can say definitively I would not have deployed Pyth oracles. My reasoning would have been that the risk of a Pyth oracle failure outweighed the time-to-market advantage vs waiting on Chainlink to deploy a pull oracle. The issue is no one knew how long it would take for a Chainlink pull oracle to be deployed. Chainlink was fairly adamant they could get something working in a few months. We will never know whether they could have moved faster with Synthetix than they have with GMX, but we can certainly say the decision to push ahead with Pyth put a lot of strain on the relationship.

B2B relationships are already a complex area to navigate, but now we have B2D (business to DAO) and D2D relationships to contend with. In this case, a long-standing four-year mutually beneficial relationship between Synthetix and Chainlink was severely strained because a choice was made to move quickly rather than wait an unknown amount of time for Chainlink to implement pull oracles. Again, I will say definitively I would have made a different call, but then I am a human and I have longstanding relationships with people at Chainlink. We have worked closely for years to try to optimize these systems and push DeFi forward, not just for Synthetix and Chainlink but for the entire ecosystem. I would not have put that at risk to buy a few months. And yet today, I saw a post that quantified the likely cost my decision would have cost stakers, something on the order of $10m USD in fees from perps, and all of the commensurate momentum gained from having a working product. The value of that momentum is huge and probably far outweighs the $10m in fees to stakers, given how competitive the perps landscape is in 2023.

The question is, why did a DAO make a different decision than our hypothetical CEO in this scenario? One answer is that while DAO’s are comprised of people, a DAO is not as biased in its decision-making as a hierarchical organization, in this case, because there is diffusion of responsibility when explaining to a long-standing partner that you went with a competitor. I’m not yet sure whether this is a feature or a bug, but I have said before that I think DAOs make fewer bad decisions than CEOs by being better at incorporating information and reducing the impact of biased individual humans. This is a great test case. But we also have the consequences to contend with. Chainlink clearly saw an opportunity to leverage this for their benefit by going to a competitor and offering to design a pull oracle for them on a different network. It can be argued that incentivizing a long-standing partner to court a competitor was winning the short-term game at the expense of the long-term game. Obviously, we will never know how the counterfactual would have played out, but this is definitely a process to watch as it impacts other aspects of Synthetix, with Chainlink's CCIP a critical component of the multi-chain Synthetix V3 vision.

It is tempting to look at Perps V2 and this new hybrid push/pull oracle and conclude everything we did from 2018 to 2022 was a waste of time and effort by a bunch of lazy green circles (the color of the core contributors in Discord). But even if we could go back in a time machine to late 2018, we would fail to implement Synthetix as it is now. So many of the dependencies were either not ready or not conceived at the time. Much of what we do in crypto is adapting to a shifting environment; in that respect, flexibility and openness are the most valuable traits in a community. Even if we knew the optimal path forward in 2018, we would have had to wait years to implement it; and very few aspects of 2023 were inevitable, let alone predictable back in 2018.

Even knowing the futility of planning in a chaotic environment, we must still make decisions and challenge current assumptions. We must look for inefficiency hiding in plain sight. This is the process I’m currently embarking on as I reflect on the state of Synthetix in 2023. Already I’ve identified many potential changes. Some of these are likely dead ends, but some may unlock new paths forward. One thing is certain: we are in a completely different environment today, given the existence of a working product on Perps V2, so we must adapt our focus accordingly. Ensuring the protocol can scale as user demand grows, and even more importantly, driving user demand in the first place.

I will expand on what this entails in a future post, hopefully in the near future.

If you feel rugged by the title of this post, please consider it a reasonably low-cost lesson in the value of using ChatGPT to summarize my posts in the future.