In the past week, The Synth team has updated the CRPS scoring to encourage better miner output, primarily in response to feedback that the reward curve was too flat, reducing competition among miners. By refining the scoring mechanism, we aim to foster a more competitive environment that better reflects performance differences between miners, rewarding the top miners majority of the incentive.

Week From 2025-02-24 To 2025-03-03

During the week from February 24th to March 2nd, Bitcoin registered significant volatility. The week opened with a price of $96,277 at midnight on February 24th and closed at $94,274 at 11:59 PM on March 2nd. Bitcoin reached a peak of $96,490 shortly after the week's opening (on February 24th at 12:51 AM) and dropped to a low of $78,268 on February 28th at 8:46 AM. Following this, the price began to recover, returning to levels closer to those at the start of the week.

These sharp price movements caused noticeable volatility, with sharp spikes observed in the first half of the week that drove the price down significantly. However, during the weekend, the price surged again. Notably, an announcement by the U.S. President on March 2nd, around 3:30 PM (UTC) and again at 4:30 PM, in which Bitcoin (along with other cryptocurrencies) was included in a future U.S. strategic reserve, triggered a price surge of approximately 11.6% (from $85,130 to $94,982) within about three hours. The corresponding price movements, along with the log-returns and rolling-volatility during the week, are shown in the figure below. The times of the U.S. strategic reserve announcement are also highlighted in the figure.

As usual, six miners were selected for the purpose of this commentary: the top three based on the leaderboard scores at the beginning of the period (midnight, February 24th), and the top three at the end of the period (11:59 PM, March 2nd). Therefore, the miners under examination were as follows:

• Top miners at midnight on February 24th: Miners 4, 5, and 196;

• Top miners at 11:59 PM on March 2nd: Miners 27, 197, and 78.

Below, we will discuss the Continuous Ranked Probability Score (CRPS), Softmax scores, and Leaderboard values obtained by these miners during the week under examination. Miners who were top performers at the beginning of the period will be visualized in shades of red, while those who were top performers at the end will be visualized in shades of green. Four equidistant time points per day were selected for plotting the scores.

Figure 2 below illustrates the CRPS values scored by the miners during the week. Lower scores indicate better model performance, while higher scores signify poorer performance.

As we observed, Miners 4, 5, and 196 performed relatively well at the beginning of the week, maintaining low CRPS values compared to the other miners. However, as volatility spikes occurred, particularly between February 28th and March 1st, the models of Miners 197 and 78 seemed to respond better to volatility regime shifts. Miners 4, 5, and 196 regained their strong performance once the Bitcoin price stabilized, which happened between March 1st and the beginning of March 2nd.

Softmax scores also provide a valuable indicator of miners' performance during periods of high and low volatility. These scores are calculated by normalizing the CRPS values via softmax transformations, offering in this way a clearer view of how miners performed relative to one another. Unlike CRPS values, higher Softmax scores indicate better performance.

Based on the CRPS trends mentioned earlier, we can observe that for most of the prompts, Miners 4, 5, and 196 consistently obtained larger Softmax scores than the other miners, though their results were not exceptional (their Softmax scores never exceeded 0.1 during the week). This is likely because other miners, not considered in this commentary, were assigned better scores. On the other hand, Miners 27, 197, and 78 received lower or similar scores for most of the week’s prompts. However, after favorable prompts, their scores were significantly higher than those of the other miners: Miner 27 peaked above 0.4, Miner 78 around 0.4, and Miner 197 surpassed 0.15. It should be noted that the performance in Softmax scores was not uniform throughout the week; different miners appeared to dominate for specific prompts during short periods.

As a reminder, Leaderboard scoring is the step to include an exponential moving average (EMA) of the normalized scores to balance short-term and long-term miner performance.

All miners under examination experienced an initial period of downward trending leaderboard scores, likely in contrast to miners not included in this analysis. However, as the week progressed, the Softmax scores assigned to Miners 27, 78, and 197 in the second half of the week significantly boosted their leaderboard scores. Miners 4 and 5, on the other hand, saw their scores continue to deteriorate during the weekend, and, more slowly, Miner 196's score also declined, although it stabilized around 0.01. Miners 197 and 78 showed similar trends, with both experiencing an upward trajectory in their scores, peaking around February 28th (with some lag between them), but their scores decreased again during the weekend. Miner 27, however, saw a significant increase in their scores starting on March 1st and became the leader by the end of the week, with a score of 0.04.

The continuous shifts in volatility regimes of Bitcoin prices during the week caused significant changes in the leadership of the subnet. This was especially visible in the prompt-specific Softmax scores, where the best-performing miners seemed to dominate for isolated episodes rather than maintaining consistent performance across the entire week. As a result, leaderboard scores also shifted frequently, with miners who led during certain periods seeing their scores decline in favor of others.

We also observed that miners are quickly adapting to the new incentive regime following the CRPS scoring update. The fact that some miners have managed to catch up to the established leaders is a testament to how the more competitive environment is driving miners to enhance their performance.

Several factors contributed to these fluctuations:

• Changes in scoring parameters (for example, β=-0.005 for Softmax scores), which stresses the performance of models with lower CRPS scores.

• Week-specific volatility that made the Bitcoin price trend particularly challenging to forecast.

• Miners’ models that struggled to adapt quickly to the volatility swings and changing volatility regimes.

The Synth subnet team is continuously working to improve the scoring mechanism, ensuring that it more accurately reflects the models' ability to predict Bitcoin's price movements. Additionally, we highly value the suggestions and feedback from our miners and are committed to making ongoing improvements to provide a more consistent and rewarding experience for all participants. We encourage miners to keep refining their models and look forward to collaborating with the community to drive innovation and enhance performance over time.

Mode is building Full Stack DeFAI, the L2 scaling DeFi to billions of users through AI agents and AI-driven financial applications. Bolstered by a $6 million grant from Optimism, Mode continues to push for an open, efficient, and inclusive financial future on the Superchain.

Website: mode.network

Forum: forum.mode.network

Community TG Groups: Join our Telegram groups

In the past week, The Synth team has updated the CRPS scoring to encourage better miner output, primarily in response to feedback that the reward curve was too flat, reducing competition among miners. By refining the scoring mechanism, we aim to foster a more competitive environment that better reflects performance differences between miners, rewarding the top miners majority of the incentive.

Week From 2025-02-24 To 2025-03-03

During the week from February 24th to March 2nd, Bitcoin registered significant volatility. The week opened with a price of $96,277 at midnight on February 24th and closed at $94,274 at 11:59 PM on March 2nd. Bitcoin reached a peak of $96,490 shortly after the week's opening (on February 24th at 12:51 AM) and dropped to a low of $78,268 on February 28th at 8:46 AM. Following this, the price began to recover, returning to levels closer to those at the start of the week.

These sharp price movements caused noticeable volatility, with sharp spikes observed in the first half of the week that drove the price down significantly. However, during the weekend, the price surged again. Notably, an announcement by the U.S. President on March 2nd, around 3:30 PM (UTC) and again at 4:30 PM, in which Bitcoin (along with other cryptocurrencies) was included in a future U.S. strategic reserve, triggered a price surge of approximately 11.6% (from $85,130 to $94,982) within about three hours. The corresponding price movements, along with the log-returns and rolling-volatility during the week, are shown in the figure below. The times of the U.S. strategic reserve announcement are also highlighted in the figure.

As usual, six miners were selected for the purpose of this commentary: the top three based on the leaderboard scores at the beginning of the period (midnight, February 24th), and the top three at the end of the period (11:59 PM, March 2nd). Therefore, the miners under examination were as follows:

• Top miners at midnight on February 24th: Miners 4, 5, and 196;

• Top miners at 11:59 PM on March 2nd: Miners 27, 197, and 78.

Below, we will discuss the Continuous Ranked Probability Score (CRPS), Softmax scores, and Leaderboard values obtained by these miners during the week under examination. Miners who were top performers at the beginning of the period will be visualized in shades of red, while those who were top performers at the end will be visualized in shades of green. Four equidistant time points per day were selected for plotting the scores.

Figure 2 below illustrates the CRPS values scored by the miners during the week. Lower scores indicate better model performance, while higher scores signify poorer performance.

As we observed, Miners 4, 5, and 196 performed relatively well at the beginning of the week, maintaining low CRPS values compared to the other miners. However, as volatility spikes occurred, particularly between February 28th and March 1st, the models of Miners 197 and 78 seemed to respond better to volatility regime shifts. Miners 4, 5, and 196 regained their strong performance once the Bitcoin price stabilized, which happened between March 1st and the beginning of March 2nd.

Softmax scores also provide a valuable indicator of miners' performance during periods of high and low volatility. These scores are calculated by normalizing the CRPS values via softmax transformations, offering in this way a clearer view of how miners performed relative to one another. Unlike CRPS values, higher Softmax scores indicate better performance.

Based on the CRPS trends mentioned earlier, we can observe that for most of the prompts, Miners 4, 5, and 196 consistently obtained larger Softmax scores than the other miners, though their results were not exceptional (their Softmax scores never exceeded 0.1 during the week). This is likely because other miners, not considered in this commentary, were assigned better scores. On the other hand, Miners 27, 197, and 78 received lower or similar scores for most of the week’s prompts. However, after favorable prompts, their scores were significantly higher than those of the other miners: Miner 27 peaked above 0.4, Miner 78 around 0.4, and Miner 197 surpassed 0.15. It should be noted that the performance in Softmax scores was not uniform throughout the week; different miners appeared to dominate for specific prompts during short periods.

As a reminder, Leaderboard scoring is the step to include an exponential moving average (EMA) of the normalized scores to balance short-term and long-term miner performance.

All miners under examination experienced an initial period of downward trending leaderboard scores, likely in contrast to miners not included in this analysis. However, as the week progressed, the Softmax scores assigned to Miners 27, 78, and 197 in the second half of the week significantly boosted their leaderboard scores. Miners 4 and 5, on the other hand, saw their scores continue to deteriorate during the weekend, and, more slowly, Miner 196's score also declined, although it stabilized around 0.01. Miners 197 and 78 showed similar trends, with both experiencing an upward trajectory in their scores, peaking around February 28th (with some lag between them), but their scores decreased again during the weekend. Miner 27, however, saw a significant increase in their scores starting on March 1st and became the leader by the end of the week, with a score of 0.04.

The continuous shifts in volatility regimes of Bitcoin prices during the week caused significant changes in the leadership of the subnet. This was especially visible in the prompt-specific Softmax scores, where the best-performing miners seemed to dominate for isolated episodes rather than maintaining consistent performance across the entire week. As a result, leaderboard scores also shifted frequently, with miners who led during certain periods seeing their scores decline in favor of others.

We also observed that miners are quickly adapting to the new incentive regime following the CRPS scoring update. The fact that some miners have managed to catch up to the established leaders is a testament to how the more competitive environment is driving miners to enhance their performance.

Several factors contributed to these fluctuations:

• Changes in scoring parameters (for example, β=-0.005 for Softmax scores), which stresses the performance of models with lower CRPS scores.

• Week-specific volatility that made the Bitcoin price trend particularly challenging to forecast.

• Miners’ models that struggled to adapt quickly to the volatility swings and changing volatility regimes.

The Synth subnet team is continuously working to improve the scoring mechanism, ensuring that it more accurately reflects the models' ability to predict Bitcoin's price movements. Additionally, we highly value the suggestions and feedback from our miners and are committed to making ongoing improvements to provide a more consistent and rewarding experience for all participants. We encourage miners to keep refining their models and look forward to collaborating with the community to drive innovation and enhance performance over time.

Mode is building Full Stack DeFAI, the L2 scaling DeFi to billions of users through AI agents and AI-driven financial applications. Bolstered by a $6 million grant from Optimism, Mode continues to push for an open, efficient, and inclusive financial future on the Superchain.

Website: mode.network

Forum: forum.mode.network

Community TG Groups: Join our Telegram groups