# Synth Subnet - Miner Performance Review #12 By [Synth](https://paragraph.com/@synthdata) · 2025-05-29 --- **Data From 2025-05-19 To 2025-05-25** Starting this week, Synth has begun scoring miners on their Ethereum (ETH) predictions in addition to their Bitcoin (BTC) forecasts. This report presents a combined analysis of miner performance across both assets. **BTC and ETH Volatility** -------------------------- Figure 1 displays the hourly log-returns and volatility observed throughout the week. It clearly shows that ETH experienced higher volatility compared to BTC. Furthermore, both assets exhibited volatility spikes at similar times, reflecting their correlation. ![](https://storage.googleapis.com/papyrus_images/9536e910b506bdfa8626b66f714a24aa0f55ed0ba3c013e40e7397c8d4972adb.png) **Miners Performance** ---------------------- This week’s analysis focuses on six miners: the top three on the leaderboard at the beginning of the week (**Miners 46, 241, and 250**), and the top three at the end of the week (**Miners 33, 103, and 252**). We assess their performance using two key metrics: CRPS (based on forecasts for both BTC and ETH) and Leaderboard Scores. **Continuous Ranked Probability Score (CRPS)** ---------------------------------------------- Lower CRPS values indicate better predictive accuracy. As shown in Figure 2, **Miners 46, 241, and 250 — the early frontrunners — received identical CRPS values throughout the week.** Notably, they completely failed to predict ETH outcomes in the initial prompts. This suggests they were producing identical forecasts, likely due to shared model architectures or coordinated strategies. The Synth team suspects that this group may include additional miners and is currently refining the scoring system to address this issue. In contrast, Miners 33, 103, and 252 generally performed better in the middle and latter parts of the week, especially in BTC predictions. Their models seem to have adapted more effectively to the changing volatility regime. For ETH, performance between the two groups varied throughout the week, but over the weekend, Miners 33, 103, and 252 clearly outperformed. ![](https://storage.googleapis.com/papyrus_images/7da174185a84140693556bd709ab2fe681e920c8c524c8b812b93fb0c6f9164e.png) **Leaderboard Scores** ---------------------- Leaderboard scores are an exponentially weighted average of past CRPS performance over a 7-day window, with a half-life of 3.5 days. Lower scores are better. Thanks to their consistent CRPS performance, Miners 33, 103, and 252 took the lead on May 20, surpassing the previous top scorers. While all miners saw slight score increases over the week, the new top trio experienced only a modest rise (from ~100 to ~140), whereas Miners 46, 241, and 250 saw a sharp increase (from ~60 to over 220). ![](https://storage.googleapis.com/papyrus_images/9c48266e2ba08bc7368aac56b60037124dad06430736e05cff9635fce83a2da8.png) **Miners’ Strategies** ---------------------- This section analyzes summary statistics, volatility, kurtosis, and skewness to identify potential strategic advantages among the top miners. These metrics were collected and assessed for both BTC and ETH. **Top Miners Statistics Distribution** * **Volatility** indicates variability in forecasted returns. * **Kurtosis**: Kurtosis reflects the thickness of the distribution tails, indicating the degree to which miners expect extreme events * **Skewness** measures the asymmetry of return distributions (a skewness of 0 implies a balanced distribution around its average). Figure 4 shows these distributions across the week for the six miners. ![](https://storage.googleapis.com/papyrus_images/37893c9f99a0d3c32b7f76cbfaf85302e74843d8db1ff9455d799cab448d3d10.png) The data supports the hypothesis that Miners 46, 241, and 250 share a common model, given their identical distributions for both assets. In contrast, Miners 33, 103, and 252 displayed: * Higher and more variable volatility: indicating more dynamic modeling. * Lower kurtosis: suggesting reduced expectations of extreme outcomes. * More symmetrical skewness: reflecting more balanced forecasts. This highlights the risks of using undifferentiated models, especially for organizations managing multiple miners, since shared models can underperform collectively during market shifts. The observed diversity in forecast behavior highlights the drawbacks of a lack of model differentiation. For entities managing multiple miners, homogeneous forecasts increase vulnerability to performance degradation during regime shifts. **Miners Ranking vs. Summary Statistics** ----------------------------------------- All miners who submitted forecasts during the week were ranked by total weekly rewards. Figure 5 shows how summary statistics vary across different ranking tiers. ![](https://storage.googleapis.com/papyrus_images/53a1fcf79399dcccf0e4c39f04e7eabf3d03fb56db03e068ea2554f7c61c4255.png) **Key Insights** ---------------- * **BTC**: Volatility modeling is improving across the board. Compared to past weeks, lower-ranked miners are making noticeable efforts to align their volatility modeling more closely with that of top-tier performers. * **ETH**: Modeling remains less developed. As competition is still in early stages, miners need more time to optimize their approaches. * **Kurtosis**: Still largely overlooked by lower-ranked miners. Many exhibit flat kurtosis near zero, indicating a lack of attention to tail risks—likely contributing to poorer results. **Takeaways** ------------- * **Identical Forecast Clusters:** Some miners are submitting indistinguishable forecasts, likely due to shared models. While this may offer short-term advantages, these miners are more vulnerable when model performance degrades. * **Value of Diversification:** Entities managing multiple miners should diversify their forecasting models. Relying on a single model can expose all associated miners to simultaneous underperformance. * **Volatility Modeling Progress:** Lower-ranked miners are making gains in modeling BTC volatility. ETH modeling, however, still lags as miners continue to adjust their strategies in this early stage of competition. * **Neglected Kurtosis:** Despite progress in volatility modeling, kurtosis remains underutilized among lower-ranked miners. **Ongoing Improvements:** The Synth team is addressing the issue of forecast copying. One potential solution under consideration is increasing the steepness of the reward curve. Further updates will follow. --- *Originally published on [Synth](https://paragraph.com/@synthdata/synth-subnet-miner-performance-review-12)*