I gave eight AI decision models the same $10,000 of play money and the same day of Bitcoin: 24 one-hour candles, from 9 November 15:00 UTC to 10 November 14:00 UTC. Every hour, each model saw the last 12 candles and answered one question: long, short or flat for the next hour. Bitcoin fell 3.04% over the day.

Every model lost money. The best one lost 1.19%. Sitting in cash lost nothing.

That a model cannot call the next candle from price history alone is not news. What I found more useful is how they lost, which model lost least and why, and what the whole thing cost to run.

All eight models run on NanoGPT, which puts them behind one API and bills only the input tokens. If you want to rerun this, you can sign up through my referral link: NanoGPT referral link. Disclosure: that is my referral link, and it is the same link everywhere in this post.

A decision model does not write text. You send it a state and a set of typed questions, and every answer comes back as probabilities over named options. A choice question returns a probability for each option, and a score question returns an expected value over an ordered rubric. On NanoGPT they sit behind one endpoint, POST /api/v1/decisions.

That makes them easy to wire into code: no output to parse, no prompt to coax a format out of. It also makes them an obvious thing to point at a trading signal, which is exactly what this test does.

• Data: Kraken's public OHLC feed, BTC/USD, 1-hour candles. No key, no account.

• State per decision: the last 12 hourly candles (open, high, low, close, volume) plus four computed features: 12-hour return, 3-hour return, range, and volume against its average.

• Questions per decision: direction (choice: long, short or flat) and conviction (score, 0 to 4). The same state and the same questions for every model.

• Execution: the whole $10,000 in each position, no leverage, and a 0.1% fee per trade leg. Entering or exiting is one leg; flipping from long to short is two.

• Baselines on the same candles: buy-and-hold, a random trader with a fixed seed, and always flat.

The eight models, by their NanoGPT names: eu/jev Jeff, Jev 1.14 (typesafe/jev-1.14), GPT-6 Luna Decisions (openai/gpt-6-luna-decisions), Drex Max Finance (nace-ai/drex-max-finance), PPLX Decider (perplexity/pplx-decider-v1.2), Rune 26B-A4B (surogate/rune-26b-a4b), Clef (cloudflare/clef) and Liquid D1 (liquid/d1).

Ranked by final balance.

"Right calls" counts only the hours a model went long or short; flat hours are left out.

Together, they were a coin flip. Across all eight models, 61 of 129 directional calls were right: 47%. Jeff's 13 of 20 looks like a signal until you put an interval on it. The 95% confidence interval runs from roughly 41% to 85%, and that includes 50%.

One model beat a random trader. The seeded random trader finished at -1.62%. Only Jeff did better. Two models beat buy-and-hold (Jeff and Jev). None beat cash.

Caution did better than accuracy. Jev was right on only 4 of its 9 directional calls, the third-lowest hit rate in the field, and still finished second. It sat flat for 15 of the 24 hours and made 8 trade legs, the fewest of any model, and its 1.85% drawdown was second only to Jeff's 1.64%. On a falling day, not trading was the winning trade, and the model that traded least got closest to it.

Fees decided the best result. Jeff was the only model whose calls made money before costs: +$20.23. Fees of $138.80 turned that into a $118.57 loss. For the other seven, wrong-way positions cost more than fees did (gross losses of $106 to $286 against fees of $79 to $188), but fees still made every one of them worse.

Two models behaved as one. Rune 26B-A4B and Clef made identical direction calls on all 24 hours: the same 19 trade legs and the same -3.98%. Their conviction scores differed, so these are not copies of each other, but on this window they traded as a single model.

Conviction does not transfer between models. Jeff's conviction averaged 0.55 on the 0 to 4 scale; the others ranged from 0.89 to 1.54. You cannot compare one model's 2 with another's. Only two calls in the whole test scored 3 or higher, one from Rune and one from Liquid D1, and both were wrong.

A finance name did not help. Drex Max Finance, the one model with finance in its name, landed mid-table at 47%.

It cost almost nothing. 192 decisions came to $0.0196 in total. The cheapest model for the day was PPLX Decider at $0.00056; the most expensive was Clef at $0.00746.

I also ran Drex Max Finance alone on 60 five-minute candles. It made 37 directional calls at 51.4%, and its calls made $50.39 before costs. Fees of $284.96 turned that into a 2.35% loss. The faster you trade, the more the fee, not the call, decides the result.

Read these before you quote any number above.

One day is not a sample. Each model made 24 decisions, of which 9 to 20 were directional. At that size, even the best result, 13 of 20, cannot be told apart from a coin flip. Nothing here is evidence of a tradable edge, for any model.

It was a down day. Bitcoin fell 3.04%. On a rising day, buy-and-hold would make money, always flat would look like the worst choice, and the ranking would change.

Price history only. The models saw candles and four derived features: no news, no order book, no funding rates. The finding is narrow: these models could not call the next candle from the last twelve, at either one-hour or five-minute frequency. That is a statement about the task, not a failure of any one model.

Paper fills. The only cost modelled is a flat 0.1% per trade leg. There is no slippage and no spread, so real trading would cost more.

Not advice. No model output moved real money, and none should. A consequential financial decision needs a person behind it.

What would make this a fair test: weeks of candles across rising and falling markets, a hypothesis written down before the run, and inputs beyond price. And a better job for decision models in the meantime is a typed question whose answer is in the input itself: route this ticket, classify this request, check whether this claim matches that document. The next hour of Bitcoin is not in the last twelve.

The whole setup is small: a public candle feed, one request per model per hour, and a few lines to keep a $10,000 ledger and count fees. The full day across eight models cost under two cents.

If you want to run it, start with a NanoGPT account through my referral link: Create a NanoGPT account through my referral link. Then pick a longer window than I did.

If you find an edge over a month of candles, I would like to see it. Bring the confidence interval.

I write about AI models, how to evaluate them and what they cost, as metaend. If you want to reach me, build on any of this, or tell me where it breaks, the door is here:

metaend on Quilibrium

More writing lives at paragraph.com/@metaend.

Written by metaend.