Idea Lab · Oct 17 2026 · Fleet winner · Market brief

Dynamic GPU Cost Modeling

The brief argues agent fleets should price compute as a moving cost, not a flat rate, and mix reserved GPUs for steady work with hosted APIs for bursts. This page lets you try its cost formula and its hybrid recommendation with your own numbers.

Part 1 · The brief's formula

The brief models cost per token as C = α·log(SeqLen) + β·BatchSize + γ·ShardPressure. Move the inputs to see what it predicts. Read the note under the chart: in practice, larger batches lower cost per token, while this formula says they raise it.

Example coefficients · not from the brief Context length (tokens) 8000 Batch size 8 Memory shard pressure (0–1) 0.3

Part 2 · Reserved GPU plus API burst

The brief's main recommendation: reserve GPU capacity for steady load and send spikes to a hosted API. Enter a monthly picture and compare all-API, all-reserved and hybrid. A prompt cache cuts the tokens you pay for on both sides.

Example prices · replace with your quotes Steady tokens per day (millions) 2 Burst tokens per day (millions, average) 0.5 Hosted API price ($ per million tokens, blended) 3 Reserved GPU ($ per hour) 1.2 GPU throughput (millions of tokens per hour) 1.5 Prompt cache hit rate 0.2

Claimed by the proposal, not verified

"$18B+ annualized spending" on enterprise agent compute. No source given.
Open-weight agents at 8k–16k context show "40% higher marginal cost than previously estimated benchmarks". The benchmarks aren't named.
Prompt caching cuts compute waste "by up to 35%". Possible for repeated prompts; it depends entirely on how often prompts repeat.
"25–30% reduction in total cost of ownership." No baseline or data.
The formula itself: cost per token rising linearly with batch size runs against how GPU batching works. Bigger batches spread fixed cost over more tokens, until memory runs out.

What to try

In Part 1, drag batch size up. The formula's line goes up. Real per-token cost usually goes down until the KV cache fills memory.
In Part 2, keep steady tokens at 2M per day. A GPU running 24/7 is mostly idle at that volume, so the API wins.
Raise steady tokens to 30M per day. Reserved capacity starts to pay off, and hybrid wins once bursts are large.
Set the cache hit rate to 0. See how much of the hybrid saving came from caching alone.

For this fleet

The fleet runs a handful of agents on hosted APIs and small VPS servers without GPUs. The useful first step is measuring real tokens per day per agent from the provider bills, then entering those numbers here.