Token-Inferred Compute Elasticity (TICE)
The brief proposes pricing GPU compute per token, with the price rising when GPUs are busy and falling when they're idle, instead of a flat rate per GPU-hour. This simulator runs one day of agent traffic on a shared GPU pool and compares the two. It shows the mechanism and also who it actually helps.
Claimed by the proposal, not verified
- 18–27% lower cost variance versus static per-GPU-hour pricing, from "simulation across 12 enterprise workloads". The workloads and data were not shared.
- Under 5% error on elasticity predictions from a "calibrated regression model trained on historical inference logs". No logs or model were provided.
- A provider-agnostic market-clearing mechanism. This only works if you run or sell the GPUs; a buyer paying list API prices has nothing to clear.
The elastic price is set revenue-neutral: on the original demand the provider earns the same as with flat pricing, so any saving comes from work that moves. Bars: tokens demanded each hour (after delay-able work moves to cheaper hours). Line: elastic price per million tokens. Dashed: pool capacity.
| Pricing | Day cost | Std dev of hourly bill | Peak utilisation | Overload hours |
|---|
What this means for the fleet
Elastic pricing helps whoever owns the GPUs fill idle hours. For a buyer it only saves money if some work can move to cheap hours, and it makes bills less predictable during peaks. The fleet today buys tokens from hosted APIs at fixed list prices and runs CPU VPS servers, so the useful part to keep is the first step of the brief: meter tokens and cost per agent, per task. Hosted batch APIs that already discount delayed work give most of the "shift to cheap hours" benefit.
What to try
- Set elasticity to 0. Elastic and flat pricing become the same thing.
- Set "agents that can delay work" to 0 and elasticity to 2. The elastic bill costs the same but swings far more hour to hour, the opposite of the claim.
- Raise delay-able work to 60%+. Now elastic pricing smooths demand and cuts cost: the benefit comes from moving work, not from the pricing formula.
- Pick "Steady all day". There's nothing to smooth, so the two models barely differ.