The most expensive thing nobody measures

Demo data today: CoverOps does not read GPU utilisation from your cloud yet. Every team has a dashboard that says whether the training job is running. Almost none has one that says whether the accelerators are doing anything. The money sits in that gap.

On-demand and spot, per accelerator-hour
AcceleratorOn demandSpot
T4-16GB$0.53$0.16
L4-24GB$0.98$0.31
A10G-24GB$1.21$0.38
A100-40GB$3.13$0.98
A100-80GB$4.09$1.31
H100-80GB$9.89$3.42

The price list the demo costs every run against.

Idle time billed while you have been reading this site

.₹0.00

One training node: 8× H100-80GB, accelerators 30% utilised, starved on data loading.

Node run rate
₹50.8 lakh ($57,758)/mo
Recoverable idle share
₹24.9 lakh ($28,301)/mo
Over a year
₹2.99 crore ($339,615)

Demo figures. Pauses whenever this tab is in the background, and starts again at zero on your next visit.

Worked from public on-demand pricing, not measured at a customer. How the number is built

Accelerator

$9.89/hr on demand, $3.42/hr spot

Accelerators in the node

Under 35%: flagged as starved on input, not compute-bound.

Recoverable per year

₹2,98,86,093

₹2.99 crore ($339,615 on a dollar bill)

Node run rate
₹50,82,669/mo
Idle accelerator share
70%
Unavoidable-idle discount
×0.7
Recoverable
₹24,90,508/mo

Arithmetic on our own accelerator price list, using the formula wasteReport() runs in the product. It is not a measured customer outcome. Rupees at ₹88 to the dollar.

Where GPU money goes

Three patterns cover most accelerator overspend, so these are the three the control plane models.

  1. Starved runs

    The job is alive and the accelerators are waiting.

    A run under 35% utilisation is not compute-bound. It is waiting on the input pipeline, and the GPUs stay allocated and billed between batches.

    How it is caught
    Utilisation tracked per run against the accelerators it holds. Below 35%, the idle share is priced as recoverable.
    What fixes it
    Raise batch size or dataloader workers, or move the run to a smaller accelerator.
  2. Spot without checkpoints

    One preemption throws away every epoch you paid for.

    Spot costs about a third of on-demand, which makes it tempting for long runs. Without checkpointing, a single preemption restarts the run from zero.

    How it is caught
    Flagged wherever spot is on and checkpointing is off. The exposure is money already spent, not a forecast.
    What fixes it
    Checkpoint periodically to object storage. Writing checkpoints costs a rounding error next to the exposure.
  3. Idle serving

    Paying for a warm GPU between requests.

    An endpoint under 25% utilisation is holding an accelerator for traffic that is not arriving. Overnight and at weekends that is close to pure loss.

    How it is caught
    Utilisation and requests per second per endpoint, costed against scale-to-zero or a smaller accelerator.
    What fixes it
    Turn on scale-to-zero, or move to an accelerator sized for the traffic you actually get.

A finding names the run and the money

Not “GPU utilisation is low”, which is a ticket someone has to go and investigate. This:

churn-propensity 8× A100-80GB, 31% utilised
$8,412/mo recoverable
fix: raise batch size or dataloader workers
Start your free scan

Shipping agents rather than training models? The same engine prices LLM agent spend.