Negotiating GPU compute
GPUs are the new electricity. And like electricity, steady access at predictable cost matters more than getting the absolute cheapest rate. If you’re spending over $1M annually, you have room to negotiate.
The cheapest per-hour option is always an exclusive reservation - a hard commitment. You’re removing risk from the provider’s balance sheet, and they discount you for it.
But before you sign, understand what you’re negotiating against.
Provider economics
GPU providers carry three cost layers: hardware (GPUs, hosts, networking, racks), operations (power, cooling, staff, space), and financing. They’re heavily leveraged—loans fund the GPUs you’re renting, and a portion of every invoice services that debt.
Their central constraint is the two-year utilization cliff. Nvidia releases new generations on even higher cadence, and older GPUs depreciate fast. Providers need to recover their capital before the next architecture takes over.
This means utilization is their core metric. “Use it or lose it” contracts are variance removers for them. If they recoup within 24 months, everything after is margin.
They would love your two-year commitment. They would also happily sell you time on two-year-old silicon.
When you negotiate
Forget timing around GPU launches - everyone tries that, demand spikes, and providers play hard to get.
In practice, you negotiate when your current contract is ending anyway.
What to protect
Some requirements aren’t negotiable. YMMV. It may be CUDA/firmware versions, failure rate SLAs (including replacement timelines and penalties), intra-node bandwidth specs, telemetry access, and guaranteed capacity.
Where to push
These levers are valuable to you and cheap for them to give:
Step-downs after 18 month — the right to scale down GPU count.
Larger upfront payment — reduces their financing cost; extract a discount.
SKU flexibility — ability to shift between H100 and A100.
Burst provisions — pre-negotiated terms for temporary capacity spikes.
Hour banking — carry forward unused compute (capped).
Storage costs — for training with images/videos storage costs surprise many.