Per Million Tokens on Paper. $29 in Reality.

Same GPU.
$0.14 per million tokens.
Or $29.

I've computed both. Neither number is wrong.

The $0.14 version:
→ 8× H200 node, ~$370k, amortized over 3 years
→ + power, cooling, network, 20% ops
→ ≈ $2.58 per GPU-hour, fully loaded
→ 50% utilization: $5.15 per productive hour
→ Qwen-32B in FP8: ~5,600 output tokens/s, ~12,400 input tokens/s
→ $0.26/M output, $0.12/M input, $0.14/M blended

Looks great in a business case. Cheaper than any API.

The $29 version:
→ a real enterprise chat app, 263M tokens/month
→ 4× H200 for high availability across 2 zones
→ same $2.58 per GPU-hour: ~$7,500/month
→ GPUs busy about 2% of the time
→ ≈ $29 per million tokens

Same hardware. Same hourly cost. A 200× gap.

The difference is one variable: utilization.

A GPU you pay for 24/7 and use 2% of the time doesn't get cheaper because your model is efficient. It gets cheaper when it's busy.

What actually moves price per token, in order:

1. Utilization (at 25% you pay double compared to 50%)
2. Amortization period (3 vs 5 years ≈ 40%)
3. Quantization (FP8 means bigger batches)
4. Batch size vs latency SLO
5. Your real input:output ratio

My rule for build-vs-buy:
Self-host when you can keep GPUs busy most of the day (batch jobs, internal traffic, multiple teams sharing a pool). Below that, pay per token and stop pretending.

And renting GPUs by the hour doesn't escape this. An idle rented GPU bills like an idle owned one. Only per-token APIs charge for what you actually use.

What utilization did your last self-hosting business case assume? And did anyone measure it after go-live?