A100 40GB SXM vs H100 SXM

A100 40GB SXM vs H100 SXM

The H100 triples dense FP16 throughput and adds FP8 at roughly 2-3x the hourly rate. For FP16 work that fits in 40 GB, the A100 40GB is often the better cost per step.

Workload fit

A100 40GB SXM: mid-size fine-tuning, batch inference, 70b-class serving. H100 SXM: llm pre-training, 70b+ fine-tuning, high-throughput inference.

Which is cheaper to rent, A100 40GB SXM or H100 SXM?

By today's published on-demand rates: A100 40GB SXM starts at $1.000/GPU/hr (median $4.08, 4 providers) and H100 SXM starts at $0.890/GPU/hr (median $5.49, 77 providers). The cheapest live listing is the H100 SXM at SpotGPUs. Figures recompute from the panel on every refresh.

A100 40GB SXM or H100 SXM - which should I rent?

A100 40GB SXM: The 40 GB SXM A100 is the original 2020 Ampere flagship module: same silicon and tensor throughput as the 80 GB part, half the memory and bandwidth. H100 SXM: The H100 SXM is the chip the current generation of frontier models was trained on, and the benchmark every other datacenter GPU is priced against. The side-by-side spec table and live price rows above give the full picture.

Where can I rent A100 40GB SXM and H100 SXM?

4 providers publish A100 40GB SXM rates and 77 publish H100 SXM rates on Compute Cafe. Every row links to the provider's own published price - committed-use and reserved rates are excluded.