GPU MODELS / A100 40GB SXM

A100 40GB SXM cloud GPU pricing

16 listings across 4 providers. Cheapest published rate: $1.000/GPU/hr at RunPod. Sorted by per-GPU hourly price. Every row links to the provider's own published price.

WHAT IS THE A100 40GB SXM

The 40 GB SXM A100 is the original 2020 Ampere flagship module: same silicon and tensor throughput as the 80 GB part, half the memory and bandwidth. It trains and serves exactly like an A100 until the model crosses 40 GB - which in 2026 most serious models do. What remains is a capable mid-size workhorse that rents at a real discount to its 80 GB sibling.

MAKER
NVIDIA
ARCHITECTURE
Ampere
LAUNCHED
2020
MEMORY
40 GB HBM2
MEMORY BANDWIDTH
1,555 GB/s
TENSOR COMPUTE
312 TFLOPS FP16/BF16 tensor dense (624 sparse)
POWER DRAW
400 W (SXM4)
INTERCONNECT
NVLink 3.0 600 GB/s

WHAT IT IS GOOD AT

Mid-size fine-tuning
Strong fit - full BF16 training of 7B-class models, or LoRA on larger, with real NVLink for multi-card runs.
Batch inference
1,555 GB/s of HBM keeps 13B-30B quantized models serving at good throughput.
70B-class serving
Poor fit - 40 GB forces multi-card splits for models an 80 GB card holds alone.

WHO SHOULD NOT RENT THE A100 40GB SXM

Models over ~35 GB of weights - the 80 GB A100 or H100 removes the multi-card tax.
Memory-bandwidth-bound serving where the 80 GB variant's 2 TB/s matters.
ProviderConfigVRAMLocation$/hr$/GPU/hrSource
RunPodRunPodA100 SXM 40GB × 140 GBGlobal$1.000$1.000receipt
ACCACCA100 40GB NVLink × 140 GBSingapore$1.960$1.960receipt
Microsoft AzureMicrosoft AzureA100 40GB NVLink [ND96asr_A100_v4] × 840 GBUS West 2$27.197$3.400receipt
Microsoft AzureMicrosoft AzureA100 40GB NVLink [ND96asr_A100_v4] × 840 GBUS East$27.197$3.400receipt
Microsoft AzureMicrosoft AzureA100 40GB NVLink [ND96asr_v4] × 840 GBUS West 2$27.197$3.400receipt
Microsoft AzureMicrosoft AzureA100 40GB NVLink [ND96asr_v4] × 840 GBUS East$27.197$3.400receipt
Microsoft AzureMicrosoft AzureA100 40GB NVLink [ND96asr_A100_v4] × 840 GBUS West 3$27.197$3.400receipt
Baidu BCCBaidu BCCA100 40GB SXM × 140 GBChina$3.876$3.876receipt
Microsoft AzureMicrosoft AzureA100 40GB NVLink [ND96asr_v4] × 840 GBUS South Central$32.630$4.079receipt
Microsoft AzureMicrosoft AzureA100 40GB NVLink [ND96asr_A100_v4] × 840 GBUS South Central$32.630$4.079receipt
Microsoft AzureMicrosoft AzureA100 40GB NVLink [ND96asr_A100_v4] × 840 GBUS Gov Virginia$34.003$4.250receipt
Microsoft AzureMicrosoft AzureA100 40GB NVLink [ND96asr_v4] × 840 GBUS Gov Virginia$34.003$4.250receipt
Microsoft AzureMicrosoft AzureA100 40GB NVLink [ND96asr_A100_v4] × 840 GBEU West$35.357$4.420receipt
Microsoft AzureMicrosoft AzureA100 40GB NVLink [ND96asr_v4] × 840 GBEU West$35.357$4.420receipt
Microsoft AzureMicrosoft AzureA100 40GB NVLink [ND96asr_v4] × 840 GBIT North$35.357$4.420receipt
Microsoft AzureMicrosoft AzureA100 40GB NVLink [ND96asr_A100_v4] × 840 GBIT North$35.357$4.420receipt

THE A100 40GB SXM MARKET, BY THE NUMBERS

PROVIDERS
4
CONFIGURATIONS
16
MEDIAN $/GPU/HR
$4.08
SPREAD
4.4x

4 providers publish 16 A100 40GB SXM configurations on the panel today. Published per-GPU hourly rates run from $1.000 (RunPod) to $4.42 (Microsoft Azure), a 4x spread between the cheapest and the most expensive published rate for the same chip. The median listing sits at $4.08/GPU/hr. 15 rows were reported in stock by the provider at last observation; the rest are published catalog rates. All rates are on-demand - committed-use and reserved pricing is excluded.

CHEAPEST A100 40GB SXM BY GEOGRAPHY
singaporeACC$1.960/hrunited statesMicrosoft Azure$3.400/hrchinaBaidu BCC$3.876/hritalyMicrosoft Azure$4.420/hr
A100 40GB SXM PRICE TREND, DAILY PANEL SNAPSHOTS

Across every A100 40GB SXM row on the panel, the daily floor held flat at $1.000/GPU/hr over 11 days of tracking. Snapshots run daily; the full history is public in the repo.

A100 40GB SXM PROVIDER NOTES
RunPod · 1 config · 1 region · 1 in stock
Sets the panel floor for this model.
from $1.000/hr
ACC · 1 config · 1 region · 1 in stock
1 of 1 configs reported in stock at last check.
from $1.960/hr
Microsoft Azure · 13 configs · 7 regions · 13 in stock
Widest regional footprint here - 7 regions.
from $3.400/hr
Baidu BCC · 1 config · 1 region
Published catalog rates.
from $3.876/hr
BUYING A100 40GB SXM: WHAT THE PANEL SAYS

At the current floor, one A100 40GB SXM running around the clock costs about $730 a month (730 hours of on-demand arithmetic). Node shape matters: the single-GPU floor is $1.000/GPU/hr against $3.400/GPU/hr on an 8-GPU node (+240.0%). Per-GPU floors by config size: 1x $1.00 · 8x $3.40. Compare the alternatives below before committing - the same budget often buys more than one chip class.

A100 40GB SXM VS THE ALTERNATIVES

A100 40GB SXM vs A100 80GB
Same compute, double the memory and 31% more bandwidth. If the model fits in 40 GB the cheaper 40 GB part is the same speed; if it does not, no discount compensates.
A100 40GB SXM vs H100 SXM
The H100 triples dense FP16 throughput and adds FP8 at roughly 2-3x the hourly rate. For FP16 work that fits in 40 GB, the A100 40GB is often the better cost per step.

Compare live rates: A100 40GB SXM vs A100 80GB · A100 40GB SXM vs H100 SXM

A100 40GB SXM PRICING FAQ

What is the cheapest A100 40GB SXM cloud GPU right now?
The cheapest published A100 40GB SXM rate on the panel is $1.000 per GPU-hour at RunPod (A100 SXM 40GB x 1, Global). Every price links to the provider's own published rate.
How much does a A100 40GB SXM cost per hour?
Across 4 providers tracked today, published per-GPU hourly rates for A100 40GB SXM run from $1.00 to $4.42, with a median of $4.08. Committed-use and reserved rates are excluded; everything shown is on-demand.
How many providers rent A100 40GB SXM GPUs?
4 providers publish 16 distinct A100 40GB SXM configurations on Compute Cafe. 15 of those rows were reported in stock by the provider's live endpoint at last observation.
Where is A100 40GB SXM cheapest?
By published per-GPU hourly rate, the cheapest geography for A100 40GB SXM today is singapore at $1.96/hr (ACC). The spread across 4 tracked geographies runs to $4.42/hr in italy.
Are A100 40GB SXM prices going up or down?
Over the 11 days Compute Cafe has tracked the panel, the A100 40GB SXM floor is holding flat: $1.000 to $1.000/GPU/hr (+0.0%). Daily snapshots are public in the repo, so the trend is auditable.
What does a A100 40GB SXM cost per month?
At the current floor of $1.000/GPU/hr, a A100 40GB SXM running 24/7 costs about $730 per month (730 hours of on-demand arithmetic on the cheapest published rate). Committed-use contracts can price lower; the panel tracks on-demand rates only.
Is a full 8-GPU A100 40GB SXM node cheaper than a single card?
On today's panel: the single-GPU floor is $1.000/GPU/hr and the 8-GPU node floor is $3.400/GPU/hr - more expensive per GPU on a full node (+240.0%). Check the table for the exact configs behind each floor.
Is the A100 40GB still worth renting?
Yes when it prices meaningfully below the 80 GB part and your model fits. Compute is identical to the bigger A100 - you are only giving up memory headroom and bandwidth.
Can it fine-tune a 70B model?
Not on one card beyond aggressive quantization. QLoRA on 70B wants the 80 GB variant; full fine-tuning wants a multi-GPU node either way.

See also alternatives to the A100 40GB SXM · cheapest GPU-hour per model · inference chips · training chips · by location