RTX 5090: what actually fits in 32GB
A 32GB card with 1792 GB/s of memory bandwidth. Below: what each model needs at Q4_K_M, how fast it should run, and what the tokens cost.
| Memory | 32 GB GDDR7 |
|---|---|
| Bandwidth | 1792 GB/s |
| Power (TDP) | 575 W |
| Reference price | $1999 |
| Efficiency factor | 0.29 |
MSRP reference, Nov 2026. Street prices often higher.
| Model | Weights (Q4) | KV @ 8K | Total needed | — | est. tok/s | Cost / 1M tokens | Max context that fits |
|---|---|---|---|---|---|---|---|
| Llama 3.3 70B Meta · Dense · GQA | 42.4 GB | 2.7 GB | 46.5 GB | No | — | $— | — |
| Qwen3 32B Alibaba · Dense · GQA | 19.7 GB | 2.1 GB | 23.3 GB | Fits | 26 | $5.53 | 36k |
| Mistral Small 24B Mistral AI · Dense · GQA | 14.2 GB | 1.3 GB | 17.0 GB | Fits | 37 | $3.98 | 91k |
| Qwen3 14B Alibaba · Dense · GQA | 8.9 GB | 1.3 GB | 11.7 GB | Fits | 59 | $2.49 | 122k |
| Qwen3 8B Alibaba · Dense · GQA | 4.9 GB | 1.2 GB | 7.6 GB | Fits | 106 | $1.38 | 162k |
| Llama 3.1 8B Meta · Dense · GQA | 4.8 GB | 1.1 GB | 7.4 GB | Fits | 108 | $1.35 | 183k |
All throughput figures are estimates derived from memory bandwidth and published measurements. VRAM figures are computed from model architecture and quantization assumptions. Verify against your own runtime before buying hardware.
What it costs to run
The GPU is paid once but depreciates every year, and it draws power whether or not you are generating. At low volume the fixed cost dominates: a $1,799 card producing 978M tokens over three years works out to about $2.08 per million tokens — competitive with cheap APIs, far above the cheapest ones.
All throughput figures are estimates derived from memory bandwidth and published measurements. VRAM figures are computed from model architecture and quantization assumptions. Verify against your own runtime before buying hardware.