RTX 4090: what actually fits in 24GB
A 24GB card with 1008 GB/s of memory bandwidth. Below: what each model needs at Q4_K_M, how fast it should run, and what the tokens cost.
| Memory | 24 GB GDDR6X |
|---|---|
| Bandwidth | 1008 GB/s |
| Power (TDP) | 450 W |
| Reference price | $1799 |
| Efficiency factor | 0.29 |
Reference street price, Nov 2026. Check before buying.
| Model | Weights (Q4) | KV @ 8K | Total needed | — | est. tok/s | Cost / 1M tokens | Max context that fits |
|---|---|---|---|---|---|---|---|
| Llama 3.3 70B Meta · Dense · GQA | 42.4 GB | 2.7 GB | 46.5 GB | No | — | $— | — |
| Qwen3 32B Alibaba · Dense · GQA | 19.7 GB | 2.1 GB | 23.3 GB | Tight | 15 | $8.69 | 7k |
| Mistral Small 24B Mistral AI · Dense · GQA | 14.2 GB | 1.3 GB | 17.0 GB | Fits | 21 | $6.25 | 45k |
| Qwen3 14B Alibaba · Dense · GQA | 8.9 GB | 1.3 GB | 11.7 GB | Fits | 33 | $3.92 | 76k |
| Qwen3 8B Alibaba · Dense · GQA | 4.9 GB | 1.2 GB | 7.6 GB | Fits | 59 | $2.17 | 111k |
| Llama 3.1 8B Meta · Dense · GQA | 4.8 GB | 1.1 GB | 7.4 GB | Fits | 61 | $2.13 | 126k |
All throughput figures are estimates derived from memory bandwidth and published measurements. VRAM figures are computed from model architecture and quantization assumptions. Verify against your own runtime before buying hardware.
What it costs to run
The GPU is paid once but depreciates every year, and it draws power whether or not you are generating. At low volume the fixed cost dominates: a $1,799 card producing 978M tokens over three years works out to about $2.08 per million tokens — competitive with cheap APIs, far above the cheapest ones.
All throughput figures are estimates derived from memory bandwidth and published measurements. VRAM figures are computed from model architecture and quantization assumptions. Verify against your own runtime before buying hardware.