AI
LocalAI Calc
VRAM + cost per token
Runs in your browser · no signup · no data sent

Local LLM Hardware Guide: What to Buy in 2026

How much VRAM you need for 8B, 32B and 70B models, which cards make sense, and where the money is wasted.

Updated 2026-11-10 6 min read

Start with the model, not the card

Hardware decisions go wrong when they start from a budget instead of a model. Pick the model you actually want to run, then work out the memory it needs. The card is the last decision, not the first.

The numbers below use Q4_K_M, which is what most people run. Weights are calculated at 0.60 bytes per parameter — the measured figure, not the theoretical 0.50. KV cache is calculated at 8K context unless stated otherwise.

What each size class actually needs

Model sizeWeights (Q4)KV at 8KTotalMinimum card
8B~4.9 GB~0.5 GB~6.9 GB12 GB
14B~8.9 GB~0.7 GB~11.1 GB16 GB
24B~14.2 GB~0.6 GB~16.3 GB24 GB
32B~19.7 GB~2.0 GB~23.2 GB24 GB (8K only)
70B~42.4 GB~2.5 GB~46.4 GB48 GB

That last row is the one people underestimate. A 70B at Q4 does not fit in 24 GB, and no amount of configuration changes the arithmetic. The dedicated guide on this site walks through the numbers.

Four hardware paths

1. One 24 GB card (RTX 3090 or 4090)

The sweet spot for 8B to 24B models. A used 3090 is the cheapest way to get 24 GB, and 936 GB/s of bandwidth is enough for roughly 55 tok/s on an 8B Q4 model. A 4090 adds bandwidth and lower latency but costs about twice as much.

2. One 32 GB card (RTX 5090)

Buys you 32B models at comfortable context, and 8B models with a large batch. The card is fast, but the price per gigabyte is worse than a used 3090 — you are paying for bandwidth, not capacity.

3. Apple silicon (Mac Studio or Mac mini)

Unified memory lets you hold a 70B model in 64 GB without a 48 GB GPU. The trade-off is bandwidth: 546 GB/s on an M4 Max is roughly half an RTX 4090, so tokens per second are lower. For batch work, long-context document processing and low power draw, it is a reasonable choice. For interactive chat at speed, it is not.

4. Two cards, or a 48 GB professional card

The entry point for 70B. Two used 3090s cost less than a single 48 GB card, but you add power draw, noise, and the need for a motherboard and PSU that can carry both. If you only need 70B occasionally, renting cloud GPU time is usually cheaper than owning.

Three mistakes that cost real money

  • Buying for the theoretical Q4 size. At 0.50 bytes per parameter a 32B looks like 16 GB. At the measured 0.60 it is 19.7 GB, and with KV cache you are at the edge of a 24 GB card. Always calculate with 0.60.
  • Ignoring the KV cache. A model that fits at 4K context can fail at 32K. If you plan to feed long documents, budget the cache before you buy.
  • Paying for bandwidth you do not need. A 4090 is twice the price of a 3090 for roughly 8 % more bandwidth. If you are running an 8B model a few hours a day, that premium does not pay back.

What to buy first

If you are unsure, buy one used 24 GB card and run 8B to 14B models for a month. Measure your actual usage, then decide whether you need more memory or more speed. The cost calculator tells you what that card costs per million tokens at your usage level.

Read next
Why 70B Won't Fit in 24GB VRAM (Despite the Tutorials)
Read →
Local LLM vs Cloud API: Real Cost Per Million Tokens
Read →
Questions
Why does Q4 need 0.60 bytes per parameter, not 0.50?
Q4_K_M stores most weights at 4 bits but keeps scales, zero-points and some layers at higher precision. Measured files land around 0.60 bytes per parameter — Qwen3-32B is 19.8GB for 32.8B parameters. Using 0.50 underestimates memory by about 20% and is the main reason people buy a card that turns out too small.
Why calculate KV cache separately?
KV cache scales with context length, not with model size alone. Qwen3-32B needs 256 KiB per token at FP16 — 8GB at 32K context. A 24GB card holding 19.5GB of weights has roughly 3GB left, which caps you near 8K. That is why a model can load fine and then fail the moment you paste in a long document.
Are the tokens/sec numbers measured?
No. They are estimates derived from memory bandwidth, using a calibration factor of 0.29 taken from published RTX 4090 measurements on Llama 3 8B Q4 (about 62 tok/s against a 214 tok/s bandwidth ceiling). Real throughput varies with batch size, backend and quantization. Treat them as a planning number, not a benchmark.
Does a 70B model really not run on 24GB?
At Q4_K_M, 70B weights alone are about 42GB. On a 24GB card the remainder has to sit in system RAM and stream over PCIe, which drops generation to roughly 1–3 tok/s. It technically runs; it is not usable for interactive work. See the dedicated guide for the numbers.

All throughput figures are estimates derived from memory bandwidth and published measurements. VRAM figures are computed from model architecture and quantization assumptions. Verify against your own runtime before buying hardware.