Local LLM VRAM Calculator

20 open models against 30 GPUs, Macs and multi-card builds. Architectures come from each model's own config file, and the key-value cache is sized by key-value heads rather than query heads — which is why these figures are smaller than most calculators give.

Two numbers decide whether a model runs. The weights are fixed and easy. The key-value cache grows with every token of context and is where the surprises live — on a long-context model it can end up larger than the weights themselves, so the model loads fine and then fills the card mid-conversation.

Grouped-query attention, and why it matters

Modern models share one key-value pair across several query heads. The cache is sized by the key-value count, so a model with an 8-to-1 ratio needs an eighth of the cache a naive calculation predicts. This is the single commonest error in VRAM estimates.

Grouped-query attention ratios and per-token cache cost
Model	Query / KV heads	Cache saving	Per token
Qwen3 235B-A22B	64 / 4	16×	188.0 KB
Qwen2.5 3B	16 / 2	8×	36.0 KB
Qwen3 30B-A3B	32 / 4	8×	96.0 KB
Qwen3 32B	64 / 8	8×	256.0 KB
Llama 3.3 70B	64 / 8	8×	320.0 KB
Qwen2.5 72B	64 / 8	8×	320.0 KB
Qwen2.5 7B	28 / 4	7×	56.0 KB
Qwen2.5 Coder 7B	28 / 4	7×	56.0 KB

Every model

Memory requirements for every model at Q4_K_M
Model	Parameters	Weights Q4_K_M	Total at 8K	Runs on
Llama 3.2 1B	1.2B	0.7 GB	1.7 GB	30/30
Qwen2.5 3B	3.1B	1.6 GB	2.7 GB	30/30
Llama 3.2 3B	3.2B	1.7 GB	3.4 GB	30/30
Mistral 7B v0.3	7.3B	3.8 GB	5.6 GB	30/30
Qwen2.5 7B	7.6B	4.0 GB	5.2 GB	30/30
Qwen2.5 Coder 7B	7.6B	4.0 GB	5.2 GB	30/30
Llama 3.1 8B	8.0B	4.2 GB	6.0 GB	30/30
Gemma 2 9B	9.2B	4.8 GB	8.3 GB	30/30
Phi-4	14.7B	7.7 GB	10.0 GB	29/30
Qwen2.5 14B	14.8B	7.7 GB	10.0 GB	29/30
DeepSeek-R1 Distill Qwen 14B	14.8B	7.7 GB	10.0 GB	29/30
Gemma 2 27B	27.2B	14.3 GB	17.9 GB	18/30
Qwen3 30B-A3B	30.5B	16.0 GB	17.5 GB	18/30
Qwen2.5 32B	32.8B	17.2 GB	20.0 GB	17/30
Qwen2.5 Coder 32B	32.8B	17.2 GB	20.0 GB	17/30
Qwen3 32B	32.8B	17.2 GB	20.0 GB	17/30
Mixtral 8x7B	46.7B	24.5 GB	26.3 GB	13/30
Llama 3.3 70B	70.5B	37.0 GB	40.3 GB	10/30
Qwen2.5 72B	72.7B	38.1 GB	41.4 GB	10/30
Qwen3 235B-A22B	235.1B	123.2 GB	125.4 GB	1/30

What decides whether a model fits

Memory, not speed, decides whether a model runs at all, and three things fill it.

Weights: parameters times bits

Every billion parameters takes about 1.86 GB at FP16 / BF16, 0.52 GB at Q4_K_M and 0.45 GB at Q3_K_M. For Qwen2.5 32B that is the difference between 61.0 GB of weights and 17.2 GB.

The cache: context times architecture

Among models that accept a 32K context, the key-value cache costs from 32 KB per token (Llama 3.2 1B) to 320 KB (Qwen2.5 72B) — 1.0 GB against 10.0 GB at that length. In 6 of the 20 models the cache at the full context window is larger than the Q4_K_M weights. Llama 3.1 8B holds 4.2 GB of weights and 16.0 GB of cache at 128K.

Usable memory

Not all of a device's memory is available to a model. The GeForce RTX 4090 and the Mac (M4, 24 GB) both have 24 GB, but offer 22.1 GB and 16.1 GB. At the reference settings the card runs 16 of the 20 models and the Mac 11.

Reading a model or hardware page

Yes, tight or no. Tight means the model fits but uses more than 95% of usable memory, which in practice fails as soon as anything else needs the device.
The longest context that fits is solved from what is left after the weights and a 0.8 GB runtime allowance, capped at the model's own maximum.
Mixture-of-experts models are sized by every parameter they hold: Qwen3 235B-A22B needs memory for 235.1B parameters, although about 22.2B work on each token.
2 models use sliding-window attention, which is not modelled, so their cache figures are an upper bound.

Common mistakes

Sizing a model by its parameter count alone. At long context the cache can be larger than the weights.
Counting all of a Mac's memory. The Mac (M4, 24 GB) offers 16.1 GB of its 24 GB to a model.
Sizing a mixture-of-experts model by its active parameters. Memory follows the total.
Treating "tight" as "fits". A model at the limit loads, then fails mid-conversation.

Where to go next

Open a model above, or your hardware below, for every quantisation and context length.
GPU running costs — what these graphics cards cost to run, by state.
PSU wattage calculator — the power supply a build around one of these cards needs.
Methodology — the memory formula, the bit widths and the sources.

Frequently asked questions

How do I work out how much VRAM a model needs?

Weights plus cache plus a runtime allowance. Weights are the parameter count times the bytes per weight — a 7B model at Q4_K_M is about 4 GB. The key-value cache is 2 × layers × key-value heads × head dimension × context × bytes, which grows linearly with how long a conversation gets. Then add roughly a gigabyte for the runtime itself.

Why do other calculators give much bigger numbers?

Usually because they size the cache by the query head count instead of the key-value head count. Nearly every modern model uses grouped-query attention, sharing one key-value pair across several query heads — Llama 3.2 1B shares 16-to-1 at the extreme. Using the query count overstates the cache by exactly that factor.

Which quantisation should I pick?

Q4_K_M for most people — small quality loss, and what most GGUF downloads default to. Note that K-quants are not their nominal bit width: Q4_K_M averages about 4.5 bits per weight once the block scales and higher-precision attention tensors are counted, so a file is roughly 12% larger than "4-bit" implies. The general rule is to fit the largest model you can and only then step the quantisation down.

Can a Mac run models a graphics card cannot?

Yes, and it is the main reason Apple Silicon shows up here. Unified memory is shared between CPU and GPU, so a 192 GB machine can hold models that no consumer graphics card — capped at 32 GB — can touch. Generation is slower than a discrete card of similar capacity, but a model that does not fit does not run at any speed.

Does context length really matter that much?

On long-context models it decides the answer. Qwen3 235B-A22B needs 125.4 GB at 8K, but the cache grows every token — on models with a 128K window the cache at full context can exceed the quantised weights. Plenty of setups load a model successfully and then run out of memory once the conversation gets long.

Model config.json files published on Hugging Face. GGUF K-quant effective bit widths as produced by llama.cpp. Manufacturer specifications for VRAM and unified memory.

An estimate of memory, not a benchmark. It counts model weights, the key-value cache at the context you choose, and a fixed runtime allowance. Actual usage moves with the runtime, the batch size, whether flash attention is on, and how much the operating system has already taken from a shared memory pool. Treat a result within a gigabyte of your card's capacity as 'probably not' rather than 'just fits'.

Data on this page last verified 2 September 2026. How these figures are calculated