Local LLM VRAM Calculator

20 open models against 30 GPUs, Macs and multi-card builds. Architectures come from each model's own config file, and the key-value cache is sized by key-value heads rather than query heads — which is why these figures are smaller than most calculators give.

Two numbers decide whether a model runs. The weights are fixed and easy. The key-value cache grows with every token of context and is where the surprises live — on a long-context model it can end up larger than the weights themselves, so the model loads fine and then fills the card mid-conversation.

Grouped-query attention, and why it matters

Modern models share one key-value pair across several query heads. The cache is sized by the key-value count, so a model with an 8-to-1 ratio needs an eighth of the cache a naive calculation predicts. This is the single commonest error in VRAM estimates.

Grouped-query attention ratios and per-token cache cost
Model	Query / KV heads	Cache saving	Per token
Qwen3 235B-A22B	64 / 4	16×	188.0 KB
Qwen2.5 3B	16 / 2	8×	36.0 KB
Qwen3 30B-A3B	32 / 4	8×	96.0 KB
Qwen3 32B	64 / 8	8×	256.0 KB
Llama 3.3 70B	64 / 8	8×	320.0 KB
Qwen2.5 72B	64 / 8	8×	320.0 KB
Qwen2.5 7B	28 / 4	7×	56.0 KB
Qwen2.5 Coder 7B	28 / 4	7×	56.0 KB

Every model

Memory requirements for every model at Q4_K_M
Model	Parameters	Weights Q4_K_M	Total at 8K	Runs on
Llama 3.2 1B	1.2B	0.7 GB	1.7 GB	30/30
Qwen2.5 3B	3.1B	1.6 GB	2.7 GB	30/30
Llama 3.2 3B	3.2B	1.7 GB	3.4 GB	30/30
Mistral 7B v0.3	7.3B	3.8 GB	5.6 GB	30/30
Qwen2.5 7B	7.6B	4.0 GB	5.2 GB	30/30
Qwen2.5 Coder 7B	7.6B	4.0 GB	5.2 GB	30/30
Llama 3.1 8B	8.0B	4.2 GB	6.0 GB	30/30
Gemma 2 9B	9.2B	4.8 GB	8.3 GB	30/30
Phi-4	14.7B	7.7 GB	10.0 GB	29/30
Qwen2.5 14B	14.8B	7.7 GB	10.0 GB	29/30
DeepSeek-R1 Distill Qwen 14B	14.8B	7.7 GB	10.0 GB	29/30
Gemma 2 27B	27.2B	14.3 GB	17.9 GB	18/30
Qwen3 30B-A3B	30.5B	16.0 GB	17.5 GB	18/30
Qwen2.5 32B	32.8B	17.2 GB	20.0 GB	17/30
Qwen2.5 Coder 32B	32.8B	17.2 GB	20.0 GB	17/30
Qwen3 32B	32.8B	17.2 GB	20.0 GB	17/30
Mixtral 8x7B	46.7B	24.5 GB	26.3 GB	13/30
Llama 3.3 70B	70.5B	37.0 GB	40.3 GB	10/30
Qwen2.5 72B	72.7B	38.1 GB	41.4 GB	10/30
Qwen3 235B-A22B	235.1B	123.2 GB	125.4 GB	1/30

Frequently asked questions

How do I work out how much VRAM a model needs?

Weights plus cache plus a runtime allowance. Weights are the parameter count times the bytes per weight — a 7B model at Q4_K_M is about 4 GB. The key-value cache is 2 × layers × key-value heads × head dimension × context × bytes, which grows linearly with how long a conversation gets. Then add roughly a gigabyte for the runtime itself.

Why do other calculators give much bigger numbers?

Usually because they size the cache by the query head count instead of the key-value head count. Nearly every modern model uses grouped-query attention, sharing one key-value pair across several query heads — Llama 3.2 1B shares 16-to-1 at the extreme. Using the query count overstates the cache by exactly that factor.

Which quantisation should I pick?

Q4_K_M for most people — small quality loss, and what most GGUF downloads default to. Note that K-quants are not their nominal bit width: Q4_K_M averages about 4.5 bits per weight once the block scales and higher-precision attention tensors are counted, so a file is roughly 12% larger than "4-bit" implies. The general rule is to fit the largest model you can and only then step the quantisation down.

Can a Mac run models a graphics card cannot?

Yes, and it is the main reason Apple Silicon shows up here. Unified memory is shared between CPU and GPU, so a 192 GB machine can hold models that no consumer graphics card — capped at 32 GB — can touch. Generation is slower than a discrete card of similar capacity, but a model that does not fit does not run at any speed.

Does context length really matter that much?

On long-context models it decides the answer. Qwen3 235B-A22B needs 125.4 GB at 8K, but the cache grows every token — on models with a 128K window the cache at full context can exceed the quantised weights. Plenty of setups load a model successfully and then run out of memory once the conversation gets long.

Model config.json files published on Hugging Face. GGUF K-quant effective bit widths as produced by llama.cpp. Manufacturer specifications for VRAM and unified memory.

An estimate of memory, not a benchmark. It counts model weights, the key-value cache at the context you choose, and a fixed runtime allowance. Actual usage moves with the runtime, the batch size, whether flash attention is on, and how much the operating system has already taken from a shared memory pool. Treat a result within a gigabyte of your card's capacity as 'probably not' rather than 'just fits'.

Data on this page last verified 2 September 2026.