Qwen3 235B-A22B VRAM Requirements
235.09B parameters, 94 layers, 4 key-value heads. At Q4_K_M with a 8K context it needs 125.4 GB — 1 of 30 accelerators here can hold it.
A mixture of experts: 128 experts per layer, 8 of them running per token. All of them have to be in memory, so the 235.1B above is what you need to hold — but only 22.19B is doing work at any moment, which is what sets the speed.
Running Qwen3 235B-A22B
Will not fit on a GeForce RTX 4090
- Weights
- 123.2 GB
- KV cache
- 1.5 GB
- Headroom
- -103.3 GB
- Max context
- 0
Where the memory goes
| Component | Memory |
|---|---|
| Model weights (Q4_K_M) | 123.2 GB |
| KV cache at 8K tokens | 1.5 GB |
| Runtime allowance | 0.8 GB |
| Total | 125.4 GB |
Weights are only half the question
A 235.1B model at Q4_K_M is 123.2 GB of weights. That part is fixed — it is the file on disk, and it does not change while the model runs. What changes is the key-value cache, which grows with every token of context you give it.
Qwen3 235B-A22B spends 188.0 KB per token. At 8K that is 1.5 GB, comfortably smaller than the weights.
The reason the cache is as small as it is comes down to grouped-query attention. Qwen3 235B-A22B has 64 query heads but only 4 key-value heads, and the cache is sized by the latter. Calculators that use the query head count overstate it 16-fold, which is where the wildly pessimistic numbers people quote usually come from. Out of reach of any single accelerator here even at 3-bit, which is the answer most people are looking for. Four key-value heads across 94 layers keep the cache modest for the size, so it is the weights, not the context, that decide.
By quantisation
| Format | Bits/weight | Weights | Total at 8K |
|---|---|---|---|
| FP16 / BF16 | 16 | 437.9 GB | 440.2 GB |
| Q8_0 | 8.5 | 232.6 GB | 234.9 GB |
| Q6_K | 6.6 | 180.6 GB | 182.9 GB |
| Q5_K_M | 5.5 | 150.5 GB | 152.8 GB |
| Q4_K_M | 4.5 | 123.2 GB | 125.4 GB |
| Q3_K_M | 3.9 | 106.7 GB | 109.0 GB |
By context length
| Context | KV cache | Total |
|---|---|---|
| 2K tokens | 0.4 GB | 124.3 GB |
| 4K tokens | 0.7 GB | 124.7 GB |
| 8K tokens | 1.5 GB | 125.4 GB |
| 16K tokens | 2.9 GB | 126.9 GB |
| 32K tokens | 5.9 GB | 129.8 GB |
Will it run on your hardware?
| Accelerator | Memory | Runs it | Headroom | Max context |
|---|---|---|---|---|
| GeForce RTX 5090 | 32 GB | No | — | — |
| GeForce RTX 4090 | 24 GB | No | — | — |
| GeForce RTX 3090 | 24 GB | No | — | — |
| GeForce RTX 5080 | 16 GB | No | — | — |
| GeForce RTX 4080 SUPER | 16 GB | No | — | — |
| GeForce RTX 5070 Ti | 16 GB | No | — | — |
| GeForce RTX 4070 Ti SUPER | 16 GB | No | — | — |
| GeForce RTX 5070 | 12 GB | No | — | — |
| GeForce RTX 4070 | 12 GB | No | — | — |
| GeForce RTX 3060 12GB | 12 GB | No | — | — |
| GeForce RTX 4060 Ti 16GB | 16 GB | No | — | — |
| GeForce RTX 3080 | 10 GB | No | — | — |
| Radeon RX 7900 XTX | 24 GB | No | — | — |
| Radeon RX 7900 XT | 20 GB | No | — | — |
| Radeon RX 9070 XT | 16 GB | No | — | — |
| 2× RTX 3090 (48 GB) | 48 GB | No | — | — |
| 2× RTX 4090 (48 GB) | 48 GB | No | — | — |
| 4× RTX 3090 (96 GB) | 96 GB | No | — | — |
| Mac (M4 Max, 64 GB) | 64 GB | No | — | — |
| Mac (M4 Max, 128 GB) | 128 GB | No | — | — |
| Mac (M4 Pro, 48 GB) | 48 GB | No | — | — |
| Mac (M4, 24 GB) | 24 GB | No | — | — |
| Mac (M3 Max, 128 GB) | 128 GB | No | — | — |
| Mac Studio (M2 Ultra, 192 GB) | 192 GB | Yes | 18.6 GB | 40K |
| Mac (M1 Max, 32 GB) | 32 GB | No | — | — |
| Mac (M2, 16 GB) | 16 GB | No | — | — |
| NVIDIA A100 40GB | 40 GB | No | — | — |
| NVIDIA A100 80GB | 80 GB | No | — | — |
| NVIDIA H100 80GB | 80 GB | No | — | — |
| NVIDIA L40S 48GB | 48 GB | No | — | — |
At Q4_K_M with a 8K context. "Only just" means the model uses more than 95% of usable memory — it will load and then fall over as soon as anything else touches the device.
Frequently asked questions
How much VRAM does Qwen3 235B-A22B need?
What GPU do I need to run Qwen3 235B-A22B?
Does context length change how much memory Qwen3 235B-A22B needs?
Why is the cache smaller than other calculators say?
Which quantisation should I use?
Architecture: Hugging Face config.json. GGUF K-quant effective bit widths as produced by llama.cpp.
An estimate of memory, not a benchmark. It counts model weights, the key-value cache at the context you choose, and a fixed runtime allowance. Actual usage moves with the runtime, the batch size, whether flash attention is on, and how much the operating system has already taken from a shared memory pool. Treat a result within a gigabyte of your card's capacity as 'probably not' rather than 'just fits'.
Data on this page last verified .