CalcuGrid

What LLMs Can I Run on a 2× RTX 3090 (48 GB)?

48 GB of VRAM, of which about 44.2 GB is usable for a model. That runs 19 of the 20 models here , up to Qwen2.5 72B.

Largest model that fits: Qwen2.5 72B at Q4_K_M, needing 41.4 GB and leaving 2.8 GB spare. Its context can run to 17K tokens before memory runs out. Two cards, one model split across them. The classic home build for 70B, and cheaper than any single card that could hold it.

Every model, on this hardware

Which models run on a 2× RTX 3090 (48 GB)
Model Size Needs Runs Max context
Llama 3.2 1B 1.2B 1.7 GB Yes 128K
Llama 3.2 3B 3.2B 3.4 GB Yes 128K
Llama 3.1 8B 8.0B 6.0 GB Yes 128K
Llama 3.3 70B 70.5B 40.3 GB Yes 21K
Qwen2.5 3B 3.1B 2.7 GB Yes 32K
Qwen2.5 7B 7.6B 5.2 GB Yes 32K
Qwen2.5 14B 14.8B 10.0 GB Yes 32K
Qwen2.5 32B 32.8B 20.0 GB Yes 32K
Qwen2.5 72B 72.7B 41.4 GB Yes 17K
Qwen2.5 Coder 7B 7.6B 5.2 GB Yes 32K
Qwen2.5 Coder 32B 32.8B 20.0 GB Yes 32K
Qwen3 32B 32.8B 20.0 GB Yes 40K
Mistral 7B v0.3 7.3B 5.6 GB Yes 32K
Gemma 2 9B 9.2B 8.3 GB Yes 8K
Gemma 2 27B 27.2B 17.9 GB Yes 8K
Phi-4 14.7B 10.0 GB Yes 16K
Qwen3 30B-A3B 30.5B 17.5 GB Yes 40K
Qwen3 235B-A22B 235.1B 125.4 GB No —
Mixtral 8x7B 46.7B 26.3 GB Yes 32K
DeepSeek-R1 Distill Qwen 14B 14.8B 10.0 GB Yes 128K

Q4_K_M at 8K context. "Max context" is the longest context that still fits once the weights are loaded.

Memory decides this, not speed

For inference, memory capacity decides what runs and bandwidth decides how fast. A slower card with more memory beats a faster card with less, every time, because a model that does not fit does not run at any speed. That is why a 16 GB RTX 4060 Ti is a more useful local-LLM card than a faster 12 GB one.

The other half is context. Weights are a fixed cost, but the key-value cache grows with every token, and on a long conversation it can outgrow the weights. That is why the table above gives a maximum context per model rather than a yes or no — plenty of models load fine on this hardware and then run out of room mid-conversation.

Frequently asked questions

What LLMs can a 2× RTX 3090 (48 GB) run?
19 of the 20 models on this site, at Q4_K_M with a 8K context. The largest is Qwen2.5 72B at 72.7B parameters, needing 41.4 GB of the roughly 44.2 GB usable.
Why is usable memory less than the 48 GB on the box?
The framework, the driver and, on a card driving a display, the screen itself all take a share before the weights get any. This page assumes about 44.2 GB of the 48 GB is available, which is why a model that looks like it fits on paper sometimes does not.
Does a bigger model at a rougher quantisation beat a smaller one?
Usually, yes. A 32B model at Q3_K_M generally outperforms a 7B at Q8_0, even though the smaller model is barely quantised at all. The practical rule is to fit the largest model you can and then step the quantisation down only as far as you must — which is what the "best format" column above is showing.
How does splitting a model across two GPUs work?
The layers are divided between the cards and activations pass over the bus between them. Memory adds up, which is the point — two 24 GB cards hold a model no single consumer card can. Speed does not add up: the cards work in sequence rather than in parallel for a single request, so throughput is closer to one card than two.
How accurate are these figures?
They count weights, the key-value cache at the chosen context, and a fixed runtime allowance. Real usage moves with the runtime, the batch size and whether flash attention is enabled. Treat anything within a gigabyte of capacity as "probably not" rather than "just fits" — which is why the table says "only just" rather than "yes" past 95% of usable memory.

Manufacturer specifications for VRAM and unified memory. Model config.json files published on Hugging Face.

An estimate of memory, not a benchmark. It counts model weights, the key-value cache at the context you choose, and a fixed runtime allowance. Actual usage moves with the runtime, the batch size, whether flash attention is on, and how much the operating system has already taken from a shared memory pool. Treat a result within a gigabyte of your card's capacity as 'probably not' rather than 'just fits'.

Data on this page last verified .

    ↑↓ to move ↵ to open