Can I Run This LLM? GPU Compatibility Checker

Can I run it?

your card, any open model · fits or does not, and what to do when it does not

Your GPU

Model

Quantisation

KV cache

Context length 8,192 tokens

– –

needs–

card has–

spare–

How the answer is worked out

Three things have to share your card's memory, and only one of them is the model:

The weights — parameters multiplied by the bytes each one takes at your chosen quantisation. This is the big number and the one people already know.
The KV cache — the conversation so far, held in memory so the model does not re-read it every token. It grows linearly with context, and on long documents it can rival the weights.
Runtime overhead — roughly half a gigabyte of context and buffers, plus about 4% of the weights for activations.

The total is compared against 95% of what your card can address, leaving the rest for your desktop and for memory that fragments rather than packs. Mixture-of-experts models load every expert, so their full parameter count counts against memory even though only the active experts are read per token — which is why they are large and fast at the same time.

Speed is a separate question with a separate formula: generating a token means reading every active weight out of memory, so tokens per second is bandwidth divided by active weight size, at 70% efficiency. The speed estimator does that one on its own, and the VRAM calculator breaks the memory into its three parts.

When the answer is no

Most tools stop at the word "no". This one lists what would change it, with the number attached to each option, because the fixes are not equally good:

A smaller quantisation is the first lever and the cheapest. Q8 is near-lossless, Q5 and Q4 cost a little, Q3 costs enough that a smaller model at Q4 is usually the better trade.
A shorter context is free if you were not using it. If you were, it is the most painful cut on the list, because it changes what the model can do rather than how well it does it.
Quantising the KV cache to Q8 roughly halves the cache and is the option people forget. It is often the difference between a tight fit and a comfortable one.
A larger card is the honest answer when the gap is wide. The checker names the smallest one that works rather than pointing at the top of the range.
A different model is often the right call. A well-chosen smaller model at good precision beats a larger one crushed into three bits.

If none of those appeal, the fifth option is not to own the card at all. Renting the memory for the hours you actually use it is cheaper below a certain number of hours per month, and the build vs rent calculator will tell you where that line falls for your own numbers.

Straight to your card

Every card in the checker has a page of its own: the full model list with fits and speeds, how much context is left once the weights load, where the card stops, and an honest verdict on whether it is worth owning.

Or approach it from the other side: all 13 cards compared, and which models fit on 8 to 96 GB.

Questions

How do I know if a model will fit on my GPU?

A model fits when its weights, its KV cache and the runtime's own overhead together stay inside your card's memory. The checker adds those three up for the exact model, quantisation and context length you choose, and compares the total against 95% of your card's memory — the remainder covers the desktop and fragmentation.

What happens if a model does not fit in VRAM?

It still runs, and it becomes unusable. The runtime keeps the overflow in system RAM and moves it across the PCIe bus for every token, which is ten to fifty times slower than reading from the card. A model that generates 40 tokens per second when it fits can drop to one or two when it does not.

Can an RTX 4090 run Llama 3.3 70B?

Not at Q4_K_M with 8k of context: it needs about 45.8 GB and a 4090 has 24 GB, so it is roughly 23 GB short. The checker lists what would work — a smaller quantisation, a shorter context, a larger card, or the biggest model a 4090 does run.

Does the tool account for Apple unified memory?

Yes. macOS makes about 75% of system RAM addressable by the GPU by default, so a 128 GB machine is treated as roughly 96 GB. Note that these figures describe generation speed; prompt processing on Apple silicon is markedly slower than on an equivalent Nvidia card.

Are these numbers measured?

No, and no page on this site pretends otherwise. Memory is a stated formula over published architecture values, and speed is memory bandwidth divided by the weights read per token at 70% efficiency. Real runtimes land within roughly 20% either way, and flash attention or paged KV usually come in under the estimate.

Memory: weights + KV cache + runtime overhead, the formula set out in the VRAM calculator. Architecture values come from each model's published config.
Speed: 70% × memory bandwidth ÷ active weight bytes, single stream, prompt processing and batching excluded.
Card memory and bandwidth are manufacturer specifications. Apple machines are counted at 75% of unified memory, the macOS default for GPU-addressable RAM.
Not modelled: flash attention and paged KV (which reduce real use), batch sizes above one (which multiply the cache), vision and audio encoders, and sliding-window attention.