How the estimate works
VRAM needed = model weights + KV cache + framework overhead. Weights depend on parameter count and quantisation — our bytes-per-parameter figures are measured from real GGUF files on Hugging Face, so Q4_K_M correctly shows ~5% larger than naive 4.5-bit maths. The KV cache depends on the model's attention architecture and your context length: dense GQA models grow linearly, while hybrid-attention models (Gemma 4, Qwen 3.5/3.6, GPT-OSS) and latent-compression models (DeepSeek V4, Mistral Large 3, GLM-5.2) cache dramatically less — this calculator uses each model's real architecture from its config.json. We then add ~10% plus 1GB for the inference framework. Full formulas on the methodology page.
Deliberately simple: one user, one machine, one clear answer. No batch sizes, no multi-GPU serving clusters — if you're planning production serving infrastructure you need a different tool.
Frequently asked questions
Can I run a 70B model on 24GB of VRAM?
Not fully on the GPU: Llama 3.3 70B needs about 48GB at Q4_K_M once you add the KV cache and overhead. On a 24GB card like an RTX 4090 or 3090 you can partially offload layers to system RAM at reduced speed, or run a smaller model — 27B–35B class models fit comfortably at Q4–Q5 on 24GB.
What is quantisation and how much quality do I lose?
Quantisation stores model weights at lower precision to shrink them. Q8 is near-lossless, Q6 and Q5_K_M lose very little, and Q4_K_M — the most popular size/quality trade-off — is still close to the original for most uses. Below Q4 degradation becomes noticeable. Our size figures use the actual measured file sizes of each quantisation format, not theoretical bit-widths.
Why does context length change how much VRAM I need?
The model keeps a KV cache — attention keys and values for every token in the conversation — in VRAM alongside the weights. For classic dense models it grows linearly with context, which can add many gigabytes at long contexts. Many 2026 models (Qwen 3.5/3.6, Gemma 4, GPT-OSS, DeepSeek V4) use hybrid or compressed attention that keeps the cache far smaller — this calculator models each architecture correctly rather than assuming the worst.
How accurate is this calculator?
Weight sizes are tuned against real GGUF files published on Hugging Face and land within about 1% of the actual downloads. KV-cache maths follows each model's real attention architecture from its config.json, and the total includes a typical inference-framework overhead of about 10% plus 1GB. Treat the result as a realistic planning figure: actual usage varies slightly by runtime and settings.
How much of my Mac's unified memory can the GPU use?
By default macOS lets Metal use roughly 75% of unified memory for the GPU — about 96GB on a 128GB machine. That is the figure this calculator uses for Apple Silicon. Advanced users can raise the limit with the iogpu.wired_limit_mb sysctl, but you still need to leave several GB for macOS itself.
Do MoE models like GPT-OSS or Qwen A3B need less VRAM?
Not for the weights: a mixture-of-experts model must hold all experts in memory, so its total parameter count decides VRAM, not the smaller "active" count. The active count is what makes them fast. GPT-OSS 120B still needs a ~80GB-class device even though only 5.1B parameters are active per token.
Per-model VRAM guides
DeepSeek: DeepSeek V4 Flash · DeepSeek R1 Distill Qwen3 8B
Mistral: Mistral Large 3 · Mistral Small 4
Meta: Llama 3.1 8B · Llama 3.3 70B · Llama 4 Scout · Llama 4 Maverick
Alibaba: Qwen3.5 4B · Qwen3.6 27B · Qwen3.6 35B-A3B · Qwen3.5 397B-A17B · Qwen3 Coder 30B-A3B
Google: Gemma 4 E4B · Gemma 4 12B · Gemma 4 26B-A4B · Gemma 4 31B
OpenAI: GPT-OSS 20B · GPT-OSS 120B
Z.ai: GLM-5.2
Microsoft: Phi-4
More tools
Token counter: count tokens and input costs for any text. API pricing calculator: what the hosted alternatives cost per request and per month.
Last updated 2026-08-03. Model architecture verified against each model's published config.json; quantisation sizes verified against actual GGUF files; see themethodology.