About Mistral Small 4
Mistral Small 4 is Mistral's 119B-parameter model released in March 2026, with a 256K-token context window. It is a mixture-of-experts model: all 119B parameters must sit in memory, but only ~6.5B are active per token, which is what makes it fast for its size. It uses a classic dense-attention design whose KV cache grows linearly with context: its KV cache is about 4.8 GB at an 8K context, 77.3 GB at 128K, and 154.6 GB at the full 256K window (FP16 cache).
Mistral ships FP8-quantised weights as the primary release; community GGUF quants (Q4–Q6) are what most people run locally. Totals above include the KV cache and a realistic framework overhead, so they are what you should expect to see in practice rather than just the download size. Weight sizes are calibrated against real GGUF files — see themethodology.
Frequently asked questions
How much VRAM does Mistral Small 4 need?
At Q4_K_M with an 8K context, Mistral Small 4 needs about 86 GB (weights 73 GB + KV cache + overhead). The smallest common hardware that fits is a RTX PRO 6000 Blackwell.
Can an RTX 4090 (24GB) run Mistral Small 4?
Not fully in VRAM. Mistral Small 4 needs about 86 GB even at Q4_K_M, so a 24 GB card would have to offload layers to system RAM at a large speed penalty.
Can a Mac run Mistral Small 4?
Yes — Apple Silicon with 128 GB of unified memory or more (macOS lets the GPU use ~75% of it, ~96 GB) runs Mistral Small 4 at Q4_K_M.
Related
VRAM calculator for any model ·Token counter · Mistral Small 4 API pricing
Last updated 2026-08-03. Architecture figures from the model's published config.json; see themethodology.