Models › MiMo-V2.6-Distill-Qwen-9B

MiMo-V2.6-Distill-Qwen-9B: VRAM needs and speed by GPU

At Q4_K_M, MiMo-V2.6-Distill-Qwen-9B needs about 6.9 GB of VRAM. No measured runs yet. Speeds below are estimates. Submit a run.

8.95B parameters · 262,144-token context · GGUF published Sep 21, 2026 · GGUF files on Hugging Face

Memory needed per quantization

QuantFile sizeNeeds (estimate)Smallest card it fits
IQ2_M3.5 GB4.6 GB8 GB cards
Q2_K3.6 GB4.7 GB8 GB cards
IQ3_XXS4.1 GB5.2 GB8 GB cards
Q3_K_S4.3 GB5.3 GB8 GB cards
IQ3_XS4.3 GB5.3 GB8 GB cards
Q3_K_M4.5 GB5.5 GB8 GB cards
Q3_K_L4.7 GB5.7 GB8 GB cards
IQ3_M4.8 GB5.9 GB8 GB cards
IQ4_XS5.2 GB6.2 GB8 GB cards
Q4_05.5 GB6.5 GB8 GB cards
Q4_K_S5.5 GB6.5 GB8 GB cards
IQ4_NL5.8 GB6.8 GB8 GB cards
Q4_K_M5.8 GB6.9 GB8 GB cards
Q4_15.9 GB7.0 GB8 GB cards
Q4_K_L6.2 GB7.2 GB8 GB cards
Q5_K_S6.5 GB7.5 GB12 GB cards
Q5_K_M6.9 GB7.9 GB12 GB cards
Q6_K_S7.5 GB8.5 GB12 GB cards
Q6_K7.8 GB8.8 GB12 GB cards
Q6_K_L8.1 GB9.1 GB12 GB cards
Q8_09.5 GB10.6 GB12 GB cards
BF1617.9 GB18.9 GB24 GB cards

Expected speed on popular hardware

HardwareBest quant that fitsExpected tok/s
NVIDIA RTX 3060Q8_032.3 estimate
NVIDIA RTX 4060 TiQ4_K_M41.9 estimate
NVIDIA RTX 3090Q8_068.7 estimate
NVIDIA RTX 4090Q8_076.1 estimate
NVIDIA RTX 5090Q8_0120 estimate
AMD Radeon RX 7900 XTXQ8_076.5 estimate
Apple M4 Pro (20-core GPU) (64 GB)Q8_024.3 estimate
Apple M3 Max (40-core GPU) (64 GB)Q8_035.4 estimate
AMD Ryzen AI Max+ 395 (Radeon 8060S)Q8_021.8 estimate

Check your own card

Too big for your card?

Run MiMo-V2.6-Distill-Qwen-9B on a rented 24-80 GB GPU by the hour.

Rent a bigger GPU on RunPod

Affiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.