Models › gemma-3-12b-it

gemma-3-12b-it: VRAM needs and speed by GPU

At Q4_K_M, gemma-3-12b-it needs about 8.4 GB of VRAM. No measured runs yet. Speeds below are estimates. Submit a run.

11.77B parameters · license: gemma · 131,072-token context · GGUF published Mar 12, 2025 · GGUF files on Hugging Face

Memory needed per quantization

QuantFile sizeNeeds (estimate)Smallest card it fits
UD-IQ1_S3.1 GB4.1 GB8 GB cards
UD-IQ1_M3.2 GB4.3 GB8 GB cards
UD-IQ2_XXS3.6 GB4.7 GB8 GB cards
UD-IQ2_M4.4 GB5.5 GB8 GB cards
Q2_K4.8 GB5.9 GB8 GB cards
Q2_K_L4.8 GB5.9 GB8 GB cards
UD-IQ3_XXS4.8 GB5.9 GB8 GB cards
UD-Q2_K_XL4.9 GB6.0 GB8 GB cards
Q3_K_S5.5 GB6.5 GB8 GB cards
Q3_K_M6.0 GB7.1 GB8 GB cards
UD-Q3_K_XL6.1 GB7.2 GB8 GB cards
IQ4_XS6.6 GB7.6 GB12 GB cards
IQ4_NL6.9 GB8.0 GB12 GB cards
Q4_06.9 GB8.0 GB12 GB cards
Q4_K_S6.9 GB8.0 GB12 GB cards
Q4_K_M7.3 GB8.4 GB12 GB cards
UD-Q4_K_XL7.4 GB8.5 GB12 GB cards
Q4_17.6 GB8.6 GB12 GB cards
Q5_K_S8.2 GB9.3 GB12 GB cards
Q5_K_M8.4 GB9.5 GB12 GB cards
UD-Q5_K_XL8.5 GB9.6 GB12 GB cards
Q6_K9.7 GB10.8 GB12 GB cards
UD-Q6_K_XL10.6 GB11.6 GB16 GB cards
Q8_012.5 GB13.6 GB16 GB cards
UD-Q8_K_XL14.4 GB15.5 GB16 GB cards
BF1623.5 GB24.6 GB32 GB cards

Expected speed on popular hardware

HardwareBest quant that fitsExpected tok/s
NVIDIA RTX 3060Q6_K31.9 estimate
NVIDIA RTX 4060 TiQ3_K_M40.7 estimate
NVIDIA RTX 3090Q8_052.4 estimate
NVIDIA RTX 4090Q8_058.1 estimate
NVIDIA RTX 5090Q8_091.8 estimate
AMD Radeon RX 7900 XTXQ8_058.3 estimate
Apple M4 Pro (20-core GPU) (64 GB)Q8_018.5 estimate
Apple M3 Max (40-core GPU) (64 GB)Q8_027.0 estimate
AMD Ryzen AI Max+ 395 (Radeon 8060S)Q8_016.6 estimate

Check your own card

Too big for your card?

Run gemma-3-12b-it on a rented 24-80 GB GPU by the hour.

Rent a bigger GPU on RunPod

Affiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.