Models › Gemma 3 4B IT (QAT)
Gemma 3 4B IT (QAT): VRAM needs and speed by GPU
At Q4_0, Gemma 3 4B IT (QAT) needs about 3.4 GB of VRAM. We have 2 measured speed results for it.
3.88B parameters · GGUF files on Hugging Face
Measured speeds
| Hardware | Generate tok/s | Prompt tok/s | Quant | Backend | Test | Source |
|---|---|---|---|---|---|---|
| Apple M2 Ultra | 134 | 2,924 | Q4_0 | Metal FA on | tg32 | llama.cpp benches details |
| NVIDIA DGX Spark (GB10) | 81.0 | 5,949 | Q4_0 | CUDA FA on | tg32 | llama.cpp benches details |
Memory needed per quantization
| Quant | File size | Needs (estimate) | Smallest card it fits |
|---|---|---|---|
| Q4_0 | 2.5 GB | 3.4 GB | 8 GB cards |
Expected speed on popular hardware
| Hardware | Best quant that fits | Expected tok/s |
|---|---|---|
| NVIDIA RTX 3060 | Q4_0 | 122 estimate |
| NVIDIA RTX 4060 Ti | Q4_0 | 96.9 estimate |
| NVIDIA RTX 3090 | Q4_0 | 260 estimate |
| NVIDIA RTX 4090 | Q4_0 | 288 estimate |
| NVIDIA RTX 5090 | Q4_0 | 455 estimate |
| AMD Radeon RX 7900 XTX | Q4_0 | 289 estimate |
| Apple M4 Pro (20-core GPU) (64 GB) | Q4_0 | 91.6 estimate |
| Apple M3 Max (40-core GPU) (64 GB) | Q4_0 | 134 estimate |
| AMD Ryzen AI Max+ 395 (Radeon 8060S) | Q4_0 | 82.3 estimate |
Too big for your card?
Run Gemma 3 4B IT (QAT) on a rented 24-80 GB GPU by the hour.
Rent a bigger GPU on RunPodAffiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.