Models › Qwen3-0.6B
Qwen3-0.6B: VRAM needs and speed by GPU
At Q4_0, Qwen3-0.6B needs about 1.2 GB of VRAM. No measured runs yet. Speeds below are estimates. Submit a run.
0.75B parameters · license: apache-2.0 · 40,960-token context · GGUF published Apr 28, 2025 · GGUF files on Hugging Face
Memory needed per quantization
| Quant | File size | Needs (estimate) | Smallest card it fits |
|---|---|---|---|
| Q4_0 | 0.4 GB | 1.2 GB | 8 GB cards |
| Q8_0 | 0.8 GB | 1.6 GB | 8 GB cards |
| F16 | 1.5 GB | 2.3 GB | 8 GB cards |
| BF16 | 1.5 GB | 2.3 GB | 8 GB cards |
Expected speed on popular hardware
| Hardware | Best quant that fits | Expected tok/s |
|---|---|---|
| NVIDIA RTX 3060 | Q8_0 | 257 estimate |
| NVIDIA RTX 4060 Ti | Q8_0 | 204 estimate |
| NVIDIA RTX 3090 | Q8_0 | 547 estimate |
| NVIDIA RTX 4090 | Q8_0 | 606 estimate |
| NVIDIA RTX 5090 | Q8_0 | 957 estimate |
| AMD Radeon RX 7900 XTX | Q8_0 | 608 estimate |
| Apple M4 Pro (20-core GPU) (64 GB) | Q8_0 | 193 estimate |
| Apple M3 Max (40-core GPU) (64 GB) | Q8_0 | 282 estimate |
| AMD Ryzen AI Max+ 395 (Radeon 8060S) | Q8_0 | 173 estimate |
Too big for your card?
Run Qwen3-0.6B on a rented 24-80 GB GPU by the hour.
Rent a bigger GPU on RunPodAffiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.