Home › Models › TinySwallow 1.5B Instruct
TinySwallow 1.5B Instruct: VRAM needs and speed by GPU
We have 1 measured speed results for it.
1.54B parameters · license: apache-2.0 · GGUF files on Hugging Face
Measured speeds
| Chip | Generation tok/s | Prompt tok/s | Quant | Backend | Test | Source |
|---|---|---|---|---|---|---|
| Apple M4 Max (40-core GPU) | 328 | – | MLX-4bit | MLX | short-chat decode (warm median) | john-rocky apple-silicon-llm-bench (MIT) details |
Memory needed per quantization
| Quant | File size | Needs (estimate) | Smallest card it fits |
|---|---|---|---|
| Q5_K_M | 1.1 GB | 2.0 GB | 8 GB cards |
| Q8_0 | 1.6 GB | 2.5 GB | 8 GB cards |
Expected speed on popular hardware
| Hardware | Best quant that fits | Expected tok/s |
|---|---|---|
| NVIDIA RTX 3060 | Q8_0 | 187 estimate |
| NVIDIA RTX 4060 Ti | Q8_0 | 149 estimate |
| NVIDIA RTX 3090 | Q8_0 | 398 estimate |
| NVIDIA RTX 4090 | Q8_0 | 529 estimate |
| NVIDIA RTX 5090 | Q8_0 | 697 estimate |
| AMD Radeon RX 7900 XTX | Q8_0 | 443 estimate |
| Apple M4 Pro (20-core GPU) (64 GB) | Q8_0 | 141 estimate |
| Apple M3 Max (40-core GPU) (64 GB) | Q8_0 | 205 estimate |
| AMD Ryzen AI Max+ 395 (Radeon 8060S) | Q8_0 | 126 estimate |
Too big for your card?
Run TinySwallow 1.5B Instruct on a rented 24-80 GB GPU by the hour.
Rent a bigger GPU on RunPodAffiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.