GPUs & Macs › NVIDIA Quadro P620
NVIDIA Quadro P620: local LLM speed
2 measured results for the NVIDIA Quadro P620.
Measured results
| Model | Generate tok/s | Prompt tok/s | Quant | Backend | Test | Source |
|---|---|---|---|---|---|---|
| Llama 3.2 1B Instruct | 24.4 | 539 | Q4_K_M | llamafile | LocalScore avg | LocalScore details |
| Llama 3.1 8B Instruct | 1.3 | 45 | Q4_K_M | llamafile | LocalScore avg · part CPU | LocalScore details |
What fits on the NVIDIA Quadro P620
Fit assumes the whole model in GPU memory with ~8K context.
| Model | Best quant that fits | File | Expected tok/s |
|---|---|---|---|
| Qwen3.5-0.8B | Q4_0 | 0.6 GB | – estimate |
| Qwen3-0.6B | Q4_0 | 0.4 GB | – estimate |
| granite-docling-258M | Q8_0 | 0.2 GB | – estimate |
| Qwen3-0.6B-Base | Q8_0 | 0.6 GB | – estimate |
Similar hardware
NVIDIA GT 1030 · NVIDIA MX150 · Nvidia A100 · Nvidia Tesla T10 · NVIDIA GTX 980 · NVIDIA GTX 1050 Ti · Nvidia Tesla K40 · NVIDIA GTX 1650 Ti