Home › GPUs & Macs › NVIDIA A100 80GB PCIe vs NVIDIA RTX PRO 6000 Blackwell
NVIDIA A100 80GB PCIe vs NVIDIA RTX PRO 6000 Blackwell for local LLMs
Across 4 tests measured on both, the NVIDIA RTX PRO 6000 Blackwell is faster in 4 and generates a median 2.32× the tokens/s of the NVIDIA A100 80GB PCIe.
On Llama 2 7B (Q4_0, llama-bench) the NVIDIA A100 80GB PCIe generates 201 tokens/s and the NVIDIA RTX PRO 6000 Blackwell 281 tokens/s (measured).
Same model, same quant, same test
| Model | Quant / test | A100 80GB PCIe tok/s | RTX PRO 6000 Blackwell tok/s | Faster | Sources and run details |
|---|---|---|---|---|---|
| Llama 2 7B | Q4_0 llama-bench | 201 CUDA | 281 CUDA | RTX PRO 6000 Blackwell +40% | llama.cpp #15013 A100 80GB PCIe · llama.cpp #15013 RTX PRO 6000 Blackwell |
| Llama 3.1 8B Instruct | Q4_K_M LocalScore | 37.1 llamafile | 85.8 llamafile | RTX PRO 6000 Blackwell +131% | LocalScore A100 80GB PCIe · LocalScore RTX PRO 6000 Blackwell |
| Llama 3.2 1B Instruct | Q4_K_M LocalScore | 81.9 llamafile | 191 llamafile | RTX PRO 6000 Blackwell +134% | LocalScore A100 80GB PCIe · LocalScore RTX PRO 6000 Blackwell |
| Qwen2.5 14B Instruct | Q4_K_M LocalScore | 20.8 llamafile | 64.0 llamafile | RTX PRO 6000 Blackwell +208% | LocalScore A100 80GB PCIe · LocalScore RTX PRO 6000 Blackwell |
Generation speed in tokens/s, best measured backend for each chip (shown under the number). Backends can differ between the two chips; check them before drawing conclusions. Measured results only, nothing estimated.
Specs
| NVIDIA A100 80GB PCIe | NVIDIA RTX PRO 6000 Blackwell | |
|---|---|---|
| Memory | 80 GB | 96 GB |
| Memory bandwidth | 1935 GB/s | 1792 GB/s |
All results for the NVIDIA A100 80GB PCIe · All results for the NVIDIA RTX PRO 6000 Blackwell · More comparisons