GPUs & Macs › NVIDIA DGX Spark (GB10)

NVIDIA DGX Spark (GB10): local LLM speed

Llama 2 7B (Q4_0) generates 60.4 tokens/s on the NVIDIA DGX Spark (GB10) (Vulkan), and processes prompts at 3,322 tokens/s.

Measured results

Model	Generate tok/s	Prompt tok/s	Quant	Backend	Test	Source
Llama 2 7B	60.4	3,322	Q4_0	Vulkan FA on	tg128	llama.cpp #10879 details
Llama 2 7B	58.5	2,790	Q4_0	Vulkan FA off	tg128	llama.cpp #10879 details
Llama 2 7B	57.2	3,062	Q4_0	CUDA FA off	tg128	llama.cpp #15013 details
Llama 2 7B	56.7	3,661	Q4_0	CUDA FA on	tg128	llama.cpp #15013 details
Llama 3.2 1B Instruct	132	7,127	Q4_K_M	llamafile	LocalScore avg · n=2	LocalScore LocalScore details
gpt-oss 20B	86.0	4,453	Q4_K_M	CUDA FA on	tg32	llama.cpp #16578 details
gpt-oss 20B	84.2	4,118	MXFP4	CUDA FA on	tg32 · n=3	llama.cpp benches llama.cpp #15396 llama.cpp #16578 details
Gemma 3 4B IT (QAT)	81.0	5,949	Q4_0	CUDA FA on	tg32	llama.cpp benches details
gpt-oss 120B	62.4	2,444	Q4_K_M	CUDA FA on	tg32	llama.cpp #16578 details
Qwen3 Coder 30B-A3B Instruct	61.1	2,987	Q8_0	CUDA FA on	tg32	llama.cpp benches details
gpt-oss 120B	58.0	2,130	MXFP4	CUDA FA on	tg32 · n=2	llama.cpp benches llama.cpp #15396 details
Nemotron 3 Nano 4B	52.9	2,762	Q8_0	CUDA FA on	tg32	llama.cpp benches details
GLM-4.7-Flash	48.7	2,364	Q8_0	CUDA FA on	tg32	llama.cpp benches details
Llama 3.1 8B Instruct	34.2	1,998	Q4_K_M	llamafile	LocalScore avg · n=2	LocalScore LocalScore details
Qwen2.5 Coder 7B	29.4	2,250	Q8_0	CUDA FA on	tg32	llama.cpp benches details
Nemotron 3 Super 120B-A12B	19.9	769	Q4_K	CUDA FA on	tg32	llama.cpp benches details
Qwen2.5 14B Instruct	18.3	1,094	Q4_K_M	llamafile	LocalScore avg · n=2	LocalScore LocalScore details

Vendor	NVIDIA
Memory	128 GB
Memory bandwidth	273 GB/s
Spec source	nvidia.com
Spec note	NVIDIA spec table

Check prices

Amazon Newegg

As an Amazon Associate I earn from qualifying purchases.

Rent one by the hour

Rent a GPU on RunPod

Affiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.

What fits on the NVIDIA DGX Spark (GB10)

Estimates for this chip are scaled from a measured run: Qwen2.5 Coder 7B Q8_0 at 29.4 tok/s (CUDA, source). Fit assumes the whole model in GPU memory with ~8K context.

Model	Best quant that fits	File	Expected tok/s
Qwen3.8-27B	Q8_0	29.0 GB	8.2 estimate
gemma-4-12B-it	Q8_0	12.7 GB	18.8 estimate
Qwen3.5-9B	Q8_0	9.5 GB	25.0 estimate
Qwen3.5-4B	Q8_0	4.5 GB	53.2 estimate
Qwen3.6-35B-A3B	Q8_0	36.9 GB	37.7 estimate
Qwen3.8-Flash-Next	UD-IQ3_XXS	82.0 GB	2.9 estimate
gemma-4-26B-A4B-it	Q8_0	26.9 GB	28.8 estimate
MiMo-V2.6-Distill-Qwen-9B	Q8_0	9.5 GB	25.0 estimate
gemma-4-E4B-it	Q8_0	8.2 GB	29.1 estimate
Muse-Glimmer-30B	Q8_0	29.6 GB	8.0 estimate
Qwen3-4B	Q8_0	4.3 GB	55.7 estimate
gemma-4-31B-it	Q8_0	32.6 GB	7.3 estimate
gemma-4-26B-A4B-it-qat-q4_0-unquantized	UD-Q4_K_XL	14.2 GB	54.4 estimate
gemma-4-12B-it-qat-q4_0-unquantized	UD-Q4_K_XL	6.7 GB	35.5 estimate

Open in the picker

Similar hardware

NVIDIA Jetson AGX Orin 64GB