GPUs & Macs › Apple M3 Ultra (80-core GPU)

Apple M3 Ultra (80-core GPU): local LLM speed

Llama 2 7B (Q4_0) generates 92.1 tokens/s on the Apple M3 Ultra (80-core GPU) (Metal), and processes prompts at 1,471 tokens/s.

Measured results

Model	Generate tok/s	Prompt tok/s	Quant	Backend	Test	Source
Llama 2 7B	92.1	1,471	Q4_0	Metal FA off	tg128	llama.cpp #4167 details
Llama 2 7B	63.9	1,488	Q8_0	Metal FA off	tg128	llama.cpp #4167 details
Llama 2 7B	39.8	1,538	F16	Metal FA off	tg128	llama.cpp #4167 details
Llama 3.2 1B Instruct	178	5,300	Q4_K_M	llamafile	LocalScore avg · n=2	LocalScore LocalScore details
gpt-oss 20B	116	2,816	MXFP4	Metal FA on	tg128	llama.cpp #15396 details
Llama 3.1 8B Instruct	63.0	1,086	Q4_K_M	llamafile	LocalScore avg · n=2	LocalScore LocalScore details
Qwen2.5 14B Instruct	36.3	574	Q4_K_M	llamafile	LocalScore avg · n=2	LocalScore LocalScore details

Vendor	Apple
Memory	Unified (depends on configuration)
Memory bandwidth	819 GB/s
GPU cores	80
Spec source	Wikipedia (Apple M3)
Spec note	Wikipedia 819.3; corrected from 800 (llama.cpp #4167 table)

Check prices

Amazon

As an Amazon Associate I earn from qualifying purchases.

Rent one by the hour

Rent a GPU on RunPod

Affiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.

What fits on the Apple M3 Ultra (80-core GPU)

Estimates for this chip are scaled from a measured run: Llama 2 7B F16 at 39.8 tok/s (Metal, source). Fit assumes the whole model in GPU memory with ~8K context.

With 16 GB unified memory

Model	Best quant that fits	File	Expected tok/s
Qwen3.8-27B	UD-IQ2_XXS	7.3 GB	73.8 estimate
gemma-4-12B-it	Q6_K	9.8 GB	54.8 estimate
Qwen3.5-9B	Q8_0	9.5 GB	56.3 estimate
Qwen3.5-4B	Q8_0	4.5 GB	120 estimate
Qwen3.6-35B-A3B	UD-IQ1_M	10.0 GB	311 estimate
gemma-4-26B-A4B-it	UD-IQ2_M	10.0 GB	174 estimate
MiMo-V2.6-Distill-Qwen-9B	Q8_0	9.5 GB	56.2 estimate
gemma-4-E4B-it	Q8_0	8.2 GB	65.4 estimate

With 36 GB unified memory

Model	Best quant that fits	File	Expected tok/s
Qwen3.8-27B	UD-Q6_K	22.0 GB	24.4 estimate
gemma-4-12B-it	Q8_0	12.7 GB	42.3 estimate
Qwen3.5-9B	Q8_0	9.5 GB	56.3 estimate
Qwen3.5-4B	Q8_0	4.5 GB	120 estimate
Qwen3.6-35B-A3B	UD-Q5_K_S	24.9 GB	125 estimate
gemma-4-26B-A4B-it	UD-Q6_K	23.2 GB	75.2 estimate
MiMo-V2.6-Distill-Qwen-9B	Q8_0	9.5 GB	56.2 estimate
gemma-4-E4B-it	Q8_0	8.2 GB	65.4 estimate

With 64 GB unified memory

Model	Best quant that fits	File	Expected tok/s
Qwen3.8-27B	Q8_0	29.0 GB	18.5 estimate
gemma-4-12B-it	Q8_0	12.7 GB	42.3 estimate
Qwen3.5-9B	Q8_0	9.5 GB	56.3 estimate
Qwen3.5-4B	Q8_0	4.5 GB	120 estimate
Qwen3.6-35B-A3B	Q8_0	36.9 GB	84.8 estimate
gemma-4-26B-A4B-it	Q8_0	26.9 GB	64.9 estimate
MiMo-V2.6-Distill-Qwen-9B	Q8_0	9.5 GB	56.2 estimate
gemma-4-E4B-it	Q8_0	8.2 GB	65.4 estimate

With 128 GB unified memory

Model	Best quant that fits	File	Expected tok/s
Qwen3.8-27B	Q8_0	29.0 GB	18.5 estimate
gemma-4-12B-it	Q8_0	12.7 GB	42.3 estimate
Qwen3.5-9B	Q8_0	9.5 GB	56.3 estimate
Qwen3.5-4B	Q8_0	4.5 GB	120 estimate
Qwen3.6-35B-A3B	Q8_0	36.9 GB	84.8 estimate
Qwen3.8-Flash-Next	UD-IQ3_XXS	82.0 GB	6.5 estimate
gemma-4-26B-A4B-it	Q8_0	26.9 GB	64.9 estimate
MiMo-V2.6-Distill-Qwen-9B	Q8_0	9.5 GB	56.2 estimate

Open in the picker

Similar hardware

Apple M1 (7-core GPU) · Apple M1 (8-core GPU) · Apple M1 Pro (14-core GPU) · Apple M1 Pro (16-core GPU) · Apple M1 Max (24-core GPU) · Apple M1 Max (32-core GPU) · Apple M1 Ultra (48-core GPU) · Apple M1 Ultra (64-core GPU)