Models › gpt-oss 20B

gpt-oss 20B: VRAM needs and speed by GPU

At Q4_K_M, gpt-oss 20B needs about 13.2 GB of VRAM. We have 30 measured speed results for it.

20.91B parameters (mixture-of-experts, ~3.6B active) · license: apache-2.0 · 131,072-token context · GGUF published Aug 5, 2025 · GGUF files on Hugging Face

Measured speeds

Hardware	Generate tok/s	Prompt tok/s	Quant	Backend	Test	Source
NVIDIA RTX PRO 6000 Blackwell	287	11,522	MXFP4	CUDA FA on	tg128	llama.cpp #15396 details
NVIDIA RTX 5090	283	9,848	MXFP4	CUDA FA on	tg128	llama.cpp #15396 details
NVIDIA RTX 4090	255	9,194	MXFP4	CUDA FA on	tg128 · n=3	llama.cpp #15396 llama.cpp #15396 details
NVIDIA RTX PRO 6000 Blackwell Max-Q	250	9,481	MXFP4	CUDA FA on	tg128	llama.cpp #15396 details
NVIDIA RTX 5080	229	10,404	MXFP4	CUDA FA on	tg128 · n=4	llama.cpp #15396 llama.cpp #15396 details
NVIDIA RTX 5070 Ti	189	6,340	MXFP4	CUDA FA on	tg128	llama.cpp #15396 details
NVIDIA RTX 4080 SUPER	187	8,171	MXFP4	CUDA FA on	tg128	llama.cpp #15396 details
NVIDIA RTX 5080	178	12,995	F16	CUDA FA on	tg128	llama.cpp #15396 details
NVIDIA RTX 3090	162	5,171	MXFP4	CUDA FA on	tg128	llama.cpp #15396 details
Apple M2 Ultra	130	2,713	MXFP4	Metal FA on	tg32	llama.cpp benches details
NVIDIA Tesla V100 16GB	118	3,527	MXFP4	CUDA FA on	tg128	llama.cpp #15396 details
Apple M2 Ultra (76-core GPU)	116	2,191	MXFP4	Metal FA on	tg128	llama.cpp #15396 details
Apple M3 Ultra (80-core GPU)	116	2,816	MXFP4	Metal FA on	tg128	llama.cpp #15396 details
NVIDIA RTX 5060 Ti	112	3,839	MXFP4	CUDA FA on	tg128	llama.cpp #15396 details
AMD Radeon RX 7900 XT	102	4,252	BF16	ROCm,RPC FA on	tg128	llama.cpp #15396 details
Apple M4 Max	92.4	1,277	MXFP4	Metal FA on	tg128	llama.cpp #15396 details
NVIDIA DGX Spark (GB10)	86.0	4,453	Q4_K_M	CUDA FA on	tg32	llama.cpp #16578 details
NVIDIA DGX Spark (GB10)	84.2	4,118	MXFP4	CUDA FA on	tg32 · n=3	llama.cpp benches llama.cpp #15396 llama.cpp #16578 details
Apple M1 Max	75.2	995	MXFP4	Metal FA on	tg128	llama.cpp #15396 details
Apple M1 Max	75.2	995	MXFP4	Metal,BLAS FA on	tg128	llama.cpp #15396 details
Apple M1 Pro	45.7	516	MXFP4	Metal FA on	tg128	llama.cpp #15396 details
NVIDIA RTX 4060	42.3	1,642	MXFP4	CUDA FA on	tg128 · part CPU	llama.cpp #15396 details
NVIDIA RTX 4060	32.7	1,430	F16	CUDA FA on	tg128 · part CPU	llama.cpp #15396 details

Memory needed per quantization

Quant	File size	Needs (estimate)	Smallest card it fits
Q4_K_M	11.9 GB	13.2 GB	16 GB cards
MXFP4	12.1 GB	13.4 GB	16 GB cards
BF16	13.8 GB	15.1 GB	16 GB cards
F16	13.8 GB	15.1 GB	16 GB cards

Expected speed on popular hardware

Hardware	Best quant that fits	Expected tok/s
NVIDIA RTX 3060	Does not fit
NVIDIA RTX 4060 Ti	Does not fit
NVIDIA RTX 3090	Q4_K_M	161 estimate
NVIDIA RTX 4090	Q4_K_M	179 estimate
NVIDIA RTX 5090	Q4_K_M	283 estimate
AMD Radeon RX 7900 XTX	Q4_K_M	180 estimate
Apple M4 Pro (20-core GPU) (64 GB)	Q4_K_M	57.0 estimate
Apple M3 Max (40-core GPU) (64 GB)	Q4_K_M	83.2 estimate
AMD Ryzen AI Max+ 395 (Radeon 8060S)	Q4_K_M	51.2 estimate

Check your own card

Too big for your card?

Run gpt-oss 20B on a rented 24-80 GB GPU by the hour.

Rent a bigger GPU on RunPod

Affiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.