Models › gpt-oss 120B

gpt-oss 120B: VRAM needs and speed by GPU

At Q4_K_M, gpt-oss 120B needs about 66.0 GB of VRAM. We have 11 measured speed results for it.

116.83B parameters (mixture-of-experts, ~5.1B active) · license: apache-2.0 · 131,072-token context · GGUF published Aug 5, 2025 · GGUF files on Hugging Face

Measured speeds

HardwareGenerate
tok/s		Prompt
tok/s		QuantBackendTestSource
NVIDIA RTX PRO 6000 Blackwell1965,518MXFP4CUDA FA ontg128llama.cpp #15396 details
NVIDIA RTX PRO 6000 Blackwell Max-Q1845,449MXFP4CUDA FA ontg128 · n=2llama.cpp #15396 llama.cpp #15396 details
Apple M2 Ultra85.61,649MXFP4Metal FA ontg32llama.cpp benches details
Apple M2 Ultra (76-core GPU)79.71,245MXFP4Metal FA ontg128llama.cpp #15396 details
NVIDIA DGX Spark (GB10)62.42,444Q4_K_MCUDA FA ontg32llama.cpp #16578 details
NVIDIA DGX Spark (GB10)58.02,130MXFP4CUDA FA ontg32 · n=2llama.cpp benches llama.cpp #15396 details
AMD Radeon 780M19.4219MXFP4ROCm FA ontg16llama.cpp #15396 details
NVIDIA RTX 406018.9333MXFP4CUDA FA ontg128 · part CPUllama.cpp #15396 details
NVIDIA RTX 406017.4201F16CUDA FA ontg128 · part CPUllama.cpp #15396 details

Memory needed per quantization

QuantFile sizeNeeds (estimate)Smallest card it fits
Q4_K_M63.0 GB66.0 GB96 GB cards
MXFP463.4 GB66.4 GB96 GB cards
F1665.4 GB68.4 GB96 GB cards

Expected speed on popular hardware

HardwareBest quant that fitsExpected tok/s
NVIDIA RTX 3060Does not fit
NVIDIA RTX 4060 TiDoes not fit
NVIDIA RTX 3090Does not fit
NVIDIA RTX 4090Does not fit
NVIDIA RTX 5090Does not fit
AMD Radeon RX 7900 XTXDoes not fit
Apple M4 Pro (20-core GPU)Does not fit
Apple M3 Max (40-core GPU)Does not fit
AMD Ryzen AI Max+ 395 (Radeon 8060S)Q4_K_M37.8 estimate

Check your own card

Too big for your card?

Run gpt-oss 120B on a rented 24-80 GB GPU by the hour.

Rent a bigger GPU on RunPod

Affiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.