Models › NVIDIA-Nemotron-3.5-Bolt-74B-A9B

NVIDIA-Nemotron-3.5-Bolt-74B-A9B: VRAM needs and speed by GPU

At UD-Q4_K_XL, NVIDIA-Nemotron-3.5-Bolt-74B-A9B needs about 52.9 GB of VRAM. No measured runs yet. Speeds below are estimates. Submit a run.

74.04B parameters (mixture-of-experts, ~9.0B active) · license: other · 262,144-token context · GGUF published Oct 14, 2026 · GGUF files on Hugging Face

Memory needed per quantization

QuantFile sizeNeeds (estimate)Smallest card it fits
UD-IQ1_S18.5 GB21.2 GB24 GB cards
UD-IQ1_M20.3 GB23.0 GB24 GB cards
UD-IQ2_XXS22.0 GB24.7 GB32 GB cards
IQ2_XXS22.0 GB24.7 GB32 GB cards
UD-IQ2_M24.6 GB27.3 GB32 GB cards
UD-Q2_K_XL28.8 GB31.4 GB32 GB cards
UD-IQ3_XXS31.6 GB34.2 GB48 GB cards
UD-Q3_K_XL39.0 GB41.7 GB48 GB cards
UD-Q4_K_XL50.2 GB52.9 GB96 GB cards
UD-Q5_K_XL58.9 GB61.6 GB96 GB cards
UD-Q6_K_XL72.4 GB75.0 GB96 GB cards
Q8_078.9 GB81.5 GB96 GB cards
BF16148.2 GB150.9 GBmulti-GPU / large unified memory

Expected speed on popular hardware

HardwareBest quant that fitsExpected tok/s
NVIDIA RTX 3060Does not fit
NVIDIA RTX 4060 TiDoes not fit
NVIDIA RTX 3090UD-IQ1_M133 estimate
NVIDIA RTX 4090UD-IQ1_M147 estimate
NVIDIA RTX 5090UD-IQ2_M192 estimate
AMD Radeon RX 7900 XTXUD-IQ1_M148 estimate
Apple M4 Pro (20-core GPU) (64 GB)UD-IQ3_XXS30.2 estimate
Apple M3 Max (40-core GPU) (64 GB)UD-IQ3_XXS44.0 estimate
AMD Ryzen AI Max+ 395 (Radeon 8060S)Q8_010.8 estimate

Check your own card

Too big for your card?

Run NVIDIA-Nemotron-3.5-Bolt-74B-A9B on a rented 24-80 GB GPU by the hour.

Rent a bigger GPU on RunPod

Affiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.