Models › MLExpert

MLExpert: VRAM needs and speed by GPU

At Q4_K_M, MLExpert needs about 24.0 GB of VRAM. No measured runs yet. Speeds below are estimates. Submit a run.

34.66B parameters (mixture-of-experts) · license: apache-2.0 · 262,144-token context · GGUF published Oct 18, 2026 · GGUF files on Hugging Face

Memory needed per quantization

QuantFile sizeNeeds (estimate)Smallest card it fits
IQ2_XXS10.8 GB12.5 GB16 GB cards
IQ2_XS11.3 GB12.9 GB16 GB cards
IQ2_S12.0 GB13.6 GB16 GB cards
IQ2_M13.2 GB14.9 GB16 GB cards
Q2_K13.8 GB15.5 GB24 GB cards
IQ3_XXS15.6 GB17.3 GB24 GB cards
IQ3_XS16.3 GB18.0 GB24 GB cards
Q3_K_S16.3 GB18.0 GB24 GB cards
Q3_K_M17.3 GB19.0 GB24 GB cards
Q3_K_L18.3 GB19.9 GB24 GB cards
IQ3_M19.0 GB20.6 GB24 GB cards
IQ4_XS19.9 GB21.6 GB24 GB cards
Q4_019.9 GB21.6 GB24 GB cards
Q4_K_S20.9 GB22.6 GB24 GB cards
Q4_122.0 GB23.6 GB32 GB cards
IQ4_NL22.3 GB24.0 GB32 GB cards
Q4_K_M22.3 GB24.0 GB32 GB cards
Q4_K_L24.1 GB25.8 GB32 GB cards
Q5_K_S25.1 GB26.8 GB32 GB cards
Q5_K_M27.0 GB28.7 GB32 GB cards
Q6_K_S29.4 GB31.0 GB32 GB cards
Q6_K30.7 GB32.4 GB48 GB cards
Q6_K_L32.3 GB34.0 GB48 GB cards
Q8_036.9 GB38.6 GB48 GB cards
BF1669.4 GB71.0 GB96 GB cards

Expected speed on popular hardware

HardwareBest quant that fitsExpected tok/s
NVIDIA RTX 3060Does not fit
NVIDIA RTX 4060 TiDoes not fit
NVIDIA RTX 3090Q4_K_S31.3 estimate
NVIDIA RTX 4090Q4_K_S34.7 estimate
NVIDIA RTX 5090Q5_K_M42.6 estimate
AMD Radeon RX 7900 XTXQ4_K_S34.8 estimate
Apple M4 Pro (20-core GPU) (64 GB)Q8_06.3 estimate
Apple M3 Max (40-core GPU) (64 GB)Q8_09.2 estimate
AMD Ryzen AI Max+ 395 (Radeon 8060S)Q8_05.6 estimate

Check your own card

Too big for your card?

Run MLExpert on a rented 24-80 GB GPU by the hour.

Rent a bigger GPU on RunPod

Affiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.