RigBench data 2026-10-22

Home › Models › Hy3

Hy3: VRAM needs and speed by GPU

At Q4_K_M, Hy3 needs about 185.2 GB of VRAM. We have 2 measured speed results for it.

Ranked #23 among open-weight models by the Artificial Analysis Intelligence Index, checked 2026-10-22T16:10:10Z. All leading open models.

298.79B parameters (mixture-of-experts, ~21.0B active) · license: apache-2.0 · 262,144-token context · GGUF published 2026-07-11T13:07:31.000Z · GGUF files on Hugging Face

Measured speeds

ChipGeneration tok/sPrompt tok/sQuantBackendTestSource
AMD Ryzen AI Max+ 395 (Radeon 8060S)18.4168UD128Vulkan decode (llama-bench)YanissAmz/Hy3-295B-A21B-GGUF model card (Hugging Face) details
AMD Ryzen AI Max+ 395 (Radeon 8060S)12.4105UD128ROCm decode (llama-bench)YanissAmz/Hy3-295B-A21B-GGUF model card (Hugging Face) details

Memory needed per quantization

QuantFile sizeNeeds (estimate)Smallest card it fits
IQ1_S64.1 GB67.1 GB96 GB cards
IQ1_M71.1 GB74.1 GB96 GB cards
IQ2_XXS82.1 GB85.1 GB96 GB cards
IQ2_XS91.1 GB94.1 GB96 GB cards
IQ2_S92.8 GB95.8 GBmulti-GPU / large unified memory
IQ2_M102.1 GB105.1 GBmulti-GPU / large unified memory
Q2_K106.7 GB109.7 GBmulti-GPU / large unified memory
Q2_K_L107.2 GB110.2 GBmulti-GPU / large unified memory
UD128116.7 GB119.7 GBmulti-GPU / large unified memory
IQ3_XXS126.1 GB129.1 GBmulti-GPU / large unified memory
Q3_K_S131.3 GB134.3 GBmulti-GPU / large unified memory
IQ3_XS137.4 GB140.4 GBmulti-GPU / large unified memory
Q3_K_M137.5 GB140.5 GBmulti-GPU / large unified memory
Q3_K_L143.1 GB146.1 GBmulti-GPU / large unified memory
Q3_K_XL143.5 GB146.5 GBmulti-GPU / large unified memory
IQ3_M143.7 GB146.7 GBmulti-GPU / large unified memory
IQ4_XS160.8 GB163.8 GBmulti-GPU / large unified memory
IQ4_NL169.8 GB172.8 GBmulti-GPU / large unified memory
Q4_0170.3 GB173.3 GBmulti-GPU / large unified memory
Q4_K_S175.5 GB178.5 GBmulti-GPU / large unified memory
Q4_K_M182.2 GB185.2 GBmulti-GPU / large unified memory
Q4_K_L182.5 GB185.5 GBmulti-GPU / large unified memory
Q4_1187.8 GB190.8 GBmulti-GPU / large unified memory
Q5_K_S206.0 GB209.0 GBmulti-GPU / large unified memory
Q5_K_M212.8 GB215.8 GBmulti-GPU / large unified memory
Q6_K257.2 GB260.2 GBmulti-GPU / large unified memory
Q8_0317.7 GB320.7 GBmulti-GPU / large unified memory

Expected speed on popular hardware

HardwareBest quant that fitsExpected tok/s
NVIDIA RTX 3060Does not fit
NVIDIA RTX 4060 TiDoes not fit
NVIDIA RTX 3090Does not fit
NVIDIA RTX 4090Does not fit
NVIDIA RTX 5090Does not fit
AMD Radeon RX 7900 XTXDoes not fit
Apple M4 Pro (20-core GPU)Does not fit
Apple M3 Max (40-core GPU)Does not fit
AMD Ryzen AI Max+ 395 (Radeon 8060S)IQ2_XXS18.0 estimate

Check your own card

Too big for your card?

Run Hy3 on a rented 24-80 GB GPU by the hour.

Rent a bigger GPU on RunPod

Affiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.