RigBench data 2026-10-22

Home › Models › Inkling Small

Inkling Small: VRAM needs and speed by GPU

At UD-Q4_K_M, Inkling Small needs about 165.5 GB of VRAM. We have 1 measured speed results for it.

Ranked #21 among open-weight models by the Artificial Analysis Intelligence Index, checked 2026-10-22T18:00:53Z. All leading open models.

263.66B parameters (mixture-of-experts, ~11.9B active) · license: apache-2.0 · 1,048,576-token context · GGUF published 2026-07-30T10:20:13.000Z · GGUF files on Hugging Face

Measured speeds

ChipGeneration tok/sPrompt tok/sQuantBackendTestSource
AMD Ryzen AI Max+ 395 (Radeon 8060S)21.0122UD-IQ2_MROCm tg32llama.cpp #25731 details

Memory needed per quantization

QuantFile sizeNeeds (estimate)Smallest card it fits
UD-IQ1_S74.8 GB77.8 GB96 GB cards
UD-IQ1_M78.8 GB81.8 GB96 GB cards
UD-IQ2_XXS82.3 GB85.3 GB96 GB cards
UD-IQ2_M82.4 GB85.4 GB96 GB cards
UD-Q2_K_XL87.9 GB90.9 GB96 GB cards
UD-IQ3_XXS97.9 GB100.9 GBmulti-GPU / large unified memory
UD-IQ3_S107.5 GB110.5 GBmulti-GPU / large unified memory
UD-Q3_K_M119.4 GB122.4 GBmulti-GPU / large unified memory
UD-Q3_K_XL119.6 GB122.6 GBmulti-GPU / large unified memory
UD-IQ4_XS127.4 GB130.4 GBmulti-GPU / large unified memory
UD-IQ4_NL130.1 GB133.1 GBmulti-GPU / large unified memory
UD-Q4_K_S152.3 GB155.3 GBmulti-GPU / large unified memory
MXFP4_MOE158.0 GB161.0 GBmulti-GPU / large unified memory
UD-Q4_K_M162.5 GB165.5 GBmulti-GPU / large unified memory
UD-Q4_K_XL163.3 GB166.3 GBmulti-GPU / large unified memory
UD-Q5_K_S184.2 GB187.2 GBmulti-GPU / large unified memory
UD-Q5_K_M196.1 GB199.1 GBmulti-GPU / large unified memory
UD-Q5_K_XL196.7 GB199.7 GBmulti-GPU / large unified memory
UD-Q6_K218.9 GB221.9 GBmulti-GPU / large unified memory
UD-Q6_K_XL239.7 GB242.7 GBmulti-GPU / large unified memory
Q8_0280.3 GB283.3 GBmulti-GPU / large unified memory
UD-Q8_K_XL288.4 GB291.4 GBmulti-GPU / large unified memory
BF16527.4 GB530.4 GBmulti-GPU / large unified memory

Expected speed on popular hardware

HardwareBest quant that fitsExpected tok/s
NVIDIA RTX 3060Does not fit
NVIDIA RTX 4060 TiDoes not fit
NVIDIA RTX 3090Does not fit
NVIDIA RTX 4090Does not fit
NVIDIA RTX 5090Does not fit
AMD Radeon RX 7900 XTXDoes not fit
Apple M4 Pro (20-core GPU)Does not fit
Apple M3 Max (40-core GPU)Does not fit
AMD Ryzen AI Max+ 395 (Radeon 8060S)UD-IQ2_M28.0 estimate

Check your own card

Too big for your card?

Run Inkling Small on a rented 24-80 GB GPU by the hour.

Rent a bigger GPU on RunPod

Affiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.