Models › Qwen3 4B Thinking 2507

Qwen3 4B Thinking 2507: VRAM needs and speed by GPU

At Q4_K_M, Qwen3 4B Thinking 2507 needs about 3.4 GB of VRAM. We have 1 measured speed results for it.

4.02B parameters · license: apache-2.0 · GGUF files on Hugging Face

Measured speeds

HardwareGenerate
tok/s		Prompt
tok/s		QuantBackendTestSource
NVIDIA GTX 106022.7506Q4_K_Mllamafile LocalScore avgLocalScore details

Memory needed per quantization

QuantFile sizeNeeds (estimate)Smallest card it fits
UD-IQ1_S1.1 GB2.0 GB8 GB cards
UD-IQ1_M1.1 GB2.0 GB8 GB cards
UD-IQ2_XXS1.3 GB2.2 GB8 GB cards
UD-IQ2_M1.5 GB2.4 GB8 GB cards
Q2_K1.7 GB2.6 GB8 GB cards
Q2_K_L1.7 GB2.6 GB8 GB cards
UD-IQ3_XXS1.7 GB2.6 GB8 GB cards
UD-Q2_K_XL1.7 GB2.6 GB8 GB cards
Q3_K_S1.9 GB2.8 GB8 GB cards
Q3_K_M2.1 GB3.0 GB8 GB cards
UD-Q3_K_XL2.1 GB3.0 GB8 GB cards
IQ4_XS2.3 GB3.2 GB8 GB cards
Q4_02.4 GB3.3 GB8 GB cards
IQ4_NL2.4 GB3.3 GB8 GB cards
Q4_K_S2.4 GB3.3 GB8 GB cards
Q4_K_M2.5 GB3.4 GB8 GB cards
UD-Q4_K_XL2.5 GB3.4 GB8 GB cards
Q4_12.6 GB3.5 GB8 GB cards
Q5_K_S2.8 GB3.7 GB8 GB cards
Q5_K_M2.9 GB3.8 GB8 GB cards
UD-Q5_K_XL2.9 GB3.8 GB8 GB cards
Q6_K3.3 GB4.2 GB8 GB cards
UD-Q6_K_XL3.7 GB4.6 GB8 GB cards
Q8_04.3 GB5.2 GB8 GB cards
UD-Q8_K_XL5.1 GB6.0 GB8 GB cards
F168.1 GB9.0 GB12 GB cards

Expected speed on popular hardware

HardwareBest quant that fitsExpected tok/s
NVIDIA RTX 3060Q8_072.0 estimate
NVIDIA RTX 4060 TiQ8_057.2 estimate
NVIDIA RTX 3090Q8_0153 estimate
NVIDIA RTX 4090Q8_0170 estimate
NVIDIA RTX 5090Q8_0268 estimate
AMD Radeon RX 7900 XTXQ8_0170 estimate
Apple M4 Pro (20-core GPU) (64 GB)Q8_054.1 estimate
Apple M3 Max (40-core GPU) (64 GB)Q8_079.0 estimate
AMD Ryzen AI Max+ 395 (Radeon 8060S)Q8_048.6 estimate

Check your own card

Too big for your card?

Run Qwen3 4B Thinking 2507 on a rented 24-80 GB GPU by the hour.

Rent a bigger GPU on RunPod

Affiliate link: if you sign up, RigBench may earn a referral credit. It never affects rankings or numbers.