▮▮▮ RigBench

New local LLMs that fit your GPU · week of Oct 21, 2026

16 open models were published as GGUF in the last 7 days. Here is where each one fits. Sizes are real file sizes from Hugging Face; speeds marked est. are estimates scaled from measured runs.

Fits 8 GB cards (RTX 4060, RX 7600, 16 GB Macs)

Orbit-3.2-Nano
2.7B · Q4_K_M · needs ~2.6 GB

~108 tok/s est.

Spark-X2.6-4B
4.11B · Q4_K_M · needs ~3.5 GB

~71.9 tok/s est.

Fits 12 GB cards (RTX 3060 12 GB, RTX 4070)

HybridAgent-14B-Agentic
14.77B · Q4_K_M · needs ~10.5 GB

~33.1 tok/s est.

Fits 24 GB cards (RTX 3090, RTX 4090, RX 7900 XTX)

Tess-4.5-27B
27.32B · Q4_K_M · needs ~18.9 GB

~37.6 tok/s est.

Tessera-27B-B-CJK
27.32B · Q4_K_M · needs ~18.9 GB

~37.6 tok/s est.

jebadiah-27b-v2-1
27.32B · Q4_K_M · needs ~18.9 GB

~37.6 tok/s est.

basal-omni-1.0-ultra
30.7B · Q4_K_M · needs ~21.1 GB

~33.6 tok/s est.

Fits 48 GB (RTX 6000 Ada, two 24 GB cards, 64 GB Macs)

MLExpert
34.66B MoE · Q4_K_M · needs ~24.0 GB

~30.7 tok/s est.

MiniCPM-V-4.7-35B-A3B
34.66B MoE · Q4_K_M · needs ~24.0 GB

~179 tok/s est.

Fits 96 GB+ (RTX PRO 6000, 128 GB Macs, Ryzen AI Max+ 395)

NVIDIA-Nemotron-3.5-Bolt-74B-A9B
74.04B MoE · UD-Q4_K_XL · needs ~52.9 GB

~88.0 tok/s est.

Kolibri-1
78.1B · Q4_K_M · needs ~52.5 GB

~21.6 tok/s est.

NVIDIA-Nemotron-3.5-Super-VL-120B-A12B
120.0B MoE · UD-Q4_K_XL · needs ~89.9 GB

~61.8 tok/s est.

Ornith-1.5-122B
124.64B MoE · UD-Q4_K_XL · needs ~82.7 GB

~13.5 tok/s est.

Needs more than 96 GB

Agnes-3.0-Qwen (~130 GB), Beam-1.0-501B-A23B (~310 GB), DeepSeek-V4.1-Flash-1017 (~504 GB)

New measured benchmarks

This is the first issue, so the whole leaderboard is new: 1,127 measured results from 8 public sources.

Fastest consumer hardware, Llama 2 7B Q4_0 (tg128):

1. NVIDIA RTX 5090 CUDA

300 tok/s · source

2. NVIDIA RTX 5080 Vulkan

192 tok/s · source

3. NVIDIA RTX 4090 D CUDA

192 tok/s · source

4. AMD Radeon RX 7900 XTX Vulkan

191 tok/s · source

5. NVIDIA RTX 4090 Vulkan

190 tok/s · source

6. NVIDIA RTX 5070 Ti CUDA

182 tok/s · source

7. Apple M5 Ultra (80-core GPU) Metal

179 tok/s · source

8. NVIDIA RTX 3090 Ti CUDA

172 tok/s · source

Llama 3.1 8B Q4_K_M on LocalScore (averaged over long prompts):

NVIDIA RTX 3090 Ti: 110 tok/s (source)
NVIDIA RTX 4090: 105 tok/s (source)
NVIDIA RTX 5090 Laptop: 97.8 tok/s (source)
NVIDIA RTX 3090: 96.3 tok/s (source)
NVIDIA RTX 3080 Ti: 90.5 tok/s (source)

Sponsor
This spot is open. One sponsor per issue, clearly labelled. Sponsor the next issue.

Check any card: the picker · add your own benchmark