About & methodology

RigBench collects measured speeds for running open language models on your own hardware, and shows where every number came from.

Where the numbers come from

LocalScore (localscore.ai): 585 rows
llama.cpp Vulkan scoreboard (#10879): 237 rows
llama.cpp CUDA scoreboard (#15013): 112 rows
llama.cpp Apple Silicon thread (#4167): 96 rows
llama.cpp ROCm scoreboard (#15021): 46 rows
llama.cpp discussion #15396: 19 rows
llama.cpp benches/ (maintainer runs): 6 rows
llama.cpp benches/ (maintainer runs): 6 rows
llama.cpp discussion #15396: 4 rows
llama.cpp discussion #15396: 4 rows
llama.cpp benches/ (maintainer runs): 2 rows
llama.cpp discussion #15396: 2 rows
llama.cpp discussion #15396: 2 rows
llama.cpp discussion #16578: 2 rows
llama.cpp discussion #15396: 1 rows
llama.cpp discussion #15396: 1 rows
llama.cpp discussion #15396: 1 rows
llama.cpp discussion #16578: 1 rows

We copy published measurements as they appear in the source, record the source URL, build/commit and contributor where given, and never edit a number. Each table row links to its source.

Checks

Generating a token means reading every weight from memory once, so a card cannot generate faster than memory bandwidth ÷ model size. We drop any result more than 10% above that ceiling. This release rejected 4 rows; they are listed in rejected.json with the reason.

How estimates work

Anything not measured is labelled estimate.

Fit: GGUF file size (real, from Hugging Face) + about 0.8 GB for the runtime + an allowance for an ~8K-token context that grows with model size (capped at 3 GB). Discrete GPUs lose 0.5 GB to the driver and display. Macs and APUs can give about 75% of unified memory to the GPU by default.
Speed: generation speed scales with how many bytes are read per token. For chips with a measured run we take that run's effective bandwidth (tok/s × model size) and divide by the new model's size. Without a run we use spec bandwidth × the median efficiency we measured for that vendor (NVIDIA 0.66, AMD 0.72, Apple 0.83, Intel 0.55). Very small models are limited by fixed per-token overhead, so we never assume fewer than 1.2 GB read per token. Mixture-of-experts models only read their active experts plus shared layers each token; we assume twice the active-parameter share is read (e.g. a 30B model with 3B active reads about 20% of its file per token).
The picker recommends the highest-quality quantization that fits, starting at Q8_0 (F16 is rarely worth the halved speed).

Tests explained

tg128: tokens per second while generating 128 tokens (what you feel while chatting). pp512: tokens per second while reading a 512-token prompt. FA: flash attention. LocalScore runs llamafile over several prompt/output lengths and reports averages, so its numbers are lower than tg128 and are shown separately.

One sponsor per week on the site and in the newsletter, always labelled. Hardware makers, cloud GPU providers and local-AI tools are a good fit. Contact details coming at launch.

Affiliate links

Links to RunPod (cloud GPU rental) are referral links: if you sign up through them, RigBench may earn a credit or commission at no cost to you. They are marked sponsored, shown only as an option next to the data, and never change rankings or numbers. As an Amazon Associate I earn from qualifying purchases. Amazon links carry our Associates tag; if you buy through them, RigBench may earn a commission at no cost to you. They never change rankings, which are sorted by measured speed. Newegg links are plain searches with no tracking.

Corrections

Spotted a wrong number? Every row links to its source; if the source says something different, tell us with the correction form and we will fix it.