About GAUNTLET Bench
GAUNTLET Bench runs local models through real workloads — agentic, live, verified — and publishes the evidence.
One standard, one host
Every score on this site was measured on the same machine: MacBook Pro, Apple M5 Max, 128 GB unified memory, running macOS, served by LM Studio. That's deliberate. Cross-host benchmark numbers are mostly noise — quant behavior, runtime, thermal envelope, and memory bandwidth all move the result. Holding the host constant is what makes a 79 and a 91 comparable.
GAUNTLET is Apple-Silicon-first because that's where serious local inference is happening at the high-memory end. A second host row is planned; when it lands it will be a separate, labeled column of evidence — never blended into this one.
Generated, not handwritten
Every page on this site derives from the data pipeline or it doesn't exist. Scores, radars, suite tables, model specs — all rendered from the same machine-generated dataset that the harness emits after judging. There is no hand-edited number anywhere on this site. If the pipeline didn't produce it, we don't publish it.
The current build was generated from pipeline data stamped 2026-08-17 02:58 UTC —
the same timestamp in the footer of every page.
What the letters mean
Eight axes, one per letter of GAUNTLET: Generalist, Agentic, Understanding, Needle, Thinking, Live, Engineering, Throughput. The full breakdown — suites, test counts, scoring, and the contamination policy — is on the methodology page.
Who runs this
A single operator with a benchmark harness, a controlled machine, and a low tolerance for numbers that don't replicate. The project is public at github.com/Yoda-quelltask/gauntletbench — test ideas and model requests are welcome via the contribute page.