Round-by-round writeups from the GAUNTLET local-model benchmark: what ran, what broke, and what the scores actually mean.https://gauntletbench.com/https://gauntletbench.com/rounds/production-replay/https://gauntletbench.com/rounds/production-replay/We replayed a real production agent's nightly documentation work through four local models and scored them against what actually shipped — including a truthfulness check on their own work reports. One model fabricated its numbers. Only one answered all twelve tasks.Sun, 16 Aug 2026 00:00:00 GMThttps://gauntletbench.com/rounds/browser-verification/https://gauntletbench.com/rounds/browser-verification/A new browser-artifact suite produced a near-perfect 79/80 static score — then a human loaded the artifacts. The cloth sim doesn't respond to the mouse, the car is drawn backwards, and all four models in the field failed the same interaction the same way.Sat, 15 Aug 2026 00:00:00 GMThttps://gauntletbench.com/rounds/quant-ladder/https://gauntletbench.com/rounds/quant-ladder/We ran the same 27B model at five quantization levels through the same 21-test gauntlet at temp 0. Quality was not monotonic with bits: Q6_K took the all-time roster record while Q8_0 — 12 GB heavier than Q4 — bought exactly zero extra points.Sat, 15 Aug 2026 00:00:00 GMThttps://gauntletbench.com/rounds/sideloaded-mlx-drop/https://gauntletbench.com/rounds/sideloaded-mlx-drop/Two MLX builds of the same model kept stalling at settings that worked fine on their GGUF siblings. The reason: LM Studio accepts the reasoning-effort parameter for sideloaded MLX models at the API — then silently discards it.Sat, 15 Aug 2026 00:00:00 GMThttps://gauntletbench.com/rounds/xhigh-minus-39/https://gauntletbench.com/rounds/xhigh-minus-39/Qwen3.8-27B ships with reasoning effort set to xhigh. Head-to-head at a 49K token budget, xhigh scored 218/260 against medium's 257/260 — 5.2x the reasoning tokens, 3.5x the wall time, for a net minus-39.Sat, 15 Aug 2026 00:00:00 GMT