← Leaderboard

Gemma-4 E4B 4B 8bit MLX

Keeper Rank #16 of 33 · 4/8 GAUNTLET progress
Generalist — 82.5 / 100 G Agentic — 93 / 100 A Understanding — pending U Needle — pending N Thinking — 100 / 100 T Live — pending L Engineering — pending E Throughput — 46.5 / 100 T

Hollow markers & dashed spokes: axis not yet scored

G
82.5
A
93
U
N
T
100
L
E
T
46.5

Specification

Parameters
4B
Architecture
gemma4
Size on disk
8.97 GB
Quantization
8bit
Format
MLX
Reasoning (CoT)
No
Internal ID
M1b
Mean speed
76 tok/s across suites
Stall census
0 stalls in 21 observed tests (0.0%)
Reasoning appetite
721 tokens mean · 2,398 max
Model card
lmstudio.ai

Suite results

Suite Score Avg / 20 Tests tok/s
General capability (13-task real-workload suite) 1a 214.5 / 260 16.5 13/13 82
Agentic tool-calling & protocol adherence 1b 148.8 / 160 18.6 8/8 70

Each test is LLM-judged 0–20 against a fixed rubric; suite max = tests × 20. Where fewer tests ran than the suite total, unrun tests count as zero toward the axis score. See methodology.