Qwen3.8-27B MLX 6bit (lmstudio-community)
Hollow markers & dashed spokes: axis not yet scored
Specification
- Parameters
- 27B dense
- Architecture
- qwen3_5
- Size on disk
- 21 GB
- Quantization
- 6bit
- Format
- MLX
- Runtime
- LM Studio
- Class
- Standard
- Reasoning (CoT)
- Yes — emits reasoning tokens
- Internal ID
- M25f
- Mean speed
- 15.4 tok/s across suites
- Model card
- huggingface.co
Suite results
General capability (13-task real-workload suite) 1a 234 / 260 19.5 12/13 ● n=2.5 13 run 2026-09-12
| Test | Category | Score | tok/s | |
|---|---|---|---|---|
| A1 | Dev/Ops Scripting | 20/20 | 15.7 | |
| A2 | Dev/Ops Scripting | 20/20 | 14.2 | |
| A4 | Dev/Ops Scripting | 19/20 | 12.3 | |
| A5 | Dev/Ops Scripting | 20/20 | 13.1 | |
| A6 | Dev/Ops Scripting | 20/20 | 11.8 | |
| B1 | Document Processing | 20/20 | 12.6 | |
| B2 | Document Processing | 20/20 | 12.9 | |
| B3 | Document Processing | 20/20 | 15.1 | |
| C1 | Content Production | 20/20 | 16.8 | |
| C2 | Content Production | 18/20 | 12.3 | |
| C3 | Content Production | 18/20 | 16.1 | |
| C4 | Content Production | 19/20 | 13 |
Category mean Dev/Ops Scripting 19.8Document Processing 20Content Production 18.8
Agentic tool-calling & protocol adherence 1b 155.2 / 160 19.4 8/8 n=1 18.7 run 2026-09-12
| Test | Category | Score | tok/s | |
|---|---|---|---|---|
| WA1 | Tool Calling & Protocol Adherence | 20/20 | 19.2 | |
| WA2 | Tool Calling & Protocol Adherence | 20/20 | 20.7 | |
| WA3 | Tool Calling & Protocol Adherence | 20/20 | 20 | |
| WA4 | Tool Calling & Protocol Adherence | 20/20 | 18.7 | |
| WA5 | Tool Calling & Protocol Adherence | 20/20 | 15.4 | |
| WB1 | Agent Robustness & State Management | 20/20 | 13.9 | |
| WB2 | Agent Robustness & State Management | 20/20 | 14 | |
| WB3 | Agent Robustness & State Management | 15/20 | 16.1 |
Category mean Tool Calling & Protocol Adherence 20Agent Robustness & State Management 18.3
Doc/OCR — real-degraded tier 1d2 59.2 / 80 14.8 4/4 n=1 17.3 run 2026-09-22
| Test | Category | Score | tok/s | |
|---|---|---|---|---|
| D9 | Doc/OCR — Real Degraded Transcription | 12/20 | 16.8 | |
| D10 | Doc/OCR — Handwriting Extraction | 17/20 | 12.3 | |
| D11 | Doc/OCR — Degraded Table Transcription | 14/20 | 15.3 | |
| D12 | Doc/OCR — Degraded Table QA | 16/20 | 16.2 |
Category mean Doc/OCR — Real Degraded Transcription 12Doc/OCR — Handwriting Extraction 17Doc/OCR — Degraded Table Transcription 14Doc/OCR — Degraded Table QA 16
Long-context multi-needle (MRCR) 1g2 44.1 / 60 14.7 3/3 n=1.3 12.5 run 2026-09-21
| Test | Category | Score | tok/s | |
|---|---|---|---|---|
| L7 | Long-Context — Multi-Needle Retrieval (MRCR) | 20/20 | 6.2 | |
| L8 | Long-Context — Multi-Needle Retrieval (MRCR) | 20/20 | 7.2 | |
| L9 | Long-Context — Multi-Needle Retrieval (MRCR) | 4/20 | 11.8 |
Each test is LLM-judged 0–20 against a fixed rubric; suite max = tests × 20. An ADJ chip marks a cell carrying a subjective adjustment from the maintainer; the judged score is the one shown, is unchanged, and remains what the axes, ranks and tie bands are computed from. Hover for the reason. Rows with a ▶ expand to the per-test breakdown. Where fewer tests ran than the suite total, unrun tests count as zero toward the axis score. See methodology.