LM Studio Silently Drops Reasoning Effort on Sideloaded Models
TLDR
- Qwen3.8 has a reasoning effort switch with three levels: low, medium and xhigh. When nobody sets it, the model runs at xhigh, the most verbose one.
- LM Studio only passes
reasoning_effortinto the chat template for models that ship with a hubmodel.yamldeclaring that control. Curated downloads have one. Sideloaded GGUFs do not. - For a sideloaded Qwen3.8 GGUF, LM Studio accepts the field, validates it, and does not use it. Low, medium and xhigh render the same prompt. The model always runs at xhigh.
- Ollama applies the setting to the same file. On our 13-task suite the same bytes scored 19.6 on Ollama and 17.5 on LM Studio. Most of that gap is this one setting.
- You can check your own setup in about a minute. The test is at the bottom and does not need a benchmark.
The plain-English version
Think of the effort switch as a sticky note the chat template puts at the top of the conversation. At xhigh the note says: think carefully, validate your assumptions, consider alternatives. At low it says: keep it brief. At medium there is no note at all.
You choose the level by sending a setting with your request. LM Studio only forwards that setting to the template when the model has a small descriptor file (model.yaml) that says “this model has an effort control, and here is the template variable it sets.” Models downloaded through LM Studio’s curated catalog have that file. A GGUF you drop into the models folder yourself does not, so LM Studio reads the setting, nods, and throws the note away. When nothing arrives the template falls back to the xhigh note, so a sideloaded model runs long-winded by default and cannot be turned down through either API route we tried.
Nothing errors. Your results look fine. On the tests we compared, the model thought about three to twelve times longer than the same weights did on Ollama at medium.
How we know
We asked each runtime to render the prompt for the same one-line question under different effort values, and generate a single token. The prompt token count each runtime reports shows how much text the template actually added.
Model: Qwen3.8-27B GSQ-RCO IQ3_S (GGUF). The identical sideloaded file was served by both runtimes. LM Studio headless (llmster, lms CLI 0.0.15-2) with llama.cpp runtime 2.43.0. Ollama 0.35.1. macOS 27.0.1. Temperature 0.
| Effort sent | LM Studio, sideloaded GGUF | Ollama, same GGUF |
|---|---|---|
| nothing | 62 | 62 |
| low | 62 | 50 |
| medium | 62 | 20 |
| xhigh | 62 | the string xhigh is rejected; think: true renders the default, 62 |
chat_template_kwargs low or xhigh | 62 | not applicable |
an illegal value (bogus) | rejected with an error | not tested |
Ollama’s count moves with the setting: 62 at the default, 50 at low, 20 at medium. LM Studio sits at 62 for everything. The 62 is the xhigh text. The error on a nonsense value shows LM Studio reads and validates the field. It just does not reach the template.
The curated variants behave differently
We then probed the curated Qwen3.8-27B downloads in LM Studio with the same three values.
| Model in LM Studio | none / low / medium prompt tokens | Effort applied? |
|---|---|---|
| Sideloaded GSQ-RCO IQ3_S GGUF | 62 / 62 / 62 | No |
| Curated variants (three of our entries) | 62 / 20 / 50 | Yes |
The curated model carries a model.yaml in LM Studio’s hub folder with a customFields entry named reasoningEffort, whose effect is setJinjaVariable. That one entry is the wiring. The sideloaded file has no such entry, so there is nothing to set the variable.
Our harness probe confirmed the Ollama side for every Ollama model we use that has an effort setting.
The template itself says it plainly. The relevant lines, abridged, from the model’s own chat template:
{%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %}
...
{%- if resolved_reasoning_effort == 'xhigh' %}
reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully ...'
{%- elif resolved_reasoning_effort == 'low' %}
reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief ...'No branch for medium means no added text. If the variable never arrives, the default branch wins.
What it did to our numbers
Same weights, same 13 tests, same temperature, same Mac.
| Ollama (effort applied: medium) | LM Studio sideloaded (effort not applied: xhigh) | |
|---|---|---|
| Suite score, 13 tasks | 19.6 | 17.5 |
| Contract checks passed | 4 of 4 | 3 of 4 |
| Test A1 | 20 | 0 (spent all 16,383 reasoning tokens, never answered) |
| Test A4 | 19 | 11 |
| Test B3 | 20 | 17 (16,131 reasoning tokens) |
Then the control. We kept Ollama and changed one thing: we asked for the default (xhigh) instead of medium. Approximate thinking tokens on three tests:
| Test | Ollama medium | Ollama forced to xhigh | LM Studio sideloaded (default) |
|---|---|---|---|
| A2 | 1,137 | 2,398 | 3,075 |
| A4 | 968 | 2,590 | 8,448 |
| A5 | 530 | 8,096 | 6,320 |
On A5 the setting alone takes Ollama from 530 to 8,096 thinking tokens, a fifteen-fold swing and right where LM Studio lands. On A2 it closes about two thirds of the gap. On A4 it closes about a fifth; the rest is the normal wobble of greedy decoding on a heavily quantized model, which we have not separated out.
The obvious workarounds did not work
reasoning_effortin the request: accepted, not applied, on the sideloaded model.chat_template_kwargswith a reasoning effort: accepted, not applied.- Pasting the template’s own low-effort sentence into the system prompt: it does not replace the xhigh note, it sits next to it. On A5 the model reasoned for 10,533 tokens, against 6,320 without it. One test, so treat it as “no evidence it helps”, not proof it never does.
- Editing the saved per-model config headlessly: LM Studio’s docs describe a Prompt Template box in the app (right-click a model, choose “Always Show Prompt Template” in the settings). That is a graphical control. On our headless install, writing a replacement template into the model’s saved config was ignored, and a hand-written
model.yamlfor the sideloaded file was never picked up bylms ls, even after a server restart. We did not test the graphical box, so we do not claim it works.
How a bench misses this
Our harness sent reasoning_effort: medium and then wrote reasoning_effort_used: medium into every result file. That records what we asked for, not what the model received. Weeks of stalls at the token cap got explained by other things: a “reasoning wall” near five thousand tokens, stream deaths, a termination hypothesis. For the sideloaded model, all of it was downstream of a setting that was never applied.
An earlier note had already found that LM Studio drops the setting for sideloaded MLX builds. Nobody checked the GGUF path on the same assumption. The test that finds it takes one token and one minute.
We now run that probe automatically before any test that requests an effort level, stamp the result into every result file, and print a loud warning when the setting is not applied.
If you publish numbers from LM Studio, log the rendered prompt size, not the request you sent.
What we do not know
- This is one model family. Qwen3.8 is the one whose template reads
reasoning_effortthat we sent an effort to. We have not tested other models that expose a thinking control. - Two of our other LM Studio entries are sideloaded GGUFs from another publisher. They are unprobed because their weights are no longer on disk. By the mechanism above they probably have the same problem, but we have not shown it.
- We tested one LM Studio build and one llama.cpp runtime on a headless install. The graphical app may behave differently, and the Prompt Template box may be a working fix.
- The control run covers three tests, not the suite. Greedy decoding wobbles, and one test (A4) is only partly explained.
What you can do today
- Run the one-minute test below on every model where you rely on an effort or thinking setting, especially anything you downloaded by hand.
- Prefer the curated download when LM Studio offers one for the model you want. In our probe it applied the setting.
- Do not compare LM Studio and Ollama scores for a model whose template has an effort switch unless you have confirmed both runtimes render the same prompt.
- Treat the effective setting as xhigh for a sideloaded Qwen3.8 in LM Studio and budget tokens accordingly. A token cap that looks generous at medium is not generous at xhigh.
- If you need medium on a sideloaded file, use a runtime that applies it. On Ollama,
think: "medium"rendered 20 prompt tokens in our test.
The one-minute test
Replace the model name and port as needed. It generates a single token, so it costs nothing.
for e in none low medium xhigh; do
if [ "$e" = none ]; then X=''; else X=", \"reasoning_effort\": \"$e\""; fi
printf '%-7s ' "$e"
curl -s localhost:1234/v1/chat/completions \
-H 'Content-Type: application/json' \
-d "{\"model\":\"YOUR-MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"What is 17 times 23?\"}],\"max_tokens\":1,\"temperature\":0$X}" \
| python3 -c 'import sys,json; print(json.load(sys.stdin)["usage"]["prompt_tokens"])'
doneIf the four lines print the same number, the setting is not reaching the model. If the numbers differ, it is.
For Ollama, send "think": "low" or "medium" to /api/chat with "options": {"num_predict": 1} and read prompt_eval_count.
Sources: LM Studio docs, Prompt Template.