Qwen 3.8 27B is excellent, but it defaults to overthinking things

All current era models overthink as it's a product of their RL incentives (or distillation of models with them...)

From my reading of the Fable 5 and Opus 5 System cards, my reconstruction is something like:

Finish the task → make externally observable evidence that it is finished → check your own work → fix problems → don't stop prematurely → satisfy the evaluator comprehensively.

That is fantastic for SWE benchmarks and autonomous agents. It also naturally creates pathologies:

under-answering is expensive; over-answering is cheap.

I feel like the current “reasoning” that LLMs are doing has got to be a dead end eventually. Every time I have to read another answer with “but wait” and “Actually,” as they “reason” their way to a (sometimes) better answer, I feel like there’s got to be a way to just shortcut to the actual correct answer instead of burning all these token going in circles mimicking actual thought

To me, the amazing thing is that we now have local models that rival the reasoning of high end models from about a year ago.

I hope this trend continues.

According to the paper "Stealing reasoning traces from proprietary llms" [0] all frontier models overthink.

Thinking is good.

You just don't see it in proprietary harnesses because it's literally cryptographically hidden from you.

[0] https://arxiv.org/pdf/2608.09867

I have this branch of llama.cpp that among other things (like patching the template to not break the kv cache, and saving conversations to disk so you can resume quickly days after) also accept the reasoning effort flag here https://github.com/alainnothere/llama.cpp/tree/disk-cache-ev...

I did testing and the reasoning effort can be set per message, I was not aware of the option of none mentioned by @xscott, I tested but didn't see any change, I think there are just 3 values, xhigh, medium and low as per https://huggingface.co/Qwen/Qwen3.8-27B-FP8 , I did testing and the thing can do it's "I'll speak 10 million words to myself to ensure I'm not missing something" and then switch to a faster model, then switch... I did a test and the thing keep coherence and follow it's train of though-kens, you can see the result here... https://github.com/alainnothere/llama.cpp/blob/disk-cache-ev...

The big problem with overthinking on a dense model is obviously the speed hit you take. Going from Qwen 35BA3B to 27B for me is about 7-8x slower (should be ~9x?). This makes me a lot less patient for useless thinking tokens.

I’d want to compare this to the new Muse 30B model which is super terse and has a whole different way of thinking (no “Wait,”) and in my experiments was way more token efficient to the point that the absolute tok / s didn’t really matter.

This is true, but I think it understates the problem. I did a task I've done with a bunch of small models lately (https://github.com/swelljoe/flar/pull/17), and it did an excellent job, the best of any self-hostable model. But, it took eleven (11!) hours on my dual GPU setup. It really chewed on it, and spent a lot of time checking and re-checking. It is by far the slowest model I've used for the task. GPT 5.5 did a similar task in about 20 minutes. Most big models took about an hour or so, and most small models needed a couple of hours (but did a worse job).

I do agree that Qwen 3.8 27B is excellent but slow and very token inefficient. My benchmark places it near opus 4.6 and codex 5.3 performance. 3.6 27B couldn't even complete the benchmark. Please see below for details:

https://gist.github.com/nharziro/aed0c364ce2f295a493494c6f1b...

when you distill a thinking LLM past its capacity it will default to overthinking because during training that was the only way for a chance at a reward on many tasks.

you can generally avoid this if you specialize it on a domain that is within its capacity.

Am I the only one who enjoys it when LLMs overthink everything?

Opus 4.8 would spend like 10 minutes thinking and then go out there and do an excellent job. Only Fable 5 seems to be smart enough to just know everything it needs to immediately start working without any reasoning or verification. Opus 5 tries to be relentless like Fable, but it's not as smart as Fable and I have to constantly challenge and correct its unfounded assumptions. Sol is somewhere between Fable and Opus 5, it's smart but it's not Fable, it keeps making assumptions that I have to correct.

After trying all these models, I find that I miss Opus 4.8's overthinking. Sure it's slow, but it actually gets things right.

To be fair, Opus 5 overthinks things on a regular basis. I interact with the LLM almost entirely through the prompt interface vs. some agentic harness, so I have a lot of granular exposure to its reasoning. For almost every code analysis, it flags all the important issues and at least one non-issue. It suggests some impractical and unnecessary fix for the non-issue that would categorically be a regression.

I've learned that medium effort can improve the outcome relative to higher settings. But I suspect the phenomenon is an artifact of a misguided effort to fix inherent LLM limitations. At least some of its reasoning will miss the target, and more bad reasoning is not the remedy.

It won't satisfy the people who just want to drop a model into their existing toolset and run, but I think there are a lot of ways to deal with this overthinking problem.

For instance, it's a step backward, but I put {"reasoning_effort":"none"} and led it by the nose:

   User: We're going to make <silly demo>.  Please create a plan, but do not write code yet.

   Agent: <short and reasonable plan>

   User: Now please follow that plan and write the code.  No other chat.

   Agent: <reasonable code in reasonable time>

Maybe this can be fixed with Jinja templates or something, or maybe it's a hack to your harness, but it shows you can get the model to reason reasonably.

I went back to Glimmer 30b for my 20GB of VRAM. Just a better experience fit-wise and speed-wise and tone-/voice-wise.

--thinking-budget and --thinking-message is all you need in llamacpp to keep it progressing.

the message can be some combination of tool calling, summarizing, etc. It's overthinking often is a bunch of recursion, so simply stopping t and redirecting is all you need to do.

If someones building a harness for llamacpp, you can set this per message, so it's possible to dynamically control it by watching for the expansion of the thinking traces, and redirecting it.

I use the message to tell it to use subagents, add additional logging and to use opencode's dynamic context pruning.

As such, we'll just whisper here _skill issue_.

For agents, token efficiency is an operating cost. I’d rather have a terse model that escalates hard cases than one that overthinks every tool call.

You have 4 thinking levels.

You can disable it. It's well known issue in Qwen, previous releases I would disable it by default.

Also xhigh seem a new thing.

A no-thinking pelican! I hope to see more, it's surprisingly good for just 2 minutes.

I wonder if this can be fixed with LORAs.

I hope Apple does end up moving to HBM. Unified memory has been a huge godsend, but the low memory bandwidth is just such a killer. Even/especially on M5, where the available compute is starting to starve incredibly badly on ML workloads.