HI version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
73% Positive
Analyzed from 626 words in the discussion.
Trending Topics
#model#agent#results#critique#opus#compare#against#here#doesn#paper
Discussion (14 Comments)Read Original on HackerNews
Multiple model vendors is key here, the cascade pattern doesn't need it, but the critique pattern does.
Last Nov, my team wrote a paper ("Team of Rivals") on the difference between using an OpenAI model to Critique an Anthropic model's output vs running a self-review agent loop on the same vendor.
The ablations [1] proved that neither company alone was better than using both.
The paper was a general response to "What does your company do that Anthropic can't?" but more so a demonstration of how to make something 90%+ good with models which eval at 60% or so (& Gas Town post unblocked our "this is a trade secret" argument about the paper).
[1] - https://github.com/t3rmin4t0r/critique-evals
This is also a reason why comparing "naked models" for which weights are available and frontier where providers can do whatever they want behind the scenes is unfair.
Specifically, all it took to boost Qwen3.8-27B to get 10% more points on SWEbench Pro and Terminal Bench 2.0 with a proxy that has just these basics: - tweaks few decode settings like slightly higher temperature - detects when model gets stuck and tells it to "go on" - detects responses cut in the middle, empty responses that contain only reasoning, formats not passing verification etc and tells the model to "try better"
And that is it. 10% more. I admit on a subset of tasks, but results are results, even on a subset.
It's so effective and helps catching so many design flaws, implementations misses etc ... that i'm wondering how people manage to build complex/large projects with agents without this kind of process. Well, i actually built this thing because i couldn't get good results so i had to find a way.
I'm gonna open source the whole thing but it needs some cleanup, there's a basic landing page here https://kodfactory.com if anyone wants to be notified when it's released on github. Yeah i know, the world really needs another software factory :-)
HyDRA: Hybrid Dynamic Routing Architecture for Heterogeneous LLM Pools : https://arxiv.org/pdf/2605.17106
It is explicitly not a foreman, task routing, or an orchestrator agent. It has a bias toward direct action and is instructed to only delegate when necessary.
I've found that this approach yields significantly faster results, without much of a quality trade-off, than an agent whose primary impulse is to delegate.
In contrast, HydraFusion starts with a task routing step, then sequential planning, execution, and review stages. My guess is that this workflow is best for people who are prioritizing cost over speed for the same level of quality.
> In controlled offline evaluations, HydraFusion’s selective coding workflows matched or exceeded the evaluated Opus 5 baseline
Opus 5 (in practice) is not a good baseline to compare against.
I'm shocked that they chose to primarily compare against Opus 5 in all the article's charts. It's pretty disingenuous that they're claiming "frontier" quality, but didn't compare against Fable or Sol.