SWE-1.7 Reach Near GPT 5.5 and Opus Intelligence

118

mmekpro about 2 hours ago 79 commentsRead Article on cognition.com

HI version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

90% Positive

Analyzed from 1911 words in the discussion.

Discussion (79 Comments)Read Original on HackerNews

pants2•about 2 hours ago

Kinda funny that their "cost-vs-performance" chart looks the same as the one for Composer 2.5[1], except that it includes Composer 2.5 at a completely different spot.

What are the chances that CursorBench ranks Cursor's model highest, and Cognition's bench ranks Cognition's model highest? Both are to be RL'd from Kimi as a base model, BTW.

I'd posit that it's not deliberate deception, but for both companies their training data and benchmarks come from the same dataset (Devin/Cursor interaction logs) so they naturally overfit.

1. https://cursor.com/blog/composer-2-5

culi•about 1 hour ago

I think it's also telling that they left out the usual hallmarks of the Pareto distribution: GLM 5.2, Qwen 3.7, Minimax M3, and Mimo 2.5

https://arena.ai/leaderboard/code/webdev/pareto

petesergeant•about 1 hour ago

> they left out ... GLM 5.2

They did not.

bluelightning2k•40 minutes ago

Good observation.

I actually started typing the same point that the chances are actually high because of train/eval overlap then realised you answered your own question with that same observation.

It is interesting though!

Perhaps in some way this means we should decide which eval set aligns best with our taste?

Back to the blog post. This is an excellent write up of an excellent technical achievement.

I have a lot of respect for the Cognition/Devin (always "Windsurf" to me) and Cursor teams.

I found it interesting - but justified - that they referred to themselves as a foundation lab rather than a dev tools company.

kgeist•30 minutes ago

On artificialanalysis.ai, Kimi 2.7 Code is way worse than GLM 5.2 at everything (general intelligence, coding, agentic tasks).

But here, both Kimi 2.7 and its derivative SWE-1.7 are ahead of GLM 5.2. This tells me the benchmarks they use are cherry-picked.

ryandvm•about 2 hours ago

Okay, let's give software engineers a break for a bit and focus on obsoleting other high-linguistic context occupations.

znsnjanwnwwn•10 minutes ago

And to do that you’ll need development so until we’re all out of a job they’ll keep pushing. Once automating is automated it’s done.

silvertaza•about 1 hour ago

Like diplomacy? heh

yousif_123123•about 2 hours ago

We need more models that optimize for coding and that can be cheaper than frontier models, like what SWE 1.7 and composer 2.5 are trying to do. I don't think there's an effort to make something GLM-5.2 level but focused only on coding.

jstummbillig•about 1 hour ago

Defining what "coding" means now, and how quickly we fall off the capability cliff seems increasingly important.

Today my "coding" sessions often enough begin with real life problems, where I discuss domain or inter-domain things, ranging from business, economics, psychology, etc. Being able to do all of that with one model is something I am willing to pay a premium for.

Of course not having to pay the premium, because the routing is smart or whatever, would be great. I just don't want to have to think about it.

UncleOxidant•about 2 hours ago

Qwen was doing something like this with their coder models. But alas, they seem not to be releasing those anymore. Last one was Qwen3-coder-next.

yousif_123123•about 1 hour ago

Its crazy that OpenAI and Anthropic themselves aren't doing that. No attempts at reducing inference cost for code as far as I know from them.

andai•about 1 hour ago

OpenAI do have codex models, which are half the price. I haven't used them enough to comment on the quality though.

I remember them saying a few years ago that, they didn't think it was worth specializing models for code, because their general purpose models kept beating them. I guess they changed their mind? Since they did start making codex models again.

gsibble•about 1 hour ago

I use this model. It's pretty good but not Opus 4.8 or Fable levels obviously. I'm really hoping we get more models like it (and better) soon. I run it locally and it's great that way.

UncleOxidant•about 1 hour ago

Qwen3-coder-next is very usable. But I don't think it's as good as Qwen3.6-27B (though it does run faster on my hardware). It would be great if we could get a Qwen3.7-coder, but I'm not going to hold my breath.

anentropic•about 1 hour ago

https://devin.ai/pricing

Apparently 'free' on the $20/mo Devin plan (presumably within some quota still)

and that is "via Cerebras at 1000 TPS" according to the announcement

I live on Opus 4.8 High and their benchmark scores SWE-1.7 slightly higher ... if at all realistic that sounds like a great deal ... too good to be true?

Mitchem•about 1 hour ago

While I am skeptical of the results here, I am very excited for this new trend of making models faster. Running capable models at 1k TPS is more valuable for me than running better models at 30 TPS. I can only imagine the trend continues to move from "let's only make models smarter" to just incremental intelligence gains but with step improvements in speed.

lnenad•about 1 hour ago

Why? I'm personally on the opposite end. Less babysitting/higher quality means more time goes back to me/the user. 1000tps of bad code means you have to keep validating the output and circling back.

aunty_helen•6 minutes ago

High tps is good for deeper agent thinking loops and openclaw etc. I was running cerebus recently doing some data heavy tasks, it managed to crash the server I was submitting posts to. 6 hour task down to ~1hr

anthonypasq•27 minutes ago

id rather iterate multiple times than wait 15 minutes to notice it made a mistake.

lnenad•19 minutes ago

Again, my point is exactly the opposite. Higher quality implies a mistake isn't made in a significant % of cases.

unshavedyak•11 minutes ago

So i agree with you, but there's no SOTA model that i don't have to babysit. I'm not going to just throw a large pile of code in there unreviewed, and so what i want is faster iteration on code in logical, reviewable chunks. Ie just like i'd normally write myself; small, logical commits.

Faster iteration means i mentally checkout less and am more involved with the code being created.

My hope is that in the far far future, we can get LLMs so fast that i can work in my IDE like normal and the LLM will just be an extension of autocomplete. I can state a goal, rough out functions, code, etc, and it'll just work around me like a very fast pair programmer / autocomplete.

The chat interface is an intermediate step that frankly i hate. The faster it is the less i wait.

Now for vibe-slop i'm making on the side, yea i don't care about speed. But that's not something i'm employed to do or anything i truly care about. It's a different workflow entirely.

holoduke•about 1 hour ago

Indeed. For me opus 4.8 is good enough. If only it would be 100 times faster. You could run it in self verification loops much much faster. It sometimes takes 15 minutes for me to complete a simple task. For example configuring AWS agentcore and deploying an agent on it. Takes forever with Claude with constant issues it tries to solve.

taf2•about 2 hours ago

Not finding anything about this while searching huggingface: https://huggingface.co/search/full-text?q=SWE-1.7 i assume this is another closed source model?

mirekrusin•about 1 hour ago

Open weight models should have GPL-like license where it says if you train model on it, it needs to be open weight as well.

joecot•about 1 hour ago

Yes, and not only that but you can't even access it via API, you can only use it in Devin (formerly Windsurf).

I'm an OpenCode user, but I'll fall back to Claude Code if I want to use Opus end to end for something, given my company has a subscription. But I'm not using yet another tool and subscription for a model that isn't even winning.

harmonic18374•about 2 hours ago

A company whose first demo was completely fraudulent announces that its model beats GPT-5.5, on its own benchmark? I’m gonna wait a little before I trust this.

This whole company seems to optimize for raising money and impressing VCs. Lying about their products, ignoring consumer market to target enterprise, bragging about how they work their employees like slaves, and writing these posts full of intimidating technical jargon...

achandra03•about 1 hour ago

To be fair it does seem like most AI startups are now like this (particularly when it comes to constantly mentioning how hard they work and ignoring consumer markets).

parineum•about 1 hour ago

> it does seem like most AI startups are now like this

Remember when AGI was going to replace all jobs in 6 months? It's always been like that.

sigbottle•35 minutes ago

I highly respect many people at cognition but yeah that's put a sour taste in my mouth.

I want to work in the AI space on actual AI research, at any part of the stack. Even if I'm developing training infra - as long as people are advancing knowledge of what intelligence could be.

But it seems like either it's big labs or grifters, that's it, and even the big labs, at least publicly, seem very grifty at times. Not like I have the technical chops probably, but still.

SubiculumCode•about 2 hours ago

Link for this?

jeffnv•about 2 hours ago

https://news.ycombinator.com/item?id=40008109

andy99•about 2 hours ago

Is it just me or does all that* seem pretty tame by today’s standards? Not saying it’s right, but it barely raises eyebrows. Sounds like a pretty typical startup demo.

* Based on the first comment in the link that claims to summarize the video.

oa335•about 2 hours ago

> "A company whose first demo was completely fraudulent"

Could you expand on this?

giancarlostoro•about 2 hours ago

Would love to see these companies use benchmarks done by third parties.

anthonypasq•about 2 hours ago

they are right there? it shows swe-bench multilingual and terminal bench

w4yai•about 2 hours ago

What happened ?

akshaydeshraj•about 1 hour ago

Would have been worth a consideration if it could have been used beyond it's own harness. Unfortunately, doesn't seem to be the case.

https://x.com/theodormarcu/status/2074896486047834380

esafak•about 1 hour ago

Ugh, that changes everything. If I wanted an arranged marriage I could go back to Claude Code.

nibbleyou•about 2 hours ago

Unrelated: what's the point of "*equal contribution"? Why would someone specify this

edot•about 2 hours ago

Because papers are often referred to by the first author’s name, and often the first author is the primary researcher and therefore deserves the extra credit. When two or more primary authors are equally involved, they’ll often do a random ordering but annotate this so that no one thinks one did more than the others.

nibbleyou•about 2 hours ago

Interesting. Thank you

hudo•32 minutes ago

How do i use it from opencode/openrouter?!

2001zhaozhao•41 minutes ago

Wait Devin has a CLI?

Time to support it in my agent IDE just like Cursor's...

spate141•about 2 hours ago

Feels like they discovers that if you build your own benchmark, you can win it

londons_explore•about 1 hour ago

Pretty sure most benchmarks are being gamed by people training on the test set deliberately or accidentally anyway.

throwaw12•about 2 hours ago

Open source for the win!

Imagine how far community might have pushed if 2 past versions of 'morally superior' Anthropic and 'completely Open AI' open sourced their models for the community to build on top of them

spott•about 2 hours ago

Is this open source? I can't find a link to download the weights.

UncleOxidant•about 2 hours ago

It's based on an open weight model (Kimi 2.7) so shouldn't it also be open weight?

NitpickLawyer•about 1 hour ago

> so shouldn't it also be open weight?

Should as in "would it be nice?" - yeah. Should as in they have to? No.

> Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so

You can do pretty much anything you want with an MIT license.

andy99•about 2 hours ago

There is no obligation to do that. I think the landscape would be very different now if one of the big labs had released an earlier “frontier” model under copyleft that requires sharing fine tunes. I hope it still happens.

akshaydeshraj•about 1 hour ago

Not open source. Also, not available beyond it's own harness.

hedgehog•about 2 hours ago

Heads up to anyone else curious, I installed the Devin CLI and SWE-1.7 is not currently available there.

fallinditch•about 2 hours ago

I'm looking forward to trying this out. I've been using SWE 1.6 quite a lot for grunt work alongside Opus for higher level planning and tricky stuff - a good combo.

As a (former) Windsurf user I'm pretty happy with the progress of the Cognition/Devin ecosystem after they took over Windsurf, now known as Devin Desktop.

llmslave•about 2 hours ago

These models are never as good, the benchmarks dont tell the full story

haritha1313•about 2 hours ago

The reality is most people building their own models and providing that alongside SOTA ones don't really care about how great these models are. They just prove that 'hey we are smart enough to build our own models so you can trust us instead of going with a single provider like Claude via Claude Code', also a cheap alternative for cost sensitive/free users - at least this was the case for Windsurf, not sure if Devin Desktop still has that tier. They just need to hillclimb the benchmarks and show something reasonable enough there.

spate141•about 2 hours ago

Benchmarks are just vibes with error bars... wake me up when it survives a week on a real codebase without hallucinating a package that doesn't exist.

SubiculumCode•about 2 hours ago

Funny, the cheerleading at HN for leading Chinese models, but a non Chinese lab (building on top of a Chinese model) gets dissed here.

mirekrusin•about 1 hour ago

It's simple: close weights = not welcome.

sosodev•about 2 hours ago

It's almost as if HN users aren't all the same.

llmslave•about 2 hours ago

all the open source models are a waste of time relative to the bleeding edge from openai/anthropic

wongarsu•about 2 hours ago

At work I wouldn't want to use anything else. Compared to my salary a Claude subscription (or two) is cheap

For hobby projects I've completely switched to DeepSeek v4 pro. I spend less than on a $10 Claude plan and am not subjected to quota limits (when I have time and motivation, the last thing I want is a 5 hour quota running out). And the difference in model performance is fine for those smaller projects, most of which will end up abandoned or in a state of "good enough" anyways

And for utility tasks, those 30b models are also great. I'm a big fan of gemma4

pixel_popping•about 2 hours ago

Not true since a few months, genuinely try GLM 5.2 and Minimax M3, especially in adversarial/gating... as a general model, I can agree, but as a coding model, they are not bad, comparable to maybe Opus 4.5 in real usage which is quite impressive.

achierius•about 2 hours ago

I've always had mixed feelings about Cognition. Obviously they have some very, very smart people working there (I even know a few), and they do make real products. But at the same time, they've made suspicious marketing claims more than once and even been caught making outright fabricated ones; and while they certainly seem to have shaped up from that, I still find their claims to be in a sort of grey area where they seem to avoid unfavorable comparisons and lean on their own benchmarks. Certainly when I've tried their models they have not been nearly as useful as comparable versions of Claude, GLM, etc. -- though I haven't had a chance to try SWE-1.7 yet.

petesergeant•about 1 hour ago

I think showing the API prices for competitors that people don't really pay for that way is all that useful. I do like that it's provisioned by Cerebras though. I think I'd have leant towards focusing on the TPS.