Discussion (3 Comments)Read Original on HackerNews

varispeed•20 minutes ago

These tests are pointless when often models get nerfed few days after release. Astra today is way dumber than just few days ago.

cbg0•3 minutes ago

Not Astra but Sol hasn't been nerfed: https://marginlab.ai/trackers/codex/

d_tr•3 minutes ago

How and why do they get nerfed? To save money?

Cognition's SWE-2 achieves 92.8 on Terminal-Bench 2.1