GPT-6 Astra makes major gains in the Artificial Analysis Coding Agent Index

wwertyk about 2 hours ago 5 commentsRead Article on artificialanalysis.ai

DE version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

50% Positive

Analyzed from 66 words in the discussion.

Discussion (5 Comments)Read Original on HackerNews

ekojs•about 1 hour ago

Well, seems like ECI [0] and the AA index is diverging quite a bit. Benchmarking LLM is tough and I think we are seeing the limitations of current benchmarks and applicability to real tasks.

[0]: https://x.com/EpochAIResearch/status/2095602754282783108

NiekvdMaas•about 1 hour ago

Title: "major gains"

First chart: from score 61 (GPT-5.6 Sol) to drumroll 61 (GPT-6 Astra)

flyaway123•about 1 hour ago

Indeed. Though to be fair it is referring to "Artificial Analysis Coding Agent Index", from 65 to 67.

dist-epoch•about 1 hour ago

I think they mean cost per task, where Astra is now on the Pareto frontier.

Readerium•about 1 hour ago

more like 5.7 not 6

lostmsu•about 1 hour ago

5.6.1