Ask HN: Is there a metric for AI code quality?

ffractalf about 8 hours ago 2 comments

I've tried many different models and without doubt the code coming out of them differs a lot when it comes to "quality". Some of that is subjective for sure, but there are objective sides to "good" code.

I wish this was a metric for the AI benchmarks so I could choose a model based on this, because honestly it's one of the things I care most about.

Problem: How can you measure such things, whats the metrcis?

...maybe there just isn't a way to do it, since that metric isn't in the charts..

⚡ Community Insights

Discussion Sentiment

100% Positive

Analyzed from 78 words in the discussion.

Discussion (2 Comments)Read Original on HackerNews

spgorbatiuk•about 2 hours ago

Not sure if I got the question right, but there are benchmarks like SWE pro and stuff. There's whole another debate whether you can trust it or not, and whether the labs are training on those benchmarks, but that's one way to measure that.

Other than benchmarks, I'd say that's your own test suite

verdverm•about 6 hours ago

Why would a metric for code quality be different depending on how the code got to to a file? In other words, if there was a good measure, would it not exist already for us? How do we measure the quality of our own code?