Back to News
Advertisement
Advertisement

⚡ Community Insights

Discussion Sentiment

50% Positive

Analyzed from 124 words in the discussion.

Trending Topics

#target#lines#code#differences#methods#generated#human#same#size#matching

Discussion (1 Comments)Read Original on HackerNews

fwlrabout 1 hour ago
Near the end of their methods:

    For each likely AI-generated target, we search human controls from the same repository and same programming language, controlling for project-level practices … We also apply size matching: a control must be within 20% lines of code of the AI target
And then from the start of their results and analysis:

    Comparing the lines of code (LOC) between AI-generated and human-written files, we observed no statistically significant differences
Here I would expect some sort of caveat that this result is a verification that their size-matching was effective in removing differences in lines of code between the target and control. However, there is no such thing, indicating to me that the authors are unaware that what they are detecting are the effects of their own methods. This leads me to strongly discount their stated insight regarding structural differences, and frankly the rest of their paper too.