ES version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
63% Positive
Analyzed from 5774 words in the discussion.
Trending Topics
#more#using#productivity#study#code#don#where#things#thing#point
Discussion (111 Comments)Read Original on HackerNews
METR reran the study early this year and, while they caveat it, this time they found a speedup, which is consistent with subjective estimates of productivity also having increased -- the simplest explanation is that subjective estimates exaggerate, but there's still a speedup with current models: https://metr.org/blog/2026-02-24-uplift-update/#wider-adopti...
(Nobody seems to cite the followup since it's not such a fun counterintuitive finding.)
That's more than a caveat. That says, this study is unreliable. And why? "The primary reason is that we have observed a significant increase in developers choosing not to participate in the study because they do not wish to work without AI."
So, maybe people are fooling themselves, just like this article suggests (I almost wrote asserts, but saying, "I think..." isn't really an assertion)
Why do you think this is? Who wants to spend an afternoon gluing API calls together when the robot can do it in 5 minutes?
If you want to talk about studies, we need to be talking about current studies with current tools. Not studies about how the situation was in the past with different tools.
At least the author admit he is biased..
This also explains why an independent dev can proclaim a 10x productivity boost or whatever, and actually seem to be showing that - while major companies dumping obscene amounts of $$$ on tokens don't seem to be have much to show for it.
They may also be seen as a revolutionary game-changer by people who actually kinda suck at their job.
And a lot of software engineers actually kinda suck at their jobs.
Preposterous. I can tell 5.6 Sol to do something like "optimize this entire subsystem to have no ongoing memory allocations, fix these 5 bugs, implement these 2 features" and go work on other code or do something else entirely, and it does it all flawlessly. All things I could have done myself but nowhere near as fast and right the first time with such thorough test cases. We've reached the point where the AI is usually faster and always less effort overall compared to the old days, even if you have to send a few followup prompts.
Serious question: How do you assess this? It seems to me that it would take very significant time to establish that conclusion.
Did you feel the same way about 5.5? The same general sentiment keeps being expressed with every model release. The prior generation is immediately cast aside with a vague “well yes, actually we didn’t mean it last time” attitude.
I think I even heard Theo the T3 guy suggesting that his work comes to halt if he can’t use the latest generation model, despite heaping significant praise on each generation previous and suggesting it does all of his programming tasks.
The boundless energy with which each model generation is said to be revolutionary and leaving everyone behind is tiresome even when incremental progress is real and measurable.
Which ones? That there is no evidence (not personal anecdotes but the numbers) of productivity increase? Or that our subscriptions are heavy subsidized (then try to compare produced value against real price of tokens)? Or that AI companies are burning tonn of cash without clear plan for monetization?
Article has good points which are worth discussing rather than discarding. We need to bring arguments from both sides of the fense.
The blog tries to cite a 2025 METR study as evidence of no productivity increase, but they ignored the newer 2026 study by the same group that did show a productivity increase with the newer tools.
I also think it's funny that there have become these hard demands for studies and proof of increased productivity. We never saw the same standard of evidence applied to previous advancements like different programming languages or using git for version control. We don't argue with people when they say they're personally more productive in emacs than in VS Code or vice versa. It's only when the topic of AI comes up that the bar for evidence gets raised to the sky
The ones that shout about that one METR study, use it to imply that millions of developers are deceiving themselves in believing that tools they use every day are useful, then admit that they don't like and don't use the tools themselves.
This opinion piece is more what he wants to happen rather than what's actually happening on the ground
> This is why I'm not using AI. I'm not using AI on moral, ethical grounds.
The funny thing is - they are using AI and they will be using even more AI as it soaks into world economy even more; they just won‘t be using it directly. Well, not for a while at least, until practicality overcomes subjective morals.
So you disagree with the quote from Feynman? (the quote that forms the basis of the article)
I'm genuinely curious to know what part of the article you feel is "so far off from reality"?
And so if someone like the author is trying to argue the contrary, that leads me to believe they either: (1) have not bothered to spend even 5 minutes validating their belief or (2) do not care to test their beliefs against reality, for whatever reason. Either one is a bit of a red flag.
I'm somewhat agnostic about the whole thing, and I find the utter confidence you display strange and arrogant. Some really good devs say AI is not that great. Some other really good devs say it is great. Why is one side of them automatically delusional? Could they know something you don't?
But imagine that Ctrl+F was just introduced as a feature and you had a bunch of people saying "All these people saying Ctrl+F is increasing their productivity are fooling themselves", it's just a silly thing to say.
Lol! Feynman's quote is in fact for everyone. Refusing to consider that it might apply to yourself is proof of that!
> AI will just be turned off.
So do you disagree that AI companies are in very large amounts of debt? And do you also disagree that debt will have real consequences (and one of those consequences might be either the lights going out or a buyout or merger)?
Zitron has been predicting an imminent bubble pop since 2023 (although in less and less falsifiable terms), as OpenAI and Anthropic revenues have steadily grown 3-10x per year, but people keep listening. Zitron has done a lot to misinform the public but at some point you have to conclude it's demand-driven, and people won't re-evaluate his credibility because he tells them what they want to hear.
Useful: https://www.theargumentmag.com/p/ais-biggest-critic-has-lost...
it's a combination of wishlful thinking and straightup delusion to think that AI will just poof and disappear (maybe the specific LLM approach eventually, but not the general idea of AI/the industry/product as a whole)
The same thing as you, man. Die in the chaos.
- a lot of folks use ai poorly and innudate you with slop, reducing your own efficiency
- because you can just prompt it doesn't mean you should. many let the machine crank out tokens until it works, rather than thinking about the problem first.
of course, if AI solves my second point automatically, it'll solve the first as well, and we'll call that AGI. But we don't have that yet.
Yeah this article is safe to ignore
Just the other day there was data corruption discovered and a relatively urgent fix required. Just a few years ago I would have been working deep into the night to make sure everything was fixed for the morning. Within an hour Fable had it all wrapped up neatly and the fix deployed…
>data corruption...urgent fix required
Do you people hear yourselves?
This "Anti AI Social Club" is just the latest example, and reality has nothing to do with it. Denying reality might just be the whole point.
It's a very human thing really.
It ended up with investors losing their shirts time and time again - https://en.wikipedia.org/wiki/Railway_Mania and https://en.wikipedia.org/wiki/Panic_of_1893 for example - but we did get some very useful infrastructure out of it.
I could use current models forever. LLM does things for me that couldn't be done in 2023. I don't want to go back to a world without it, but I'm also ok with progress stalling right here.
As for the productivity benefits, I find there is a kind of skeptic that just has his eyes closed. Open an AI page, ask it to write you an app that renames all your scientific papers so they have their title as the filename instead of the weird long number.
Without AI you could easily waste an hour looking for the right PDF library and working out how to use it. Or fiddling with the file search mechanism so that you only get the articles and not every file in your downloads. Now you will have the files renamed in a few minutes, and a tool that keeps them correct every time you have download some more.
How is this not a productivity win? It clearly is. Do we really need a double blind test to check this kind of thing?
Whether it gets squandered in modern orgs is another matter, but the core win is clear for everyone to see.
Do you think that your perceived increased productivity offsets the true, unsubsidised cost of AI?
Writing emails and summarizing other emails...
Ironically, the author would have been better asking an LLM to write the article. At least it would have found better sources and developed more convincing arguments.
I'd be much more curious on a study that used regular developers on brand new-to-them codebases for significant features or migrations. I'm personally encountering what I'd describe as a 200%+ productivity boost.
That kills all arguments from the article instantly.
December 2025 through today GenAI has become massively better. I’ve built things with Claude in contracted spans of time that would have been man-years in the before times.
> Maybe that was true in 2025, but it's clearly not true any more - I feel so productive now!
Even if you are right, this is a completely baseless argument. The people in the study felt the exact same way! And "it's 6 months old, so I won't listen to it" is an insanely high standard. You're eliminating all the evidence!
Opus and similar models completely invalidated that skepticism, especially when context windows increased.
In the last two weeks I designed and deployed a new programming language that compiles in a browser. Impossible in the before times.
I'll just have to move up a level of abstraction to get that same feeling, I guess.
I wish this were how things are going but Uber is still surviving. I figure at least one AI company is going to come out of this alive and kicking.
Would you commission a study about whether cars are really a faster form of locomotion before deciding to purchase one?
There is empirical data right in front of your face. You're just choosing to exclude it because of where you already stand.
That's exactly what Feynman was warning about.
People said this a year ago, then all the companies admitted they were incinerating cash and had to increase to token pricing. Consumers basically mutinied and managed to push back a bit, but that is the struggle right now - to keep prices from increasing 25+ times, not to see them decrease.
If you're premising this on some future technology like ASICs, even if that works out then we probably shouldn't be spending a trillion dollars on datacenters that it will make obsolete.
A year ago Opus 4.1 was $75 per million tokens, and now Opus 4.8 is $25 and Fable is $50.
Why do you feel this way? Are you talking about frontier models from US companies or smaller weight open source models?
From the perspective of US companies, it seems pretty clear they're severely under water and will need some way to become profitable or risk collapse, so how else will they do that or they than jacking up prices or some unknown future miracle advancement?
I could see the argument applied to self-hosted/OSS models tho.
I hear this a lot and it seems to be far too optimistic. Call me skeptical but plan pricing continues to increase with tighter usage constraints. We’re still at the point in time that companies like Anthropic, OpenAI, and GitHub charge seemingly reasonable prices in order to get people hooked onto AI before raising prices to get their return on investment.
> Meanwhile, a study (late 2025) seems to report that although participating developers felt they completed tasks faster using AI, they where around 19% slower.
I think you might be fooling yourself if you build your entire worldview concerning the productivity benefits of AI-assisted programming around that one study from one organization that confirms your priors.
(I had an argument on Lobste.rs the other day about why only believing aspects of software engineering that have survived peer review doesn't make sense to me: https://lobste.rs/s/5kgenk/reviewing_ai_code_is_not_viable_a... )
Here are my personal anecdotes for the week. I occasionally ask AI to write some SystemVerilog test bench code. This is an area where I have deep and long experience. AI has never once saved me time with that. Just yesterday, however, I was writing a tricky Makefile, something I rarely do, and AI helped a lot.
Aside from productivity, this article mentions ethical and financial issues with AI. That doesn't stick with the subject that the title of the essay implies, but it's a more concrete thing to discuss and I wish we'd spend more time on that. I think that's where people really are fooling themselves.
But, I don't understand the position of that has no value or that is going to be "turned off", that is just doomsday prophecy at this point.
We focus so much in code, but AI have changed many if not most of the work. It made possible to take a chance in challenges that would once be dismissed of the bat.
I do think we are still get used to the new ways of work, which IMO is a moving target. Once that we overcome that, results may come clear, but that may not be in companies making 10x more revenue.
1. I am not yet convinced that I am more productive with AI for complex tasks. But, I am fairly certain for trivial tasks, or bureaucratic tasks AI has lessened my cognitive load (data point of one).
2. Does AI run on money, or does it run on energy? Money is the means by which we facilitate exchange. But in reality, the energy in the universe is ginormous. The fundamental thing to keep AI going is available in excess, all other things are negotiation.
3. Is AI bad for the environment or is it the way we obtain energy bad for the environment? The trend seems to be that energy is getting cheaper and cleaner. The world is not static, but will move to accommodate a world with AI.
If investors are looking for a 50% return, they need the investment to be worth about $1.5 trillion. If they're looking for a 100% return, it needs to be worth about $2 trillion. That's still an enormous hurdle, but it's much smaller than suggesting AI needs to somehow generate trillions of dollars in cash simply to recoup the original investment.
Thats a pretty strong outcome. It implies that not only are GPUs that power AI too expensive long term, but they cost too much to operate even if they were free.
Seems to me the more likely outcome is a wave of dotCom style bankruptcies wiping out equity holders for companies who contracted to buy chips and datacenters at MSRP, and a second wave for the groups that step in after to operate whats left without the absurd financing charges and lower capex.
They are free. You can download glm-5.2 and run it on your own hardware. Even you got hardware for free, electricity would cost you more than the sub, in most places.
And if you don't know anything about programming, or don't care that the solution it'll come up with is potentially flawed, then it's a lot quicker to vibe code up some sort of hack job answer than learn how to do everything manually. I'm pretty sure that my non-technical parents or grandparents could ask Claude or ChatGPT to build a simple website for their small business or charity, and have something workable within an hour or two.
For a software engineer, AI might not make them any more productive. For a non-engineer, it's basically magic.
The unsustainability thing is more of a concern though, and the external effects of AI are a huge problem. It's just that pure productivity wise, a lot of people will be significantly more productive with these systems than without them.
Some people still suggest that AI flat out isn't useful though, and that angle to me doesn't hold much water. I was a slow AI adopter, but I have found plenty of benefit from using AI in various applications recently. I haven't come close to using on the level of some people who orchestrate agents and rarely read or touch the code they are writing, but there have been clear times when I popped into Claude and had it help me very successfully with something that would have taken longer to do without that help. So increasingly I am skeptical of anyone positing that there is no potential benefit to appropriately using AI assistance. I think the author could rewrite around the angle of their perceived negative externalities outweighing potential advantages, but to deny or ignore those potential benefits ultimately distracts or actively takes away from a serious discussion of said negative externalities.
I don't put much stock in the very very frequently cited study about AI productivity included in this blog. It's a single study. The study itself cites multiple other studies that did find a productivity boost in writing software with AI assistance. I don't see the methodology as being very sound, the sample size isn't very big, some of those developers had likely never used AI before so of course they were slower as they explored using a new tool for the first time (which the study more or less writes off as a non-factor). They also used "Claude 3.5/3.7 Sonnet", which is not the strongest model. I personally find the Sonnet family of models not that useful and much more inaccurate than something like Opus or Fable.
Sure - it was light on the meat, But it's an opinion blog post, not a full article. the author ventures to cite their refs, acknowledges potential bias.. way more than you get with a tonne of front page articles (many of which probably should have been short opinion pieces like this)
Sure - it's not particularly notable or high entropy enough to save, but it's far from offensive or bothersome ...yet all I'm seeing (at time of posting) is defensive dismissals, ad hominems, and echo chamber pablum
Anybody actually using AI in anger (to get paid) will tell you AI was pretty bad early 2025. It wasn’t usable till June. By December it was writing most of my code - with significant reviews by me. Now it’s basically getting it right. YMMV.
One thing I've noticed is, that coding manually is exponentially more stressful than the slow AI approach. I'm not going any faster than I normally would, but the burnout doesn't show up on the fifth day of the week. The hyper-focus is kind of fun but utterly exhausting.
With the fast approach I can create impressive demos and POCs that are nowhere near ready for production. But sometimes that's exactly what I need.
I find the exact opposite. Typically the AI will make too many mistakes (even with latest advanced models), they usually aren't major/program breaking mistakes, but they still require fixing if I want to keep the code quality good enough, and make future changes possible without a falling house of cards. In order to describe the correct approach I have to write more English to describe the problem than the amount of code I could write myself. I could maybe see it working by writing rough pseudo code and having the AI translate, but then I loose out on any code analysis or tab completion as I write. It's also just innately frustrating to have to babysit and correct somethings work all the time. This all may be in part caused by the nature of my current work for the last couple years, which has mostly been on library code rather than application code.
I suspect I'd have a very different experience if I was working on CRUD related tasks like making new API endpoints, interfacing with a DB, or writing FE components. Not to make light of those tasks, I enjoy that work, though it is much more repetitive.
I agree with the parts about using AI to ask questions and gain better understanding though. I do that constantly for APIs and languages that I don't have a clear mental model of. It's like an interactive search.
Also agree on POCs, sometimes there are small tools, demos, or prototype features that I don't need to care deeply about.
I can have an agent put together a shitty prototype of something just to get a general idea of what I'm looking at, and save probably half a day of work and avoid physically typing potentially hundreds of lines of code. And I can rubber duck with it, and very quickly search through software packages and literature, produce summary reports, dig through my own company source code and wiki and Slack messages to figure out how some specific detail works, etc. These things are important but take my time and energy away from focusing on the hard parts of what I'm supposed to be doing.
My new favorite thing with AI is that my pile of random helper functions is now a thoughtfully-designed library with an extensive test suite for every single thing. I'm not auditing every line of code but I'm reading the plans carefully and I do take a glance at what the agent wrote before I start using it. Most of the time it's not perfect or it's ugly in some way I wouldn't have done, but it's good enough to do research with, and it's way better than what I could have put together by hand, and in much less time. I trust my own research results more because I can put together these kinds of helper libraries much faster, and if I have to make changes I have a big test suite to catch me when I make a mistake.
The only hard part is managing the context switching where instead of putting my work on pause to go write a helper routine, I now have to account for periodically interrupting my work in order to keep an eye on what the agent is doing. For now my approach has been to just ignore it for a while and then take a break to cycle through AI work sessions/chats.
The requirements didn’t really change, opus just couldn’t see all the real world potential issues with its various suggestions.
But it was helpful for me to talk it out. So I guess it fell in the (very) slow category of productive use.
You do have to be very careful of when the AI might be confidently wrong, or leading you down some kind of weird narrow path.
When searching the web to gather info on a topic, all have a tendency to find 2-4 search results and fixate on those as if they represented the entire scope of the topic. No surprise about the fixation because that's kind of how LLMs work (X, Y, and Z are in context -> more tokens about X, Y, and Z). But you have to be aware of it, and I often have to prompt it to do another search and keep turning up more options.
Unfortunately this recent experience wasn’t that as much as it producing rapid-fire misses when it did have the constraints. It regularly just violated X to kinda-solve Y. Or made up a solution to both that solves neither. And this wasn’t some wildly complicated scenario.
Sure made me solidify some opinions (we ended up pretty close to where I had started without it), but woo-boy… it was a painful journey.
I just tell Claude Claude code, "commit all, grouping logically" after a tranche of work.
Also, once you build something with slow AI, it becomes more practical to actually rewrite it by hand quickly and refactor it in a way that makes more sense for humans. It’s “humanwashing” the code.
This is largely an infrastructure problem, and it will be solved just as similar problems were solved with storage, network capacity, and computing power.
There will be bankruptcies, but the technology will prevail, just as the web did after the dot com bubble. Personally, I see the loss of craft and practical skill, which can only be built through doing, as a much greater danger. I still practise LeetCode from time to time so I do not lose the ability to write code myself.