HI version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
57% Positive
Analyzed from 6581 words in the discussion.
Trending Topics
#code#llms#llm#don#open#projects#more#software#source#why
Discussion (146 Comments)Read Original on HackerNews
I think its fascinating just how much of a gap there is between what's being claimed, and the verifiable observable data of the open source world. Major open source projects are by and large starting to ban LLMs now, because the contributions made by LLM users have been universally terrible and unhelpful. There doesn't appear to be a single major project that's found generating code to lead to major productivity speedups, and the consensus appears to be that its just lead to a lot of crappy contributions that are harder to spot immediately as being obvious crap
I regularly see people claim that they are now 10x more productive with LLM code generation, and I just wonder where all the code is. Is it somehow true that these gains are only being realised in proprietary projects, and not a single one of them has put even a small fraction of their new found engineering powers into eg Godot? Why do only the poor quality LLM code generation users make PRs to open source projects, and never the engineers that know how to really use it correctly?
If you look in the open source major project space, you can find almost no evidence that AI code generation exists at all. Go browse your favourite critical tool and look for AI generated PRs that have landed in the codebase, its probably a tiny handful of them in comparison to the human written PRs prior to an LLM ban. It turns out that once you have a verifiable, open quality review bar, for some reason almost no LLM commits really meet the level of quality necessary
I strongly suspect that what we're seeing is that much of the tech code-writing economy had already become completely performative prior to AI turning up. It no longer matters in the current age if your code is good, or works, because your job is to give the illusion of product development while the stock market price gets pumped, until you all cash out your share value, get bought, or hop jobs in 2 years. For many companies it literally does not matter if you produce anything that generates value (or works), because the illusion of progress is all that matters. AI is absolutely incredible at creating the illusion of progress, because it looks a whole lot like real code, it just appears to have failed the bar of making actual projects that work. If that was never the goal in the first place, it probably really is a 10x productivity boost
It could be that the engineers who are extremely productive with LLMs are landing PRs that look indistinguishable from good, hand-written PRs.
It seems likely that the gains from generating tons of code are being offset by the debt incurred to understanding what you're doing. We see lots of greenfield projects one-shotted with GPT or GLM or whatnot, but very little on the side of projects with long-term maintenance goals. This is telling, to me, that the _effective_ gains are much lower than perceived (it's lots of fun to see the thing crank out code at breakneck pace, probably contributing to this). Still quite nice, and very useful, but not a totally new paradigm.
My guess is that open source folks overindex on being anti-AI.
Instead, software is plodding along exactly the same as it did prior to LLM code generation, and there's no evidence of superprogrammers making superprojects in 1/10th of the time. With a 10x productivity gain, what used to take a year should take a month
LLMs are superhuman at short term coding such as debugging and writing tests though and you're missing out by not using them there.
Commercial and industrial electricians are LLMs now?
According to this list of top Github repos by stars[1], of the top 10, nine of them are informational and one is OpenClaw, which certainly has plenty of AI generated code.
The next 10 include react which has a CLAUDE.md and a .claude, and lots of landed PRs that look like AI. And linux, which, we all read Linus' stance. He sees the value. And superpowers, ECC, and hermes-agent. All of which are AI stuff.
I think it would be reasonable to ask which major open source projects aren't using AI assistance?
> Major open source projects are by and large starting to ban LLMs now, because the contributions made by LLM users have been universally terrible and unhelpful.
That is not why. Or at least not the majority of it. The biggest part is that maintainers just can't keep up. They don't have time to do in depth reviews for the number of PRs coming in to find out if it: works, does what it says it does, meets a need, has docs, is well written, meets criteria, etc. That does not mean all those PRs are trash. Some might be trash, some might be gold. Just like human generated ones.
What it comes down to is vibes. Anti-AI folk don't like AI for reasons (that are valid) and they don't want to find any value in it. So they don't. But lots of people are. Lots of really cool, interesting, clever, new software is being written and getting used and building communities but people see CLAUDE.md and go "slop, no thanks" and won't even give it a try.
[1]: https://github.com/EvanLi/Github-Ranking/blob/master/Top100/...
Perhaps the biggest issue hasn't been the speed of development all along but people working on wrong things, hence why progress isn't as evident. LLM helping to build something that isn't needed faster isn't going to productivity more visible.
So 0<whatever<1
Weeks for each feature in a greenfield project... Yeah, LLMs make unworking code way faster than that.
There are two incentives for contributing to open source. The first is to make the app better (add a feature, improve the code, fix a bug, etc.) It's possible that LLMs don't meet the bar but if a human has put the effort in it's not always obvious that it's AI. More likely LLM code is accepted when it's good and rejected when it's bad. The system works.
However, the second incentive is that open source contributions are seen as a 'ahortcut' to making a name for yourself. Being a contributor on a big project goes on resumes and GitHub profiles. Often people who use AI for that don't review the code or even check it does what they say it does. That slop needs to be banned. The downside is losing the good contributions, but it's still a net win.
It's not just productivity, it's life changing.
AI lowers the bar to making submissions which permits flawed subissions to be thoughtlessly submitted by people who don't really understand what they are doing, technically or culturally.
It does, however, enable people who put in the effort to work on something significant. Those people are fewer in number, but exist. They do get to see the animosity that unwitting novices receive.
I don't think it would be surprising for those people to opt out of engaging with a toxic environment.
Selection bias? OSS has a thing proprietary projects don't - an endless cohort of opportunists trying to wedge in a "contribution" for personal gain, be it a Hacktober t-shirt or resume boost or an occasional vulnerability (with the resume boosting being by far most likely). The good LLM-based contributions, you probably won't recognize as AI-assisted unless the author explicitly decided to label them, and if they're really good and use LLMs responsibly, they probably don't even have any ethical obligation to label LLM involvement, much less any benefit.
OSS involvement in general stopped being an indicator of skill once Github activity became a factor for job applications.
Edit:
There's also a second factor: many people (myself included) use LLMs to code one-off personal tools with no intent of them having more than one user, ever. Get an itch, have an LLM scratch it for you, carry on with your life. You'll see little trace of it, because it's hard to distinguish a single-user program that's good enough for specific purpose from a pile of autogenerated slop that probably doesn't work, and neither of these will show up as PRs to big OSS repos anyway.
Instead, if you check out something like ImGui, we can see that its largely just Omar as usual with a bunch of PRs. Nothing much seems to have changed, feature development carries on at the same pace as it did previously. This is replicated across nearly every single major long term open source project I can find
The only reason developers now feel the need to hide their LLM usage is because the LLM contributions have all been shockingly bad, so you're reversing cause and effect. If LLM contributions had been incredible, they wouldn't be frowned upon socially now. Surely someone would blow the lid on this huge covert conspiracy about how incredible their contributions to.. SFML have been?
This effect also wouldn't prevent LLMgineers from creating their own open source projects that should be absolutely outstripping the existing open source space. After all how old is ImGui? With a 10x productivity gain, it should take a year of development to easily match its features if those gains are truly real now
>if they're really good and use LLMs responsibly, they probably don't even have any ethical obligation to label LLM involvement
The copyright issues means that non disclosure is inherently unethical
You're dismissing scale of things. OSS communities were having high-profile discussions about gatekeeping and discrimination way before LLMs were a thing, because already at that point the amount of human generated slop was becoming untenable. And yet, no one argued that this must be evidence that all human contributions are bad and humanity is a bubble waiting to pop.
> ImGui
Weird choice of an example. ImGui is common infrastructure. Who in their right mind would want to mess with that? Especially when building something to solve some end-user problem (whether N=1 or N=1 billion), ImGui is a component you include, not one you rewrite.
There's hardly any reason anyone has to contribute to that project, with or without LLMs.
I suspect the ban of AI in established projects to be a very complex decision. Even if core developers would like to use AI, they don't want to review all of the AI-generated code from the larger community. So the only consistent way to preserve sanity is to declare that nobody would use it. Good luck enforcing it though!
Nothing's stopping core developers from adopting LLM generated code for themselves, while banning it for external contributors
And if they're doing that, why would they tell you, or anyone?
And if they're doing that responsibly - collaborating with AI, and reviewing the code - they don't even have an ethical reason to tell about AI involvement, any more than telling about the StackOverflow answers or blog posts they read before coming up with some implementation.
Two years ago, I couldn't just roughly describe my backlog and then have the code fixed. I had to type it out myself, run it, look at logs, fix toolchain issues, and so on. It was tedious. Or I could get a junior to do it.
Now can get these things done quite fast, without concentrating nearly as hard.
Clearly, it isn't vapour.
It delivers something. That something we have yet to figure out the best way to use, but there's definitely something there that works.
I get the feeling a lot of people are frustrated because the little gains are lost in organisational chaos, rather than the tools not working.
LLMs will certainly be an aide, but assuming adoption of them is required across a whole _field_ ignores whole classes of problems, domains, and constraints the production of software covers.
Rather, we should be wary of allowing our skills and hard-earned knowledge to atrophy by over reliance on new technology that is far from perfect, reliable, or even universally available. These tools more than any before suffer from the junk-in-junk-out problem and I’d much rather work with someone who knows their fundamentals than someone who knows their way around a the LLM of the month.
They aren't saying LLM adoption is required across 100% of the field. They are pointing out that LLMs have reached an impressively capable state, and someone insufficiently inclined to test new tooling that they still dismiss LLMs as "just fancy autocomplete" is exactly who gets left behind when technology advances.
Two things can be true at the same time, 1) that LLMs are not required across a whole field, and 2) that software engineers unwilling to acknowledge their utility have as much a place in the future as the software engineers of 50 years ago who refused to use a compiler.
If they're so easy to use, wouldn't the opposite be true? i.e. people who over rely on LLMs become weaker at the core competency? Nobody gives a rat's ass if you did something in an hour or took all night. The deadline is still next week because of all the meetings. What's the point? Anyone at any point in human history can tell you that doing too much too early has extremely high odds of failure. You're much better off thinking about the business concerns at hand instead of getting lost in the weeds playing with the code.
There's also zero chance of the cadence speeding up because software engineers were never the bottleneck, and LLMs don't work so well for everyone else's job.
If you can't pull rabbits out of hats when the time is right without an LLM, you're already at a disadvantage compared to everyone else who can (anyone who isn't a junior dev today).
Edit: although you might be subject to an NDA... But this is pretty much my test for "AIs will take all the jobs": can it write truly safety-critical software yet?
Most programmers are not experienced enough to write such code themselves off-hand, but more importantly, the safety-critical systems are made safe by following a strict process, not by skills of individuals, which makes it orthogonal to involvement of LLMs.
Also, weird choice of example. 99.9% of coding is not safety-critical, so whatever reservations this would imply (even if it actually doesn't imply any), don't apply anyway.
Why? This has already done via existing processes. My point was to illustrate that if the future is LLMs then surely these processes wouldn't be needed anymore? After all, the LLM would just... Do it itself.
> ... the safety-critical systems are made safe by following a strict process, not by skills of individuals, which makes it orthogonal to involvement of LLMs.
And an LLM that is going to take all the jobs wouldn't be able to execute that process independently and with little oversight?
If my example is, to you, not a good one, what would you rather I use? Most of the common ones can be overly trivialized/minimized (particularly by someone who is uninterested in admitting that LLMs can't do something). That is not to imply that the gp is this kind of individual, but far too many people who I ask to do this (or something similar) are exactly that kind of person: believing that LLMs are insanely great and can't admit (or see) the cons.
People keep repeating your sentiment here but I simply can't follow, are we even on the same planet? Or did everyone switch to just not caring about maintainability and code quality anymore? Or are your work tasks simply so mindnumbingly, stupidly simple that even an AI can oneshot them properly?
I mean this honestly btw, not dismissively like some sister comments. The gap between the productivity increases people report on HN and what I experience myself is insane. In fact, if I factor in the procrastination I find myself doing on dotting the i's on a supposedly "one-shotted" AI implementation of a nontrivial feature, I think the AI actively slows me down.
The only way I've found that I can actually use AI productively and sustainably is in very small tight loops and, well, at that point it's not that much faster than just typing in the code (with the occasional "Cursor Tab" complete).
Am I doing something wrong?
A big problem is that these codebases rot. Agents move incredibly fast at first, but then as you pay less attention (or perhaps no attention at all) to the architecture, they slowly fall to bits. So then you decide, I'll use AI to rewrite it! And it gets better for a while until, well, you get it.
That's not to say there isn't value here, there absolutely is, just — chill. A little.
Where things go off the rail is when you want it to plan AND implement features. The blind spots of LLMs are not where they are for humans and way more work to anticipate. You have to stay on top of the bucking bronco, but you CAN move much faster if you can architect your system so more tasks fall in the "obvious" bucket - that is where the art of engineering still lives. Human understanding remains the goal.
Claude Opus 5 is insane. Still not able to do big complex projects well start to finish but it can, with guidance, implement large parts of them.
There are now open weights models approaching it in performance too.
I don't recommend web work, the last ten years or the next ten. Wouldn't touch the stuff. I skipped the phone app era, too.
It also wrote some unit tests that validated the kernel sampling weights, and wrote some Jupyter notebooks to go along with the kernel algorithms as comparisons.
It's not just web dev... It helps (a lot in some cases) if you ask very specific things rather than just "make this vague thing", but I'm more and more coming round to the conclusion it is now a useful dev tool (until two months ago I was a sceptic).
So an unimportant, personal project. People seem to extrapolate being able to do something cool into being able to do useful work, which is what this whole discussion is about.
I do agree that the current pace of abandonware creation is wild. I think a lot of projects are generated top down from an idea, and not bottom up through usage. If you have a new experimental project, and you don't have recent commits, I assume you are not using it and thus it has no value.
I think agentic use is mostly only valuable as author extension (help you search, rubber duck, generate code but needs very heavy review). Over time with enough usage of a process I do think some of them can turn into author automation but not from simply writing down an idea. It takes lots and lots of executions, iterations, generalizations, specifications, basically lots of work to get any decently hands-off intelligent automation through agents.
The problem is really that the promise was so big but the management and implementation was so painful, coupled with non-stop updates, coupled with far too big of an implementation surface.
A classic "lets solve 15 problems at once" situation, but none of them really well.
There are definitely guys in some companies just pointing an agent swarm at 50 Github tickets and saying "go get 'em boys" and hence need a lot of orchestration tools but for the rest of us do we really need the AI to do all the build steps and the testing steps... or do we just need the new feature that runs when the button is clicked and then we can check it out in the browser ourselves. The code gen is what I need the AI for, not the 'smoke tests' and the tsc finagling. I'm willing to spend some mental calories myself in figuring this feature out step by step
Actually this makes me wonder if some people are just not very into building step by step like "okay first let's check the JSON shape the endpoint returns", "okay now let's make a simple form", etc
That's accurate. There are a lot of people (some of them quite senior and/or skilled) who operate in the way you described.
I still don‘t know if this is real. If it works like that, where are the companies that only kept every tenth, 10x‘d AI enhanced dev? They should have significantly higher margins. The companies that keep all devs and 10x‘d them, finally bringing that backlog to 0 should have exploding revenue and profit.
Where does AI create value? I can‘t see it.
I use it every day, but nothing got faster.
Why fire anyone in this scenario? Growth is only going to keep coming.
I don't think devs are that expensive relative to a big company.
Airbnbs revenue last year was 12b. If they have 1000 engineers,maybe average 200k/yr that's only 200m. If they got rid of their engineers still only a fraction of of the revenue
I still think of what I'm doing as software engineering, and I'm glad that I had many years of professional and hobby development before using agents since I think that's given me the ability to make good architectural decisions (and helps me resteer the LLMs when they want to do something suboptimal), but my involvement in actually writing code is quickly going to zero. That said, they aren't perfect and they still introduce bugs, but I believe the quality of my current product is higher than what I would have created pre-agentic coding.
Things I've found helpful in keeping quality high:
- Visual regression tests (detect UI bugs before you commit them)
- Fuzz testing of interfaces and app behavior
- Automatically add regression tests for any bug that I/the LLM fixes
- Logging/alerting that tracks an errors/invariant violations triggered in the app
- Performance metrics that are surfaced in a dashboard.
All of these are very easy to add since the LLM can create this infrastructure for you. The fuzz testing in particular is something very few products I've previously worked on have since most people don't know how to implement it. I ran the fuzzers for a few minutes and they quickly caught multiple subtle bugs that I was not aware of.
This is a real product that helps a real, non-VC funded service business, and although I could have made something similar myself it would have taken me a lot longer, be harder to use, and probably be less reliable.
Edit: while it's true that you can quickly blow through the $20/month plan, the $200/month plan allows you to get a lot done and is basically sufficient for my needs. It's also very cheap when you consider what it would cost to pay someone to do similar work.
(I wouldn't suggest basing my evaluation of the entire field of coding agents around that particular product.)
There's no such thing as a free lunch, and anyone trying to build software without significant guardrails and insights into the process will have to give up control of their codebase.
Ray Myers does a good job of exploring this topic in a recent Software Should Work talk, trying to answer: "Is code for people or AI?" [0]
[0] https://www.youtube.com/watch?v=mZgglPK8Rg0
We've seen similar waves with new technologies before: overexposing the pros, dismissing the cons, hyper-optimism, and people using a lot of jargon without saying much of substance. The difference this time is the scale of the impact and the volume around it.
May I fix that for you?
The product is being launched before the value is there.
This is one of the reasons that I've simply not bothered with a lot of these types of AI products. It feels like gambling. Maybe I'll spend $20 on tokens and end up with something awesome. Or maybe I'll spend $20 on tokens and end up with nothing useful and then I'll be glad it was only $20 I lost.
That statement somehow amazes me. Obviously, LLMs need plenty of steering, but I don't get how a company can be built on that.
That's like saying smart watches are useless based on trying out the smart watch made by will.i.am's tech company rather than Apple's.
I'll share my experiences and how I do things, but I'm not about to engage in arguments about it. If you don't want to use it, then don't. What I do I care?
I almost wonder if there's a campaign to start suppressing public awareness of where SOTA capabilities really are.
It's nothing short of magic. And in terms of productivity, features that took months are now being done in days. I'm not even sure what the author is talking about frankly.
Am I missing something?
Again, are we still on this ridiculous concept. I stopped reading at that sentence.
Is that going to be the revisionist future that the industry tells itself - ""one day, LLMs came along and suddenly computers could program themselves, and it was an absolute revolution, but it made no difference at all to anything because "The hard part of software engineering was never writing the code." so people stopped using LLMs because of that. And that's how LLM's came to an end after being a short trend.""
The second part was that the test was being administered by a gatekeeper who was hired before those practices were put in place.
The 3rd was what? Amazon or Netflix burning your ass out because they are anti-human companies?
God this times 1000. I'm so sick of having AI chase a bug and having to lead it by the hand like a toddler to try and help me solve bugs. The difference is, a toddler learns when you explain. An LLM holds your responses in context and uses them to generate more authentic sounding garbage, but next session, foomf, the lesson is unlearned again.
I had a tiny problem with prerendering not working on a react site last night, I must have spent close to an hour running commands it asked me to run like a moron, verifying dumb things I had already checked like "did the file actually upload to ftp" or had I restarted Apache or was there some magic htaccess file somewhere. Eventually I just gave up on Claude and tried some different paths in Apache virtual host settings and it worked.
Of course if I went back to Claude and reported that, as would be my instinct with a human, it would confidently explain why that was the bug, why it couldn't spot the issue, and how smart I am for finding it. And then it would forget everything tomorrow
This hasn't been my experience at all. I can give it a vague description of the problem and have it find it pretty easily without any more input from me.
I do think there's a pretty big variation in tools and setups and what output people are getting. I'm now just using OpenAI Codex in VS Code and it churns through problems like they're nothing.
Sometimes I have to get it to not over-engineer a solution; not because it's necessarily wrong but because I don't need that much correctness.
Better read https://blog.senko.net/code-was-never-the-hard-part-is-an-in...
That Nvidia finances a lot of this should give anyone with a clue on finance pause, that it doesn’t tells you you’re in the middle of a bubble.
I can absolutely see where this technology works; but like EVERYTHING THAT THESE VC FUNDED FASCISTS IGNORE, just because it works in the lab does not mean it works in real life. I can see exactly where this technology fails: it lets you spool out so much thread into the abyss that you're just going to keep refactoring everything, all the time and when it blows a hole in your project, whelllp! you wanted to refactor that too. Imagine having 100 side projects grossly built and still, you just want to move on.
Fiishing projects are still going to need work and the explosion of "this is _my_ engineering harness" is everyone twiddling their thumbs cause real projects of significant value are still about scope and "total addressable market" and not "I made an AI to diddle children, so you dont have to" crowd.
No. Anyone making this claim is being disingenuous.