RU version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
30% Positive
Analyzed from 3051 words in the discussion.
Trending Topics
#vulnerability#bugs#reports#llms#security#find#report#bug#software#more
Discussion (79 Comments)Read Original on HackerNews
Occasionally I see real security researchers on HN complaining that no one takes the disclosure seriously, or that people reply immediately with a cease and desist. But from the receiving end it's just because the spam is unmanageable.
It's hard to spot the stuff that actually matters.
(a) add a new function that does regular expressions searching / matching with a resource checker (eg a timer);
(b) write a local linter that reports an error for any use of the builtin regular expression tools;
(c) fix all the lint warnings;
(d) commit the linter.
When we were making fun of leftpad saying node is unsuitable for serious work, it was labeled elitism. What is it now?
Sure - modern AI can figure that out, but I bet in a vast majority of cases they won't.
I still check the spam folder for legitimate emails, but so far there haven't been any false positives.
Also people ironically just DGAF that much. The last actual bad exploit was log4shell in java, which given how it was introduced (i.e someone purposefully at Apache made it so a log statement can execute code, and nobody questioned it before pushing it to prod), should have been the signal for everyone to completely remove all Apache libraries from their services, but yet all the software is still being used.
Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to minimize creating bugs in the first place. That could be new or better languages, less 3rd party dependencies, more vetted code, better linters, better fuzzers, whatever. The point the new reality of bugs being easy to find will, actually must, lead to less bugs eventually because the world can't function with easy to find bugs.
I find that often bugs will be created when using an LLM, like others have said. Saying that this can then be fixed by identifying all the bugs created by an LLM with an LLM doesnt guarantee another bug is not introduced when the LLM is addressing the initial problem.
Also, what if the LLM has a blind spot. They certainly also could be incapable of finding or fixing a bug. They dont pass any benchmark at 100% right now. Also also, guaranteeing there are no bugs in your code is like saying you have 100% test coverage, all of the tests pass, and they are written perfectly. Saying that you can simply identify and fix the bugs also assumes there is enough time and energy to find all of the bugs that exist within a project and then to address them. Even LLMs use time and energy. In a sufficiently complex system that is certainly wishful thinking.
Considering the size and complexity of a lot of modern software (like web browsers, 3d modelling software, game engines, etc.) software is just too complex to not have bugs even when created and managed by LLMs.
There will continue to be bugs in code and we will simply have to live with the fact that LLMs make it easier to exploit computer systems. I mean consider a hardware bug like Spectre [0]. If bugs like this become easier to find does that mean our existing hardware will just become obsolete more quickly? that type of problem can be addressed, but at quite a high cost.
Not sure what all of this means for the future.
0. https://en.wikipedia.org/wiki/Spectre_%28security_vulnerabil...
I think we're at the point that the best LLMs can indeed write software that's far more secure than your average programmer. Partly because the average is so terrible.
Hopefully at the end of this decade, a ton of software practices have been overhauled to eliminate classes of problems. Memory-safe language use is a great start - but it’d be great to see innovation in checking for TOCTOU problems, improper/missing authn & authz, and many others.
This is an engineering problem. It won’t be solved by models that “only do dumb shit 1/10th as often, only 0.01% of the time now not 0.1%!” It won’t be solved by adding more models to do even more double-checking before and after the work. It won’t be solved by hoping humans catch it in review. It isn’t solvable by adding outer loops of any sort - though we may get close. To truly solve this will take serious CS research.
Personally i have some doubts, a lot of research has gone into the idea without much to show for it, but its a very reasonable research area.
Part of what the research shows is that correctness-by-proof has a cost in developer effort.
If there really is a vulnerability-apocalypse due to AI, and it's not just a different flavour of AI hype, the cost of having insecure software will rise to the point that the cost of dealing with insecure or incorrect code at time of creation becomes less than the cost of ignoring it until it blows up.
I doubt it'll rise so much that we'll want to face the cost of behaviour proofs for much code at all, but it's quite possible it'll rise enough that we want to do things like prove that indices are in bounds, at compile time, so vector accesses can skip checks without compromising safety.
> A requirement for staying sane while working in public as an open source maintainer is realizing that every issue, PR, and piece of feedback is a present, not an obligation. You can accept it, ignore it, and use it partially or not at all.
> Except…
> For years, as lead of the Go Security team at the time, I’ve told new team members that it doesn’t apply to vulnerability reports. No, vulnerability reports are special. Security researchers are doing us a favor by reporting things confidentially instead of doing full disclosure, so we owe them something, which is not true of regular issues opened on the issue tracker.
[...]
> It’s 2026 and none of the premises are true anymore.
I respectfully disagree.
The premise is absolutely still true: if someone discovers a critical, exploitable vulnerability in your software, the impact and tradeoffs are exactly the same as they were before LLMs started finding bugs. There are just more of them now, so they're easier to come by.
But that won't last forever, either. As LLMs find increasingly difficult-to-find vulnerabilities, there will be fewer of them to report. This is just chugging through the backlog.
All of that said, I don't think finding vulnerabilities has really been the difficult security problem for most companies (or open source projects). The difficult problem is dedicating resources to fixing those vulnerabilities instead of building software, products, and/or infrastructure that people want. That problem is absolutely still here today, but I'm optimistic that agentic security developers will be able to take the burden off of development teams in the near future.
For tokens, of course.
I think your logic is partly correct but the fact that the same LLMs are allowing an exponential increase in insecure code generated is a counterbalancing point. I do not think this phenomena will slow down.
That is not my experience at all. People will continue to high-volume spam intended behaviour as if it is a bug.
There will be fewer reports that matter as you fix things - but the volume of reports will either stay steady or go up. Making it harder to even notice the ones that matter.
I think the point is those issues are now easily discoverable and are nearly public because of it.
I wonder what the metrics are. Also, not "anyone", just the affordable.
The _demonstration_ of security impact through vulnerability reports was special. The automation of “demonstration of impact” with AI isn’t that at all. The last mile is human and always was. This isn’t to say it won’t change in the future, but that’s a fact of where we are now.
Vulnerability reports aren’t special anymore. They never were. It was the impact, the demonstration, the communication that was special.
When you realize that this is being written from the perspective of someone who does vulnerability reporting in a professional capacity, you’ll connect the dots. We took care to be kind and succinct because for many of us, we learned our skills from being on the development side of things first.
Vulnerability reports aren’t special anymore. The only ones that felt special were the ones with human touch, the ones doing their job as an adversarial thinker, and taking the care to understand that net positive outcomes require coordination even if both parties don’t see eye to eye.
Nothing has changed. It never was. You’re just inundated with AI slop; which as a practitioner who uses AI regularly I can say with absolute confidence. The end result is the same, the volume is increased, but the special thing was never the report itself.
Finding a vulnerability was always the easy but high toil part. It was the care to communicate succinctly and be invested in the outcome that was special.
Godspeed.
I’ve been screaming this from the rooftops. Impact is what was always important. No one is going to take down prod to do an emergency patch on an RCE that COULD NEVER ACTUALLY BE EXPLOITED.
I feel like we’re witnessing the result of multiple roles suddenly becoming security aware but not having the background or understanding to make any sense of it.
One flipside to this is that, because many of these bugs are "shallow" to LLMs, it's actually easier than ever to moderate the worst participants in your vulnerability program -- if someone sends you slop, you can just ban them and wait for the next, better orchestrated LLM to send you a better report for the same vulnerability.
This of course made vulnerability researchers seethe worse than aggrieved Redditors.
It turns out he was right all along.
The author also gets it wrong by assuming that regular bug reporters are not "providing a service". They are.
When I wrote up a bug report, I made sure it's thorough with detailed steps to reproduce. It takes a lot of time and I've done it professionally for projects you've absolutely heard of.
Having said that, getting them ignored repeatedly and — even worse — having my detailed PRs rejected, sometimes within minutes, as if I'm some ignorant luser is why I don't do it anymore. My time is more valuable than your hubris.
A lot of open source developers have their heads so far up their own asses they forgot that it takes a community for projects to be successful.
It’s tough staying motivated on a craft when an AI is nearly as good as you. Chess players manage to do it at least.
The 5 on earth still getting paid to play chess?
but like, if you mean literally "someone gave them money and they played a game of chess", the number becomes much bigger. Chess coaches, streamers, club instructors, exhibition players, league players, camp counselors, and titled players receiving appearance fees, etc. All told, you're looking at ten's of thousands across the world.
It’s like most of art, writing, and sports. The only way to make money is by becoming a teacher.
I don't think the gift analogy works well. In most cultures, turning down or even ignoring a gift is considered anywhere from impolite to hugely offensive. But that's the opposite of open source: there's nothing wrong with requesting changes to a PR or even closing it.
Is this even a question? You triage and fix the vulnerability just like any other one. Are truths spoken by folks one dislikes — even for perfectly valid reasons — any less true?
The only way I can imagine this somehow applying is if someone has a habit of reporting vulnerabilities which do not exist, or of exaggerating their severity. Is crying wolf a CoC violation? If so, then I can imagine that particular sort of bad behaviour justifying some consideration before acting on a report.
No they are not. Everything else can be safely ignored. The author is suffering from AI psychosis and needs to get some help.