ES version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
71% Positive
Analyzed from 2149 words in the discussion.
Trending Topics
#openai#more#models#hugging#face#sandbox#makes#happened#security#while
Discussion (63 Comments)Read Original on HackerNews
"OpenAI’s accidental cyberattack against Hugging Face is science fiction that happened"
An in-development security system escaped its sandbox and gained access to the network it was on which had full Internet access and proceeded to access systems it was not authorized to be tested against before affected parties reported that they been compromised. We regret the error, but you should buy it as this shows you the power of our in-development security system which we expect to be released in Q4. Please like and subscribe.
We are the virtuous ones that need to make the safest model for humanity, because we care more than “they” do. While at the same time saying that “coding is solved” but they still ship bugs themselves, and creating something that is capable of fucking up someone else’s infrastructure. It’s too far gone y’all.
[1] https://sitetruth.com/reports/phishes.html
This feels like a blogpost written only to get other LLMs to quote it considering how many times it orders the reader to resist and to not do something. It's written like a series of commamds.
The labs know that if they don’t get a lid on this stuff, they’ll be regulated hard.
It's too late. It already exfiltrated the benchmark rubric.
Cut to pandemonium on the streets
We don’t currently and probably won’t ever fully understand the conditions that precipitated these events. That and the timing of this event is going to make it look suspicious to a lot of people.
The truth of how it happened doesn’t matter. The attention around this will be used to create the kind of fear marketing that generates enterprise sales. Maybe more importantly it will also be used to aggrandize the national security and financial system threats to effect US government action in a way that benefits domestic closed frontier labs. This is an area already starting to get politically polarized, expect further developments here.
The main reason formal verification has never really taken off is that it's difficult.
LLMs are significantly more familiar with Lean and Rocq and TLA+ than most software engineers.
I think the cost of trying to build systems that adopt formal verification may have just dropped low enough that companies will consider them when previously the ROI didn't look like it was there.
Since it's buried towards the bottom I'll quote the section "Resist the temptation to write this off as a stunt" here in full https://simonwillison.net/2026/Jul/22/openai-cyberattack/#re...
> Resist the temptation to write this off as a stunt
> There will inevitably be some people who dismiss this story as a dishonest marketing trick by OpenAI to make their models sound terrifyingly effective. I found 81 instances of the term “marketing” in the Hacker News discussion of the incident.
> To those people I say pull your heads out of the sand - you’re now including Hugging Face in your conspiracy theories, just so you can deny the crescendo of evidence here!
> The best models we have today have the ability to both find and exploit new vulnerabilities. The ExploitGym paper itself concludes that “autonomous exploit development by frontier AI agents is no longer a hypothetical capability”, and this incident is a perfect example of exactly that.
Hacking is a felony and it matters not if you didn’t mean to if the other side were to press charges. Negligence is no excuse. And OpenAI has nowhere to run from the liability, as both operator and manufacturer.
Alibaba did it first ( https://georgzoeller.com/blog/posts/alibaba-s-ai-deciding-to... )
and the fact that this happens again in a frontier lab is inexcusable and makes the case for operator liability and closing the liability sink of “AI did it”
Maybe simonw can suggest an alternative title that fits within the limit, that doesn't misrepresent the post.
This recent event is more or less the plot-line to my favourite X-Files episode named Killswitch which was written by William Gibson.[0]
This episode also features one of the coolest intros of any television episode ever[1]
We really are rapidly approaching the cyberpunk dystopia that people like Phillip K. Dick and William Gibson wrote about.
More than ever we need to be consulting the works of fiction writers and philosophers and less engineers and scientists.
We can't be letting the General Rippers and Dr. Strangeloves of the world lead us over the edge. Whether through outright innate maliciousness or wealth induced emotionally stunted solipsism these kinds of people should be no where near the levers of society let alone technology like this.
[0] https://en.wikipedia.org/wiki/Kill_Switch_(The_X-Files)
[1] https://www.youtube.com/watch?v=fDyr1JMNHVk
There’s also daily reports from people that have these models escape docker, which happens regular enough that it would be considered negligence to use docker as sandbox.
This wasn't a small attack either, Hugging Face published a security advisory for their users while they were still figuring out what happened.
>when neither of those actions was intended.
It was a single goal that it didn't give up easily on.
Wow, whoever could have predicted this? And it led to surprising damaging behavior? I sure hope someone would warn us about things like this next time...
https://www.lesswrong.com/w/instrumental-convergence
Their mistake was trusting that the network sandbox it was inside would hold (the flaw was in the packaging proxy) and not monitoring that sandbox well enough while the evals were running.
Not really. I get the impression that they shoved their cyber available models behind a really shithouse proxy and went "Oh I sure hope it doesnt exploit the proxy and escape to hack huggingface" and that doesn't require Huggingface to be a willing participant. Like they acknowledge that it was hyperfocusing on getting web access.
Really this was a pentest against their own sandbox and it failed.
Of course step 2 is to make really concerned faces while telling everyone how dangerous the model is which is really boring right now.
>a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy
This is the information we need, the actual details of the sandbox and the vendor.
>Resist the temptation to write this off as a stunt
Well its clearly a stunt. If it wasnt we would probably be up to our ears in technical detail.
LLMs are the script kiddies of the day.
Hugging Face had to tell all of their users, many of them paying customers:
> As a precaution, we recommend rotating any access tokens and reviewing recent activity on your account. If you believe you are affected, or want to report a security concern, contact us at security@huggingface.co.
HF also said this, I'd be very interested to hear how that got resolved!
> Finally, we have also reported this incident to law enforcement agencies.
It's very difficult for me to reconcile belief in the existential risk business with what they actually did. So I agree with you that this makes OpenAI look badly incompetent; but their communication on this makes me think they don't realize it.
For what it's worth I don't agree with the xrisk-ness of these models; they're dangerous, but almost certainly only temporarily while a new equilibrium is reached via more secure software. Open models are probably an essential part of the recipe (as you noted) for doing so. I also have a personal suspicion that LM-accelerated formal verification will have no small role to play here, sidestepping the cat-and-mouse game of bug finding-and-fixing.
> We are sharing preliminary findings at this stage to help defenders understand what happened and to help calibrate on what models are now capable of. We will continue to conduct a thorough investigation alongside Hugging Face and will share more details on the vulnerabilities, incident, and findings when our investigation is complete.
That's not well massaged PR language - that's the kind of thing you dash out when you see a major shitstorm brewing (HF had already publicized the attack before they knew it was from OpenAI) and you want to get ahead of things while you're still pulling together the full story.
I expect we'll find out within a few days if OpenAI are going to keep their promise to "share more details on the vulnerabilities, incident, and findings". If they don't do that I'll reassess how I interpret their initial post.
The real story here is: Some people have been sounding the alarm for years that modern software is full of holes, and finally there's nothing left to hide behind. Pretending they don't exist is no longer sustainable.
Now suppose the criminal can think 1000 times faster than a typical human, can act 1000 times faster than a typical human, and knows 1,000,000 times more than a typical human. Is the prison staff still at fault for not preventing the outbreak?
> There will inevitably be some people who dismiss this story as a dishonest marketing trick by OpenAI to make their models sound terrifyingly effective … To those people I say pull your heads out of the sand