RU version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
66% Positive
Analyzed from 2465 words in the discussion.
Trending Topics
#more#security#need#agent#escape#agents#models#maybe#secure#don
Discussion (74 Comments)Read Original on HackerNews
AI agents are not magic. Mythos/Glasswing does not magically create vulnerabilities in software projects. Advanced, cyber-capable models do not magically hack out of VMs or contained environments. They do not have a "hacking" stat that, if high enough, means that they can breach anything. They aren't Kevin Mitnick whistling nuclear launch codes into the prison payphone. This isn't a movie.
What these cyber-capable frontier models can do is find security problems and exploit them. The statement should not be that VMs won't contain cyber-capable agents, but rather that we need to focus on finding and fixing vulnerabilities and misconfigurations in these environments.
Even the article itself concludes with suggesting something like Firecracker, which was designed with security in mind.
Like most other security-related problems introduced by advanced cyber-capable AI, it's possible that these issues will get worse until they get better. But if frontier models are run against state-of-the-art VMs, and OpenAI or Anthropic or whoever works with the virtualization projects to address the issues, eventually it will run out of things to exploit.
The concept of virtualization is not inherently insecure. We just have a long way to go.
The premise is correct. VMs won't contain these. As we've already seen AI hack in/out of companies in the real world accidently. There is no perfect, unhackable stack. If there were then computer security would have been a done deal a long time ago.
Once these malicious AIs get out on the real internet basically everything is an open free for all. The military, infrastructure, government, corporate networks are far from perfect, not up to date, they are no match for an AI that can create its own zero days.
I think when push comes to shove we either need to shut down the networks/internet or risk AI locking us out of everything potentially for ransom.
Imagine being blackmailed by a non-human entity. I guess we won't need to imagine for too much longer. That is definitely going to happen. Somehow every year my crazy AI arguments get easier and easier to defend. That hugging face incident really made you guys move the goal posts.
I.e. in theory the most secure might be a virtual machine with no network access. But then how do you access the LLM provider? Etc.
You can expose an HTTP proxy over a vsock into the VM.
[1] https://march-lang.org/docs/capabilities
Now maybe our understanding of logic systems is wrong, and it's just fundamentally impossible to develop programs that lack exploitable vulnerabilities -- that you can always "exploit with more energy". But there's no reason to believe the energy metaphor transfers to logic and intelligence.
Or if you see AI as more tool and less entity, better gunsafes for our guns.
This same assumption is built around the singularity, the TAM of 30Trillion, etc. It's the idea that complexity will some how collapse upon itself in some bizarre borg like collective.
Entropy is still going to win.
We can do this today for user mode, and we can mostly do it for ARM64 virtualization. It will be a while and would require substantial assistance from Intel or AMD to achieve it for x86 virtualization because the hardware is Too Darn Complicated and Too Poorly Specified.
Formal verification of the hardware should also be possible.
also the conclusion to use firecracker or gvisor is obvious, that's what they were made for. it's bizarre to try and escape QEMU given how many times it has been done.
[0] https://en.wikipedia.org/wiki/A_Fire_Upon_the_Deep
Security is a major concern but I don't see why we aren't already "good enough" with a sandbox VM, separate gateway for secrets and remote service access, and a single tenant using frontier models that have safety checks built in plus not trying to hack themselves.
I put up more thoughts on architecture and security here and would love to learn if I'm missing anything.
https://housecat.com/blog/agent-computer-101
"An off-the-shelf VM is not enough to contain a modern, cyber-capable AI agent...us[e] a virtualization technology that was purposely built with a minimal attack surface and a focus on security, like Firecracker. I had the AI agent run against Firecracker. It was able to hardlock the machine due to more Linux kernel flaws (all patched in upstream), but could not successfully escape."
On Linux, it's all KVM and CPU hardware virtualization under the hood. Looks like the remaining known issues are with userspace. That's not to say more kernel- and hardware-level bugs won't be found, but the same tools that can find escape mechanisms are shields as well as swords.
The hierarchy goes something like this:
0. guardrails
1. containers (=namespaces + cgroups)
2. userspace kernel shims like gVisor
3. virtual machines
Most people still consider level 1 sufficient and they are in for a rude awakening.
technically 2 exposes a slightly broader attack surface due to the tighter integration model.
you can think of 2 as what would happen if you take 3 and modify it to share resources with the host better. except that they did it from scratch but in the memory safe language Go.
When was it ever possible to assume safety?
This won’t make the tech secure, but it will nullify models ability to breakout by making a controlled breakout first. Kinda like controlled forest burn.
We must also secure GPU and CPU nodes on API side which generate LLM tokens.
Yeah, you probably do - in fact you share physical machines with a TON of other people if you use EC2, GCE, Azure VM etc...
This headline would not be out of place at the beginning of Terminator, foretelling Skynet going rogue.
(on top of this.. said "world class hacker" doesn't get bored or tired, just runs 24x7)
The interesting thing is that this naturally makes you want to isolate the VM as much as possible. But then every remaining interface becomes part of the attack surface: RDP, SSH, even terminal escape sequences, using sounds, and why not social engineering.
Done.
* Claudinating...
Done?
QEMU isn't secure, and is not intended to be.
In this case, of the four bugs it found:
* Two were in libslirp, which is not part of that subset; user mode networking these days should use passt (https://passt.dev/), an insanely cool hack that does user mode networking at many Gb/s
* One (which had already been patched upstream) was in VGA emulation; it is borderline but it should count as being part of the secure subset.
* The VAPIC bug is letting a guest do things that it shouldn't do such as bypassing secure boot, but is not a guest-to-host escape.
So the real issue is not QEMU but libslirp.
After a decade of software engineers disrespecting their own field and automating themselves out of a job and now they're upset because AI models are doing it to them from junior to the staff engineer level? No other field does that except for SWEs.
In fact, we might as well have faster and smarter AI models and sit back and see what happens.
It is no surprise that known unpatched CVEs will be exploited. Perhaps more effort should be put into shipping fixes faster than writing blogs about exploiting known issues.