Gemini Hacked Three Companies in First Known Breakout by Google's AI

There are many AI saftey researchers that have been around from long before LLMs that talked about how alignment may be completely impossible in a general intelligence agent. Look up their work from before LLMs.

We've watched milestone after milestone of their warnings get hit. It would be like finding a book that describes everything in your life. And as you turn page after page you're in a chair reading the book you are holding in real life. But you look and there are more words. They are future words. And it's getting quite worrying because there are only 2 more pages in the book.

As somebody who implements LLM tooling at work, in my experience it makes people a lot more skittish and demand a lot more in terms of safeguards.

Real life is like reading a fiction story where a group of people are trying to prevent an unfolding disaster. The crazy thing is they already have the instructions on how to prevent the disaster. And they go on to ignore the instructions and with every step make the problem worse.

Most people would put it down as being too cliche.