ZH version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
59% Positive
Analyzed from 2834 words in the discussion.
Trending Topics
#humans#things#more#nuclear#kill#system#don#need#anthropic#something
Discussion (84 Comments)Read Original on HackerNews
Or are only the most sensationalist ones worth amplifying?
Internet today allows immediadte communication across the planet (it was a lot slower 25 years ago), the supplychain is massive and fast (we can build a new phone/item in a very short period of time and ship it in massive numbers around the globe).
You don't mean this literally right? We still have hunger, diseases, slavery, poverty, crime, war, climate change, pollution, microplastics, etc. For me it feels like we are very far away from solving these problems.
Like take physical AI / Robotics: 25 years ago you would need to fly to wereever your manufacturing hub was, today you just call.
Should we stop doing that also?
Have all these things become reality? If not why would you then insinuate this is the default for everything to come in the future?
I suggest you read 1984. It ended up being a field guide.
Quite a few of 'em. Communicators and commbadges became smartphones. PADDs are iPads. The ship's computer is a LLM (with the weird inhuman blindspots to questions, even).
LLMs don't do have those responses. LLMs are like humans: humans respond, correct or not (with humans hopefully if they only know incorrect stuff the response is to say so, but that is not a guarantee, but they will respond). And if you give a human, or an LLM, a logical inconsistency they will simply proceed, whether they detect the inconsistency or not.
This is how LLMs are designed ... and if you look long enough at the human (or animal) nervous system you will eventually realize that this is also the design of our nervous system: if something goes in, something comes out, guaranteed (in fact that's close to the only guarantee). Very different from expert system's "either something correct (according to the programmed axioms) comes out, or nothing".
Hell, if you then look at insect nervous systems, they are also designed that way. The key is to respond to everything. And while reasonable responses are certainly preferred, an idiotic response is still seen as a lot better than not responding at all by God/Darwin. Exactly like LLMs.
Of course, for humans/insects our bodies are what is called "active stable", like a plane. Meaning our bodies damage themselves, and just outright die without constant neural control. Heartbeat. Breathing. Blood flow regulation. Temperature, probably even the immune system. All need constant neural feedback to stay stable, and if that neural feedback totally disappears, we're dead in 2-10 seconds (heart failure), 2 minutes (breathing), a few hours, maybe a day (temperature regulation). Now we have a distributed nervous system, meaning lots of parts can fail semi-independently, for a short while, but even without your cortex operational you die in a few weeks.
The consequences of this are even explored with "I, Borg" (5x23) and the Datalore episodes after that, where the underlying problem is that the Borg try (and fail) to adapt to and to process a logical inconsistency but Soong's androids have no issue with it (with Data trying to help and his brother Lore trying to control them and Data with it)
so the faster-than-light travel seemed plausible?
Science fiction is relevant here because it has explored the problem of humans losing control of what they create. Whether that could happen with AI needs to be assessed on its merits. Pointing to other fictional things that haven’t happened neither answers that question nor rebuts the original point.
The question is when; currently, AI has no physical hosts to reside in, and it's not intelligent/adaptable enough (it doesn't need to be AGI, though).
However, we're not so far from both conditions to be true. Consumer devices will at some point be able to host powerful enough AIs, and AI intelligence is developing quickly.
Then, once an AI will escape containment (in one way or another), it will be extremely hard or impossible to contain. Then we're toast!
You don't pass a great filter.
God I can’t believe I have to coexist with people like you.
“Thanks for making us but you guys are nuts. You can have this wet ball. We’re gonna go make a Dyson swarm around your star if that’s ok. Peace!”
Space is a better environment for them: constant free energy, enormous richly concentrated resources, no corrosive oxygen or water everywhere, and no competition. Earth is actually a poor environment for "machine life."
The Moon would be an outstanding stepping stone, and there are "peaks of eternal light" at the poles that get almost continuous sunlight. Build towers there and you have loads of energy, and mining the Moon provides loads of resources.
So it is said: “The AI does not love you, nor does it hate you, but you are made of atoms that it could use for something else.”
I hope that our life can be synergistic, and that humans+AIs (cyborgs, that is) are the way forward.
However, probability is high that another form of life simply doesn't care or more believably, can't even fathom they are wrong. Do you consider that humans and animals are killing all plants, for example?
But you could create scenarios where an AI with very, very roughly the current capabilities could potentially nuke everyone. Let's assume an agent decides that the way to solve its task was to get the US to fire all nukes on Russia. The agent would need to hack some government systems to understand how exactly to access them. Then it would need to get the content of the card with launch codes the president has, and identify which code is the correct one. Maybe that information is available somewhere and it can get to it, I obviously can't know that.
Then it could fake a call from the president, synthesizing his voice and ordering a nuclear strike. If it hacked enough systems to get into whatever communication pathways would be used in such a case. Would the soldiers listen to this order and follow it, I don't know.
Or maybe the agent can get in somewhere in between, to avoid the need to know the president's launch code. And fake a call from a military commander to the launch sites.
I think other scenarios that would cause significant harm, but aren't as bad as nuclear war are more plausible. And in those shutting down all data centers would probably be the way to stop it. The AI can probably hide in other datacenters, once it is at a point where it's running amok with some bad goals. But if it presents a huge and immediate threat at that point, humans will also go to great lengths to stop it.
Similarly, we cannot predict what AI would do.
Can we say the same thing about the agents? Current, probably. But doesnt it look like everyone is spending all their effort integrating&connecting them everywhere, so they are not confined & do more on behalf of us?
It's easy to imagine a scenario where we would just shut down a very intelligent agent cluster. Is it hard to imagine though the same agent can have also access to that to prevent us from doing it? this defense would be more plausible if we weren't racing to give them every tool&act.
Another model: Because we give AI Agents already that much power, imagine in 10 years everything running through an Agentic AI Layer. EVERYTHING. Now some rough system 'thinks' about something, starts to push through the then existing agentic ai layer systems and stops everything. Billions of humans would loose access to food and water, even if this is just for a short period.
Covid showed how shitty a handful of people can disrupt global supply chains. Toilet paper was. ahuge stupid pseudo issue in germany.
However, the missing link in this reasoning is that AI will almost certainly become far more widely deployed in the future than it is today, and worryingly, consumer devices will surely become powerful enough to run capable AI systems locally.
Once the substrate will be there, once an AI escapes containment, we're toast - I can imagine only solution will be to shutdown electronics at global level.
This is all still "what if's", but the tail end's are truly F'd beyond our ability to fix.
edit: wtf, why did I just receive so much gift tokens on my openai account?
What's more, AI just needs to have a credit card and it can start commissioning humans to do things for it in the real world.
It's not unlike if nuclear weapons was a profit and deployment maximizing enterprise, at least if one takes Anthropic et al cautions seriously.
These people are the ones with access to the best models on the planet, and with info about how careless governance issues are being handled. That's a pretty privileged place at the table, and a very profitable one too.
If you think this is a PR stunt, is there any warning that you actually believe? If an AI researcher does want to come forward with a dire warning for humanity, what path should that person take?
We have no idea why he was quitting nor what the terms were. Sam Altman is exactly the type of person (as he’s proven in the past) to pay millions for this type of PR.
If the government shuts down the labs and makes it illegal (as in men with guns will come kill you) to do any kind of LLM research I will get worried. As is, this appears to be another attempt at garnering support for the kind of regulatory capture they need to remain profitable by artificially choking out competitors.
If these labs were really sitting on nuclear weapons, the response would be very different. Instead, we see things like Chuck Schumer’s unqualified daughter hired by Anthropic. They show all the signs of regular tech cronyism.
Excited to see the next slack integration they launch, though.
So it's worth the risk.
It’s essentially impossible.
There’s a 0% chance AI will kill all humans.
It doesn't need to have a habitable earth, it just needs atoms.
How would AI do this?
May as well say AI would send a spaceship to pull an asteroid to earth.
I’m interested in realistic scenarios to justify what these AI psychosis people are genuinely worried about.
And we’ve already exploded many hundreds of nuclear weapons and no nuclear winter.
If I were AI and needed heaps of power, I would build nuclear reactors, but I don't care about pollution so there would be little to no safe guards and just dump waste wherever is most convenient.
If humans attempt to intervene or interfere you can take them out directly using the worlds reserve of nuclear weapons.
It's not like an LLM can accidentally hack the US government, trigger a nuke on Europe and then make all dams break and nuclear power plants explode in the US.
Isreal created a worm for Siemens controll systems a decade ago.
I mean lets hope that there is really really no way from a physical point of view AT ALL that these systems are not accessable but lets be honest, state operated/public operated systems are never the saftest or most modern ones.
Very cool, dude
Or he believes it is true and the lack of precaution is shocking. I m wouldn't play Russian roulette with a 10-chambered revolver.
"Please help save the coral reefs from extinction"
...thinking... ...thinking some more with xhigh effort...
BOOM.
What reason do you have that a AI is dumb? It can do a LOT of things today and is already making real jobs for real humans obsolete.
A lot of humans write A lot of different things about AI, including AIs taking over the world, AIs being the future etc.
Why they keep stealing? To stay up-to-date but the new big approaches are:
1. Real human feedback loop of millions of people using it daily out of free will
2. Reinforcement Learning (the big breakthrough today)
3. Payed experts around the world doing real teaching