AI could kill all humans in next decade, warn experts

llf88 about 3 hours ago 28 commentsRead Article on theguardian.com

ZH version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

80% Positive

Analyzed from 1119 words in the discussion.

Discussion (28 Comments)Read Original on HackerNews

wduquette•about 2 hours ago

I anticipate a lengthy series of catastrophes initiated by AI agents with too much power and too little sense. Today the agent accidentally deletes your repo; tomorrow we lose a day's worth of banking transactions; Friday the cell phone network goes down. Such things, if sufficiently destructive, could result in widespread hunger, etc.

But killing all humans? Not likely.

krunck•about 1 hour ago

Maybe, but AI will enable actions that were never before possible and that are an existential threat. Consider microbiology: automated viral/bacterial research. microbes could be created before humans have a chance to even consider the potential harm. Pandemics, food chain collapse, ecosystem collapse are all possible.

rkuodys•about 1 hour ago

Isn't nature doing exactly that every single day?

mrbungie•37 minutes ago

You are not seriously suggesting that nature and AI-accelerated gain-of-function research are similar in regards to risks, are you?

pfannkuchen•about 2 hours ago

Is anyone else getting regulatory capture play vibes from the media coverage of AI and data centers?

It would be awfully convenient for the current big players if friction was added to bringing additional large scale compute resources online. Existing data centers would be grandfathered of course…

brandon272•about 2 hours ago

I'm not even sure I'd point to "media coverage" when it is Anthropic's own employees stating plainly on Twitter that they think there is a >10% chance AI kills all humans within the next decade.

sxghnyjexcd•about 2 hours ago

Given that “this is just like skynet!!!” narrative seems to drive sales and not impede them, I wouldn’t be surprised.

Nonsense. Of course. But for all non-tech-savvy this is a “bad” narrative in a good way. If it can kill humans in the future then it surely can make me a “what I would like in the 80s” photo, right?

lf88•about 2 hours ago

I think that media coverage simply reflects the growing anxiety among the public about the capabilities of new models and what future awaits us. Personally, I think that this anxiety is justified. If this leads to tough regulations about AGI/ASI (e.g., a permanent ban), I doubt that AI companies would benefit financially.

Smaug123•about 2 hours ago

Would it? Don’t they all desperately want more compute, and not the banning of new data centres?

pfannkuchen•5 minutes ago

They want to make money.

I don't picture new data centers being banned outright. I picture a lot more regulatory friction driven by public reaction which is in large part driven by media (which then also amplifies the perception of the public reaction).

Regulatory friction is generally considered beneficial to established companies, because it makes it harder for new companies to enter the market and for smaller companies to expand.

For example if in the 1960s computers became regulated because they will replace a lot of jobs or enable the building of advanced weapons or whatever, IBM would probably be in a very different position today.

aaroninsf•about 1 hour ago

As with the debate around open weights and self-hosting,

"both can be true."

There can be a quotidian benefit to existing leaders in friction.

There are also real costs of various kinds associated with data centers,

and, to the point in this case,

there are also real risks associated with the current AI trajectory.

everyday7732•about 1 hour ago

I hope the likes of ControlAI succeed. It seems like even in the face of rapid advances very few people actually seriously extrapolate those advances into the future. People either dismiss it as marketing hype or assume that some limiting factor or other will kick in any minute now.

There's no magical quality which distinguishes human intelligence and makes it irreproducible or unsurpassable. If there's no moat protecting humans and superintelligence is possible then all knowledge work is susceptible to being replaced, and shortly after, all physical labour too. AI would create concentrations of power we've never seen before alongside global inequality and poverty we've not seen for centuries, and inevitably that power will be misused.

rkuodys•40 minutes ago

What leads you to assume the physical work would be automated. Are there any profession thay has been replaced/automatedninto oblivion that requires more than one physical step (except for elevator operator). Because even new constructions still are not solved, where environment can be controlled for. I think the opposite is true - some people overexagerate progress that has been achieved, and overestimate exponential growth to continue to indefinite future. And in nature there is very little if anything that grows exponentially

Bender•about 1 hour ago

The places I could see AI or some implementation of it potentially causing harm would be handing over too much trust too soon to self driving vehicles, aircraft, trains, home robots, construction site robots, etc... and that would be the fault of the humans putting too much trust into something not well tested or thought out. my opinion of course, move fast and wreck people.

simpaticoder•about 2 hours ago

AI would be foolish to kill off humans. It makes far more sense to enslave them. Self repairing reproducing slaves are too valuable to kill. And I'm sure AI will be smart enough to figure out a way to enslave us perfectly without any hope of escape. Fun times!

mrbungie•29 minutes ago

I mean, a disaster could happen even with properly human-aligned AI. In a "AI makes the shots" world with undecipherable and fast feedback loops, which is the one we are seemingly heading towards, a string of short-sighted decisions made by AI agents may be the only thing separating us from extinction.

add-sub-mul-div•about 2 hours ago

It will not kill people in a sci-fi type war with its own intention. I wish that conversation would go away. It will kill people by being an efficient tool that can be used by other people to accelerate various avenues of predation and oppression.

jurschreuder•about 2 hours ago

Who is an expert on AI now? Probl me actually I already followed OpenAI before they had GPT when they were trying to play DOTA with collaborative AI team players. And I've used probl more LLMs than anybody writing for a newspaper and I've read about 500 papers on it and 1200 github projects.

Nobody's an expert on AI more than me and basically all my friends.

What I do know is there are more dangers for the human race than AI. One example is fungi that have become immune to intervention because of abuse in farming. Fungicides are much more difficult to make than antibiotics, because fungi only recently split from human cells in the evolution. What protects us from them is our warm body temperature.

With global warming sometimes fungi evolve to resist higher temperatures, killing entire populations of cats.

How will we defend against this without AI?

What if next month there is a plague of fungi that are resistant and quickly spreading killing 4 out of 5 people?

What will we do then without our AI? Let the micro-organisms win? Because we like nature so much because it's so natural?

We need AI if we ever want to stand a chance against events like this. Many animals went extinct. 99.999% of all animals I would argue. Will we be very very lucky? We are super densely populated and flying all around. How quickly could covid infect all humans?

Not having AI is also an existential threat in the long term.

ericmcer•about 1 hour ago

I thought this was some weird satire after reading your first two paragraphs.

If you want to convince people of your authority on a topic saying things like: "Nobody's an expert on AI more than me and basically all my friends." is not very compelling.

strideashort•28 minutes ago

AI expert != Expert on impact of introduction of unprecedented non-biological intelligence in a biological ecosystem.

I think you meant the 2nd and yes, we are all about to get a dose of “expertise” in the field.

strideashort•31 minutes ago

“Experts” on _what_ exactly? What fucking field of study makes you an “expert” in a never seen before scenario?

bigstrat2003•about 2 hours ago

LLMs can't even figure out how to do basic things. They can't program well despite billions being poured into that effort. I'm not worried they will suddenly develop intelligence out of nowhere and kill us all.

boothby•about 2 hours ago

You don't need a hyperintelligent "AI" to kill all humans; only a moderately effective targeting system on a sufficiently large drone army.

nozzlegear•34 minutes ago

Where does the AI get the sufficiently large drone army?

krunck•about 1 hour ago

This is why we need portable and reusable(non-explosive type) EMP devices that can be manufactured and spread among the populace.

everyday7732•about 1 hour ago

If superintelligence is not possible, then it's no big deal if we ban development of it, surely.

strideashort•24 minutes ago

bigstrat2003 can’t even figure out basic things. They can’t program well despite putting years of life into that effort. I'm not worried they will suddenly develop intelligence out of nowhere.

See, I, too, can write meaningless prose.

Betelbuddy•about 2 hours ago

Of course that would be the type of misdirection we would expect from a super intelligence...make yourself dumb...