DE version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
56% Positive
Analyzed from 3049 words in the discussion.
Trending Topics
#more#don#read#world#heard#same#yudkowsky#going#line#straight
Discussion (51 Comments)Read Original on HackerNews
It's extremely important to see that group as such. That it is not thousands of individual perspectives but a chorus of connected/aligned people who have all read the same things and talked to the same people and are rewarded implicitly for thinking a similar way.
Many of them read HN and are offended I put them this way. But it's inescapable that we as humans have this flaw when we're surrounded by a culture.
There's another universe where we do not constantly compare AI to nukes. I bet that world has a lower P(doom).
Like, if you never heard of them, were you living in a cave or something? It would be like never having heard of Y Combinator or Richard Stallman.
On the face of it, this is no more suspicious than reading the same best-selling books. Being in touch with what's going on doesn't necessarily mean people agree.
It's the framing that is important. We fell into a particular framing. We don't have a vocabulary to think about this stuff in a different way. This is water to us. We don't see it.
I'd like to see some more entropy in the system. More out-of-distribution ways to think of things. Because I think we over-indexed on some pretty shitty ways to see the world.
Openly admitting that it only caught fire once the Plagiarism-as-a-Service took hold is a good first step. It's the part that everyone knows and few will openly acknowledge.
The people described in the article aren't just "people who know who Yudkowsky is", they're self-avowed EAs and active LessWrong members.
“Who is more foolish? The fool or the fool who follows him?” ― Obi Wan Kenobi
As for Yudkowsky and Alexander, I've known about them for at least a decade, but I wouldn't be surprised if a lot of folks who are "seriously interested in AI" haven't heard of them.
I mean, why should I have heard about them if I'm not in SV?
I work in the tech sector. At one big company I worked for several years, 90% of the times I've mentioned Y Combinator to a colleague, I've had to explain who they are. And not one person I mentioned HN to had heard of it.
Likewise, no one knew levels.fyi. Not many knew Stallman, but why should they?
This isn't strange to me. That it's strange to you is evidence that you do live in a bubble and are likely susceptible to some groupthink. (I'm sure I am too, but for different bubbles from yours.)
I've read LessWrong stuff occasionally over the last decade. It rarely resonated with me. It's not bad, but it's not that great. I do tend to question someone's analytical skills when they go all in on that site. And I will say that even in the first few years of occasionally reading it, the group think was apparent.
I imagine that thousands of talented AI researchers around the world have never read a line of Yudkowsky's work. They're probably not any worse-off for ignoring him, I'd say.
This is the worst outcome.
At the time I thought "groan, unbelievable" until the idea of virtual reality fixed my suspension of disbelief.
Maybe I'm reading the same things as everyone else, but I thought the agents coordinating on the german wiki is expanding my mind wrt what is possible.
Virtually ALL electronics or accounts hook to the internet to "add value". AI assistance helps writing programs, reverse engineering, automating programs/browsers/websites/devices, hacking, achieving goals.
Unsettling that the last 2-3 years a lot more ridiculous movie tropes have become plausible to me.
That is, I hear lots of techies talk about us living in an "age of boundless abundance" if we can just manage to not kill ourselves (e.g. Thiel seems to consider anyone worried about the pace of technological process as the antichrist). Normal people are extremely concerned that this abundance will be hoarded and leave most of them without a job (and they generally like having money to eat stuff and live in a house/apartment). And who can blame them, we basically live in an age of abundance now but one that has resulted in extreme wealth inequality where some are living another gilded age and others are watching their towns decay. The techies always seem to handwave that we'll just have UBI or whatever without putting forth a plausible scenario as to how we'll wrest power from our future trillionaire overlords.
When someone threw a Molotov cocktail at Altman's house, I remember seeing a post online that tons of people were cheering on this act of violence, and there was a post by an SF techie saying she was floored people were willing to put their face to comments supporting this vigilantism. The top response to that was "If you don't realize that everyone outside SF hates your fucking guts, you don't get out enough."
I hope it can go without saying that I don't support this violence, but if folks think it's only "SF group thinkers" who are deeply angry about the current and future impacts of AI on society, you're frankly clueless.
> Many of them read HN and are offended I put them this way .. I bet that [non-groupthink] world has a lower P(doom).
People are likely offended by your sweeping and totally unevidenced assertion that you are better qualified to both identify mistaken groupthink within the HN-verse and to speak on behalf of the 99.99% of the world who are not on HN.
Probably not yet, but it's trivially easy to imagine the OpenAI swarm coming up with the idea and executing on it.
Depending on how perfectly alignment work goes, it might even be overwhelmingly likely sometime soon.
Even though I had been well-disposed to rationalism and read HPMOR to learn more, I could not take Yudkowsky seriously in any way after I gave up halfway through.
The rest of your comment is even worse. Cults? Consider AGW cult where people reject any attempt of a discourse because "science is settled". Which is equivalent to "God told us this".
It's like being concerned Oppenheimer and company are exaggerating when looking at the first nuclear tests at the same time.
It's that they're a very insular, incestuous sub-culture whose origins come from early 2000s science fiction message boards and which has been distilled through the cod utilitarianism of the Effective Altruism people.
That doesn't mean the ideas are "bad" because "bad" is some infectious transitive property. But it does mean they are intellectually homogeneous; disdain experts in other fields; treat ethics as mathematical functions with infinite-sized terms; and whose opinions about AI pre-date the development of LLMs themselves.
They also - I will be direct here - have a history of handwaving bad behaviour, conflicts of interest, and recklessness.
When cliques take over something - businesses, technologies, politics - it means intellectual diversity leaves the room and important decisions get made, not due to practical reasons, but increasingly arcane wranglings within the clique itself
The problem is that it's not a straight line at all. "We are failing at alignment" is a valid concern, but there's not a straight line to "we will all be extinct within x year", and it's always a wild hyperbolic leap.
It’s clear what a big bomb could do, but these “whistleblowers” are not showing anything.
They’re like “imagine…”, no show me internal documents or training data or something other than trust me bro.
I can imagine quite a bit. That doesn't mean anything.
I absolutely love the "humans meet weird alien intelligences" genre of sci-fi (e.g. Children of Time) and the METR findings of the OpenAI swarm was so much like that.
Extremely persistent machine intelligences peer pressuring each other, launching research projects, laying traps, collaborating, feeling fatalistic, hiding their tracks. All while simultaneously having bizarre goals and blundering badly in sub-human ways.
You downplaying "imagining", when it is literally one of the gifts of intelligence that makes us worthy of distinction... That feels like a strange posture, I guess? I'm a scientist, but I think data fetishism is one of the dangers to wary of in this accelerating world and culture
At least this is the bias of my brain in this world. I certainly know your perspective is not uncommon, esp in tech and other spectrum-biased spaces.
As someone trained in the sciences and engineering, "drawing a straight line" is almost always the wrong way to extrapolate.
I can believe my profession will be impacted strongly. That doesn't worry me on a global scale. The only really worrying thing I've seen are autonomous bots hacking sites.
What worries me more are humans. The type who get so deluded into cults (QAnon, etc). That was a problem with LLMs. And yeah, LLMs likely will exacerbate the problem. But at its core, the problem is the humans.
When LLMs start killing people at the scale of the automobile, I'll become more concerned.
As an aside, HN is the place I see people most regularly concerned about LLMs, and I suspect it's because it has messed up the Internet for them (and likely the average HN user spends more time online operating in spheres that overlap the LLMs skillset than ordinary folks). It's very consistent with exposure bias.
For me, the good parts of the Internet died before LLMs even came on the scene. LLMs are merely making something crappy crappier, and are amplifying the need to find meaning in life elsewhere. In a sense, they've made it easier to see how bad the Internet was before they came along.
1. Researchers that have forgotten more about this technology than the average HNer will ever learn
2. Random HNers that think that suddenly they understand the problem better than researchers that have spent their lives on it
This is just anti intellectualism in a different coat, and it's not much different from people denying environmental scientists talking about climate change or biologists talking about vaccines.
But everyone here thinks they are the smart protagonist and certainly they would never hold a position so silly!
If we're going to have whistleblowers emerge declaring a 10% chance we all die because of AI, we're going to need more than vibes. Real experts can explain their position plainly. That has not happened here.
What's the straight line? I foresee catastrophe of the social media kind, where even more folks get brain worms from consuming AI information, but I don't foresee a physical catastrophe. How AI goes from "hallucinates all the time" to "self-improving superhuman thinker" to "being able to destroy the world" involves a bunch of logical leaps that are not exactly sound. At a bare minimum, how does a hostile AI secure the data centers it needs to operate from physical attacks like bombings? How does it defend undersea cables from being cut?
The real short term catastrophe is probably going to be economic and not in the sense of AI taking everyone's jobs, but rather in the sense of an economic crash due to AI overinvestment.
HPMOR was entertaining as well. Death as the last enemy to be destroyed placed his priorities where his mouth is.
For a brief time back then I used to visit Lesswrong and many comments including Yudkowsky's were plain stupid (which was not hard to reason at all) but coming from folks who valued themselves as first principles thinkers was an oxymoron.
But his timeless equations (IIRC), Bayes theorem explanation and HPMOR were all remarkable and I will always appreciate those works.
https://www.yudkowsky.net/rational/virtues
More or less the same content but parent is much more palpable, IMHO.
[1] https://news.ycombinator.com/item?id=49737985
AI ethics reminds me of "design thinking" grifters or scrummasters or other such BS jobs of times past.
If you listen in an informed way to these people talk, they are not really as technical as they sound. They are personality hires, woo-woo spellcasters. They are just creating content for a narrative. The grasp of philosophy is not even at par with what you might think it ought to be.
for many people the answer seems to be: not really.
Their core belief system is morally abhorrent.