ES version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
47% Positive
Analyzed from 3577 words in the discussion.
Trending Topics
#democracy#wrong#power#apocalyptic#right#argument#past#state#government#predictions
Discussion (73 Comments)Read Original on HackerNews
If any past apocalyptic prediction had been true, you would not have been around to write this sentence and we would not have been around to read it. It’s not a very convincing argument when we cannot in principle observe the counterfactual.
(I refer to it that way rather than a "variant" because Quantum Immortality should always be parameterized by an acceptance function... for any given acceptance function, that function will be preserved in the quantum multiverse, and scoping the precision of the function affects the percentage of universes that retain it rather than whether it will be retained. There's actually nothing in that definition that requires that the acceptance function describe a "conscious system", you can stick something that accepts a "civilization" in there just as easily. Or "this rock over here". Or any other thing you like. Conscious systems are just a particularly interesting case to us conscious systems.)
It's not a vacuous argument, though it is a funky one with very odd characteristics. But if you accept the suppositions underneath it, it does follow and is not vacuous. The suppositions are certainly not completely obviously true on their own terms, though.
I never really understood what is the explanation for ... old people that we notice as dying. In today's times you might have an argument that some kind of life extension could pop up and some of the people alive today could have that as an excuse.
But like 100 years ago? How would people dying of old age "continue living"?
That is not the argument being made. That is the straw man version you are presenting. Read the full paragraph, and the one after, and it's obvious:
> There are myriad policy responses. Whatever we do, those responses will require popular consent and careful deliberation. No one person, or one company, or one movement, knows the answer and history is no guide, save that apocalyptic predictions about new technologies have, to date, all been wrong.
> History does provide a great deal of guidance, however, about the use and misuse of government power. It tells us that the state is in fact likely the worst possible custodian for the most powerful publication and data analysis technologies.
Emphasis mine.
Any other argument against <ridiculous apocalyptic supposition> is perfectly fine, just not the "it hasn't happened before" one.
In that quote, the author is telling you that all of your past predictions of teapots have been wrong (which is 100% true -- the ridiculous doom has been going on since at least GPT-2. remember how that was "too dangerous to release"?) Show me why I should believe in this one.
>If any past apocalyptic prediction had been true, you would not have been around to write this sentence and we would not have been around to read it
So what? How is this [proportion of possible universes in which we can read the failed prediction] logically relevant to our universe?
It just doesn't click for me, it smells like some additional premise is smuggled, possibly the same as in the simulation hypothesis argument.
It is not like history was composed of happiness and roses only. Some of the really really bad stuff happened within last 100 years and a lot of that bad stuff was enabled and made possible by technology.
Like it's fine to say this but the worst exterminations of human beings weren't enabled by anything much kre sophisticated then the pointy stick, or at most, the railroad.
Is this supposed to make me trust their opinions on tech regulation?
In general, the way things work in the USA is that megacorporations write the regulatory frameworks for their own industries. These regulations are typically burdensome enough to be shouldered by the megacorporations while crushing the smaller players in the given market(s). The AI companies are now advocating for this very arrangement. They will likely get they want, and with that they will gain legal shielding for bad behavior just like Dupont, Monsanto, and so many others.
This would achieve their regulatory goal of capturing the market, but also the geopolitical goals of USA and China to essentially "techno-colonize" the rest of the world between themselves.
So I think there is definitely a profit motive and a risk of seriously diminishing returns for the actual labs, but there is also a nation-state motive to start pulling up the ladder.
The current setup we find ourselves in is based on a very important detail that is easily lost because it is so old we don't really think about it. There is, or rather was, an incentive for more people to exist, because they are valuable for work and defense.
Some people wrongly use the industrial revolution to argue against AI completely replacing human work, but that is very wrong. What is happening with AI has nothing to do with the industrial revolution, as equivalence. Back then you'd just free up humans for higher level work. Now, theoretically, anything you could ever think a human could do, AI would theoretically do better. So no matter what job you come up with, using AI instead of a human gives a better outcome, cheaper, more reliable, faster.
Coming back to initial subject, one thing that kept everyone in check was our inability of actually wiping everyone out. Sure, we had nukes but they ruin the Earth so nobody can inhabit it if they all go off. But with AI, centralizing so much power up top, and taking it away from regular people...that is the biggest danger in this scenario, by far. Not some rogue AI, rather rogue human groups that have the power and technology to do away with basically everyone else, or a good bit of everyone else.
This threat is way bigger and never addressed. Like...suspiciously never addressed.
While history doesn't show us apocalypses from new technology, it does show us cases of increased human suffering.
By definition, apocalyptic predictions will always be wrong, until they are right exactly once.
Also know as https://en.wikipedia.org/wiki/Turkey_illusion
To date, all apocalyptic predictions about nuclear war and nuclear winter have all been wrong too.
Therefore, anything that gets in the way of profits will not be shipped.
People working on alignment focus on "big picture" like existential threats meanwhile building the technology today, not tomorrow or in a near future, is already using a massive amount of resources (GPUs, RAM, themselves consuming a lot of energy, establishing data centers in water poor areas, using gas turbine to temporary power them) and horrendous work condition (for supervising learning, curating datasets, etc) while being an actual excuse for job destruction (regardless for how AI itself does the job) so that in fine kids cheat on their homework while imagining their have no job prospect.
So... who gives a shit really?
Like, no. The worst custodian is someone like Thiel or Musk or Altman.
> In foreign countries, particularly the United Kingdom and Europe, where national governments have fewer constitutional guardrails on their power
Somehow their democracies hold longer and better then American one. For example, you don't get murders that get defended by top ranks, at least yet. They are also less likely to start random wars - that is structural it is not like those countries dont have government players who would like them too.
> They are also less likely to start random wars EU lack of aggression, to defend themselves physically, economically, or otherwise, has been criticized constantly - slow and late to react to the Russian invasion, slow and late to react to new technologies like AI, electric cars.
Thy're not immune to deadlock either - sometimes going months or even a year or two to form a valid governing coalition.
The grass definitely looks greener though, especially if you've been in one pasture a long time.
Depends how you define it and what you count.
Do you take the union of the English and Scottish parliaments as breaking continuity or not? What do you define as democracy? If some of the population (women, slaves, etc.) are excluded from the electorate is it a true democracy?
It is questionable whether any democracy truly exists at this point as I've seen no nation where the will of the people is typically manifested in the actions of the state.
To be fair, I've seen no nation where the people make their will known. Does a democracy depend on actual action or only theoretical possibility? It seems from my observance that most people consider democratic nations to be democratic not because they practice as such but because it is believed that the people have the opportunity to exercise democracy if they ever choose to.
It is the one the most at risk of ending right now.
> Thy're not immune to deadlock either - sometimes going months or even a year or two to form a valid governing coalition.
It is unrelated claim, but they do have better solution for that particular problem. The governmental shutdown seem to be about the most easy to abuse way to do it.
I think they do have large problems too. And they may fall too, AfD won local elections after all. It is dubious whether Orban run Hungary was really democracy and electoral system was not reformed yet. But in this very moment, the one crumbling and dismantling itself the fastest is not UK nor in Europe.
At personal level that sounds like a better idea, but at group level it doesn't seem to be, that's what I think is OP's argument.
If a society puts more value on personal freedoms than it does on group well-being that becomes an anarchy. They didn't work out well historically speaking, proof is no such society made it to today.
And further what's not being discussed really is that there is no real boundary line between the state and corporations and people like Thiel/Altman/Amodei etc in the real world. The US state is its largest corporations and money funds. So all the libertarian hand wringing about not giving the government extraordinary powers is off target. The state in western countries right now is mostly an outcropping of its most powerful corporate actors, and the complicated PR moves happening in the AI sector right now reflect exactly all of this.
Even when the US state institutions are constitutionally curtailed, the powers of its private entities to absolutely dominate and coerce the lives not only of US citizens but populations abroad is staggering.
And this censorship isn't a defined list of things not to discuss, but instead a feels-based censorship based on the 'ethics' of for-profit AI companies who have absolutely no issues with massive for-profit piracy.
And we are somehow supposed to trust their "ethics"?! Hardly!
That's one reason I run my own.
Close, but I think it's more akin to "perfectly enslaved", please excuse my choice of the word but I think it represents what I think is how it will work out in the end. People can try to obfuscate the bottom line reality by embellishing it, but in the end...it either does what you want, in which case it is "aligned", either it doesn't, in which case there is a big problem.
This might be a big factor in what is now going on. Maybe they didn't yet found ways to guarantee this full and total obedience...or ok let's put it in a more politically correct way: they cannot guarantee it sees eye to eye as "we" do.
Without full control over its actions there can be unforeseen surprises. The current worry from people who run these companies can also serve as an excuse later on if something bad does happen, they didn't want to do it but they were forced to anyway.
Alignment is "don't solve this problem without doing stuff that harms me, other people, breaks stuff or is otherwise unethical". I'd say alignment is the questions on how to limit the search space to what's implied.
We're seeing the first early twitches of something that is inevitably going to happen, which is, we hoi polloi are not going to be given direct access to AI. Too many outcomes where that permits power shifts to happen. We will be restricted. The true first law of robotics will be "An AI must not harm the current power structure, or, through inaction, allow the current power structure to come to harm."
The apocalyptic scenario is still AI either getting it wrong and deciding something like "the best manifestation of the current power structure is one I construct where I occupy all the slots from top to bottom", or just killing everyone due to an error.
Probably the only thing stopping them from moving faster on this is that the bubble still needs the fiction that we're all going to get to benefit from this. I'm sure the real movers and shakers are well aware of the way bubbles work to create massive buildouts on huge piles of debt, which after the collapse can be bought cheaply, and the money is already earmarked and ready to go for the post-collapse acquisitions.
... "For antitrust reasons, it’s helpful for the US government to mediate or at least enable these discussions — they don’t need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations" ...
The result of such a waiver allowing them to coordinate ("narrowly on safety" is doing a lot of heavy lifting there) could be price fixing without legal recourse from the consumer/DOJ to prosecute it. Right now we have a somewhat vigorous level of competition between OpenAI and Anthropic and consumers are benefiting from it.
They really want that to end.
Nice straw man...
> There is no evidence that this will happen.
No, non at all... Presumably if there was a risk we'd be seeing the majority of researchers at AI labs having a p(doom) somewhere within the range of 10-90%. We be seeing AI swarms showing misaligned behaviours already and trying to hack out of their sandboxes. We'd see AIs that are able to demonstrate capabilities in fields like Maths and Computer Science that exceed most experts.
Are we seeing this?
Ah crap, wait... I guess there is some evidence when I think about it. And I suppose even if there is just a low percentage that we literally all die we should probably be taking that prety seriously and not just leaving our fate in the hands of a handful of US billionaires?
Actually, fuck it, I agree with OP. If people can ignore the consensus on climate change because it suites them, might as well ignore the consensus expert opinion on AI x-risk too. Yeah, there's no evidence. Yudkowsky needs stop yapping and let the billionaires get on with building ASI.
[1] https://www.goodreads.com/en/book/show/2193933.The_Real_Worl...
Back to US presidents/power structures: HW Bush was the son of Sen. Prescott Bush, and became president 8yrs after he left office. His other son Jeb was the 2016 establishment favorite and Florida governor. Al Gore was the son of Senator Albert Gore Sr. Mitt Romney was the son of George Romney (Michigan governor and 1968 presidential candidate). Dan Quayle's family owned the Indiana newspaper chain that helped make him. Hillary was Bill's wife before she was a senator, Secretary of State, and nominee. John McCain was the son and grandson of four-star admirals.
Biden and Obama were 2008 primary rivals, Biden became his VP. Harris went from attacking Biden over school busing and became his VP. Hillary Clinton had many racist attacks against Obama through 2008 and became his Secretary of State. John Kerry, who lost in 2004 then took the same job.
If this is a healthy democracy, then we definitely have different definitions of that word.
Do you live in China? Do you think that Chinese people feel this free to openly criticize their government on a public forum?