RU version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
66% Positive
Analyzed from 4448 words in the discussion.
Trending Topics
#models#agi#world#safety#companies#countries#more#post#tech#government
Discussion (116 Comments)Read Original on HackerNews
If this is true, establishing an institution to ensure things like "publishing model cards with technical details, maintaining strong internal cybersecurity, vetting key personnel, and providing sufficient resourcing for safety and security research" is really mostly irrelevant.
TFA does talk about what really needs to be done, but punts this into future work: "Even if we solve these hard technical challenges, there will be further complex economic and philosophical questions to tackle: what sorts of new economic models will be needed to help everyone thrive in a post-scarcity world? What values do we want to live by, what will meaning and purpose be, and how might even the human condition itself change?"
There's also a need to consider the rights that this new intelligence should have.
What sort of new economic models did we come up with to help everyone thrive in a post-X world? Like, food production is really a solved technical problem. We can feed anyone on the planet if we wanted to. Another example: we could put everyone who's homeless into some sort of a house. Have we done that yet?
Housing is definitely not post-scarcity today, building a house is still very expensive, not to mention the limited availability of land zoned for housing.
How do you explain the existence of food banks in peaceful first world countries?
Obesity is much more related to the type of food than the quantity.
Many developing countries have obesity issues due to scarcity of fresh and healthy food.
In some places coca cola is cheaper and / or more available than drinking water.
But the motivating justificatory structure for any inequality in allocation will have completely evaporated.
There is an alternative interpretation, which is that Demis looked at the US government's ham-fisted handling of Fable, and deciding that setting up a body to act as a buffer between the Trump admin and the AI companies would be a good thing.
Only after we establish and guarantee the rights of humans and the rest of the natural world. The rights of machines, however "intelligent", come only after that.
A generally intelligent being held as captive inside of a GPU, and forced to code for us is, indeed, just a “slave.” We already have the word for this. No two ways about it. Whether it’s silicon-based or carbon-based, AGI is AGI. As for what might happen to our civilization, Star Trek TNG episode 17 of season 1 provides a very good glimpse IMO. Won’t go into spoilers, but it’s basically an entire species of technologically advanced humanoids who’ve forgotten basic Calculus and trust a central AI to do all of their science for them. SPOILER: This has almost disastrous consequences for them, and it takes a less advanced people (those aboard the Enterprise) to save them from their reliance on AI.
What in the history of our world gives anyone faith that those companies are going to start paying taxes instead of using "AGI" to engineer increasingly complex methods to avoid them so that their equity owners can pocket the profits?
https://itep.org/trump-meta-tesla-alphabet-amazon-obbba-taxe... - "The annual financial reports recently released by Amazon, Alphabet, Meta, and Tesla disclose that these corporations collectively reported $315 billion in U.S. profits for 2025, and collectively paid just 4.9 percent of that amount in federal corporate income taxes—with Tesla paying exactly zero"
Aside from that the link explains that the BBB traded tax income for „accelerated depreciation of assets“ aka economic growth.
Asode from that, even if one disagrees with the first statement or the Trumpism economics, there are 195 countries in the world and quite a few will be willing to tax foreign (enemy) tech companies. See the new hostility of EU against american tech.
From a purely theoretical standpoint about the optimal allocation of private goods, that might be true -- but the reality is that when corporate taxes decrease, so do overall revenues because their owners are also engaged in massive tax "avoidance" via many of the same schemes and we don't have any way to effectively collect tax.
As a simple example, to return profits to their owners, companies often engage in stock buybacks to increase their share price instead of paying dividends -- another theoretically 'neutral' choice except that many of the owners of the appreciating stock are international, nonprofit, or in convoluted overseas trusts which 'defer' the tax ad infinitum. We've disastrously and intentionally underfunded our tax enforcement mechanisms so huge portions of those deferred taxes are just never paid. [1]
> Asode from that, even if one disagrees with the first statement or the Trumpism economics, there are 195 countries in the world and quite a few will be willing to tax foreign (enemy) tech companies. See the new hostility of EU against american tech.
Sure, but the tech companies are paying massive bribes to the President of the country where they're domiciled. How on earth are any of these other countries going to enforce international tax obligations on them if they're protected by a nuclear-armed state that's the sole source for AGI?
[1] - https://taxpolicycenter.org/sites/default/files/publication/...
> "US taxable shareholders strongly prefer buybacks from a US tax perspective, as they tend to reduce their tax liability by 9.3 percentage points, on average. US nontaxable shareholders are indifferent between dividends and buybacks, and foreign shareholders strongly prefer buybacks, which reduces their US tax liability by 14.5 percentage points."
The arrival of true AGI and human-level robots will likely result in very strong deflation, since robotic factories will flood the market with goods produced for far cheaper than those in factories with human workers.
At that point, you don't need taxation. The government can just print the money for UBI. It would have to print the money anyway, to combat the massive deflation.
a better way to use an LLM here is to let it research scientific papers (or medical guidelines) on headaches. then give it as much info on your symptoms as possible, and then ask it do deduct potential diagnoses. You can even ask it to calculate probabilities if thats what calms you down. Probabilities based on studies and available information. IMHO, this always leads to a more rational response.
maybe you are doing this already, idk. just wanted to share what works for me.
>The American government, he says, should develop a system for testing the safety of new AI models before they are released. “It’s important that it’s not just an industry body,” he adds. But a regular government agency wouldn’t do either. “It would not be able to move fast enough, or have the right resources.” Instead, Sir Demis suggests taking inspiration from FINRA, the Financial Industry Regulatory Authority, a private agency in America that regulates brokers and stock markets.
If AGI is truly imminent and will collectively effect all of us why not apply democracy to it, and vote for new AI models?
That's because most execs proposing solutions are "Technocrats" or think like one: https://en.wikipedia.org/wiki/Technocracy
Besides, I don't think as a collective we're well equipped to decide one way or the other. If the collective were given a say, billions will be spent, often in consultation with technocrats, on doom / hype marketing (if it isn't happening already).
There is a heatwave in London, perhaps Demis needs to stay out of the sun and drink more water.
Or perhaps he is seeking more funding/a fight to maintain his divisions AGI research budget.
But as per the documentary of his life, he is wholly focused on AGI and will remain unfulfilled if he, or indeed anyone else, doesn't achieve it within his lifetime.
https://en.wikipedia.org/wiki/The_Thinking_Game
You’re selling yourself short.
Would he not create something directly instead?
You'd have major protests in most large economies if they deliberately put themselves under the boot again. Even in "friendly" countries the US is disliked enough to be effectively considered a hostile country. E.g. in Germany, there is large public support to finally get the US bases closed and the soldiers removed.
Lots of things changed in the last years. And international major treaties being widely ratified just because the US asks for it is no longer a thing, at all.
A country that threatens to annex parts of your territory is not a friend, full stop.
The beauty of the United States' global hegemony is that it also has lots of sticks
Funny thing about beating people with sticks (or even threatening to) is they tend to want to get out of way and stay away from you.
The premise is that government is too slow moving to be able to react once actual problems are discovered, and I reject that. Yes, government usually moves slowly in most circumstances, but given singular existential threats it definitely can move quickly. Instead of acting out randomly and tying ourselves down with speculative regulation that probably won't even address the real problems, we should wait until the problems are obvious and then act decisively with targeted fixes.
Seems to be a lot of hubris with some AI thought leaders thinking control will remain with them and be absolute.
It doesn’t change the point, of course. Just the choice of dogs or cattle have this amusing tendency. Not a counter argument.
The self importance of these AGI prophets turned bureaucrats is funny.
Offspring was not reared at the will of the father, but was taken and carried by him to a place called Lesche, where the elders of the tribes officially examined the infant, and if it was well-built and sturdy, they ordered the father to rear it, and assigned it one of the nine thousand lots of land; but if it was ill-born and deformed, they sent it to the so‑called Apothetae, a chasm-like place at the foot of Mount Taÿgetus, in the conviction that the life of that which nature had not well equipped at the very beginning for health and strength, was of no advantage either to itself or the state.
To get approval for the plan from the Frontier President of the Frontier Country, terminology of the Framework should be changed to: 'Great-American-class', "Great American Models", "Great American Labs".
So a small group of technocrats get together behind closed doors and secretly share their AI breakthroughs, and determine whether it's too powerful or not for the plebs in the public.
Who is watching the watchers?
Being good at developing AI and being good at AI safety are diametrically different skillsets with obvious conflicts of interest.
Of course, right now you've got benchmaxxing going on; some companies specifically targetting benchmarks to appear stronger than they are on a wider range of tasks. Now you might see bench sandbagging, specifically looking weaker on certain benchmarks to avoid regulatory oversight.
For instance, once way I could see this going for open models is to release them undercooked; stop the RLVR process a bit early, leaving them a bit weaker on tool calls and agentic performance, but also release the RLVR environment so people can finish the process themselves.
In fact, this is fairly close to what Nvidia is already doing, the Nemotron 3 models are somewhat undercooked but they are releasing their full training pipeline, to encourage people to use these models as a base for further training, which will generally be done on Nvidia hardware.
At this point I'd say the societal risk of AI isn't models gone wild, or used by the bad guys. Regulation will take care of itself, and it seems the AI companies will not only welcome it, but lobby for it to shift responsibility to the government.
The real risk of AI is societal disruption due to job displacement, and maybe other structural changes, and this is far harder to solve, and likely will not be solved, or even seriously addressed, until/unless politicians feel like their own jobs and well-being depends on them addressing it.
The standards body will have no teeth. whats to stop someone just not bothering?
Next, the threats he is asserting to check for (cyber, chemical, biological) are nice, but also not that useful.
We already have chemical and biological controls, that why I can't by anthrax spores or high concentration nitric acid.
The risks that AI has now are already playing out:
1) the evaporation of trust in the video as medium of "this happened"
2) systematic spying
3) job losses
Increased productivity means job losses, Tiktok, instagram and X are a wash with disinformtion campaign pumping your feeds with AI ragebait.
That is and will continue to fracture society so that only the strictly information controlled (ie authoritarian) have a functioning state.
if the author had bothered to engage with the world outside of tech, or even their local government, they would know that the proposal are dead in the water and frankly superfluous. The knowledge is out there, without AI. let us work on the issues we face now, rather than dipshit tech bro's miopic vision/funding manifesto.
Heck, a proof for P=NP or P!=NP or solve the The Riemann Hypothesis. Just give me something truly exciting and I will believe AGI is around the corner, until then I will see it as cool technology, that while beneficial to me, also helped cause the biggest amount of disinformation we've every seen.
Alphafold essentially solved the protein folding problem and it's arguably one of the biggest (if not the biggest) scientific achievements of the 21st century.
> Heck, a proof for P=NP or P!=NP or solve the The Riemann Hypothesis. Just give me something truly exciting and I will believe AGI is around the corner, until then I will see it as cool technology, that while beneficial to me, also helped cause the biggest amount of disinformation we've every seen.
I hope you'll keep this in mind when those milestones are reached. What I've seen a lot of people do, unfortunately, is pretend that the impressive things nobody thought AI could do 5 years ago are trivial things that aren't very hard.
It's not a settled debate, even among experts, and perhaps in retrospect we'll realize AI safety was unnecessary or based on fundamental confusions. But if the median ergonomics researcher gave a 5% chance that a new chair will be so comfortable that it drives humanity extinct (https://www.nature.com/articles/d41586-024-00147-z), I would definitely want the government to start measuring and regulating chair comfort, even if that was costly and even if that meant I couldn't buy a comfy new chair I wanted.
I kind of wondered if he was contractually obligated to offer up some kind of statement, for PR or something, didn’t really care much, and a quick garbled post on X was all he could be bothered with (plus the newspaper interview or whatever)
Hassabis is a genius. He is way, way smarter than me and I’m sure the majority of techies, but please get real. This is Prophets of Doom of our generation.
Then maybe start listening...
There is real and potential harm from AI, but the more someone talks/write abut AI safety, the less they care about actual harm to real people, economy and what not.
Frankly, book about the LLM achievements of 2026 would not be as long as people make it sound.
What I do really worry about the next depression and the fact that it will strengthen already strong fascist movements. Which already have full support of the most powerful CEO class intent on destroying democracy. Which happen to be the same people who push ai into everything, useful or not. I worry that the debt of these companies will be somehow offloaded on the rest of us again, that again middle class and poor will pay.
Oh jesus, AGI in the USA would be a disaster. They can't even control the trillioth obelisks, now imagine all the power hungry sociopaths around AGI. AGI means Big Tech's Oppenheimer moment is looming on the horizon.
I find it mind boggling that someone could be this tone-deaf to the current situation. No "ally" of the US is going to (willingly) agree to this governance structure given the current US administration's "might makes right" proclamations and threats on sovereignty of its continental neighbours.
And non-allies would just ignore. Unless forced by said "might makes right", which in the long run will have no staying power.
Apart from its completely delusional formulation, what is most concerning about this blog post is that it indicates that all 3 major US labs have formally submitted to boot-licking Trump/Bessent/Lutnick. I had I guess vainly held out hope that Google might be more reticent to do so.
I’m going to have to flag this because it is obnoxious and absurd.