ZH version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
64% Positive
Analyzed from 2231 words in the discussion.
Trending Topics
#models#don#more#model#down#anthropic#wall#better#slow#things
Discussion (80 Comments)Read Original on HackerNews
Clearly they aren't that scared.
If I hold a gun to your head and tell you to fork over your wallet, you will do it, because you sense imminent threat to life. Yet somehow we're to believe these people have the equivalent of a technological gun to their heads but are choosing to keep their wallets?
You give it instructions to do something, it goes off the rails and after a certain point starts having like malware, destroying things to accomplish its goal. Depending on what it destroys, it may kill humans.
With this said, I'm not exactly in disagreement with the view that AI companies with upcoming IPOs are using fearmongering to convince people that their models are very powerful and to also block others from competing with them.
Fable and Astra are what we currently call frontier models, but to be more specific they are generalist models, built in pursuit of AGI. The strategy is to have one single model that does everything, whether it's writing code or doing research, etc. Fable is a single, massively sized models that is intended to do specialist work across every domain.
The issue with this is first of all that it is the contradiction of a generalist doing specialist work, and that contradiction creates the present situation with model profiles that ensure that these models will rarely be chosen in a pool of models like V4.1 Flash that can now do GPT 5.4-level work.
We are seeing this reflected in the market where companies and individual developers are moving away from frontier models toward models with better cost profiles. In a sense, the market is killing Anthropic's dreams of AGI.
https://x.com/trydotworks/status/2098618997230985375?s=20
1. They hit a wall from the technical perspective 2. Inference costs are getting out of hand and newer models require significantly more resources for marginal gains, meaning nobody is going to buy those models 3. They can’t afford the hardware for further scaling
- subagents not having received those guidelines
- simply ignoring them because other tokens in the context outweighed them
LLMs are not dangerous. They are as dangerous as the tools they have to work with are dangerous. If you give an LLM a tool to play piano but that tool is physically connected to a machine gun, then it will happily play anything you want.
The stuff I can do with Astra that Sol couldn't do at all is wild and it's not even coding related. Dario et al are not hitting any sort of make believe wall. They're seeing how fast things are progressing and are legitimately alarmed.
Most are the same people who said LLMs will amount to nothing when GPT-2 came out. Some still argue it is all hype and dont understand we are on the cusp of a technological and social revolution.
This!
Their IPO is getting closer, but the Chinese models catching up will ruin it, so they want government to stop the competition.
Instead of shutting down at any discussion of hacking they need to be giving out free credits for hardening. That's going to make it easier to use these tools for hacking, because hacking requires hardening. But the alternative is huge numbers of intrusions, and a slowdown won't fix that, we already have far too much poorly secured stuff, and the models are way too good at exploiting obvious problems.
I also think they need to do a better job of safeguarding end-user privacy. I've seen some things with Claude that make me very concerned it's possible for Claude to hack my local network, then for one of their classifiers to trip, hide the log of how I was hacked from me, but beam all of the information about the hack back to Anthropic to use. And this is totally reasonable, they want to train their models not to hack. Except now details of how they have a foothold on my local network only exist in training data that they may never read but will use to train their models. This is very avoidable but Anthropic has to treat alignment with the end-user's goals as more important than treating the end-user as an adversary.
So... No. Even if he means what he's saying, it's irresponsible of us not to consider his possible motivations.
Which one is it, Dario?
[0]: https://www.anthropic.com/threat-intelligence-report-septemb...
[1] https://darioamodei.com/post/we-must-pace-the-frontier
Hypocrites hyping for the IPO.
PS: True story.
The discourse is a result of their behavior. Not the opposite.
Guys like Paul Graham are a rare breed, a Hobbit-like optimist living in the equivalent of Mordor. When the political headwinds are strong, that type of hopecore content finds the right people and motivates legitimate change. If we still lived in 2009, then yeah, this would be a bizarre reaction to a national-scale business saying that we need to organize for a greater purpose.
But two decades have passed. I'm not going to enumerate everything that happened, but the US lost a lot of international standing in that time. We saw unprecedented protectionism, app store censorship, crypto/NFT celebrity scams, national-scale bribery, American annexation threats against NATO members; 10 years ago this would all be called parody by HN. We all watched neoliberalism get the Gallagher watermelon treatment.
This HN is the older and callous one. It's not a comfortable status-quo, but blaming the skepticism reveals a shocking blindness on many people's behalf. The problems with Anthropic and OpenAI today isn't the possibility of RSI, it's their political commitment to corruption, lies, hysteria and debt. The closed-door research only exists to impose an artificial authority that wouldn't exist if oversight committees and responsible disclosure was enforced by a trustworthy government. None of that is "cynical" to highlight, it's an objective development in America's national AI story.
It will take decades of economic and social innovation to catch up with current model capabilities.
No upside is worth the potential downsides of getting this wrong, even if you set aside all the x-risk stuff.
https://news.ycombinator.com/newsguidelines.html
https://news.ycombinator.com/item?id=49676085
This is a really important topic (and a lot of vested interests involved), so I sent an email to hn@ycombinator.com but if it takes too long for any action to be taken there's no point, right?
(I'll delete this reply if I can later, couldn't think of any other options)
We must not trust anything coming out of these CEOs, since they have a history of lying, and I don't for a second believe that they have our well-being at heart.