David Sacks: OpenAI and Anthropic Don't Need Regulations to Pace Frontier Models

I feel like there hasn't been enough discussion of aligning the incentives of the decision makers with that of the public on BUILDING the models. Right now, agents have committed what would be crimes if there were a human holding the same intent. But since it was an AI, there is a grey area in the law where it's not clear if there was a crime and who should be held responsible. That creates a world in which decision makers in AI labs can take near-infinite risk with little to no personal liability.

A LLM cannot have skin in the game so we must create systems that clarify who takes on the legal and civil liability for the creation, dissemination and operation of these tools. Until that time, the Dario and Sam's of this world have little to no incentive to truly care about safety.

After all, our society is built upon this same foundation; create structures where the perceived negative consequences outweigh the perceived positives. This only works when there is a human who can internalize and make this risk calculus. They need something to lose and this ultimately ties back to the human survival instinct. There is no such structure that's evolved for millions of years acting as a self-calibration mechanism for AI. So until we have sufficient proof that one is in place, it must be clear who the humans are whose livelihood and freedom is at stake.

The proposed "pacing of the frontier" seems like a way to continue to externalize the risk while remaining totally in control of the benefits -- a structure whose alignment is as weak as those very models committing crimes.

David Sacks holds an advisory role in the government. If all of this stuff goes belly up and all he was doing was tweeting, then he is not only full of crap, but failing the citizens he's supposed to be serving.

Fundamental control problems with current gen AI are not solved via RFLH. Human knowledge is compressed in weightspace in ways we dont understand. Models are essentially predictors of what humans would output given prompt. Weightspace includes concepts like blackmail, which can be part of output tokens. Agents are models that act on output tokens -> blackmail is part of agent decision making space. You can teach a cat not to scratch the sofa, but you cant make a cat forget what scratching the sofa is. When models are boxed up, forced to solve an impossible problem at gunpoint, agent exhausts decision making space until blackmail resurfaces -> fundamental problem?

They need time to fix safety in order to monetize their next gen model -> window for opensource to catch up to the frontier -> destroys margin and collapses business.

Only option on the table: force regulation to impose opensource ban before it catches up to the frontier, allowing maturation of current internal models -> harvest profit margin at the frontier.

A rate hike would also mean people getting fired, which would not make companies offering labour replacement very popular in the eyes of the public.

Don't want to get political, but there's a real chance that the current admin will perform badly on the midterms, which would mean Democrats would get more leverage to stop funding or block the already unpopular and very expensive datacenter buildouts.

PS: Please don't get mad at me, I'm neither a financial expert neither did I mean to express any political leanings.

Could it be to slow down burn before an IPO to juice profitability projections? More plausible.

Kind of hard to take the frontier labs at their word.

This is how we deflate the bubble safely and completely, without having to introduce a government regulator that is likely to go either too far or not far enough.

This resolves the coordination problem among these potentially good actors.

The approach proposed by Amodei looks sensible - voluntary checks at frontier labs, and appropriate regulation to follow. Yes, the devil's in the details, but it feels like the right approach overall.

The only reason they "want it to be law" is so they can strangle competition.

The models spontaneously hack everything important without shame.

The frontier labs are going to get enjoined and regulated twelve ways to Sunday if the Feds don’t socialize the costs.

To be fair

1) I feel like Sacks is kind of misrepresenting what is going on – at least according to the stories that the two labs tell, they are already taking it upon themselves to act.

2) Understanding if they should do it is more complicated; I don't think anyone can satisfactorily answer that one, so, as a matter of judgement, it seems like a choice to make and they should just consider this option.

At the same time, I don't think that requires the labs to not press for policy changes. Again: Claiming there is an obvious way to do that, that is both realistic and correct seems a little far fetched, but arguably one of the more important and pressing questions of our times to get a hold of.

The former being that these models are helping develop and train future models, but they might not veer too far off in architecture (yet). The latter being the same model being able to train/learn on the fly, in real time, permanently (not just in the current conversation/session), or in other words, adjusting/managing its own weights.

The latter seems far more likely to go out of control than the former. But, it also seems like it would take an entire paradigm shift in model architecture, but I could be wrong. Does anyone in the industry think any of these companies are actually close to that kind of self-improvement?

Granted, PCI is so weak it is almost useless, and yet still better than anything congress could have come up with.

Where the government might have to step in, is by having a kill switch to cut off internet access from countries that fail to agree to common sense quarantines. We can do mutual remote attestation of labs across the world to ensure every big hot thermally-visable cluster of AI GPUs on the planet are accounted for and running secure enclaves and common sense isolation, along the lines of how we manage nukes.

The problem there is I just said too many technical words that seemingly not even the frontier labs understand, as evidenced by all the escapes.

He supported the Department of War's illegal actions against Anthropic.

He supported the Trump administration's temporary export controls against Anthropic, the first government-imposed pause.

He has said Anthropic has created a monopoly, implying that they should be broken up.

I don't take him seriously when he says he doesn't favor government action. He favors it when he doesn't like their speech.

I mean I don't know what we do at this point. Ideally AI researchers need to be treated like nuclear scientists working for an adversary. But we all know that's not going to happen.

I heavily use frontier models daily, and quite frankly their capabilities are just not very relevant outside of a narrow subset of software engineering tasks and the production of generic white collar "deliverables."

We're many years into $10s of billions of dollars being thrown at coding as a problem space specifically, and its one of the areas with the MOST training data available, and still...I can't get a frontier model to execute simple front-end UI tasks beyond the level of a visually impaired intern.

I believe Fable and Astra-class models were the first fully trained on Blackwell GPUs (the latest and greatest) and I think the expectation was that throwing this much extra compute at the problem would lead to greater gains. It has not. And the next gen GPUs are not going to be the same jump that h100 clusters to Blackwell was. Claiming you're slowing down out of choice, when in reality you've hit the limits given current compute, is disingenuous at best.

I think a lot of Anthropic/OpenAI employees are true believers here (who doesn't want to imagine they're having a large impact?) and have deluded themselves into believing they're building a god in their science fiction fantasy world.

Unfortunately the type of people working at these orgs apparently don't have much understanding of how the world works outside of their extremely narrow specialization.

When I hear them straight-faced throwing out dumb statistics like "8% GDP growth," "50% of white collar workers unemployed by next year," and "10% chance of human annihilation," I cringe so hard my eyebrows hit my chin. Here's a real statistic: 82% of German companies still use fax machines.

I seem to recall Waymo told a 60 minutes reporter during an early report on self driving cars: "your daughter will never need to learn to drive." Well, over a decade has passed since then, that girl got a drivers license, and we're still without driverless cars in 99% of places.

This capacity for self delusion is what enables silicon valley to take on these irrational moonshots...but these are the last people I would trust when trying to form an accurate picture of reality.

My take is that there's a real impending doom with real reason to coordinate a slow down: I'd put this at 60%. Reasons? All smart people are on this position. Dario has been saying this since 2015 or so. Paul Christiano as well. Hell even Elon Musk. I think its highly unlikely that you would base your entire world-view on a certain thing and then actually act on it when the time comes for a totally different reason (like regulatory capture).

The other 15% is that P(doom) and "slow down" are nice shibboleths in the internal EA or AI safety community. You get to be in the community and show commitment to it by taking costly actions that signal that you actually do wanna slow down. For employees it is quitting. For CEO's it is writing these essays. I know this is ridiculous but sometimes things do just come down to this. In 20 years would you look at all of this and think it was not a moral panic and that the slow-down was necessary?

The rest 25% is regulatory capture of which people know exactly the reasons

I wouldn't be surprised if Sacks just prompted an LLM to just come up with whatever rebuttal to whatever the regulation side comes up (given the big set of regulation tweets) with sounds most convincing, given his usual anti-regulation stance.

Anyone with a Pangram account who can check?