Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
74% Positive
Analyzed from 1359 words in the discussion.
Trending Topics
#models#open#more#model#frontier#labs#important#weights#running#stack
Discussion Sentiment
Analyzed from 1359 words in the discussion.
Trending Topics
Discussion (29 Comments)Read Original on HackerNews
AI development speed is increasingly influenced by the quality of the model you are able to use internally. The frontier labs could easily pull ahead again if they increasingly withhold their best models (e.g. Claude Mythos) from the public entirely. They will benefit from increased R&D speed internally that cannot be matched by open labs.
Also, it seems plausible that frontier labs will eventually accumulate architecture-level improvements in their models that make them significantly more efficient than open LLMs. In that case, if the open labs cannot reverse engineer and replicate that, then their models will forever fall behind. On the other hand, any architectural innovations in the open model space can be used freely by closed frontier labs.
There's a counter-trend that favors open models however, which is that the design of model harnesses and multi-agent systems is more and more important to AI quality today relative to the intelligence of the model itself. This MIGHT mean that having a bunch of dumb, but cheap models in the right harness can actually compete very well against raw frontier intelligence in most practical tasks. (In other words, better harnesses makes models more efficient at improving their task completion by spending extra tokens.) This would give cheaper open models an advantage in any task where they're smart enough to complete at all, since a good multi-agent harness might mean they can do these tasks reliably and typically for cheaper than frontier models even if pushing a higher number of raw tokens.
It’s much more likely that performance will plateau and open weights will catch up asymptotically
I really don't think so. This almost never structurally happens.
I think it'll be more like Linux on the Desktop.
Or Ubuntu on the smartphone.
Or Firefox.
We'll have open weights, but 99% of everything will go through hyperscalers.
I think it will be Linux on the server, or the one that runs your watch, your phone, the radio or infotainment system in your car, maybe your thermostat, a bunch of medical devices and military devices, running in space shuttles and space stations and... You get the point. It's on everything.
I agree with the outcome of your premise (i.e., openness), but for different reasons:
First, isn't it the case that these bleeding edge 'newfangled' LLMs are basically variations on the same core ideas from "Attention Is All You Need" from 2017? [1]. Different scale, but still the same basic architecture. Even the "MoE" innovation keeps the Transformer attention stack while replacing or augmenting the dense feed-forward/MLP part with routed expert blocks.
And, I would argue that Engineers aren't working on new architectures. That would be Researchers, working on
That research is still open, so the outcome that you propose (openness) is likely to come to pass. Researchers/Scientists gotta publish, otherwise it's not science (to quote LeCun [2])[1] https://arxiv.org/abs/1706.03762
[2] https://x.com/ylecun/status/1795589846771147018
It could also work if you DO have competition but your compute capacity is overbooked anyway, so releasing the better model doesn't actually make you that much more money (except for raising prices for the same amount of compute, which would give limited gains).
This is pretty much the situation Anthropic is in today.
The agentic tooling and harnesses will be what's important... and nobody has a moat with those, either. At least not until the easy money runs out and the patent suits start flying back and forth.
> The collision between those two facts — that American capital paid for a moat, and that the technology no longer provides one — is the most important force in the AI industry today.
> The open-weight ecosystem did not arrive in stages. It arrived in a wave. In late 2024, a Chinese lab named DeepSeek released a model
Looking at the assertions above, anyone passingly familiar with AI over the past few years will tell you that open weights and open research were the norm until OpenAI GPT-3 came along, and even then they were forced to release GPT-OSS by the market. So what technology moat? There has never been one in AI. Training 100B+ or trillion+ parameter models in expensive runs was potentially a moat, until the chinese startups showed in short order that it could be done for $6 million a run. Even the CUDA monopoly seems to be ending.
Also, no evidence referenced to back up any of the assertions. How do they know that the bet was that the frontier models would be the next great monopoly business? Especially when there were many from the outset: GPT, Anthropic, Llama, Deepmind, etc. etc.
I'd argue that the wholesale replacement of labor was and is the driver behind the capex, not monopoly dreams.
The starting premises appear to be, well, faulty. Whither the rest of the article?
Azure Copilot can charge whatever it wants because you can't use anything else.
"running on the LangChain" ??
EDIT: look, I think the general discussion is important, so I don't want to denounce the article. I, for one, am excited for better control, ownership, and accessibility of models. The ride labs take us on can be quite frustrating. Maybe there's even signal that the model progression is stalling (ie Opus 4.7). If that's true, then some of the notions made in the article are important to discuss. Ref https://x.com/ClementDelangue/status/2046622235104891138?s=2...
EDIT: this is not a complaint about the grammar. Look at my reply in the comments.
Yeah not sure about both ollama and llama.cpp though lol
Cutting it off at "the LangChain" is like if I took the first sentence of your edit and said "look, I think the general" ?? You think the general?
And it'll only get better/cheaper.