Mozilla: The state of open source AI

For the near future it seems that the new models will consume whatever improved hardware capacity we have. And that is always beyond what is available to your individual user - unless you have tons of disposable cash (40k+) to throw at a mostly-idle homelab, and even then it's not particularly impressive.

Longer term I think we'll see these uses of AI cluster into a few groups:

- maximal code / reasoning quality, at high prices (Fable)

- typical code / agents (sub-Opus, Terra)

- cheap but decent enough quality (think Deepseek / GLM / Luna)

- so cheap I don't care about utilization (Deepseek, and friends)

And also more niche ones:

- ultra fast with high quality answers (typically sub-SOTA). Cerebras / dedicated silicon type approaches, expensive.

- ultra fast with mostly-adequate answers, and an openness to retries, moving up to better models

I think the open models will dominate all except the top 1-2 of those categories, and there will be a continuous erosion on the big player's moats. The top categories are also where all the money is, but I'm not sure it can justify those investments long-term. I also think they will have to squeeze more money out of them to justify the investments, which will also drive people down the list.

Meta is selling their now excess compute, other compute has been on the market for a while. The current hardware cost bubble is temporary, especially once people are forced to pay the real inference price instead of majorly subsidized subscriptions.

There will be plenty of model providers with prices that undercut Anthropic/OpenAI's prices.

I can't take anybody seriously when they keep declaring open models is beating frontier models. What they don't understand is that besides the huge capex to train and run inference, the real gold is in the human response to the prompt results, this is what all the Chinese companies are making their open models dirt cheap and distilling american frontier models via scraping.

The idea that we can out-parameterize frontier models is a common misconception, the true moat that Anthropic and OpenAI is why Chinese model providers are open sourcing and making it dirt cheap to keep pace through its "proxy chain operators"

https://x.com/HarshalsinghCN/status/2056626175959826692

Yes, the problem with comparing open models to open source is that open source requires humans to volunteer their time. Open models requires humans to volunteer their money.

These two types of contributions have very different behavioral profiles, and it doesn't obviously follow that the historical success of getting people to collaborate socially on building software for fun and for the benefit of the community will translate in any meaningful way to the necessity of being able to raise enormous amounts of money to pay for enormous amounts of electricity.

I don't think it needs to be framed purely as generosity. You just need a sufficiently self-interested actor that sees open ecosystems as a necessary part of reducing their own risk profile, relative to the alternative of complete reliance of a third-party business that can take an exorbitant cut and/or Sherlock them at any time.

Valve and SteamOS are a good example of what this idea looks like in practice. (Though they may also illustrate a third thing you need: a privately-run company, that has enough profit, and enough commitment from leadership to the company's vision, that they can make long-term bets without having to eventually bow to investors seeking short-term gains.)

I’m not exactly sure on the “how” but it only makes logical sense for (non-AI) companies to band together to fund the training of a shared model. Apple is a great example, AI is not their core business but they still require it.

The only thing that took us down a different path is the vast sums of VC funding pumped into the AI companies.

How does it work if people flock to open models but they're too expensive to train? What is the financial incentive to do so?

I seem to understand open models are mostly coming from China, and the benefit of training and releasing them for 'free' is a powerful geopolitical weapon against the Western/US economy that at this point depends on OpenAI & co. to succeed.

Will the West make open models illegal?

I would definitely pay a monthly subscription to help fund a non profit compete with Anthropic and OpenAI. I already pay subscriptions for myself and 2+ other people. It's a non brainer to be able to pay for the training of better models that I can then run myself for many more. I hope someone starts this, I think this model would work. I'd start it today if I had the team and initial capital to bootstrap the infr. I know VCs won't fund it, but we definitely will, enthusiastically and continually.

not really a feeling; if you listen to ed zitron and strip out the vitriol, you still get the fact that the VCs are looking for some 5 trillion dollars in 5 years.

The onlly way that happens is if America turns into zimbawe.

This is easier to say as Fable is good (even SOTA). But people have been were saying this continuously for the current model and for now the improvement are still coming.

A better question is would you settle for o3 now or pay 20$ or 200$/month for fable ? Because o3 quality is available OSS.

It is like the new IPhone, in some sort. At some point come a feature many would like to have, despite diminishing returns.

We will see how long labs can keep up and what the scaling curve look like, but I would be more worried into losing sota status to Chinese companies than letting them take the open non-sota approach.

Have they?

https://www.cnbc.com/amp/2026/07/14/apple-prismml-ai-compres...

> handed the reigns over

Oh man, they gave them free reign?

How will anyone reign them in now?

For all intensive porpoises, this is like Babe Ruth, chomping at the bat!

> The real moat aren't the models, but the tooling around the models that allow them to perform specific tasks/goals. That's what really sets apart frontier vs open. Open only has the model itself, closed have the tooling to enhance the model.

As these frontier companies have been boasting, writing software is now a negligible cost because the LLM can do it.

IOW, no, their software can't be a moat, because, according to their own arguments, you can use their LLM to trivially clone their software.

The vast majority of companies are still putting most things in the cloud and will continue to do so and this far outnumbers the must-be-on-premise companies.

Sure there will be self-hosters but hosting AI models will always be more of a challenge than running scalable database on your own hardware and specialized hyperscalers will be here.

Would that require a watershed event to clearly establish the importance/risk of privacy though? For example, right now it seems like most big software companies w/ strong security process are comfortable uploading entire codebases to Israeli cybersecurity firms for vulnerability scanning compliance purposes

This will only delay the inevitable. Sitting on some magic prompts is hardly the moat they need.

Referent of "the models are meaningfully different" reads as <top closed, top open> rather than <top closed, cheaper closed> to me, so I'm not sure why we'd be comparing Fable vs Opus/Sonnet or Sol vs Terra rather than the same against Kimi K3.

You're saying it's important to have up-to-date facts stored in parametric knowledge? It seems to me like that's grown less and less important as agentic capabilities have grown. Even if a frontier model doesn't know something, if it's out there, it can easily find it through tool use.

Grok has the biggest advantage in current events knowledge because it's integrated with X, which enough people still use even though it isn't Twitter.

I don’t even know the names of any open source search engines, but the open source models perform decently on various benchmarks and in personal experience.

Was it ever even a claim that open source search engines were trying to outperform google, let alone kill it?

Marginalia already gives me better results for many queries than Google. Because Google has sunk so low.

>There were a million cell phones before the iPhone took over the world.

You have tripped yourself up there.

iPhone took over as it introduced something innovative over standard phones, but then Open Source (Android) matched the multi-touch and software differences and Apple's branding, lock-in and design etc have managed to keep it as a big player in wealthier countries. IPhone also came on the back of the massive iPod success.

ChatGPT launched the same innovation vs Google Search, but just like Android Opensource AI is moving fast now.

Android has 72.7% market share at present, Open Source AI will do the same unless the frontier labs can continue to do something new.

The frontier labs are saddled with enormous investor and other debts. How long they can keep innovating by spending so much on R&D and paying there staff very high wages remains to be seen.

Once investors cash out via an IPO, the companies are back down to earth and playing in the real world again.

The moat is enterprise contracts and artificial friction moving between harnesses. Moat is a strong word, more of a puddle.

And to keep around a concurrent browser with full adblock abilities so they can cripple their own without too much of an outcry.

Do you think Firefox development is unsustainable without this substantial stipend?

That's a very low bar, though. Chromium is open-source too and has a bunch of privacy-focused forks such as Ungoogled Chromium, Brave and Chromite.

On my device, bars grow as I scroll. I want your feature, being able to just scroll the static page without elements jumping around.

> The bars, in fact, don't grow as you scroll. And I don't even see why they should.

On my device, they grow as I scroll to them.

> it seems to respect user's choice of "prefers reduced motion".

Cool, that I didn't check, because it is impossible to enable that setting, as it breaks _huge_ amounts of websites.

I'm not aware of a way to enable it selectively, but one could also just display the content at all times. It's a static page. It's static content. None of this makes any sense.

___

The idea behind that style of gradual reveal is probably some kind of storytelling, but the only story it tells is that mozilla is wasting donations on people with incorrect opinions that could be used on.. idk not building torment nexii?

Are they open?

I don't know of any useful model that would match the usual definition of "open source". That is, where everything needed to build the thing is provided, such as the training data and code. Not that it would be tremendously useful anyways considering how expensive training a model is.

I much prefer the "open weights" term. It is not open source in the sense that you only get the finished product, not the actual source, but it is still open in the sense that it is not only accessible as a service.

For an analogy, take Quake for instance. When it was launched, its game server was available as an executable, so you could run it your machine, but that didn't make it open source. Only much later it was released as true open source software.

Really wish websites weren't allowed to force smooth scroll on. Hijacking basic browser functionality is so hostile.

Oh that's easy: they outsource design to the LLM, which doesn't have eyeballs.

I agree 1000%, Mr. Jake.

I use hermes only ever saw their repo. Atrocious. I was sure you were exaggerating.

Particularly in this day in age when the FF market share is down to low single digits

I hope you're right and I want you to be right, but, even seeing the current hype around local models, etc... and open-source models, I think the industry is currently under a big confusion where they see the benchmarks of things like Kimi, GLM, Qwen, they play with it via opencode, and they think like: "Wow this is pretty good, I want to deploy this". But they don't understand how the KV cache grows over time and can take almost as much memory as needed for a 30B param model, they dont understand that a quantized model WILL NOT be the same as a full precision one, and they surely don't see the engineering work needed to serve inference to even tens of customers at a decent quality and latency level.

The biggest moat of these giant labs and models is increasingly shifting towards deployment capabilities and (debatably) having better (proprietary) harnesses.

The models themselves can be impressive on benchmarks, but unless they can be served reliably to customers either at scale, hosted somewhere, or even on edge with predictable latency and memory usage, then frontier will always be leading.