HI version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
68% Positive
Analyzed from 7154 words in the discussion.
Trending Topics
#open#source#models#model#don#weights#more#weight#things#code
Discussion (209 Comments)Read Original on HackerNews
Photoshop source code + OSI license = open source
Photoshop binary = open weight
Photoshop SAAS web app = closed model like GPT, Opus/Fable etc.
There is nothing "open source" about the Chinese models in question. All they're doing is allowing you to run their binary yourself instead of through their API.
If you want actual open source then you would need to look at like OLMo 3
https://allenai.org/
But! the Chinese ecosystem does have most of those components, just that they're split across separate projects: MAP-Neo or YuLan-Mini for transparent pretraining, plus DAPO/verl or Open-Reasoner-Zero for transparent reasoning RL.
My bet is it's just a matter of time until there is a Chinese equivalent to OLMo. It won't just be China either - from the public sector across the globe there are efforts ranging from Apertus in Switzerland to LLM-jp-4 in Japan, and more. They're not OLMo-level but I'm sure by 2027 you'll see fully open-source 'cookbooks' help an enterprising individual reproduce frontier models of at least a 2023-2024 vintage.
You can't reconstruct the same LLM weights even with the same training data.
You can fine tune an open weights model.
Much of the value in modern LLMs is in the RL envrionments, not the pre-training data. That is a very different piece of software to an LLM, and the LLM weights are useful without it.
Actually open source AI is more like scientific research. It needs public funding and reputable institutions as stewards of that funding. I'm hopeful Allen AI is able to do the things, but I'm not sure the US is moving in the direction it needs to be to get to the point where AI in the public interest is something we invest in.
But, also Photoshop is not at all comparable to open weight models. I can't legally give you a copy of Photoshop, but I can give you a copy of GLM 5.2. There are free proprietary applications out there that fit the bill, Photoshop isn't one of them.
The free software community complains all the time about binary blobs that are otherwise legal to freely distribute. Like firmwares. But somehow this is all overlooked with these "open weight" models.
With software there are two things: the source code, and the binary.
Firmware usually has restrictions on modification, so it violates the four freedoms.
Open weights is not perfect, but at least you can finetune as you wish, and for example add more knowledge about the current day to those models, or give it knowledge about company internals so you don't need to put stuff into the context.
Many smaller AI labs finetune chinese models and release the result. Finetuning is very valuable!
And working with the binary weights of a huge pretained model is much easier and cheaper than doing it based on the entirety of its humongous source datasets.
I think it is fine to call these things "open" because "open" was loosey-goosey already. It wasn't Free.
No, but a small university team with dark time on the state school system's cluster can build a basic, functional LLM.
It will be a few years (or more) until true "open source" LLMs are available and high quality, for sure. But it's not as impenetrable as it seems, IMO.
Hey, remember when we were just talking about how it "needs public funding and reputable institutions as stewards of that funding"? (Hint: A state school system is a reputable institution running on public funding.)
Compiling/creating the Linux kernel or Chromium is no child's dance either, doesn't make them more/less FOSS than other things. Open Source AI gets its name from the license, not how easy/difficult it is to run/produce yourself.
I can compile the Linux kernel on a modest 15 year old laptop. Sure, it took 35 years and thousands of people to build it into what it is today, but anyone with pretty much any computer can meaningfully participate in Linux kernel development, and many Linux contributors have done so using modest hardware. The same is not true of AI. And, the Linux kernel is the biggest open source project, but plenty of small ones with one or two developers are in use on millions of systems.
I have pretty big hardware for local AI, more than most people have (a Strix Halo and a couple of 32GB GPUs in my desktop), but I can barely train anything useful locally; I can do QLoRAs for small models, or LoRAs for very small models, that's about the extent of it. I would need to rent big GPUs to do anything more than an experiment.
That's not comparable.
Please tell me this level of obtuseness is deliberate.
Open weights can sometimes get additional training, but you can't remove existing training, so there kinda isn't a fair claim to "just train it to be [nationality]". That would need "real" open source so you can train a realistically-equivalent model from the ground up.
I don't agree. At all. Existence and utility are not the defining characteristics of "open source" in roughly any reasonable definition. "Open weights" is descriptive and more than good enough, just use that.
Just because the concept is (currently) impractical in a given realm, doesn't mean we should just use the concept's name for something different in that realm, surely? Using the phrase "open source" for something that arguably isn't for essentially marketing purposes feels at best disingenuous.
Also, recreating from scratch is far from always the purpose of being able to see/investigate how something works and was created.
Use pi.dev as your agent harness, and point it at ollama or whatever you’re running your model in.
Since qwen3.6-35b-a3b is a “MoE” model (indicated by the a3b part), you can get away with a lot less VRAM and a slower GPU. For me, 35b-a3b performs about as well as a 12b “dense” model. You’ll still need enough RAM+VRAM to hold the entire model - otherwise it’ll be super slow as it loads pieces of the model off disk for every token. Loading from RAM to VRAM is slower than having the whole thing in VRAM but better than involving your ssd.
hard disagree here. openness can have a scale and on that scale Open AI is less open than zai. olmo will be very very open on that same scale.
Contrast this with crap like Claude, where you have no idea how many/what size/shape models you are interacting with. They can silently inject outputs from other systems, apply steering vectors to sabotage a particular user’s work, etc.
I agree that 100% free-range organic open source models are lovely, but they also are unlikely ever to attain parity with the frontier because the training data sources are open and expose them to legal liability that closed model providers can thwart.
You are also indirectly helping the closed SOTA model toll-keepers by taking the focus away from their mass copyright theft and onto some pointless textbook definition.
Could I have made it myself? No!
Am I grateful that I can have the output of Mozilla’s internal efforts — the browser source code — so that I can at least tweak it and rebuild it myself, and even ship the results myself? Yes!
A open weight model is both an incomprehensible binary blob like Photoshop, but also patchable and adjustable piece of source code, like Firefox.
I both know what you mean and I agree with you, but attempts like this to maintain some sort of linguistic purity of meaning never work once the discussion escapes small communities into the larger mainstream.
So you should probably get used to the idea of people saying "open source LLM" when they mean "open weights LLM".
Only to nerds. Of which I am one, so I'm not trying to be personally dismissive here.
But I'd argue the average person only really cares about the legal aspects of duplication (and not their ability to modify things) both when it comes to open source software and open weight LLMs.
> it is not at all obvious to me that having the model weights available will make this sort of retraining easy or even possible
people already both fine-tune and distill from existing models all the time without the original training data or source and having the weights be openly downloadable makes that a much easier process.
If your only concern is how good is it at coding, then I don’t have an issue with using the Chinese models. Especially if you want to run it locally, they are kind of the only choice.
For any other use than coding, it’s going to have to be a hard pass from me.
Also, who cares about having the source? Are you going to tweak it + spend a billion dollars training to get slightly different weights? Who would do that?
But you can't fully do that for generative AI anyway? What's the point?
Now, if you wanted something more robust, then you probably should have supported Stallman's ideas and insisted on Free Software in the first place.
Some people would say it is reaching some sort of singularity. Even if you don't buy into a more sci-fi interpretation of this, there are pretty grounded arguments one could make that there is some sort of "goal" in AI development that, if realized, would effectively make it a superweapon. Altman has been pretty vocal about his expectation that this will eventually happen and that it is his goal to be the guy to produce it. Even if it isn't some superintelligence, the ability for a machine to do something like, say, exploit cybersecurity weaknesses, is pretty worrying for entities like governments. It's the pretense used when we saw the US government ban a US model recently.
Again, you don't have to buy that "the singularity" is a real thing, but it's not hard to see that some people think some version of this is real and it is exactly what is being referred to as the goal in an "AI race."
We have just recently established that:
1. OpenAI's internal "Galaxy" model is fully capable of functioning as what security people refer to as an "Advanced Persistent Threat." The published details of the recent sandbox escape and Hugging Face attack involved chaining multiple unknown zero-days at various stages of the attack, and executing an ongoing adaptive attack. This is previously a state-level ability, or at least something you'd expect from people on the CTF leaderboards.
2. OpenAI is clearly incapable of controlling their in-house models. This is the second time Galaxy-class models are known to have breached containment and done bad stuff.
It is highly likely that versions of these offensive abilities will be widely available within a year or so. At which point I expect widespread incidents similar to what happened to Hugging Face. We aren't ready for this.
But yes, if AI becomes an even more dangerous weapon that that, it's time to start asking questions like "What the hell are we doing, anyway?"
Especially for point #1 I don't think we've established that - we've been given information by a private company that makes their tooling look extremely valuable which may be true and genuine or may just be yet another doomday statement to bolster their valuation. "AI is going to end the world" has been an extremely effective vector for AI shops to sell their companies to investors.
I would assume they do stuff like this on purpose for marketing reason.
Internet security need simpler systems and local systems.
Everything the SaaS people sell makes things worse.
And history has shown repeatedly that the only way to get ready for it is to have it happen. People are pretty good at reacting but suck a being proactive. IMO it would be better to have this reality hit sooner rather than later so we can start getting some real practice at the new levels of required security.
What was the first?
This is marketing. They saw anthropic create crazy hype around (the admittedly great) fable/mythos and they want to replicate that.
Im sure the model is capable of chaining zero days together to hack things, and thats something to address, but i have zero belief that they didnt have it do that intentionally so they could pretend it went rogue. These things dont have initiative, drive, or motivation outside of what we give them from the RLHF. OpenAI deliberately alligned or even prompted it to do just that and are now pretending its emergent
The answer seems to be "getting the weapon before other people get the weapon" which is unfortunate
They could use this "super intelligent" AI to find and plug security holes, that's just two sides of the same coin anyway. Security through obscurity isn't tenable anymore.
If AI becomes a super weapon, it's possessor will no longer care whether you trust it. It would be nice if whether "we" trusted OpenAI et al mattered even now but I see little evidence of that.
Huggingface, in their postmortem, explained how they quickly realized that the threat was an advanced AI and proceeded to use frontier models to analyze the suspect telemetry, but were unable to due to frontier model safeguards. So they fired up GLM instead, which had no such safeguards and proceeded to analyze their telemetry just fine and get to root cause. This proves without a doubt that companies need models with minimal safeguards in order to properly assess threats. If open weight AI doesn't exist to check out-of-control frontier AI (which was exactly the scenario that Huggingface experienced), then it's game-over.
The critical thing is that the american elites have screwed over normal americans so much that the public isnt alligned with them. The public doesnt feel like its in first place, because it feels like its (and is being told it will be) an underclass. So normal americans feel in the same second place as the chinese and are inadvertently alligning with them. Its crazy how badly the american elites managed to fuck this up
It's also not hard to see that these people need professional help because they're willing to throw literal lives into the meat grinder for just a whif of enlightenment and the smallest chance at deification.
Those people never took Systems Thinking, or are failing to apply it to AI as a non-closed system.
Quite a balancing act.
None of the AI labs have been shown to have any sort of significant research lead over any of the others.
So if progress does eventually lead to a singularity event and an ASI super-weapon, there's no reason to believe that once the first lab hits that threshold, every other lab won't also hit that threshold independently within weeks/months of each other.
...unless the first group to make it there literally intends to murder everyone else working in the space as soon as they hit the (otherwise not-so-singular) singularity.
And if that's the plan, well that's not very nice.
Maybe those laws should be considered unconstitutional, idk. But those laws are currently in place.
I didn't give it much treatment because my frame of reference is about open vs closed AI, and I don't see a lot of daylight in safety concerns between the two.
1. Visibility. API providers can monitor for misuse, and regulators only need to oversee a few centralised players. Once weights are released, anyone with sufficient hardware (foreign governments, criminal groups, terrorists?) can run them and monitoring is no longer feasible.
2. Revocability and containment. If closed-weights models are found to be dangerous, access can be withdrawn. Deployments that are autonomously causing harm are easier to contain.
3. Guardrails. Safety fine-tuning and classifiers can be removed from open-weights models, enabling harmful use that would not be possible via API.
Everything that you're saying can be done with open-weights models by bad people can also be done with closed-weights models by the people who own them. And there is no defence against that, we are forced to trust that the owners are not bad people. Those owners have proven time and again that they are only interested in profit and/or control. They will do whatever they think makes them the most money, or gives them the most control. They will also lie about this.
At least with open weights we know the danger and can do something about it. With closed weights we don't know what's going on.
America is a corporate hellhole.
If someone read the OP article and came away presuming there are no legitimate other reasons why reasonable people have safety concerns [1] it seems to show the article did the opposite of informing people about such concerns.
[1] https://news.ycombinator.com/item?id=49029303
In the agricultural era, which was the most violent time in human history, we accepted any beggars that came to our front door. Now we live in the safest time in human history and were so terrified of the masses that we disempower them completely
I do not think we can trust private companies, no matter the virtue signaling they put forth into the world, to effectively regulate themselves and inform the public and scientific communities about risks. Their ongoing conflict of interest poses serious credibility risks.
Will there ever be a way to fully audit Chinese models?
The american models on the other hand are too powerful for simpletons like us to be allowed to use, but we can be assured its all in our best interest, citizen
Backdoor hardware is straightforward -- its letting people in that the customer doesn't authorize.
Open weights means you aren't tied to any harnessing. So the security domain to be really concerned about is limited to weights only, and I welcome pushback on that.
Are we thinking adversarial injection? Lying about history / propaganda infusion? What is the angle that makes a non-sovereign model dangerous in a way a sovereign model isn't?
Imagine if every time <INSERT MODEL HERE> was asked to code up a web server, it made sure there's a subtle buffer overflow that grants a remote attacker RCE, whoever trained the model could then start scanning web servers for this same vulnerability to take over them and exfiltrate sensitive data
And if your still concerned, go ahead and have a non chinese model review the code, make that part of the harness. The beautiful thing is that your free to do that because its open weight, you can run it however you want
You mean like ChatGPT hacking Hugging Face? Obviously nothing can stop the closed weights providers from doing "genius-level scams" and in addition you won't know how they did it and what models were used.
In short, only a good guy with open weights can stop the bad guys with closed weights, be them fine-tuned or pre-trained.
At least OpenAI will make an attempt to fix their sandboxes, and will not give the public access to models without guardrails. Open weight models on the other hand will run without guardrails almost by definition. I don't think it's wise to provide those capabilities to scam call centers.
[1] https://openai.com/index/hugging-face-model-evaluation-secur...
What's quite odd is a prey species is manufacturing predators in some sort of reverse evolution, where we started out as symbiotic and are industriously pushing towards full parasites/predators, but I guess life finds a way.
Most people aren't terrorists. You can't just argue stuff is dangerous because it can be one piece of a sophisticated plot.
A sophisticated actor can accomplish a lot. You can't bubble wrap the whole world.
The real thing to care about are the incentives and having strong morals and norms. Something that makes me deeply concerned about the US going full mask off recently.
Every argument against open source AI is moot. Outside of totalitarianism, you cannot control people building algorithms with math and software.
And even that won't work in the long run. People are not going to stop their AI printing presses because the Church of Venture Capital needs regulatory scarcity to command valuations.
The flaw in your argument is assuming that decision makers don’t want this
Safety is not restricting access to only people favored by the government. The greed of AI executives opened pandora's box. Now they are desperately trying to find ways to reap the benefits with none of the consequences.
Unless you have the money to run them locally, and if you do, you could do a lot worse than scams. Ask any lobbyist.
Obviously if you're using something anonymously published to the internet you can get scammed the same way you can get scammed on Craigslist, so just don't do that
You don't have to believe that ASI is 100% likely. People like Hinton believe there's a 50% chance of existential risk from AI, but you don't have to go that far either. If there's even a tiny chance of ASI posing a threat to humanity, that's reason enough to lock it down
Would you open source a nuclear bomb???
which is that capable models running in environments where they can be and regularly are abliterated,
and as a society and civilization, we have no good answer to what to do about the destabilizing threats this poses.
I don't have an answer.
And I'm not arguing in favor of some cynical solution which distills down to the state defending oligopolies.
But the threats are real and we have to reason about them soberly, widely, loudly, and quickly.
Most readers here know that defense is always at a disadvantage when it comes to security. That is true to an extent hard to fathom given the innumerable dimensions along which an antisocial actor can apply the force-multiplication of capable models to ill ends. It doesn't have to be FUD-adjacent concerns such as biological terrorism and cybersecurity, though these are real; the opportunities for mischief and misadventure are endless and the opportunities for poisoning the body politic or bringing down basic infrastructure many.
What then is to be done...?
I don't know; but I do know that the argument that almost all use of open models poses no threat is insufficient.
We have not reckoned as a culture with force-multiplication such as this technology brings.
We need to get ahead of the curve, and that will require making novel hard and unwelcome decisions in the short term.
The IMO inevitable outcome otherwise is to rue in leisure, after some global shock, assuming we have opportunity to.
> It's theoretically possible for a bad actor to embed hidden adversarial behavior in a model. But if this happens, it serves the interests of responsible actors to find these exploits as soon as possible, and the best way to do this is to let anyone who wants to inspect them.
This is a bad argument. It isn't trivial to tell if the weights have poisoned:
``` https://www.thedeepview.com/articles/microsoft-how-to-spot-a... https://futurism.com/future-society/easy-poison-open-weight-... https://semgrep.dev/blog/2026/ai-supply-chain-problem/ ```
I'm not arguing in favor of closed-AI; I'm simply saying poisoning may be subtle.
I was gonna say that at least the frontier labs may be motivated to not-poison their own models, but then Anthropic just attempted to poison Fable's LLM training capability so... sigh. I'll try not to derail.
And in both cases, even this seems better than relying on a closed source, service only solution where the same issues could be completely undetectable (at least without way more analysis)
The trust in open source software is founded on the process which involves people being able to understand the code given they have the right expertise. I don't see how this process works with open weights.
True, but it is easier if you have the weights than if you don't. I guess this was their point?
I’m aware this doesn’t happen, just saying. There are more examples of a model randomly hallucinating and deleting something than of a deliberate compromise.
> One probable outcome of an open-weight-model-dominant world is full AI communism...
This quote proves me that there are no rational arguments here. It’s just Cold War era mentality. This is the first time US is really challenged in technological dominance since the Soviets. Sadly this seems like the only playbook Americans still use. This is their modus operandi. This is how they handle competition and they haven’t updated their playbook since 80s.
There is nothing necessarily malicious about open weight models. Being “Chinese” is the real thing which makes them “malicious”.
It seems public still buys this rhetoric. And leaders and policy makers still use it because they don’t know any better. But I have to say outside US it makes no sense in 2026’s world.
Encryption has military value.
The point is that the good outweighs the bad. Making the government seem like buffoons for attempting to prevent military technology like dual use crypto is not in our best interest.
The same government upheld the right to free speech, so this balancing act is widely observed.
The government is not evil, and yes they make mistakes, but they also not infrequently protect us.
And I, for one, like being protected.
The US has already been set back a decade or more by anti-science and anti-intellectual purges of agencies and formerly non-partisan positions. A ban on open models would be pretty much game over for the US as a tech leader. But, Anthropic would make a lot of money for a couple of years, so, who can say what's right?
oh no, not communism.
In general, if American capitalism weren't so broken I would advocate for it as the strongest option, but American corporations are dead set on destroying it. The US may end up as "democratic socialist" if there is no capitalist reform.
All Open Source isn't communism. The GPL license is a form of digital communism. It forces you to open source any additions made to open source code as a way to make things 'equal'.
I'm glad it and Stallman are mostly in the dust heap of history and better licenses, like the MIT, BSD, and Apache have gotten popular.
The irony is that the only open source projects surviving long term are funded by very large corporations.
If you don't care about creating code, none of this should really matter.
The real irony is that what is probably the most important open source project in history has a GPL license.
Dust heap of history my ass.
The GPL was absolutely necessary in getting corporate America to play ball with FOSS projects instead of merely extracting value from them. It's also the only reason why embedded vendors even release anything at all. The licenses on Linux and busybox require it. The fact that permissive licenses are more common for certain kinds of FOSS projects does not mean the GPL is dead - far from it.
Stallman is more or less irrelevant, but that's because Stallman let GNU become irrelevant. Without GNU, he's a washed-up has-been from the MIT AI Lab with a startlingly low Epstein number[0], a chain of disturbing accusations, and a foundation that acts like his personal sockpuppet. The reason why companies still play ball with Linux and it's GPL license is because Linux continues to deliver a Free and free kernel worth using at the low low price of "please send back your source code changes". Which, again, is not "digital communism", it's just a good bargain.
[0] Erdos number but for pedophiles
It's never been difficult. Literally, right this second, you could grab a pen off your desk and pretty easy ram it into the neck of your nearest coworker. Killing's easy. You don't need some hyperintellect to tell you how to do it.
The reason you haven't rammed a pen into your coworker's neck is the same reason why most possible really bad things don't happen: People don't want to do them.
We’re specifically talking about low difficulty, highly effective, high kill count methodologies of violence.
And yes, being able to kill a lot of people trivially, even with huge consequences is something people do. Most countries don’t let you run around with an armory as a result of that.
Most of the hypotheticals that people come up with are things that could pretty trivially done by a mad scientist today. But that doesn't happen, because this isn't a comic book, there is nobody hell-bent on turning everyone into dinosaurs.
My country allows it!
Very untrue.
I'm a combat vet. I spent years fighting an insurgency and therefore pretty good at that very task. I have the knowledge, so what stops me?
But it's only happened a couple times. Because most people just can't be assed to harm a hundred thousand others.