ZH version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
73% Positive
Analyzed from 5428 words in the discussion.
Trending Topics
#model#models#flash#glm#don#more#chinese#opus#https#china
Discussion (207 Comments)Read Original on HackerNews
I decided to take the plunge and get myself four sparks at a decent price (and bought the QSFP cables from AliExpress because they are literally 1/2 the price of Amazon), even knowing Apple was going to release new hardware and there's probably a spark 2 on the horizon. It looks like this is going to be a decent fit for what I need. I've been experimenting with a two-node DS4 and it's _good_ at some tasks, but it really just spins its wheels when it hits the limit of what it can reason through.
I can offload mundane/basic tasks to DS4 on two sparks, but I've been pushing it harder on some novel work and it just can't run on its own at all beyond a certain complexity level.
I would love to see an Opus-4.8-level local model but TBH I just haven't got there yet. The models I've tried so far _are_ good but they aren't able to solve tough technical challenges, regardless of harness/prompting/etc.
https://artificialanalysis.ai/models/qwen3-8-27b?models=gpt-...
DS4 Flash 0731, on the other hand, wildly opposite experience. Would recommend.
GLM 5.2 - even quanted down to a hybrid 4/3 bit setup is amazing for everything but the hardest/most complex stuff in the same projects/realm.
Wow, if you don't mind me asking. How and where?
They were briefly on sale with a $200-off coupon, but they show up on warehouse deals from time-to-time as well.
$4,000 isn't priced insanely? ye gads
Reality is on a single spark I'm constantly running out of room and it being an odd size M.2 slot it's a pain to upgrade. I'm setting up a NAS over RDMA via ConnectX though, that's fun.
Agree. It doesn’t even have to be local, using models in this size class through OpenRouter will reveal their limits if you work side by side with Opus level models regularly.
There are a lot of social media posts about people cancelling their Anthropic or ChatGPT subscriptions after installing a local LLM. I’ve used local LLMs a lot and I spend a lot of time with frontier models and the difference is still huge. As far as I can tell, the social media posts about local LLMs replacing frontier models are either wishful thinking, engagement bait, or people who must be working on much simpler projects with a much higher tolerance for slop than I have.
Over the last couple years I’ve had to learn sales and understand the thought process behind this better, and I think I’m beginning to understand it
The psychology is that most people aren’t really trying to optimize for productivity (even most people who think they are) on an ROI basis, because their compensation is too decoupled from their actual raw output, and more closely coupled to how differentiated their marginal contribution is to peers. They’re much more incentivized to spend their personal/work time optimizing for being more skilled or acquiring some kind of competitive advantage relative to baseline.
Most people don’t consciously run the numbers of “I get paid $X/hr to add $Y of value” or model pay at work as something with variable inputs (eg something that can be increased with high performance), so it makes sense to them to spend 20 hours of time to save $100 or to make themselves 5% less efficient to take home 0.5% more or avoid doing something they don’t want to start doing.
NOT saying this always happens or that they’re stupid for doing so. I didn’t even realize how much I had been doing it myself until I started recognizing it, and shifted to having my own comp/performance fully aligned with the company’s P/L.
It actually makes a lot of sense IF you can accurately estimate incremental upside (which is much harder and more diffuse than modeling downside if you’re salaried a employee) or if the upfront skill/knowledge investment that looks like bikeshedding pays off in the long run.
Don't use DS4 Flash in max effort mode. It's just spinning its wheels, in my experience (I have a harness for testing models with 25 real bugs/features/etc from my real projects that I measure outcomes against) DS4 flash does _worse_ with max effort. It will literally have the right approach and reason itself away from it.
But things change real fast when you're no longer bound by costs/apis/rate limits. All of a sudden it's not about "how can I do this right and efficiently" and more about "I can poke at and test _all the things_ that might make this better".
I think most people who can't see this value in the local inference approach are likely still copy/pasting from their web LLM ui's or don't even come close to subscription quotas. Meanwhile, 1b tokens a day is a light day for me with 3 $200/m subscriptions + some level of sub at basically every frontier level provider. Had I been less frugal and ponied up for the hardware before things got crazy I wouldn't need 80% of that - just the frontier models for the most complex tasks, the open weight models would handle the rest easily _and_ I'd get to do a lot more exploratory work without concern about quotas.
The more reasonable comparison is against rented GPU's, while looking at tradeoffs in latency and upload/download/storage/instance management overhead.
Buying hardware for local models is meeting a wholly different need than buying tokens through OpenRouter or whatever.
For example, I have a small posix-shell-based LLM harness that can SSH into my NAS and run organization tasks using the local DS4Flash that I have right now. It's already been a massive help for me to keep me organized, and that's just 2x DGX Spark's worth of compute.
https://deepswe.datacurve.ai/
That's pretty solid. Smarter and cheaper than Luna xhigh, not as smart but less expensive than Luna max. Smashes deepseek v4 flash, and even worse it matches v4 pro at a tiny fraction the cost. Roughly equivalent to sol medium, at a fraction the cost.
They should've just lead with real, up to date data, because it's good, not the silly old tactics like comparing to Opus 4.8 when 5.0 is out in many of their charts.
Congrats to them!
Opus 5 consistently comes at or near the top, but outputs constant unreadable jibberish. Meanwhile GPT 5.6 Luna medium tends to be rated pretty poor on agentic tasks compared to the Chinese lab open models, but I find the latter much more likely to lose track of their own behaviour during a long-horizon task or get stuck in a doom loop.
(This isn't a comment on GLM-5.3 Flash as I've not used it!)
https://kommodo.ai/i/IFSUUQYT522uZXWvePZ1
it still fells stupid sometimes and it is benchmaxxed for sure. but its good enough that im building all the hobby projects with it.
I've probably wasted a good 40% of my weekly usage on Luna Max agents just thinking and not writing a single line of code.
The flash one?
My prompt was something like: "here's data I have, here's what matters to me, create HTML mockup".
All GPT 5.6 models were laughably bad. And I don't want to downplay it - they were just absolutely, objectively horrible. Every single attempt was what I could probably call "if json was ui".
Claude models produced... "claude look".
GLM 5.3 - somewhere between GPT and Claude.
Kimi k3 - each attempt produced beautiful UIs. It used components that I didn't even know existed and wouldn't even know to ask for. But expensive, very expensive.
ox-alpha (GLM 5.3 flash) was very close to K3. And at this price point, it's already configured as "designer" model in my oh-my-pi.
It's what people know. Opus is just the common target.
> Smarter and cheaper than Luna xhigh, not as smart but less expensive than Luna max. Smashes deepseek v4 flash
The problem with this and DeepSWE is it goes for a very specific profile. I'm not convinced DeepSWE is any accurate in actual work. It's surely a different signal (compared to some that allow cheating) but it has its own issues, e.g. weak harness.
Luna is great at following instructions but bad instructions or anything not covered = death.
Deepseek is more analytical. Good for bug tracking.
GLM is a better all rounder in some ways. Better at creativity.
July 16th: The "Kimi K3 moment" - China has caught up to Opus!
4 weeks later: GLM 5.3 - Same performance, but cut the amount of parameters and cost to a third!
12 days later: GLM 5.3 Flash - Almost GLM5.3 performance but cut the parameters in half, cut prices to a fifth and serving on Chinese chips!
Broad and perpetual license over inputs and outputs, and even your name and profile picture.
Vague prohibitions on whatever may harm Z.ai’s "interests" or even the "national interests" of any country.
Vague prohibitions on "disturbing" or "inappropriate" content, whatever that is.
Vague prohibitions on discussing Z.ai, even my posting this comment violates it.
Can ban you if you, in the "sole and absolute opinion" of Z.ai, have violated these broad terms, and if you paid for the discounted yearly plan kiss your money goodbye.
Nearly every TOS I've ever read has a "We can ban you for any reason, or no reason, are under no obligation to disclose any reason." line somewhere in it.
HN's for example
> We reserve the right, at our sole discretion, to change or modify portions of these Terms of Use at any time.
> You acknowledge that Y Combinator may establish general practices and limits concerning use of the Site,
> You further acknowledge that Y Combinator reserves the right to change these general practices and limits at any time, in its sole discretion, with or without notice.
> Y Combinator reserves the right to investigate and take appropriate legal action against anyone who, in Y Combinator’s sole discretion, violates this provision, including without limitation, removing the offending content from the Site, suspending or terminating the account of such violators and reporting you to the law enforcement authorities.
Then alternatives are:
- Grok - where I absolutely have 0 trust in X.ai's interst in "pushing humanity forward".
- OpenAI and Anthropic - which seem to try to be building the biggest moat they can by pushing to ban open models. And at the same time want to be an Arbiter of what level of intelligence I can use.
- Google and Meta - I don't need to talk about the practices of these companies.
Yes, the terms of service aren't great. But the alternatives aren't great either. I don't believe that a future which OpenAI and Anthropic are pushing for has my best interest in mind.
Chinese companies do not follow American laws and there are absolutely no consequences for violating it.
Moreover, the average American is not even aware of exactly what the legal/judicial environment is like in China. If your code and data is stolen, you can't fly to China and demand justice in the courts.
... lmk when anthropic/openai/spacex/xai are held accountable for anything. Anything at all. Hard to be when you're _writing_ the rules.
Artificial Analysis ranks GPT 5.6 Luna similar to GPT 5.4, but that never matches my real world experience. AA seems to do a good job making a single number as representative as possible but there is still so much benchmarks don't communicate.
For implementation tasks, where I have the problem already defined and researched, or just simple task, I'd definitely use something like Luna xhigh or max. If the task is vague, or involves planning, I'd rather use Sol medium, even though it's theoretically worse on benchmarks.
RIP Nivida shareholders
And I don't see a situation where serious API payers are OK with handing the Chinese state all their data. Like manufactures of decades past did and learned a hard, even existential, lesson for it. The state mantra has been "Collect and Copy" for a long time now, tech just hasn't had that moment to experience it yet.
So that leaves local hosting/leasing, but one of those has totally non-practical economics and the other doesn't have enough compute to meet any kind of real demand.
I also have yet to meet a single person who isn't neck-deep in the tech space mention a Chinese LLM. It's 100% the big American three.
If anything it's custom chips from the labs that threatens Nvidia.
https://www.ft.com/content/32a70a3c-7d28-40b4-808e-36edb58c7...
Because I genuinely can't tell if you mean Google or SpaceX/X.ai lol.
xAI is already selling spare compute, and basically exists just to gas spacex's perceived valuation.
Further quote:
"Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale."
https://z.ai/blog/glm-5.3-flash
While I think China would always get to hardware self-sufficiency eventually, all export controls have done is (1) accelerate China's development, and (2) divert revenue that would've otherwise gone to NVIDIA/AMD/etc instead.
That doesn't mean by any stretch of the imagination Nvidia will disappear. But the entire stock market valuation, not just tech, has had me scratching my head for a while.
This isn't the kind of thing you can hash out in public and go back and forth on. Once you put it out there, the other party will take steps to make sure they don't have to rely on us in the long run.
Zai is on another "export control" list outside the broader 1. Doesn't help.
Nvidia will do just fine. (Disclaimer: not a shareholder. At least, not directly.)
Edit: Ah:
> This stealth model was developed and operated by ZAI, revealed to be ZAI GLM-5.3-Flash.
> Before release, we tested GLM-5.3-Flash anonymously as ox-alpha on OpenCode and OpenRouter to gather user feedback. It quickly became the most popular model of the week — with all of this traffic served on Chinese AI chips.
I'm sure the chips are fine, but they clearly didn't have enough capacity for the demand they had (that 100T/day claim was asbolute bs)
Get a 210 strike put contract and if your thesis is that nvidias current 10 day slide continues you could make some money.
>> "They are already there on open weight models and Jensen knows that it is only a matter of time until China catches up with GPUs or other AI accelerators."
It is also why Nvidia becoming a bank for other AI companies who are unable to find VCs to fund them isn't really a good thing and that is bearish.
[0] https://news.ycombinator.com/item?id=49397204
[1] https://news.ycombinator.com/item?id=49431231
What irks me about this is that the harnesses seem to be just an afterthought here.
Don't get me wrong, I love messing around with installing Pi, getting it hooked up with OpenRouter, and just trying all kinds of different stuff, local models, etc... but when it comes to literally just setting up a productivity environment and trusting my entire machine with it, I just run Codex.
I have heard from anecdotes where people have indeed replaced their main drivers with DeepSek V4 Flash or GLM and state that "it's almost as good as... [claude/gpt]" but I never hear anyone say "yeah, this is the model/harness that I now run on my machine and don't mess with it"
* me raises hand.-
Think z code gives a token bonus though
This is pretty hefty for a "flash" model, even a 256 GB setup is insufficient at q4 - and q4 is already the worst-but-still-acceptable quant in my experience. The benchmarks look great, especially since GLM tends to be more honest than the average Chinese lab, but you’ll need to splurge to run it at home.
@edit: so many releases that I forgot to math. This fits just fine in q4, realistically the minimal hardware would be 192gb - so blazing fast on double rtx 6000 pro and usable on 256gb unified memory. You could even go with 5bit quant on 256gb.
… you’ll still need to splurge, though.
From a biased source, but would be big if true. I've had great results with GLM 5.2.
From their subscription page, the smallest plan gives you about 97M tokens weekly for 5.3 but 292M for 5.3 Flash. Not exactly 10x the limit.
It's at least close (even if not better) from the Ox Alpha runs. For the price it's definitely great.
│ https://openrouter.ai/api/v1/chat/completions model: stealth/ox-alpha auth: OPENROUTER_API_KEY status: 404 Not Found response: {"error":{"message":"Thank you for participating in the Stealth Ox Alpha testing period. This model was ZAI's GLM-5.3 Flash.
│ Use it now: https://openrouter.ai/z-ai/glm-5.3-flash","code":404},"user_...":"}
Like were executive at Google and AWS and Microsoft expecting this kind of performance from models smaller than what openai/anthropic have been doing? Are we really in a "compute desert"?
e.g. "Agent Coding Performance by Effort Level" cuts Y-axis from 0~20.
- This makes it as if GLM-5.3-Flash made a bigger jump than it claimed as the Y-axis does not increase much (stupid trick used in biz reports)
I did mention that ox was working ok for me, and having an open-weight comparable to close to SOTA makes it very compelling for me to try it out locally (well, only if I got more VRAM)
edit: nevermind. it is there in the artifical analysis scatter plot, but is greyed-out.
MUCH more interesting is that in that chart, their cost is WAY off. The actual chart shows GLM 5.3 Flash at $0.09, but their chart shows $0.045...
How is the business model of Anthropic/OpenAI will sustain?
I don't see how NVIDIA can keep their spot as belle of the ball. If LLMs and friends are truly to become as useful and ubiquitous as everyone thinks they will, then commoditization is the only option.
FWIW, people were saying "ASICs will kill CUDA demand!" since the crypto mining boom. Then a few months later, CUDA found another niche application in LLM applications.
With the mounting demand for robotics, surveillance and autonomous weapons, I don't see how Nvidia couldn't keep their spot. They have their pick of the litter with hundreds of market segments, and unlike the rest of FAANG they're not afraid to branch out.
- Input: $0.15 - Output: $0.50 - Cached input: $0.03
https://openrouter.ai/compare/deepseek/deepseek-v4-flash-073...
EDIT: Looks like they are swizzling around the pricing dynamically on that page, on both the GLM and the DS sides, so who knows.
It's too big, bright and resourceful of a country to choose confrontation instead of collaboration.
That's good. Keep going.
Now the US is behind in EVs can you guess what they're doing? [1]
[1] https://evwire.com/p/video-ford-ceo-jim-farley-says-they-fly...
If the USA wanted a copyright treaty with China bad enough, we would negotiate one. China is not breaking any laws here, international or otherwise.
"problem" indeed.
Intellectual property is part of WTO agreements but enforcement is domestic.
US companies do it too, regularly, they simply hire and poach staff from competitors.
Proving it to be IP theft is difficult unless you can prove documents being passed. But often all you need is the know-how of the hired talent.
(281 points, 118 comments)
Is the optimal formula still 20x the amount of model params in tokens for training? Could this mean we're getting a GLM with 1.5t params?
https://x.com/Zai_org/status/2092616204787626030/photo/1
> (...) Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale.
It might be one of the most actually practical tasks that AI might've done because the compounding effects of it and also its implications are/feels so immense. It feels as if Nvidia might be in a slight turbulence from it.