DeepSeek v4

You are of course specifically referring to the math optimised models, not the chat ones folks would generally encounter. Not that I’m trying to contradict you, your point is super valid and I agree with you! But I’m supplementing to help anyone following along who may make choices.

This is when it happened for anyone interested: https://binaryverseai.com/deepseek-math-v2-benchmarks-review...

You run a 671B model at home?

Vertex AI has had deep seek available via API for a while

My apologies, I thought it would be implicit that I am using the top-tier model of the time given the challenge of the tasks. GPT-5.5 was too new in this top comment (although I did test it a bit in a comment below), so I was using GPT-5.4. Gemini is Pro 3.1 Preview.

High bet on 3.1 pro. I use it a lot for math and classic engineering, it's very strong.

I feel like people suck at promoting Opus. Baseline, it's pretty on par with GPT 5.5.

But if you prompt it well - give it the reasoning behind why you're asking it to do something - it pulls far ahead.

Would love to know how GLM 5.1 stacks up in this ranking. Seems like it's on par with Kimi K2.6.

I'd be interested to know when that Opus 4.6 baseline is from given their recent recognition of performance issues. Do you have a paper posted on this review?

Very cool to hear there is agreement with (probably quite challenging?) coding problems as well.

Just ran a couple of them through GPT 5.5, but this is a single attempt, so take any of this with a grain of salt. I'm on the Plus tier with memory off so each chat should have no memory of any other attempt (same goes for other models too).

It seems to be getting more of the impressive insights that Gemini got and doing so much faster, but I'm having a really hard time getting it to spit out a proper lengthy proof in a single prompt, as it loves its "summaries". For the random matrix theory problems, it also doesn't seem to adhere to the notation used in the documents I give it, which is a bit weird. My general impression at the moment is that it is probably on par with Gemini for the important stuff, and both are a bit better than DeepSeek.

I can't stress how much better these three models are than everything else though (at least in my type of math problems). Claude can't get anything nontrivial on any of the problems within ten (!!) minutes of thinking, so I have to shut it off before I run into usage limits. I have colleagues who love using Claude for tiny lemmas and things, so your mileage may vary, but it seems pretty bad at the hard stuff. Kimi and GLM are so vague as to be useless.

Has nothing to do with context length, they have experience training math models, they have a model that would take gold in IMO and a lean prover. Both have been out for almost a year.

Yes to both, I'm paying for them and use the top-tier thinking models.

I don't want to give away too much due to anonymity reasons, but the problems are generally in the following areas (in order from hardest to easiest):

- One problem on using quantum mechanics and C*-algebra techniques for non-Markovian stochastic processes. The interchange between the physics and probability languages often trips the models up, so pretty much everything tends to fail here.

- Three problems in random matrix theory and free probability; these require strong combinatorial skills and a good understanding of novel definitions, requiring multiple papers for context.

- One problem in saddle-point approximation; I've just recently put together a manuscript for this one with a masters student, so it isn't trivial either, but does not require as much insight.

- One problem pertaining to bounds on integral probability metrics for time-series modelling.

I have plans to publish the problems, not any plans to publish how well the LLMs perform on them. The standard for publishing benchmarks is very high, and I'm really just posting vibes here. Still, I hope my experiences are useful to some people, as others experiences have been useful to me.

Agreed, they also have great documentation. There's something to be said for documentation that is so concise, well laid out, and immediately actionable for those looking to get started quickly.

>Western Models are optimizing to be used as an interchangeable product.

But so much investment in their platforms, not just their APIs?

[flagged]

Yeah I think those details are the least of most peoples concerns. I can't vouch one way or another for DeepSeek's documentation but for me what matters most when reading documentation is being able to get the information I want efficiently, not whether someone spelled "software" as "softwares", which is a very common spelling in Asia as an FYI.

I read OpenAI or Anthropic's documentation nowadays and it's just so full of useless junk and self-congratulation that makes it a miserable experience to go through. It's a real shame because OpenAI used to write stellar documentation and publish really lucid papers just few years ago.

No one cares about this kind of stuff. 99% of the devs are not English native speakers, what do you expect ? It works and we all can understand it

The tool calling Python example would have benefitted from actually parsing the tool call. As is, it explains almost nothing.

i dont think deepseek will ever recover from this. huge loss for them. they will stop the pursuit of agi cause of one hn user and a comma.

This tells me a real developer wrote the docs, instead of someone with good English writing skills but is less technical.

> they could have even used their own LLM to edit their documentation to fix grammar issues

In my experience companies who do this rarely stop at using LLMs to fix grammar issues. It becomes full on LLM speak quite fast, especially if there isn’t a native English speaker in the room who can discern what’s good and bad writing.

pedantry

i prefer it cuz it indicates they didnt use an LLM to write their documentations and that its human generated

> Example: the second sentence on the first page says “softwares” but “software” is a mass noun that cannot be pluralized.

I constantly see and hear this mistake from actual humans too.

It's fairly ironic that your own comment contains run-on sentences, speculative claims and phrasing peculiarities like "could have even" instead of "could even have". Perhaps you are less sensitive to this than you think!

Nobody cares, we're talking about quality documentation here, not a couple spelling mistakes

Yeah, of course. Making it cheap/compatible with heavy batching is exactly what they did, that's what I mean. ("with minimal performance overhead")

it's in the footnote text of the first figure of the section the link points to, where "昇腾950" means "Ascend 950"

No, some benchmarks are definitely objective, but most can be easily gamed. For example, most of the benchmarks on the model cards: they have measurable answers that don't rely on a human judge (a human made the question, but the answers are measuring some uncontroversial knowledge or capability). But because there is a single, correct answer, and those answer leak (or are randomly discovered and optimized for in training), they lose value over time, and regardless, they have a ceiling on the intelligence they can measure.

Others are purely subjective, like LMArena, which really only measures the personality and style preferences of the masses at this point, because frontier LLM technical answers are too hard for the average person to judge.

Then there are some interesting one-off benchmarks, but they lack enough rigor, breadth, and samples to draw larger conclusions from.

So we designed our benchmark with 3 goals: objective measurements (individual submissions not dependent on a human or LLM judge), no known correct answer (so simulations can scale to much higher levels of intelligence), and enough variety over important aspects of intelligence. We do this by running multiple models in cooperative/competitive environments with very complex action spaces and objective scoring, where model performance is relative and affected by the actions of other participants.

And yeah, there are some interesting results when you have a more objective benchmark. It should raise eyebrows when every single sub-release of every company's model is better across the board than its predecessor -- that isn't reality.

you are arguing with your belief instead of an objective truth. benchmark is more objective, if you don't agree with it, come up with a better one. but what you believe doesn't matter.

I agree that benchmarks are inherently subjective.

but the fact that you cite your brief as your main argument is funny - you don't even have any inherently subjective numbers to justify what you believe, you only have "I don't believe".

I think most hardcore people I know are still sticking with 4.5 for coding workflows

Take a look at the Time column in https://gertlabs.com/?mode=oneshot_coding -- this is the total time to complete a solution for a reasonably complex problem end-to-end (you would have to divide by avg submission size to estimate tok/s). It's fast in the sense that most of the smart, recent Chinese releases are quite slow, especially the DeepSeek Pro variant. Opus 4.7 is also quite fast.

If pure speed is most important for your use case, GPT-5.3 Chat is the fastest model we've tested and it's still reasonably smart. Not meant for agentic tool usage / long context, though.

So it might be more useful for business applications or non-engineering usage where you don't need exceptional intelligence, but it's useful to get fast, cheap responses.

Waiting on public API release. Once it drops, results will be up within 24 hours.

In six month deepseek won't be sota anymore und usage will be wayyyy down.

Research and training costs have to be amortized from somewhere; and labs are always training. I'm definitely keen for the financials when the two files for IPO though, it would be interesting to see; although I'm sure it won't be broken down much.

Amazingly, the current depreciation overestimates the retained value of GPUs.

In 2023, the depreciation schedule for H100s was 2 years, but they are still oversubscribed and generating signficant income.

Coreweve has upped their depreciation for GPUs to 6 years(!) now, which seems more realistic.

https://www.silicondata.com/blog/h100-rental-price-over-time

I'm pretty sure OpenAI and Anthropic are overpricing their token billed API usage mainly as an incentive to commit to get their subscriptions instead.

Google stated a while back, that with tpus they are able to sell at cost / with profit.

Aka: everyone who uses Nvidia isn't selling at cost, because Nvidia is so expensive.

This is nonsense. The real reason is because the US companies are scamming the public, as per usual.

It is possible to question the sustainability of the AI buildout and not have a dogmatic position on AI development.

There are still major unanswered questions here. For instance, all of the incremental data capacity build out is going to businesses that have totally unknown LT unit economics and that today are burning obscene amounts of cash.

The people who doubted the sustainability of dot com era bubbles were correct even though the tech was actually transformational. Personally I expect roughly the same outcome.

Before the AI bubble that will burst any time now, there was the AI winter that would magically arrive before the models got good enough to rival humans.

These models are open and there are tons of western providers offering it at comparable rates.

As this is a new arch with tons of optimisations, it'll take some time for inference engines to support it properly, and we'll see more 3rd party providers offer it. Once that settles we'll have a median price for an optimised 1.6T model, and can "guesstimate" from there what the big labs can reasonably serve for the same price. But yeah, it's been said for a while that big labs are ok on API costs. The only unknown is if subscriptions were profitable or not. They've all been reducing the limits lately it seems.

not soooo much though. It's heavily subsidized for residential consumption, but industrial power rates are almost comparable to the US (depends on the state you go to etc).

you know, if you don't have to pay insane salary for your top engineers, and don't have to pay billions for internet shills to control the narrative, then all of the labs will be insane profitable.

The claim that APIs are subsidized is very common.

eg:

Token prices are significantly subsidized and anyone that does any serious work with AI can tell you this.

https://news.ycombinator.com/item?id=47684887

(the claims don't make any sense, but they are widely held)

Yeah, subscriptions used to be extraordinarily generous. I miss those days, but the reinvigoration of open weight models is super exciting.

I'm still playing with the new Qwen3.6 35B and impressed, now DeepSeek v4 drops; with both base and instruction-tuned weights? There goes my weekend :P

When China does it it's communism. When companies in the west get massive tax cuts, rebates, incentives and subsidies, that's just supporting the captains of industry.

Comments like this are why I go to the comments! I never would have thought to check.

And while I'm here I want to note that I feel there's a big misunderstanding of what is and isn't demonstrated by DeepSeek. So far as I can tell the major (and important!) innovation is reproducing near-frontier level capabilities at a fraction of the cost, but it may be the case that iterating forward at the frontier is the costly thing and is a cost borne by Western companies and that nuance seems to get lost with DeepSeek. Which is not to say that as a matter of principle that non Western companies aren't sometimes capable of jumping into the lead (Kimi has been super impressive) but if GPT/Claude/etc "only" lead at the frontier with more expensive models, that's still a moat.

Thank you for this due diligence, I was just reading through the technical report and couldn’t find any references to the software stack or hardware mentioning Huawei either and came back here wondering about this comment that I had read earlier.

Here's a note about running entirely on Huawei chips:

https://finance.yahoo.com/sectors/technology/articles/deepse...

Not long ago the story was this:

DeepSeek’s next AI model delayed by attempt to use Chinese chips

https://www.ft.com/content/eb984646-6320-4bfe-a78d-a1da2274b...

They mention it uses MXFP4 quant which is a blackwell capability but it looks like this is also supported by ascend 950 series according to marketing material

DeepSeek is planning to use Huawei extensively for inference

“Due to constraints in high-end compute capacity, the current service capacity for Pro is very limited. After the 950 supernodes are launched at scale in the second half of this year, the price of Pro is expected to be reduced significantly.”

https://x.com/jukan05/status/2047516566149816627

I said the same thing as you and I got summarily downvoted (https://news.ycombinator.com/item?id=47888227).

That HN is quick to upvote an unsubstantiated comment ( the grandparent one, because it aligns with the anti US bias? ) and downvote fact finding one doesn't bode too well for the community as a whole. I have seen enough how polticial ideology colors everything in my home country( Malaysia), and the decline of the country is palpable, and I don't expect to find such a thing here. We are supposed to be impassioned and rational, right ?

Render to Jesus what's due to him, ditto for Caeser.

They would have moved to their own stack regardless. They've got the people and resources for it, and they've witnessed the fallout of globalization and experienced dependency on semi-hostile political powers enough to know that it's the smart move.

It's also more or less the same move that they've been using pretty much since the WTO entry: take on foreign manufacturing, copy the products, sell knockoffs as their own, build new products on top of the that knowledge.

Referring to the Dwarkesh interview clearly.

Jensen came across as incredibly defensive and intentionally close-minded, shows that even billionaires suffer from "a man can't understand something if his paycheck depends on him not understanding it."

Your assertion is silly: did Tesla selling electric cars into China stop them from delivering their own industry? They were going to develop their domestic industry regardless.

We simply don't know the counterfactual, if they had unlimited access to Nvidia chips, how far ahead would their models be?

> but if you push them they will build on an all Chinese stack

That's alright. It delays them at least.

I'm American. If the choice is between the current US direction or China, then no, I don't think the word "healthy" should be anywhere near this discussion.

I’m also a Brit and agree 100%.

We need to accept that being too close to America is harming us and start funding projects to protect our assets e.g talent leaking out to American entities.

It’s a shame your country couldn’t get back its technical edge.

america is a continent. let’s take back our vocabulary (fellow european here). the little orange man shows very well what i mean when he started giving names to the gulf of mexico.

"not perfect" is a _very_ big simplification of what China is though

Americans are also tired with what’s going on.

As a different Brit I do not accept such moral relativism.

China’s governments actions are on a completely different level - for example:

“””

Since 2014, the government of the People's Republic of China has committed a series of ongoing human rights abuses against Uyghurs and other Turkic Muslim minorities in Xinjiang which has often been characterized as persecution or as genocide.

“”” https://en.wikipedia.org/wiki/Persecution_of_Uyghurs_in_Chin...

https://www.amnesty.org/en/location/asia-and-the-pacific/eas...

Yes Trump is clearly trying Totalitarianism in America, but it is orders of magnitude different from what is happening in China.

Fellow countryman here. I came here to say the same thing

I don’t know if we’re ahead of the curve but that tired feeling has started turning into hate here in the EU. I guess being threatened with invasion does that to you.

The next decade is going to look very different with America Alone.

As someone that lived in Britain for 15 years until 2024, I'm not sure a nation with a GDP per capita lower than Poland, that is now poorer than every state in America, with a gang rape epidemic the government tried to suppress investigating should really concern itself with how other countries are ran.

> I'm tired of America with everything that's going on.

Yeah, me too. All that pesky saving the world stuff that we do on the regular is so exhausting sometimes.

> Let's see how long it takes before the big US AI companies start lobbying to outright ban use of Chinese AI, even the open source / local models. For "national security" reasons, of course.

Already do on EVs.

? Every company should lock down AI and only whitelist allowed tools.

We are "only" allowed Claude and MS Copilot for security reasons and cost reasons.

This is already happening. My company just went through this

Hopefully the US’ self imposed isolation will mean that when they do, they aren’t able to force the rest of the world to follow suit.

They already did - State Dept. launched global campaign against Deepseek.

Same thing for Chinese EVs. America can't compete in the free market anymore

In the US, yes, but Huawei has been gaining ground selling its SuperPod/Ascend turnkey solutions internationally, with some major recent wins in Thailand, Brazil, Egypt and Morocco.

This is a pretty banal comment at this point. Open source is the term used in the LLM community. It's common and understood. Nobody is going to release petabytes of copyrighted training data, so the distinction between open source vs weights is a rather pointless one.

Unfortunately Uber and the likes have been doing this but nobody batted an eye

Theoretically yes. It is entirely possible to poison the training data for a supply chain attack against vibe coders. The trick would be to make it extremely specific for a high value target so it is not picked up by a wide range of people. You could also target a specific open source project that is used by another widely used product.

However there is so many factors involved beyond your control that it would not be a viable option compared to other possible security attacks.

All China (or anyone) has to do is deliver a close to equal product at a much cheaper price and make it scaleable / usable... which is what they're doing. It doesn't have to be malicious at all. Just a good product at a good price. The US is basically in a recession that's hiding behind insane AI investments.

If there is, couldn't they exist in any model?

I don't mean that flippantly. These things are dumped in the wild, used on common (largely) open source execution chains. If you find a software exploit, it's going to affect your population too.

Wet exploits are a bit harder to track. I'd assume there are plenty of biases based on training material but who knows if these models have a MKUltra training programme integrated into them?

What about LLMs from other origins? What makes them less risky?

Backdooring software at scale.

Spearphishing.

Building reliance and exploiting it, through state subsidies, dumping, and market manipulation.

Handicapping provision to the west for competitive advantage.

There must be. The executives at my company wouldn't have banned them all for no reason after all.

I sometimes wonder is there are any security risks with using LLMs from the US.

From my experience, kinda the opposite? It's like Chinese software is... Harder to weaponize or hurt yourself on. Deepseek is definitely censored, but I've never caught it being dishonest in a sneaky way.

Is this a serious comment? It honestly reads like the last famous words.

Of course there are risks.

The model is open weights, so you can download it from the link given at the top.

Then you can run it using some inference backend, e.g. llama.cpp, on any hardware supported by it.

However, this is a big model so even if you quantize it you need a lot of memory to be able to run it.

The alternative is to run it much more slowly, by storing the weights on an SSD. There have already been published some results about optimizing inference to work like this, and I expect that this will become more common in the future.

There are cases when running slowly a better model can still be preferable to running quickly a model that gives poor results, especially when you do not use it conversationally, but to do some work with agents.

If you don't have the source code then it makes no difference. If you have the weights and are running some model via llama.cpp, then you are using whatever API llama.cpp is using, not the API that was used to train the model or that anyone else may be using to serve it.

If you found a rare 9000 card with 200+ GB of VRAM, sure

If the card supports vulkan and the model has gguf weights. llamacpp has excellent vulkan support that is being actively developed and is not that far behind CUDA where speed is concerned.

* https://github.com/ggml-org/llama.cpp/releases

The response from US corporations has been banning Chinese models claiming they’re spying or something.

>mostly taking a public paper from 2017

I love the implication that this paper just dropped out of thin air and not decades of private AI research funded by a US company.

>The Chinese made it public, among other things to show how fragile this is as a business

The Chinese distill US models, that's why they keep trailing close but never exceeding. It's easy to make things public when you didn't take on any of the cost of developing the technology. Stealing US IP and selling cheap copies has been China's MO for decades now.

The model is not "open source", but it is an open weights model.

You can download it from the link given here at the top and you can run it on your own hardware, with whichever open-source harness you prefer, without having to worry about token cost or about subscription limits or about any future degradation in performance that you cannot control.

The recent history has demonstrated that such risks are very significant.

Being open weights is important for anyone who wants to use an LLM. Being open source is important only for a subset of those, who have the will, the knowledge and the means to train a model from its training data.

Having access to the training data used by a model would be very nice, but the reality is that for a normal LLM user it is very beneficial to use an open-weights model with an open-source harness, but it would be much harder to exploit the advantage of having access to all the information about how the LLM has been created.

For me open source means that the entire training data is open sourced as well as the code used for training it otherwise it's open weight. You can run it where you like but it's a black box. Nomic's models are good example of opensource.

Look up Olmo 3, where the have open weights, checkpoints, training data, and training process.

AllenAi is the fullest open ai I know of

you talk like there isn't censorship in american AIs, like Israel topics.

I can't wait for Taiwan to peacefully reunify with the mainland so the west with its constant war waging won't even have this talking point

> China asks other country not to meddle with internal separatism > They also dont support separatism in my country

Understandable.

Quit a bit better then made to bomb little girl schools in Iran.

Just ask it for a summary of the USA’s role in Iran, Gaza, Lebanon and its recent threats against Panama, Cuba and Greenland! It might be able to keep track.

pretty sure you can ask whatever you want and it will tell you official stance agreed by almost all countries in the world that Taiwan is part of China as it's recognized by your own country (I don't even know where are you from, but there is like 98% chance I'm right)

>I asked DS itself and it denied this

Bro, seriously?

If those docs were written by Deepseek, it’s also a pretty positive review of the model.

doesn't it get tiring after a while? using the same (perceived) gotcha, over and over again, for three years now?

no one is ever going to release their training data because it contains every copyrighted work in existence. everyone, even the hecking-wholesome safety-first Anthropic, is using copyrighted data without permission to train their models. there you go.

Aww yes, let me push a couple petabytes to my git repo for everyone to download...

They are exactly open source. The training data is the internet. Don't say it's on the internet. It IS the internet.

The training scripts are in Megatron and vLLM.

Weights are the source, training data is the compiler.

On-device is incredibly far away from being viable. A $20 ChatGPT subscription beats the hell out of the 8B model that a $1,000 computer can run.

Nvidia's forward PE ratio is only 20 for 2026. That's much lower than companies like Walmart and Costco. It's also growing nearly 100% YoY and has a $1 trillion backlog.

I think Nvidia is cheap.

This is only good advice if you don’t have the need to understand what’s happening on the edge of the frontier. If you do, then you’ll lose on compounding the knowledge from staying engaged with the major developments.

This one’s been particularly hard to sit out because the executive and managerial class are absolutely mainlining this stuff and pushing it hard on the rest of the organization, and so whether or not I want to keep up, I need to, because my job is to actually make stuff work and this stuff is a borderline existential risk to the quality of the systems I’m responsible for and rely on.

I didn't express this well but my interest isn't "who is in the top spot", and is more _why and _how various labs get the results they do. This is also magnified by the fact that I'm not only interested in hosted providers of inference but local models as well. What's your take on the best model to run for coding on 24GB of VRAM locally after the last few weeks of releases? Which harness do you prefer? What quants do you think are best? To use your sports metaphor it's more than following the national leagues but also following college and even high school leagues as well. And the real interest isn't even who's doing well but WHY, at each level.

It is funny seeing people ping pong between Anthropic and ChatGPT, with similar rhetoric in both directions.

At this point I would just pick the one who's "ethics" and user experience you prefer. The difference in performance between these releases has had no impact on the meaningful work one can do with them, unless perhaps they are on the fringes in some domain.

Personally I am trying out the open models cloud hosted, since I am not interested in being rug pulled by the big two providers. They have come a long way, and for all the work I actually trust to an LLM they seem to be sufficient.

For coding Opus 4.5 in q3 2025 was still the best model I've used.

Since then it's just been a cycle of the old model being progressively lobotomised and a "new" one coming out that if you're lucky might be as good as the OG Opus 4.5 for a couple of weeks.

Subjective but as far as I can tell no progress in almost a year, which is a lifetime in 2022-25 LLM timelines

I don't trust the benchmarks either, so I maintained a set of benchmarks myself. I'm mostly interested in local models, and for the past 2 years they have steadily gotten better.

Can't argue with subjective experience, but if there were some tasks that you thought LLMs can't do two years ago, maybe try again today. You might be surprised.

Yup, can't really be used in production atm.

Its on OR - but currently not available on their anthropic endpoint. OR if you read this, pls enable it there! I am using kimi-2.6 with Claude Code, works well, but Deepseek V4 gives an error:

`https://openrouter.ai/api/messages with model=deepseek/deepseek-v4-pro, OR returns an error because their Anthropic-compat translator doesn't cover V4 yet. The Claude CLI dutifully surfaces that error as "model...does not exist"

Try interacting with it through the website, it will give an error and some explanation on the issue. I had to relax my guardrail settings.

It is tricky to build good infrastructure for prompt caching.

It’s not remotely hypothetical you’d have to be living under a rock to believe that. And the fusion with a one-party state government that doesn’t tolerate huge swathes of thoughtspace being freely discussed is completely streamlined, not mediated by any guardrails or accountability.

This “no harm to me” meme about a foreign totalitarian government (with plenty of incentive to run influence ops on foreigners) hoovering your data is just so mind-bogglingly naive.

I, personally, have never been asked for an asshole scan, but I'm interested in providing one if you can point me to a company that's offering.

As someone with Tibetan friends and as someone from India, Chinese ulterior motives are way more clear.

Lol EU pats you on the head

Its sad to see how you have regulated yourselves into a position where Mistral is your only claim.

Same

> Internet comments say that open sourcing is a national strategy, a loss maker subsidized by the government. On the contrary, it is a commercial strategy and the best strategy available in this industry.

This sounds whole lot like potatoh potahto. I think the former argument is very much the correct one: China can undercut everyone and win, even at a loss. Happened with solar panels, steel, evs, sea food - it's a well tested strategy and it works really well despite the many flavors it comes in.

That being said a job well done for the wrong reasons is still a job well done so we should very much welcome these contributions, and maybe it's good to upset western big tech a bit so it's remains competitive.

Please don't slander the most open AI company in the world. Even more open than some non-profit labs from universities. DeepSeek is famous for publishing everything. They might take a bit to publish source code but it's almost always there. And their papers are extremely pro-social to help the broader open AI community. This is why they struggle getting funded because investors hate openness. And in China they struggle against the political and hiring power of the big tech companies.

Just this week they published a serious foundational library for LLMs https://github.com/deepseek-ai/TileKernels

Others worth mentioning:

https://github.com/deepseek-ai/DeepGEMM a competitive foundational library

https://github.com/deepseek-ai/Engram

https://github.com/deepseek-ai/DeepSeek-V3

https://github.com/deepseek-ai/DeepSeek-R1

https://github.com/deepseek-ai/DeepSeek-OCR-2

They have 33 repos and counting: https://github.com/orgs/deepseek-ai/repositories?type=all

And DeepSeek often has very cool new approaches to AI copied by the rest. Many others copied their tech. And some of those have 10x or 100x the GPU training budget and that's their moat to stay competitive.

The models from Chinese Big Tech and some of the small ones are open weights only. (and allegedly benchmaxxed) (see https://xcancel.com/N8Programs/status/2044408755790508113). Not the same.

Weights are the source, training data is the compiler

I think their motives are pretty transparent, as are china’s, as ever, you have to pick the lesser of two evils.

> It’s my experience that models that perform very well on benchmarks do not necessarily perform well in real life

Well, yeah... Like Opus 4.5, 4.6, 4.7. Top of the benchmarks and yet it's a pile of crap at the moment and has been for months.

If benchmarks are all to be believed then gemini 3.1 and grok 4.2 are still in the lead pack. A laughable notion to anyone who has actually tried to use them and compared.

True, and they're being tried in a federal court of law for it. NYT v. OpenAI is still very much alive, these things just take a while. Can the same be said about DeepSeek or any other open-source model provider performing distillation?

Literally.

Altman and Amodei are so mad about muhh model when they steal our data and pollute the Internet with slop.

let's not forget that calling copyright infringement theft is hyperbole, and the claim that AI is even infringing is also dubious at best, and that the concept of intellectual property at all is also ethically dubious

I am thinking about getting some 1 year promotion as a student before defending my PhD.

How do you find the token speed of open router with kimi?

I have tasks that used to take ~3-5min with Sonnet 4.6. With OpenRouter Kimi, the same task takes 10+ min. It's also just obviously slower in opencode sessions. The results are good, and I love the lower cost, but the speed can be frustrating.

I just want to try to make something useful out of my time, that's why I'm subscribed to Codex at the moment. 20€ is affordable, not really a problem. But yes, maybe I would do me a favor unsubscribing and going back to the old ways to learn properly.

I tried OpenRouter but I feel the money flies even with these models, it is not comparable to a subscription but yes, it's very good for trying. Maybe I should test other models alongside GPT 5.5 to see which one fits me.

I had Claude make me a quick tool to combine my Claude Code token usage (via ccusage util) with OpenRouter pricing from the models API

I'm on Max x5 plan and any of the 'good' models like Kimi 2.6, GLM, DeepSeek would have cost 3-5x in per-token billing for what I used on my Claude plan the last three months

So unless my Claude fudged the maths to make itself look better, seems like I'm getting a good deal

I am not so sure, credits fly when using any model trough API if I use it as much as I use Codex.

Output is what the compute is used for above all else; costs more hardware time basically than prompt processing (input) which is a lot faster

input tokens are processed at 10-50 times the speed of output tokens since you can process then in batches and not one at a time like output tokens

Which model's best depends on how you use it. There's a huge difference in behaviour between Claude and GPT and other models which makes some poor substitutes for others in certain use cases. I think the GPT models are a bad substitute for Claude ones for tasks such as pair-programming (where you want to see the CoT and have immediate responses) and writing code that you actually want to read and edit yourself, as opposed to just letting GPT run in the background to produce working code that you won't inspect. Yes, GPT 5.4 is cheap and brilliant but very black-box and often very slow IME. GPT-5.4 still seems to behave the same as 5.1, which includes problems like: doesn't show useful thoughts, can think for half an hour, says "Preparing the patch now" then thinks for another 20 min, gives no impression of what it's doing, reads microscopic parts of source files and misses context, will do anything to pass the tests including patching libraries...

Agree with your assessment, I think after models reached around Opus 4.5 level, its been almost indistinguishable for most tasks. Intelligence has been commoditized, what's important now is the workflows, prompting, and context management. And that is unique to each model.

I found Opus 4.7 to be actually worse than Opus 4.6 for my use case

Substantially worse at following instructions and overoptimized for maximizing token usage

Is Opus nerfed somehow in Copilot? Ive tried it numerous times, it has never reallt woved me. They seem to have awfully small context windows, but still. Its mostly their reasoning which has been off

Codex is just so much better, or the genera GPT models.

This resonates with me a lot.

I do some stuff with gemini flash and Aider, but mostly because I want to avoid locking myself into a walled garden of models, UIs and company

What do you run these on? I've gotten comfortable with Claude but if folks are getting Opus performance for cheaper I'll switch.

eh idk. until yesterday opus was the one that got spatial reasoning right (had to do some head pose stuff, neither glm 5.1 nor codex 5.3 could "get" it) and codex 5.3 was my champion at making UX work.

So while I agree mixed model is the way to go, opus is still my workhorse.

I don’t know what people are doing but Minimax produced 16 bugreports which of 15 was false positives (literally a mistake).

In contrast ChatGPT 5.3 and also Opus has a 90% rate at least on this same project. (Embedded)

All other tests were the same. What are you doing with these models?

actually this is not the reason - the harness is significantly better. There is no comparable harness to Claude Code with skills, etc.

Opencode was getting there, but it seems the founders lost interest. Pi could be it, but its very focused on OpenClaw. Even Codex cli doesnt have all of it.

which harness works well with Deepseek v4 ?

The thing is, it doesnt need to beat 4.7. it just needs to do somewhat well against it.

This is free... as in you can download it, run it on your systems and finetune it to be the way you want it to be.

Tbh I was more productive with 4.6 than ever before and if AI progress locks in permanently at 4.6 tier, I’d be pretty happy

It is more than good enough and has effectively caught up with Opus 4.6 and GPT 5.4 according to the benchmarks.

It's about 2 months behind GPT 5.5 and Opus 4.7.

As long as it is cheap to run for the hosting providers and it is frontier level, it is a very competitive model and impressive against the others. I give it 2 years maximum for consumer hardware to run models that are 500B - 800B quantized on their machines.

It should be obvious now why Anthropic really doesn't want you to run local models on your machine.

Apparently glm5.1 and qwen coder latest is as good as opus 4.6 on benchmarks. So I tried both seriously for a week (glm Pro using CC) and qwen using qwen companion. Thought I could save $80 a month. Unfortunately after 2 days I had switched back to Max. The speed (slower on both although qwen is much faster) and errors (stupid layout mistakes, inserting 2 footers then refusing to remove one, not seeing obvious problems in screenshots & major f-ups of functionality), not being able to view URLs properly, etc. I'll give deepseek a go but I suspect it will be similar. The model is only half the story. Also been testing gpt5.4 with codex and it is very almost as good as CC... better on long running tasks running in background. Not keen on ChatGPT codex 'personality' so will stick to CC for the most part.

Their Chinese announcement says that, based on internal employee testing, it is not as good as Opus 4.6 Thinking, but is slightly better than Opus 4.6 without Thinking enabled.

Anthropic fans would claim God itself is behind Opus by 3-6 months and then willingly be abused by Boris and one of his gaslighting tweets.

LMAO

FWIW it's also like 10x cheaper.

There appears a flight of dragons without heads. Good fortune.

That's literally what the I Ching calls "good fortune."

Competition, when no single dragon monopolizes the sky, brings fortune for all.

I always find it an illuminating experience about the power of mass propaganda every time I see an American believe they somewhat have the moral high ground over China, despite starting a new war somewhere around the globe either for petrol or on behalf of Israel every six months.

> Still not sure how I feel about China of all places to control the only alternative AI stack, but I guess it's better than leaving everything to the US alone.

Fully agree. From a US perspective, that sucks. For everyone else it's pretty great.

At this point the world's opinions of China are better than those of the US in some polls. One country invests and helps build infrastructure on a massive scale globally, the other alienates allies, causes countless conflicts, and openly threatens to end civilizations.

Indeed, even if one isn't partial to China, there's reasons to be glad that an increasingly hostile US has powerful competition.

> This is about who will dominate the world of tomorrow.

For this you'd need a technological moat. So far the forerunners have burned a lot of money with no moat in sight. Right now Europe is happy just contributing on research and doing the bare-minimum to maintain the know-how. Building a frontier model would be lobbing money into the incinerator for something that will be outdated tomorrow. European investors are too careful for that - and in this case seem to be right.

I don't see the issue. China hosts the alternatives or the only game in town for lots of technologies. China has every interest and right to create products. Not everything that comes out of China is some devious plan to do terrible things. It's people trying to make money just like you and me.

I am not washing away the authoritarianism, but take a look at other economic super powers directionality. Or that of tech ceos as well. At least Chinese tech companies aren't going around praising wwii Germany, writing manifestos, and bombing children at school or fisherman on whims. It is difficult not to see more countries regardless of leadership putting their hat in the ring as a net positive. Especially if it increases sustainability and lowers the price, which this very clearly does. It's even open source...

Moral stances aside, I'd argue it's healthy that the US gets competition from abroad. I appreciate the boost that the world is getting from China - infrastructure and construction projects are a huge benefit to economies. Their focus on green energy has caused a huge influx of affordable solar panels, home batteries, EVs, etcetera, helping reduce the dependency on fossil fuels - while the US and especially the other big money spenders in the middle east would rather the world remain fully dependent on them. But for the past years Europe and now Asia are feeling the pain from being overly reliant on that.

China's policies and government aren't morally defensible and I do fear that they will become more aggressive in spreading their influence and policies onto other countries, but from an economic standpoint what they're doing is super effective. While the previous world power (the US) is stuck in infighting and going through cycles of fixing/undoing the previous administration's damages, instead of planning ahead.

Competition with the Soviet Union gave all the workers in the world better conditions, also advances in science and technology... (And risk of mutual destruction ;)), even if the USSR wasn't good.

The important thing is that LLMs are well-dispersed and the technology is relative open, much more open than it could have been. Alternative worthwhile LLMs will emerge from Europe and other non-US western countries once the economic incentives are there.

> mercy of authoritarian regimes.

Yet, it's the democratic regime which is causing all the chaos around the world and disrespecting the leadership of other jurisdictions, just to keep pushing the petrol dollar going up.

Do we ever think there's any subtle difference between authoritarian and democratic? Where democracy ultimately makes the world a better place?

Thankfully, DeepSeek is the most open of the model providers.

And in the hardware side, RISC-V is gaining a lot of traction in China. So the dependency on a single supplier is lower with the Chinese tech stack than with most western options.

> If China ever feels emboldened enough to go for Taiwan and the US descends into complete chaos, the rest of the world running on AI will be at the mercy of authoritarian regimes.

Alternative being the current reality and world being dominated by US. Let's ask people in Middle East/Asia/South America about how they feel about that. In this current day and age, how is this statement even relevant?

Mistral (a French company) shouldn’t be discounted.

bold to think half the comments here arent from deepseek itself :)

I personally love the bit "us initiated tech war" lol. thats right, they started making AI its their fault! bad imperialist US !

yeah, v5 will do better

You’re right… but that’s on the rest of the world not getting their shit together.

It’s this sort of example (and not properly supporting Ukraine, and not agreeing how to collectively deal with migrants, and not agreeing how to coordinate defence, and myriad other examples) that highlights what a pointless mess the EU is. It’s not a unified block - it’s 27 self-interested entities squabbling and playing petty power games, while totally failing to plan for the future with vision.

The EU could/should have ensured that a European equivalent to OpenAI or Anthropic could thrive, and had competitive frontier models already; instead, they’re years and countless billions behind.

China doesn't even care about Taiwan anymore, their saber-rattling about it is a convenient distraction while they quietly make it completely irrelevant in the next few years.

Isn’t Mistral close in the ballpark?

Come on... I was hoping that Mistral would do something and man that would be great as european but I hear NOTHING from them ever.

I don't know what the problem is. Are we europeans to stupid? Do we just not have enough money / VC money? Are we not proud enough?

:(

Not worse than having our stuff built there. Is it great to be relying on them? No, but at least more stable than US under Trump.

Are third world users opinions of lesser value?

And then westerners wonder why they're disliked in the rest of the world...

The content we see here is that you are Swedish. I am not sure what sort of moral, technical, financial authority are we supposed to be deriving from this.

Dont get me wrong, Sweden is a cool country, but still my point stands.

FWIW, I am a lifelong American citizen and I exclusively use Chinese AI models for programming because I consider Claude and Codex to be highway robbery for the price.

Trust whoever you want, I just don't have the patience (or money) for American models.

> I notice many comments being pro China but they seem to be from the third world (one mentioned a very low salary) I feel the opening of the internet was a mistake.

Yeah, I also really hate when poor people think they're allowed to talk.

Opening of the internet WAS a mistake. During times when whole countries (you know which ones) get geoblocked, the internet (especially online gaming) gets a lot better.

Honestly the China scare mongering is borderline hilarious. The US has literally attacked two countries this year in the span of weeks and is blockading another causing needless deaths. Not too mention the last 50 years of US imperialism making the world a worse place for everyone, except to benefit the few (doesn't benefit Americans, only capitalists).

The idea that China is worse than America is laughable. LMK when China invades 5 countries in a span of 20 years unimpeded by anyone else in the world and maybe I'll be scared.

Until then it's quite clear how consumers benefit from actual competition and it's not because of the US.

Also you saying you trust the US when they just threatened to invade Greenland (a threat so credible that Denmark was planning a full scale resistance against US troops).

Sorry but the curtains are truly coming down and the US will become one of the most hated nations in the world while 100s of millions will needlessly starve and die because of the actions of Americans that simply don't give a fuck.

FWIW, I'm not just talking about Trump either. Democratic politicians are just as much to blame, they champion corporatism and imperialism as much as Republicans and the only issues D leadership seems to have is that the "right process" wasn't being followed.

I say this as someone who is a literal democratic operative within the party.

China never threatened this: "A whole civilization will die tonight, never to be brought back again.". Also China never announced that it was going to attack Europe. I trust them much more compared to a malignant narcissist that doesn't care if the whole world burns now that he doesn't have much left to live.

There is no morality at Country level. The talk about values, morality, just world is only a lip service and pretty much every smart person knows this. If you still want to hold a country for moral standards, our own dear USA's standards would be pathetically low. One example - we force and demand every other country to use USD as the reserve currency. If anyone considers the alternative, we follow the usual routines (bombing, hacking, kidnapping, Tarrifs, coercing, currency sabotage, etc). If two countries in some remote corner of world want to exchange goods and transact in their own local currency, what legal rights does the USA have to stop it and force them to use USD? and punish them if they do not listen? Just because we are aligned with Western Europe, do not assume moral high grounds.

I don't see any sense in trusting the US more than China. There are arguably as many arguments to say the US is horrible as the current dominant country as China would be. If anything, a multipolar world would be more positive, specially to the EU, as currently the EU is just US's bitch, and has to live by appeasing Mr. Donny, as done in the stupid trade deal signed by Von Der Leyen.

Also, feeling the opening of the internet as a mistake show the degree of your ignorance, people from third world countries also have the right to speak as much as you do, your opinion is not more valid than anyone else's.

For context, I am Italian-Brazilian, so I pretty much have been exposed to both sides (western and non-western, even though we can argue that Brazil is more west aligned).

Deeper than the inability to digest. The incapability to comprehend it.

China's fall in the 19th century came at them for the same reason. How could these European savages be stronger, thus better than us? Our intelligence service must be out of their mind.

Because it worked on Japan in the 80s and 90s and sometimes “Americans” have a hard time telling the two cultures apart.

I'm no huge fan of America, but claiming China is as good or better at innovation is asinine.

It costs 100-1000x less manpower, money, and time to hug the heels of innovators than to actually pioneer. Say what you will about America but they absolutely lead technological innovation and it's not even remotely close.

America has been making short term and short sighted moves to try to widen a gap that cannot sustain. They have chosen the wrong strategy out of fear and greed. Cooperation is the right strategy. Isolationism will not work in the long term except for maybe the handful that drove it. The irony is that it's an anticompetitive and anticapitalist move to do what they have been doing, so it's not even on principal.

Which Monopoly? Are all large enterprises in USA? There are tons of them outside and they will run the open ones and cheapest ones to infer and those are Chinese. I run Chinese models at home and don't bother with cloud. If I could call the shots at work, we will switch 100% to Chinese models so everyone could have "unlimited" tokens.

You might be underestimating how significantly cheaper this is and how much people care about price.

Walmart is a horrible company owned by horrible people and yet it’s cheap so it dominates.

If the quality really is in the Opus 4.6 range (considering how bad 4.7 is), then it’s a pretty big deal.

Multiple independent implementations inherently virtuous. After all each individual party may innovate in ways that benefit everyone ultimately.

Also it's tech they can be sure we can't cut them out of or tariff and money flowing from Chinese companies to other Chinese companies which we appreciate the benefits of when the shoe is on the other foot.

I think you missed the bigger picture here. It’s that China has their own stack now, soon others will follow. It’s not about putting up the highest numbers, it’s about putting up the highest ROI. To them, this is it. Qwen too but being able to compete with today’s models means they are closer to competing with tomorrow’s.

Espionage has changed wildly, and the ease of taking out key people in "accidents" has dramatically increased.

They are, plenty of BYD factoring being built throughout South America and Southeast Asia as a condition of opening trade. Same is starting to happen in Europe too.

You just aren't going to this too much in the US or any countries fully aligned with the US for fear of competition. It doesn't benefit anyone really. It's not like I get richer when Ford says more vehicles or Meta makes more teenagers suicidal, so why should we care? It'll hurt the country in the long run too.

You had a chance with Bytedance. It didn't sound too great though, there was a very hard glass ceiling for all non-chinese according to Blind.

The PRC government operates extrajudicial police forces outside their borders to keep the diaspora in line. I think they disappeared Jack Ma for a while. I suspect there’s something like that that goes on in the US, but I don’t have strong evidence for that.

More like he wants to ban accelerator chip sales to China, which may be about “national security” or self preservation against a different model for AI development which also happens to be an existential threat to Anthropic. Maybe those alternatives are actually one and the same to him.

Annecotal, but I saw a tweet from someone who interviewed at Anthropic, and was explicity rejected because of cultural mismatch because they were not against open weight models.

It's hard not to see Anthropic's messaging of "this tech that we're pushing on you is going to take your job and maybe kill you" as being about anything other than regulatory capture, with the goal of the government shutting down competitors.

I think OpenAI and Anthropic are both really in a tough spot - spending so much on what is becoming a commodity product for which neither seems positioned to be low cost producer. Maybe a bit like the UK-France channel tunnel project where the product itself is a success but a bloodbath for those who invested to build it.

Jingoism: Its such a rush!

The chances of my bank account getting hacked due to the PLA backdoor in Deepseek is higher than the CIA backdoor in OpenAI.

Kimi is indeed somewhat cheap for frontier-level intelligence, but still is $4-5 per mm tokens. Deep Seek is at least an order of magnitude cheaper.

Interesting, thanks. I'll keep an eye out.

Yes, Flash doesn't seem to have the same rate limits as Pro.

I expect once the API issues are fixed, for v4-pro to be around the same level as GLM-5.

Their benchmark is full of nonsense like this and I'm amazed the fact most of their interactions on the site are promoting it hasn't gotten the account banned for spam.

They have Gemini 2.5 Flash ahead of Opus 4.6: https://aibenchy.com/compare/anthropic-claude-opus-4-6-mediu...

Absolutely worthless benchmark but every release has a comment linking to this nonsense.

It depends...

If the conclusion is: "DeepSeek v4 is this good, if you use it from DeepSeek" (which is how most people would use it anyway), then it makes sense to count API errors as failures.

But, if the conclusion must be "The DeepSeek v4 model is this good when self-hosted and ran at ideal conditions", then the model should be tested locally, and skipping all invalid calls.

I am still debating what should I do in this case, because showing a model as #1, and then people try to use it from their official provider and it fails half of the time, then that's also not a good leaderboard.

I am considering adding a "reliability" column. Retry API errors until the test completes, BUT track how many retries was needed and compute a separate reliability score. But here comes a different problem: reliability varies over time and providers, so that's tougher to test.

The pro pelican is a work of art! It goes dimensions that no other LLM has gone before.

yeah. look at these 4 feathers (?) on his bum too.

a lot of dumplings

I tried that for the GPT-5 launch - a self-improving loop that renders the SVG, looks at it and tries again - and the results were surprisingly disappointing.

I should try it again with the more recent models.

they're just late for work.

They're stressed pelicans from Hangzhou.

996 Pelican, lol

This is an excellent comment. Thanks for this - I've only ever thought about whether the frame is the right shape, I never thought about how different illustrations might map to different bicycle categories.

The Pedersen looks like someone failed the "draw a bicycle" test and decided to adjust the universe.

It was a bigger deal in the Gemini 3.1 launch: https://x.com/JeffDean/status/2024525132266688757

>Generate an SVG of a pelican riding a bicycle

at the top of the linked pages.

In the 5.5 topic

This should be the bottom comment on the pelican comment on every model release post.

I don’t know for sure, but I believe those larger models must be run on nVidia hardware (CUDA), while Deepseek-V4-* can be run on Huawei chips. My assumption is that there is less demand pressure on non-nVidia chips.

The sheer number of bugs and lack of meaningful improvements in Google products is a clear counterargument to the AI bull thesis

If AI was so good at coding, why can’t it actually make a usable Gemini/AI Studio app?

Most of these tests are one-prompt in nature. I've also noticed issues with the PDF reader in Gemini which was very frustrating, although it is significantly better now than it was even two weeks ago. On the contrary, now GPT-5 seems to be giving me issues.

In my experience, Gemini is the most insightful model for hard problems (particularly math problems that I work on).

You know, with a bit of prompting, you can instruct Gemini to output the state of the conversation into a prompt that you can enter in a new chat and continue where you left off. But now with a fresh context window.

I gave up on Gemini 3.1 Pro in VSCode after 2 hours. They fully refunded me.

Yeah if I could use Gemini with pi.dev that would be my choice. But Gemini CLI is just so, so bad.

Like a special parser? Would you mind elaborating?

How can they fix it after the release? They would have to retrain/finetune it further, no?

I guess it depends on a task. Opus is already spawning Sonnet/Haiku for simple tasks with a good success rate.

You can, but does it work well? I assume CC has all kinds of Claude specific prompts in it, wouldn't you be better with a harness designed to be model agnostic like pi.dev or OpenCode?

there's no pinging. Someone's gotta email dang

Probably usable for chat sessions, unusably slow for agentic coding.

By "completely Nvidia-free" do you mean Nvidia wasn't used for training nor inference? Because if it's only inference, we know that Opus already can run on TPUs. Not to mention Gemini.

That's like saying Raytheon would outsource building drones from Saheed makers (don't know who exactly).

Not gonna happen

Because they all (if my memory serves) did this release at the same time thing last time. I have not looked into it but I am guessing that not letting one model pull ahead for a month means everyone keeps up - which implies the “stickiness” of any one model is a lot lower than we think

"2 minutes ago" https://huggingface.co/unsloth/DeepSeek-V4-Pro

Those are quants, not distills.

I've been calling that the "streaming experts" trick, the key idea is to take advantage of Mixture of Expert models where only a subset of the weights are used for each round of calculations, then load those weights from SSD into RAM for each round.

As I understand it if DeepSeek v4 Pro is a 1.6T, 49B active that means you'd need just 49B in memory, so ~100GB at 16 bit or ~50GB at 8bit quantized.

v4 Flash is 284B, 13B active so might even fit in <32GB.

These are more like experiments than a polished release as of yet. And the reduction in throughput is high compared to having the weights in RAM at all times, since you're bottlenecked by the SSD which even at its fastest is much slower than RAM.

Do you have the links for those? Very interested

Flash is less than 160 GB. No need to quantize to fit in 2x 96 GB. Not sure how much context fits in 30 GB, but it should be a good amount.

Very carefully

That's right, need a lot of GPUs + memory, anybody experimented with Mac Studio M3 Ultra for this?

So, dual RTX PRO 6000

Jordan Keller worked at Moonshot? Or am I missing something? I thought he is the original author. https://x.com/kellerjordan0/status/1842300916864844014

But it's now mainstream thought on the left in America.

easy, you buy twitter and let people speak freely again

OpenCode? Pi?

If one finds it difficult to set up OpenCode to use whatever providers they want, I won't call them 'dev'.

The only real friction (if the model is actually as good as SOTA) is to convince your employer to pay for it. But again if it really provides the same value at a fraction of the cost, it'll eventually cease to be an issue.

You can literally run it from Claude code. Easily too

They have instructions right on their page on how to use claude code with it.