Qwen 3.8
621
RU version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
77% Positive
Analyzed from 10619 words in the discussion.
Trending Topics
#models#model#open#more#qwen#deepseek#same#china#don#chinese
Discussion (455 Comments)Read Original on HackerNews
Now the response of Alibaba is that they will also publish soon a big open weights LLM, the 2.4T parameter Qwen 3.8.
I wonder if Alibaba has always planned to make this big LLM open weights, or they have chosen to do this now, to better compete with Moonshot AI.
In any case, from this competition in LLMs, we win.
Feels pretty easy to me.
They want to turn LLMs into a commodity, and watch the US AI labs crash and burn.
There will still be plenty of customers who will pay them to host the models and run inference, even if the weights are open and others can offer competing products. (If necessary, the Chinese government can ban use of foreign inference services by Chinese citizens and businesses to give their own companies a domestic monopoly.)
When their models equal or surpass those from the Western AI labs, they can even stop releasing weights for new models, and keep all the inference revenue for themselves.
Meanwhile, they're still manufacturing much of the hardware that everyone in the world needs in order to run datacenters (see also: Spolsky's "commoditize your complement" essay).
Beyond that, it's a soft-power play. As the world keeps looking at the US more and more skeptically as an ally and superpower, Chinese companies releasing weights for competitive models is a way for China to look better and more world-minded.
Why does a debian contributor make debian free, why do they work on this thing anyone can use?
Is it because linux and debian hate windows and iOS and want to see american fail?
No, it's because most debian contributors believe software source code, information, should be free, users should be free to modify the code they use, and that they're building a thing they want to share with the world.
Maybe the chinese AI labs believe AI is powerful and useful, are proud of what they're doing, and want to share it as broadly as they can so everyone can use it.
There doesn't have to be any weird "chinese government" or "they hate the west" type vibes, it could just be the same thing as OSS, they're trying to do what they think is best for the world.
Man, imagine Darios face when suddenly, he cannot decide anymore what other people consensually do with their own hardware in their free time.
Rumpelstilzchen.
I want to watch that too.
If they take Meta and Musk with them, all the better, but that is just dreaming I am afraid.
Not for anyone who reads history.
Back in the late 18th century, England was the world's top economy, in big part due to its textile industry. England had an export ban on the technology, but textile worker named Samuel Slater brought blueprints over (Supposedly in response to a bounty posted in a newspaper by the US government!). The technology diffused rapidly because the legal environment made competition easy, and ironically the US had better sources of energy (superior water-power sites).
Arguably, China is doing the same thing in the 21st century.
By contrast, export of protected Chinese tech today frequently gets the death penalty.
History doesn't repeat itself but it does rhyme.
So it’s not a surprise why open weights are so cherished. As frontier models continue to block everyday individuals from securing their own codebase, I expect the adoption and usage of open weights to continue.
As an example, HuggingFace recently was investigating a security incident and got locked out of frontier closed APIs. Yes, HuggingFace.
https://huggingface.co/blog/security-incident-july-2026
> When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.
> This experience points to a gap worth planning for. We do not know which model powered the attacker's agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried. The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment. This is not an argument against safety measures on hosted models, and we are sharing this feedback with the providers concerned.
Yeah, big problem! Although I'm kind of surprised HuggingFace doesn't have access to Mythos? Or maybe Mythos still has some guardrails.
The tech industry along with US foreign policy has become much more zero-sum in its ideology in recent years, and I think that's a tremendous mistake.
No training budget means deceleration, or at least slower acceleration, margin compression and a completely demolished IPO valuation; path to machine god requires dollars and capable open models externalize training costs to true frontier labs parasitically.
IMHO humanity has a better chance at not destroying itself due to less than breakneck pace - but there’s a chance frontier models get sponsored by the USG and are never released publicly so they can’t be distilled and then what?
The closest we've seen to this in tech in recent decades was iOS vs Android, where Android only really was competitive for a very short window of time (approx 4.x) and it was during that period that both Android and iOS actually improved dramatically for end users. Once Android lost the plot again, and especially in the US market, all that energy started going in some very silly directions.
That premise hinges on one implicit assumption: Chinese advances are due to distillation ONLY and that Chinese model providers cannot keep advancing if they do not distill, which is a very big if. If Chinese models keep advancing in such a scenario, and they almost certainly will, they will overtake publically available models by US providers and China will dominate the LLM industry.
Sometimes, constraints, like sanctions, can also be a source if innovation.
Open weight AI is decelerationist from the perspective that all capital should be allocated to a market leaders for training, and that the market leader is fully invested in continuously making the models smarter, cheaper, faster for its users, or that distillation from this market leader is the main way to make progress.
We might reach a local optimum/equilibrium faster without open weight models, with leaders capturing more of the market faster to a point where further R&D isn't required due to lack of competition. I also doubt that distillation is the only/main way that open weight models were advancing AI research. We can name a few examples from DeepSeek around reasoning, context optimization, etc. I'm also unconvinced that the overall market capex on AI is lower given more competition (probably less specifically for US market capex, which is decelerationist from only the US perspective).
If you're worried about an AGI arms race between the U.S. and China putting AI Safety at risk, then the fact that inherently less knowledgeable/capable models (fewer and more coarsely quantized total parameters than their proprietary competitors according to commonplace rumors) are having a "decelerationist" effect is actually great news. Even better if China is actually "Yann LeCun-pilled" (verbatim from Ball's post) and doesn't really believe in early AGI. So explain to us exactly why we're supposed to ban/discourage use of these open source models? The only way that makes sense is as a transparently self-serving proposal from the chief OpenAI policy lobbyist.
"No, sir, we haven't reached the peak of this tech... It's those open models! Please, keep pumping dollars into the market!"
Not that hard to say IMO, they basically see models becoming a commodity and see value in the applications on top of them. So if Alibaba Cloud is the best place to build applications on top of Qwen, why not give the model itself away?
Yeah. Its a bit like the "open core" model in open source.
Unless you work there, your opinions are guesses, and parent is saying we cannot know, which remains true even with your guesses :)
Google: Chromium, Kubernetes, Android, TensorFlow
Meta: React, PyTorch, Llama
Microsoft: VS Code, TypeScript, .NET Core
LinkedIn: Kafka
Slotted in along these, an analogous explanation is that Alibaba needs Qwen internally (vs depending on an American company), but licensing is not part of their revenue strategy. (As a cloud vendor, they can make money on inference. The strategy is very similar to the US hyperscalers ex-Google.)
Joel Spolsky wrote in depth about this notion of commoditizing one's complement in 2002[1] using tech examples stretching back into the '80s.
1 - https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
So what would the long game be for chinese companies?
One aspect of this is making a name for yourself i.e. PR. Making a capable model open source helps a lot with that.
Why is it hard? Their government has been very clear that they plan to win on manufacturing: https://english.www.gov.cn/news/202601/08/content_WS695f1b55...
Technically they've been saying it for the last 40 years.
https://www.reuters.com/world/asia-pacific/chinas-xi-promote...
Data centers?
Not one person here has any idea what is going to happen long term.
AI being good for humanity is still an open question, but for closed vs. open models/weights, yeah it is preferred. I foresee it won't be much longer before everyone will be slicing/distilling/tuning their models once the architecture improves.
Involution is a major problem in Chinese industries [1]. Where companies will sell their products at a loss, effectively playing fiscal chicken [2] with one another to dominate a market. It is such an issue the government has had to step in to prevent EV companies from destroying themselves by more-or-less requiring companies sell their goods at a profit [3].
The straight forward line of reasoning that AI/LLM labs are applying this logic to their profit.
I think (we) Americans are reading a bit too far into this assuming government intervention, conspiracy, etc.. Chinese markets are downright cut throat. They're using those tactics to compete with US labs.
1. https://www.reuters.com/business/autos-transportation/what-i... 2. https://en.wikipedia.org/wiki/Chicken_(game) 3. https://www.theguardian.com/business/2025/aug/05/china-warns...
In fact, there are no other organizations in this world that is well suited to leverage scaled intelligence than Silicon Valley and great American companies
They've done this in other industries like solar panels, chips, and EVs. This is no different.
I keep looking at the numbers. The power use numbers are not that problematic. Ordering a burrito on DoorDash uses more power than a few days of heavy AI use. The water argument applies to some locations, and is mostly a local governance problem... if the data centers are using too much water, it means they are not being charged enough for that water. Charge them more and they'll push toward closed loop cooling.
Yet the visceral pile-on here is so extreme, it feels fake.
One thing I've learned after 40 years on this planet is: propaganda works, and much of what a large fraction of people believe across the entire political spectrum (left, right, anything else) is there because someone paid to put it there. It's depressing but it's true, and it makes sense. Propaganda is an asymmetrical attack on human cognition and discourse, and in information security the attacker always has an easier job. Crafting viral bullshit is orders of magnitude easier than fact checking. On top of this, humans are busy and don't have time to fact check and logic check everything they read. As a result, much of what we believe is "sponsored content."
People get mad when you talk about this because everyone wants to believe they're too smart to fall for propaganda.
In any case, the US AI labs deserve to lose for their stupid "safety" regulatory capture monopolization push, which ended up blowing their own feet off and handing the lead to China.
You're not going to debase the frontier labs through distillation.
https://en.wikipedia.org/wiki/World_Artificial_Intelligence_...
You're able to run quantized ~100B class models on local hardware today, but still lots of compromises when it comes to quality. I guess it ultimately depends on how far "near future" is, in a year you'd likely be able to run something like 5.6 Terra on local (~10K USD) hardware, but Sol/Fable would still be out of range, and at that point the closed-source labs probably have one or two more iterations put out at that point.
I would love to see something like a 90B A6B model that is optimized for 128GB machines e.g. strix halo, I haven’t seen anything really targeting the combination of RAM and compute these machines have, but I’m biased because I have one.
I have had my 32G mac mini for 2 1/2 years and I have enjoyed watching one technology advance after another improve the quality of work I can do locally. I bet that what I will be able to do in one year on my old hardware will be even more awesome.
It's hard to answer quantitatively, but for example Qwen3.5 -> 3.6 was a significant step in capability, arising from continued post-training of the same models. If we were at the end of low-parameter-count scaling then that would be a surprising datapoint.
Like, throw us a bone, we all know we need SOTA for lots of dev work anyways, but at least some tasks can be local.
Do we really though? Everyone is wasting resources doing almost exactly the same thing. Climate loses, we lose.
About climate, I think you overplay it. China is already investing heavily in nuclear, and we should be doing the same.
China will always benefit from a broader adoption of their models as hidden propaganda machines.
Eventually with several services relying in those tools, their answers will always be more friendly to China.
- Minimax M3 Pro (2.7T)
- GLM 5.3 (or beyond)
- Deepseek V4 Pro (current V4 Pro is preview)
- Kimi K3 weights out in 8 days
Exciting time on the open-weights frontier.
Looking forward to see what Antrophic and OpenAI does next.
It would be great if they'd release an MoE model somewhere between the 35B size of 3.6 and the 122B version of 3.5 - it could be a great balance of speed and ability for people with reasonably powerful but not insane home computers.
Yeah, this is what I'm holding out for, the NVFP4 variant of 3.5 122B is blazing fast with reasonable quality and even with max context fits perfectly within 96GB.
The fantasy is a 100B or 80B model, but MoE and highly tuned for coding.
I wholly disagree. Rather than going the "everything is a claude code skill" route, I've been hacking together purpose-built harnesses for all sorts of tasks, and in that environment a wee little baby model can do some really useful things. You end up burning lots of tokens making the thing, but then all that investment comes back when the resulting tool works perfectly fine on a dinky little model that fits on my 3060 Ti.
Europe will definitely be interested in democratizing these things if China starts losing interests; from there, there'll be more countries looking to keep their citizens entrained in their own Country's infrastructure.
It'll especially be true if the memory cartel keeps prices high and NVIDIA tries to gouge higher memory models.
It's an arms race everyone can join because PC hardware was mostly democratized in the last decade.
I dont see most model building as anything more than a pig at a slop troth, despite the level of sophistication; they're still rarely pruning the input beyond random sampling.
I have used both Qwen3.6-35B and Qwen3.6-27B locally (both Q8 quantized with llama.cpp). I have also used antirez's quant of DS4-flash. They all performed within the same tier, DS4 being a bit more efficient, but they all gave really good results, mainly used for bash scripting, debugging, python and some C++. I am curious what type of applications/langauges failed with Qwen? One thing to note, the chat templates were "broken" for qwen models and had to debug it, there are already effort on this. Tbh, the same with gemma.
Qwen-3.7-Plus is quite OK, good for subagent use. Way better then Sonnet.
Qwen-3.8-Max-Preview seems working just fine for me at the moment - I am playing with is right now but too early to say anything. At 10% of regular price it is a steal so far.
I've been using https://gitlab.com/gabriel.chamon/orisun which is my own simplified methodology, for coding web apps in python and elixir and have been very successful using qwen3.6 27b Q4 locally with help of larger models for architecture, so I get very suspicious when people talk how useless larger models are. They are either using it for a domain that models don't perform well or just not using it right.
Here standard plan has been discounted to $18.00, from $25.00/month.
It makes a huge difference if you're writing Javascript/HTML/CSS, Python, or C++/Rust.
Also the application type matters, e.g. user interfaces or scientific computing.
domain: typical web backend tier, mobile apps. not particularly complex, but requires OOP/architecture/system design.
I meant Opus 4.8 which is rather dumb and ineffective in coding harness, especially with higher thinking levels.
Such diametrically different ones.
I paid $2 for deepseek api, put the key in void editor and made a crypto tool in html.
It turned out to be around 67kb. I used sample files in CSV that were a few hundred lines.
It spent around $1.8 in the hour or two or light coding and follow up bugs.
Is it really really this much?
I can't imagine spending a month using it for a day job, it would cost more than the salary so what gives?
I understand the local ai and all that but do cloud providers cost this much?
Earlier I thought "billion tokens" but now not sure
I delegate small-medium tasks: refactors, summaries, research, writing tests + have very good codebase already + extensive history / architecture / docs / linters. so it picks up and does decent small-medium scope work. it is fast, accurate, cheap. does exactly what I want directly and does not waste time nor tokens.
definitely not "implement me complex greenfield project".
Pro is ~50% more expensive than Flash.
Both need babysitting.
Plan, split in small tasks, give it docs, types, tests, linter, best practice examples, etc.
Always start a new session when starting a task.
Do regular manual sanity checks, and tell it to find issues in the codebase.
I pay like $1,50 per day for Pro.
I would also add that I run it this way ~12hour a day non-stop. 300M / tokens per day (99.7% cache hit).
Anthropic should not have bugged their knowledge distillation attacks.
It is like one of Pizzaro's men crying that someone have stolen his precious golden dublons
As Lenin have said - "Loot the looters" (Russian: Грабь награбленное)
It boggles my mind how you can train a frontier model but not write a tweet without an obvious typo.
if youre going to use ai for everything youre gonna start losing your edge as you focus less and less on what youre doing and this isnt me just talking out my ass, like... the front page here is peppered with study after study and blogpost after blogpost about how its overuse can come to the detriment of one's own abilities and skills.
coca cola had the ad with the magical truck that changed its design, shape and amount of tires it had and if nobody noticed that before releasing it then im not sure why anyone might think that the people peddling the LLMs would somehow be immune to this phenomenon
Come off that high horse
Will probably be at Opus 4.8 level, and I find it pretty big deal because of Deepseek price...
The model is fantastic. And costs almost nothing. The only problem I see is that they will train on your data.
There are zero-data-retention providers of DeepSeek models, of which I have used openrouter (with zdr guardrails), and fireworks. But these are 3x to 5x more expensive than directly using DeepSeek, possibly due to poor caching. Thats the price to pay for zdr.
I bought it through OpenRouter and used it with Pi agent.
The model was good, but there appeared to be a pricing glitch or something, because it burned through $50 in under an hour on pretty trivial stuff.
Pi agent claimed it only used like $1. OpenRouter claimed differently and said I used all $50.
I use Openrouter for everything except Deepseek. For Deepseek I use their API directly.
I use it from pi.dev as well through the OpenCode Go $10 subscription ($5 first month).
Used more than 20M tokens at a cost of ~$20 (up to $60 is included in the $5 plan) Out of which deepseek pro had ~200 messages which is around 1.5M tokens (10+M cached)
You can check the logs in OpenRouter and see which providers it used and how many tokens you used.
Besides, in a few days, they'll change their pricing, doubling it during their peak hours, so, realistically:
- It will be 2x more expensive if you live in their time zone
- It will be 1.5x more expensive if you live in a time zone that is adjacent to theirs
- It will be the same price IF you use it while they sleep (during offpeak hours)
It's still cheap, but the price/performance ratio is not that good
DeepSeek V4 didn't produce the same impact as V3, and Huawei dropping the ball is making it worse
They had promised massive price cuts for July, so now (Huawei chips), but they had to rush the cuts because lack of momumtum (they advertised them as promotion), and are now backtracking by introducing this peak hours pricing
Trump decided to help them a little by allowing them to buy more NVIDIA chips, so what exactly is China's role in all of this?
We are supposed to blindly pat them in the back while praising them, all while handing them over our data? I thought they were dangerous competition threatening our model of society
I have been happily using DeepSeek V4 Flash for the last couple of months now. I tried GLM-5.2 for a while, but it was too slow and verbose compare to DeepSeek V4 Flash. If I have a basic skill I need to execute, DeepSeek V4 flash is still the best model for it.
Over the past few weeks while using pro from them directly I have had an increasing number of responses that are obviously from a much, much better model. It is so good that the closed model dog and pony show is already spinning fud about "dark routing" and "stolen directly from fable"
Even at their new pricing it is a genuinely ridiculous amount of value. If you are the type of person who, very reasonably, does not have time to be trying out every model, and just want to use what seems to be the best currently... don't try it. You will be sick to your stomach with buyers remorse as you start to internalize just how much more you could have accomplished had you spent the first six months of the year giving them $1200 instead of OpenAI.
Was there ever an explanation for why we never got the weights of 3.7? I would like sourced quotes and not weird/cringe accusative speculation about distillation, or your take on The Big D.
Which is why OAI and Anthropic will most probably push for more governmental control and bans. Without it their whole income model is cooked.
Anthropic and OAI can piss and moan all they want - limiting the U.S. to only their models would hurt the U.S. economy in myriad more ways than the failure of a couple of companies that scaled too quickly. If they get that outcome, the rest of the world would simply keep moving forward with access to open models and tokens at pennies on the dollar.
Then again, all of Chinese models are open. And DeepSeek even publishes research papers alongside their models that go in depth into the methodology. I guess there's not much stopping USian companies from copying
But it's not like Fable is so substantially better than the other two that I would be seriously impacted if I didn't have access to it anymore. All three are amazing models, and of the three, Fable is the only one that regularly triggers refusals.
Coordinating agents though? Fable any day.
[1] - https://news.ycombinator.com/item?id=48965243
[2] - https://huggingface.co/blog/security-incident-july-2026
"When we started the log analysis, we first used frontier models behind commercial APIs. This did not work [...] We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. [...] The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout [...]"
had mythos been just Opus 5, with the same size and price as the previous opuses, then yeah, that would be a tie-breaker. but it's not.
I'll switch to OpenAI soon because of this. I also can't wait for the day it becomes feasible to run these awesome open weight models on my own hardware.
What's more interesting is that Anthropic moat shrunk to just that model. There's zero reason to use any other model from Anthropic right now. And once they take Fable off subscription there will be zero reason to have Anthropic subscription.
Yeah, that'd be neat, but that's not what this announcement is about at all:
> With a massive 2.4T parameters
The niche for small models should be filled with medium sized labs doing distillations of the huge ones into consumer grade hardware runnable models and LORAs for the huge ones.
Looks like they're previewing the model only on their subscription plan.
We are spoiled in the LLM segment, but I would love to see an open source competitor to Flux.2, etc.
You can also try it out on Qwen chat, Its free.
I haven't tried Qwen 3.8 Max yet, looking forward to it. My hope is that its way less verbose. Another thing I experience with the Qwen models is that I do not trust their benchmark scores at all. Have anyone played with Qwen 3.8 Max and can share their experience? Which model it come close to? Sonnet 5? GLm-5? DS V4 Pro? Flash? Gemini 3.5 Flash?
I mean even the cheapest option for Luna is still more expensive than anything DS or MiMo is offering right now and I think a new Ministral model would also hit hard there because we also need some variance in model sources, we can't rely only on the US and China.
China isn’t an altruistic state. They’re an aggressor in many fields, economic and otherwise, and this one of them.
I guess we'll see if the "second only to Fable" hype pans out. In my limited experience with Kimi K3 (I signed up for a month of the $19 plan) it's slower and chews a lot more, so ends up being pretty expensive; one little feature burned through almost the entirety of my five hour limit. The $20 GPT plan is a lot more useful and includes 5.6 Sol, which is fast and token-efficient enough to be quite usable even with the small plan.
Try it yourself here: https://www.qwencloud.com/try-ai/chat
Edit: I saw online they do in fact plan to release this openly at some point – x.com/Alibaba_Qwen/status/2078759124914098291
https://xcancel.com/Alibaba_Qwen/status/2078759124914098291
That's a massive model!
The shift from "value" models to "intelligent, huge and slow" models coming from China is an interesting change in strategy.
My main issue with GLM 5.2 and Kimi 3 is that they're extremely token hungry and thus feel slow(er) to use.
Of course they train on literally everything they get their hands on, like everyone else. If you need privacy, that's what local models are for.
Whether you trust them is different, but there ARE knobs on other hosted AI companies.
I also find the model is a lot more predictable and less “glitchy” when made to think in Chinese. You can do this in the system prompt.
Too little too late imo
I had no issues with it for C++ development with https://pi.dev. I'm yet to try it with Zed Editor. I don't rely on agents too much. However, I used it on Chromium's codebase to research some functionalities, let's say for searching. Requests like: check my last commit and do the same for SetterA and SetterB; it also ran without any errors.
*within the scope of open models only
Oh, and Mira’s thinking machines lab dropped Inkling, a ~1T open weight model too.
This isn’t US vs China. This is open vs closed.
The right question is which is the speed that can be achieved on a given hardware and whether it is high enough for the model to be useful.
Until now, the speeds reported for running big LLMs with the weights stored on SSDs have ranged from as low as a token every 10 seconds or so, to as high as a few tokens per second.
With open weights models that you host yourself, you are not constrained to use any single model, because that is the one for which you pay a subscription.
You can use many models, each for whatever it is more suitable. You can use frequently a small model with a high inference speed, but for some tasks you may actually save time with a better model, even if it is much slower.
In my opinion, even at 1 token per second a big model may be useful for some tasks.
Broadly speaking, this ultimately pushes local inference towards a challenging world where you use SSD offload for weights as a matter of course; then smaller requests (or requests sharing the bulk of their context, e.g. subagent swarms) can be batched together and run quickly in aggregate, but running very large contexts will actually limit you to single-session inference and require swapping out even the KV cache itself to some external scratch SSD, further hurting your performance. Then feel free to add wide use of MTP in a probably futile effort to go back to tolerable tok/s numbers.
It is like a ping-pong game: the advantage flips back and forth between providers.
So yeah, it's the best local model I've seen. I am going to try the Qwopus 3.6 fine tune soon with the same spec and tickets and compare the output of both.
Not tried it yet but I've seen tests that suggest they've properly fixed the tool calling issues.
vLLM gives me ~7000+ tok/sec with Gemma 4's MoE model. Vs ~6000 tok/sec for Qwen 3.6 MoE.
But there’s also the quantization of DeepSeek v4 flash called dwarfstar
I have a feeling this is the next…frontier of that fight
One can only hope it eventually does as well as Linux
Made on the website, so not sure if on the API there's more thinking options...
It never was. The point of this "pelican test" was for performative reasons, or just for attention of the joke.
It is like trying to test whether if an adult elephant could actually climb up a tree and reporting that some elephants are slightly better at doing that than others while also reporting at the same time that they are all bad at tree climbing anyway.
This is an example of testing for the sake of testing. The "pelican test" tests for nothing.
And it has nothing to do with the individual, from what I can tell, 70% of the population placed in their position would become the same type of uberpath.
For example they don't even tell you anything about the tokenization. They even do random chunking and padding to avoid leaking the token strings in the streaming api after it got reverse engineered. (See: https://spylab.ai/blog/claude-tokenizer/)