Kimi-K3 Releases on HuggingFace 6/27

nnateb2022 about 2 hours ago 8 commentsRead Article on huggingface.co

ES version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

55% Positive

Analyzed from 747 words in the discussion.

Discussion (8 Comments)Read Original on HackerNews

NitpickLawyer•about 2 hours ago

This will be interesting for a few reasons. First, depending on where the median pricing settles w/ 3rd party providers will tell us what it costs to serve a 3T model. Since it's going to be mxfp4 native, it'll take ~1.5TB of VRAM to host this, which is juuust at the limit of 8xb200s (but realistically you'll need 16x for context / throughput optimisation). Won't be cheap to host, but at least we should get some range of $/MTok for a 3T model. Then we'll be able to guesstimate if "labs are subsidising tokens on API pricing".

Also interesting to see what effort it will take to fine-tune this beast. The latest AISI benchmarks on cybersec place it above glm5.2, but still way way behind SotA closed models. Some fine-tuning might be needed here. Also, interesting to see if Cursor does another training round on it, to directly compare it w/ kimi2.6/2.7 fine-tunes (composer series) and grok4.5.

Also also, interesting to see if someone takes on distilling (proper distillation, w/ training the entire distribution) from this into smaller models. (dsv4-kimi should be really good, since dsv4 is very cheap to serve)

walrus01•1 minute ago

It will be very interesting to see what kind of 'slow' performance people get from running it on a no GPU, but tons of RAM server (like a dual or quad socket xeon with 1.5 to 3TB of RAM). For the purpose of giving it longer duration tasks to generate a piece of something and come back and check on what it has done in 4 or 6 hours. Even if the output is like 5-6 tok/s, that might be usable for some purposes.

Huge price difference in what you can do with buying a used 4U rackmount server and putting 3TB of RAM in it (64GB DIMMs x quantity 32 in a quad socket xeon, you can see some benchmark prices on eBay for sets of 16 or 32 matched 64GB ECC DIMMs) for <$30,000, vs the cost of trying to run it on real GPU hardware.

vb-8448•3 minutes ago

> Then we'll be able to guesstimate if "labs are subsidising tokens on API pricing".

No, you don't. Without training cost you can infer only the marginal cost of serving this kind of models.

Moreover, you don't know the actual size of closed models (what if Fable is a 10T model? What if it's 1T?)

woctordho•34 minutes ago

Speaking of finetune, currently a common practice is LoRA over bnb 4-bit base model, but I think it's time to replace bnb with GGUF as the base model format. GGUF is actively supporting new model architectures and more aggressive quantizations.

I've made some proof of concept in https://github.com/woct0rdho/transformers5-qwen3.5-recipe . We can finetune Qwen3.5-35B-A3B in 16 GiB VRAM, and DeepSeek-V4-Flash (284B-A13B) in 90 GiB VRAM, without CPU offload. This works well on unified memory machines like Strix Halo.

Even so, larger models like Kimi-K3 still require multiple GPUs and nodes, and there are a lot more to do compare to single-GPU training.

dist-epoch•37 minutes ago

> if "labs are subsidising tokens on API pricing"

> SemiAnalysis estimates that Anthropic's current blended gross margin has risen to the mid-60% range, with the API business gross margin exceeding 80%

Of course, people will insist "they are lying", "why should we believe them, it's well known they subsidize API pricing", ...

https://newsletter.semianalysis.com/p/anthropic-3q26-profit-...

https://finance.biggo.com/news/02d45650-b569-4d12-b44d-8d6d8...

NitpickLawyer•35 minutes ago

Agreed. My (somewhat educated) guess is that top labs have healthy margins on API pricing. But this release will add another 3rd party / clear of conflict datapoint in this estimation.

knollimar•12 minutes ago

That numner blends in training or no?

davidkunz•about 1 hour ago

This is historic. For the first time, an open-weights LLM is right at the top.

We won't be able to run this ourselves, but many providers can.

embedding-shape•9 minutes ago

> For the first time, an open-weights LLM is right at the top.

Hmm, not quite true, I think that honor, for better or worse, goes to OpenAI. When they released GPT2 (or GPT1 for that matter) is was quite literally the SOTA in the ecosystem when it was released.

wwwhizz•about 1 hour ago

That would be 7/27.

broodbucket•about 1 hour ago

20270727 if we're improving dates :)

c7b•35 minutes ago

270727 if we want to save tokens :)

darkwater•24 minutes ago

Y2K100 bug for you, then ;)

fragmede•32 minutes ago

ISO 8601 ftw

m00dy•36 minutes ago

Looks like another LeetCode problem about checking for anagrams.

ra•about 1 hour ago

or 27/7 for the rest of the world

LeonM•29 minutes ago

No, 27-7 for the rest of the world.

The separator is often the only way to distinguish American notation from ISO, so please use a dash for dd-mm-yy and a forward slash for mm/dd/yy

ashwoods•1 minute ago

Not really. Spain's traditional format is dd/mm/yyyy with slashes. This applies for a good chunk of Europe. Germany/Austria uses dot, I think nordic countries embraced the dash. While you might see more adoption in offical/digital contexts, I just double checked a few popular spanish websites, all slashes.

casparvitch•26 minutes ago

Have never seen 27-7, as someone in rest-of-world

distances•15 minutes ago

I've never seen dd-mm-yy. It's usually dd.mm.yy, dd.mm.yyyy, or yyyy-mm-dd, with some dd/mm/yy sprinkled in for general confusion.

letier•26 minutes ago

Or 27.7. in some other places.

kuboble•20 minutes ago

Nope,

There are quite some countries around the world using d/m/y

https://en.wikipedia.org/wiki/List_of_date_formats_by_countr...

Algeria, Belgium, Brazil, Chile...

linzhangrun•27 minutes ago

China uses YYYY/MM/DD, which is logical.

_zoltan_•24 minutes ago

the only logical format.

signed: a hungarian :)

JSR_FDED•5 minutes ago

lpszReleaseDate ;-)

CodeCompost•about 1 hour ago

Why is there a countdown?

broodbucket•44 minutes ago

You're not having a party?

InsideOutSanta•16 minutes ago

I think it's shameful that Moonshot isn't providing us with party kits like Microsoft did with the Windows 7 Launch Party kit. How am I supposed to properly celebrate this without fun Kimi-themed quizzes for my guests?

mythz•37 minutes ago

It's a release party

m00dy•about 1 hour ago

There’s going to be a lot of competition around this model. Let’s see how low AI providers are willing to push prices.

torginus•36 minutes ago

I think the results might be underwhelming - AI providers need to turn a profit and can't subsidize, and they're working off of the commodity hardware everyone does.

I wouldn't be surprised if they started offering potentiall bad quantizations with much reduced capability at lower prices (without telling the users, of course)

Iolaum•44 minutes ago

As long as they are transparent about what quant they serve the model and any other optimization they do that also affects performance of inferred tokens.