Deep Seek v4.1 M5 Max at 17 tokens/s

AArgonautlabs about 8 hours ago 2 commentsRead Article on github.com

⚡ Community Insights

Discussion Sentiment

0% Positive

Analyzed from 100 words in the discussion.

Discussion (2 Comments)Read Original on HackerNews

Argonautlabs•about 7 hours ago

We stream DeepSeek-V4.1-Flash (518 GB, 4-bit) from NVMe on a 128 GB M5 Max, because it doesn't fit in RAM.

                        prompt processing      steady decode
    upstream, internal       16.23                 10.59
    ours, internal only      28.04  1.73x          14.38  1.36x
    ours, +1 external        36.88  2.27x          16.05  1.52x
    ours, +2 external        43.62  2.69x          17.38  1.64x

  The first fork row is the one that matters: same single internal SSD, no replicas, no enclosures.

nextaccountic•34 minutes ago

Just so you know, you are getting automatically flagged in HN (your comments are automatically removed). I think it's because you are using LLM to write them. You should not copy paste LLM output directly to comments in this website.

(I'm not talking about using LLM to generate the README.md of your project, that's also distasteful but not nearly as bad)

In either case your repository is interesting, and might be useful for people that own a M5 Max