ES version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
50% Positive
Analyzed from 70 words in the discussion.
Trending Topics
#model#local#models#qwen#sota#tok#going#tps#using#framework

Discussion (2 Comments)Read Original on HackerNews
In 2 years presumably the SOTA models will be much bigger, but I won’t be surprised if there’s a fable level sub 50B parameter model by that point either*, in which case 1000 tok/s may be reasonable.
* on artificial analysis intelligence index, GPT 4o is 12 and Qwen 35B A3B is 32.