Back to News
Advertisement
Advertisement

⚡ Community Insights

Discussion Sentiment

50% Positive

Analyzed from 70 words in the discussion.

Trending Topics

#model#local#models#qwen#sota#tok#going#tps#using#framework

Discussion (2 Comments)Read Original on HackerNews

throwitaway222•about 3 hours ago
Are we going to get 1000 tps local?
andy99•about 2 hours ago
I’ve been using local models on a Framework desktop. I get > 50 tokens/s on Qwen 3.6 35B A3B, and I find that speed ok for coding (though the model not quite). If machine could do a SOTA model with say 0.5-2T parameters like GLM 5.2 at 100 tok/s, it would be very useable.

In 2 years presumably the SOTA models will be much bigger, but I won’t be surprised if there’s a fable level sub 50B parameter model by that point either*, in which case 1000 tok/s may be reasonable.

* on artificial analysis intelligence index, GPT 4o is 12 and Qwen 35B A3B is 32.