Show HN: Reame – a CPU inference server that gets faster as it runs

ttargetbridge about 4 hours ago 2 commentsRead Article on github.com

DE version is available. Content is displayed in original English for accuracy.

Discussion (2 Comments)Read Original on HackerNews

ohadkr•about 2 hours ago

The persistent KV cache is interesting; I’d love to see how much of the speedup remains when requests share less structure.

tyzoid•about 1 hour ago

Why qwen 2.5 everywhere? Why not 3.5?