FR version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
67% Positive
Analyzed from 55 words in the discussion.
Trending Topics
#top#writing#series#understanding#llms#ground#software#engineers#last#part

Discussion (1 Comments)Read Original on HackerNews
In this post I cover what happens when you type a prompt and hit enter. How the model generates one token at a time, why that is slow, what the KV cache does about it,and how decoding strategies like temperature, top-k, and top-p shape the response.