Back to News
Advertisement
Advertisement

⚡ Community Insights

Discussion Sentiment

50% Positive

Analyzed from 113 words in the discussion.

Trending Topics

#cache#token#providers#pricing#efficiency#calculate#effect#caching#sometimes#breaks

Discussion (3 Comments)Read Original on HackerNews

cortesoft26 minutes ago
How do you calculate the effect of caching?

Sometimes, I take breaks in the middle of a session and end up losing the prompt cache which drives up the token usage a ton. Don't all the providers have different cache times and behavior? If one person takes 20 minutes between messages, some services will keep that cache while some won't. Is there a way to factor that in?

schleck88 minutes ago
OpenRouter shows the cache hit rate and calculates an effective pricing per provider if you scroll down on the model page. Note that some providers run fp4 versions of models. DeepInfra often does this
esafakabout 1 hour ago
It's easy to scrape the pricing; you have to do some leg work to estimate the token efficiency, which factors into the ultimate cost. Ideally you'd compare reasoning efficiency too; given them all the same task.