Back to News
Advertisement
ddpc94 about 4 hours ago 6 commentsRead Article on demo.frugaltokens.com
I wanted to share a project I’ve been working on called Frugal Tokens. I originally built it because I was curious to see how much all of my sessions cost and how much cache misses affected that spend. I’d noticed people had widely different spend profiles and wanted to better understand what might contribute to that.

As I’ve worked on this, the tool has grown to show more usage patterns across all of your sessions. It shows overall usage, estimated working time and overlapping sessions, and where your spend is coming from across models and cache misses. I also have a few session level metrics with percentile breakdowns, along with a list of your sessions and high level info.

Clicking into a session opens an explorer where you can see individual model calls and tool inputs and outputs. You can also jump directly to where a cache miss happened. There’s also a rough cost comparison that shows what the recorded session would have cost with another model’s pricing, or for Anthropic, with 5m vs 1h caching.

In the future, I’d love to collect more information to see which patterns might make people’s workflows more expensive, e.g. long sessions, high context usage, many turns, etc.

The tool requires deno, but is just one command to run once that is installed. The demo provided has some of the data scrubbed, but helps to show what it looks like before running it.

Would appreciate any thoughts or ideas

https://github.com/dpclark4/frugal-tokens

Advertisement

⚡ Community Insights

Discussion Sentiment

100% Positive

Analyzed from 174 words in the discussion.

Trending Topics

#token#usage#https#cache#misses#dig#local#conversation#tool#joshstrange

Discussion (6 Comments)Read Original on HackerNews

gen220about 2 hours ago
Really cool to see the difference in token usage across different harnesses.

These kind of independently-verified data points will be super important for companies (and individuals!) trying to measure RoI on token spend as they recover from letting people go wild on token usage in the past year.

stephensilberabout 3 hours ago
I have started using this to inspect some of my heavier sessions and it has helped uncover some of the parts of my workflow and my project's build pipeline that were really slowing me down.

I also had no idea how many cache misses were happening when I stepped away for an hour or more at times.

ixtliabout 4 hours ago
dug around in there briefly and the per-session explorer is the best part imo
joshstrangeabout 3 hours ago
I can highly recommend https://agentsview.io/ for a way to dig into your LLM usage. It reads your local conversation files and provides a ton of visualizations as well as the ability to dig into each conversation and see all the tool calls/etc. It's incredibly powerful.
dpc94about 3 hours ago
Looks interesting. One of the angles that I approached while building this was focusing on cache misses. Not sure if this shows much info about this category. My tool is also local only + detecting changes and syncing into a sqlite db
joshstrangeabout 2 hours ago