Show HN: Claude-thermos – keeps your Claude session warm for you

ss0ck_r4w about 4 hours ago 29 commentsRead Article on github.com

⚡ Community Insights

Discussion Sentiment

89% Positive

Analyzed from 654 words in the discussion.

Discussion (29 Comments)Read Original on HackerNews

SwellJoe•29 minutes ago

This is just making it more expensive for everyone else, right?

How Claude handles its sessions is none of my business. I'm going to let them do the best they can to provide good service for everyone, and if they can't/won't, I'll switch to a provider that can.

Using these massive models is already pretty danged extravagant, I'm not going to demand to be at the front of the queue at all times, too.

s0ck_r4w•4 minutes ago

How do a few extra requests with the same prefix make it more expensive for everyone else?

unholiness•about 3 hours ago

I've directly inspected calls for pro/Max plans and as of today they have 1hr cache expiries. This has definitely degraded to 5 min in the past but that's the behavior today.

If you're paying API rates, you can choose 5m or 1hr yourself (and pay different rates).

Keeping a 1hr cache warm could still be useful, sure, but outside that, I don't see much use of this today.

s0ck_r4w•about 3 hours ago

When using API keys the default is still 5 minutes. Setting 1 hour for the entire session is actually a lot more wasteful considering the 2x rate that it comes with

alukin•about 3 hours ago

Feels like will be shut down real quick

gogobio•about 3 hours ago

Yes, this wastes cycle. They dump your cache and deallocate the VMs so that others can use it. This will result in tokens wasted and downtime for others.

supern0va•about 2 hours ago

Can you expand on this? I can't imagine this is holding up an entire machine or materially impacting capacity. Isn't it common to have a tiered cache and to ultimately evict (or move to another tier) if there's an active inference request and no available capacity elsewhere?

Lalabadie•about 1 hour ago

Yes, however if a tool like this commoditizes a way to stay "higher" in the cache hierarchy to avoid getting dropped, then for the same total capacity, Anthropic have to become stricter about retention.

Cyberdog•42 minutes ago

Agreed, the Thermos company's lawyers will probably fire off a terse letter once they find out.

l1n•about 3 hours ago

Why? Seems fine if people want to burn more cache reads.

agluszak•about 3 hours ago

Because it makes Anthropic earn less money

munk-a•about 3 hours ago

> Because it makes Anthropic burn more money

Fixed that for you.

cadamsdotcom•about 3 hours ago

"Earn"?

Cache duration is arbitrary. What it actually does (if used en masse) is decrease the amount of oversubscription their infra can handle..

jonas21•about 3 hours ago

Is the 5-minute expiration correct? I thought it was more like ~1 hour.

s0ck_r4w•about 3 hours ago

You can set the retention globally (whole session) to 1 hour which will actually make it more expensive. The default is 5 minutes.

*UPD:* actually it appears the default is authentication-dependent. API key gets 5 minutes, subscriptions - 1 hour.

foota•8 minutes ago

Claiming that it's more expensive isn't true, it's workload dependent. Filling the cache (worst case) 12 times in an hour is much more expensive than caching it for an hour, and the cheapest is to have your 5m cache refilled by a ping like this is doing.

Imo it's their fault for not having pricing that aligns incentives.

cosmotic•about 2 hours ago

How will this not lead to tragedy of the commons?

davesque•about 2 hours ago

Exactly. I really wish people wouldn't use this. If this becomes popular, anthropic will just modify their cache policy to be much less fair. It's not like they have infinite cache.

ATMLOTTOBEER•about 3 hours ago

Glad this exists. It will force anthropic to fix their flawed cache mechanism.

SwellJoe•26 minutes ago

While Anthropic is far behind on caching and efficiency (made more dramatically apparent by how much cheaper it is to use GPT 5.6 Sol at API rates than even Opus 4.8, much less Fable), a bunch of people forcing their way to the front of the queue at the expense of everyone else isn't going to solve that.

cortesoft•about 3 hours ago

How should they fix it?

s0ck_r4w•about 2 hours ago

Make the cache TTL more adaptive. Have more tiers than just 5m and 1h. Long-running subagents become increasingly ubiquitous. There's no reason for Anthropic to not do a better job for this use case.

addaon•about 2 hours ago

There are a lot of workflows where cache is likely to be consumed exactly once (e.g. anything chat-like where a single thread is extended; if a prefix cache hit is found, and a set of new longer prefixes is inserted in the cache, it is unlikely that the original shorter prefix will be consumed in later turns). I could see allowing each session key a small number (1?) of 24-hour cache entries, where inserting a new cache entry (perhaps the maximum-length prefix, perhaps an API-tagged specified-length prefix) consumes that slot and either boots the rest of the items from cache, or demotes them to 5 minutes or something. Basically: workflow awareness, not uniform handling.

razodactyl•about 3 hours ago

Don't we pay for cache input though?

broodbucket•about 3 hours ago

You can send and receive 1 byte and refresh the cache. Keeping the cache alive isn't free, but it's close.

cortesoft•about 3 hours ago

The idea is this keeps it in cache so you don’t have to pay to re-input.

j45•about 3 hours ago

This is a good idea if it could manage it in an acceptable way.

For example, there might be something I intended to complete in one sitting, but took two sittings in the same day unexpectedly. Maybe it could just be a few cache delays per day or something, tagged in advance somehow.