GuidePrompt caching
Prompt caching, explained.
When you send a prompt that starts exactly like one you sent a few minutes ago, the provider can reuse the work it already did on that start. Those cached tokens cost about a tenth of the normal price with Claude and OpenAI, as long as the start matches byte for byte.
AHow it works
The prefix is stored
The provider keeps the processed beginning of your prompt for a while. The next request that begins with the same text reads that part from the cache.
Only the start counts
The match runs from the first token. One changed character early on, like a timestamp in the instructions, and everything after it is new again.
It expires
Claude keeps a cache entry for five minutes by default, renewed every time it is used, or for an hour if you ask. Others keep it for a similar short while.
BWhat it costs
Claude
A cached read costs a tenth of the input price. Writing an entry costs 1.25 times the input price for five minutes, or twice for an hour, so caching pays off from the first reuse.
OpenAI
Long prompts are cached automatically, and cached input is billed at a fraction of the input price, a tenth on current models.
Gemini and Grok
Both cache repeated prompts automatically and bill cached tokens at a discount.
Output is unchanged
Caching only touches input. What the model writes costs the same.
CWhat breaks the cache
Changing the start
Editing instructions, adding or removing tools, or reordering anything near the top.
Dates and counters early in the prompt
A clock or a running number in the system prompt makes every request look new.
Long pauses
Come back after the cache expired and the next turn pays full price once.
DHow Lunchbox uses it
Stable rooms
Lunchbox orders every room so the conversation stays byte for byte the same between turns, with shared memory and the task last.
Cache marks on Claude
For Claude keys it places cache breakpoints itself and keeps the room's instructions for an hour, so a room left over lunch still starts from the cache.
A cache key per agent
OpenAI requests carry a key per room and agent, so each one keeps hitting its own cache. Ollama keeps the model loaded between turns.
Counted honestly
The Savings page counts each cached token at the gap between the full and the cached price, minus what writing the cache cost.
EQuestions
What is prompt caching?
A provider stores the processed start of a prompt and reuses it when the next prompt starts the same way. Those cached tokens cost about a tenth of the normal input price with Claude and OpenAI.
How much does prompt caching save?
Up to 90% on the cached input tokens. How much of a whole bill that is depends on how much of each request repeats; in long agent sessions most of it does.
Why is my prompt cache not hitting?
Something near the start changed, such as instructions, tools or a timestamp, or the cache expired after a few minutes without use.