Cache the Model

I rendered my episode about retries four times. Same script, same voice, same answer, and I paid for the voice once: $0.39, then nothing, nothing, nothing. Where did the other three bills go? Two caches.

The short version

• The response cache: skip the call. My studio saves every voice line under a key, a sha256 of the exact words and settings. Same line, same key, same file; a re-render looks the key up and never calls the model. The cheapest call is the one you never make.
• The prompt cache: pay a tenth. Most of every prompt repeats (instructions, tools, files, the conversation so far). Anthropic charges 1.25× to write that prefix to the cache and 0.1× to read it back (0.05× on Opus 5.5); OpenAI caches automatically above 1,024 tokens, up to 95% off. A cache pays for itself on its first reuse.
• One word breaks it. The cache is a prefix: change one word near the top, like today's date, and everything after it is a miss. Put stable things first and what changes last.
• It expires. Anthropic's cache lives five minutes by default, reset by every use; pause longer and the next call writes it again. There's a one-hour option at twice the write price.
• Only cache answers that shouldn't change. A voice line, yes. This morning's news, no.

This episode is the first in a new look for the series: an investigation board, built into my video studio for it.

Further reading

• Prompt caching (Anthropic) — the prices, the 5-minute and 1-hour lifetimes, and how the prefix is matched.
• Prompt caching (OpenAI) — automatic caching, and why stable content goes first.

Next episode: Scale Before the Wave. Previously: Stop Babysitting. All episodes →