What cached input pricing actually saves
Cached input is advertised as 90% off, and the discount is real — but it applies to one column, on a byte-identical prefix, with an expiry clock running.
Every major provider now bills a discounted rate for input tokens it has already seen. The headline is usually “up to 90% cheaper”, and on the rate card that is true. On an invoice it rarely is, because the discount applies to one of three columns and comes with conditions that are easy to violate by accident.
What the cached rate is
When you send a request, the provider computes internal state for every input token. Prompt caching keeps that state for a short while, keyed on the prefix of your request. If the next request begins with a byte-identical prefix, the provider reuses the stored state instead of recomputing it, and bills those tokens at the cached rate.
The multiples in practice, from the current rate cards:
| Model | Input / 1M | Cached / 1M | Discount |
|---|---|---|---|
| DeepSeek V4 Pro | $1.32 | $0.044 | 30× |
| GPT-5.6 Terra | $2.00 | $0.20 | 10× |
| Claude Opus 5 | $5.00 | $0.50 | 10× |
| Gemini 3.1 Pro | $2.00 | $0.20 | 10× |
| Grok 4.5 | $2.00 | $0.30 | 6.7× |
| GPT-4.1 | $2.00 | $0.50 | 4× |
| GPT-4o | $2.50 | $1.25 | 2× |
| LLaMA 3.3 70B (hosted) | $1.04 | — | no cached rate |
Note the range. “90% off” is the best case, not the going rate, and the older models are markedly worse at it. The full comparison carries the cached column for every model in the catalogue, which is the fastest way to see whether a cheaper-looking model is actually cheaper for a cache-heavy workload.
A worked example
A support assistant with a 12,000-token prefix — system prompt, tool schemas, a style guide and six few-shot examples — handling 500 requests a day, each producing about 400 output tokens. On GPT-5.6 Terra: $2.00 input, $0.20 cached, $12.00 output.
Without caching:
| Tokens/day | Rate | Cost/day | |
|---|---|---|---|
| Input | 6,000,000 | $2.00/M | $12.00 |
| Output | 200,000 | $12.00/M | $2.40 |
| $14.40 |
With caching — one request pays full price, 499 hit the cache:
| Tokens/day | Rate | Cost/day | |
|---|---|---|---|
| Input (first call) | 12,000 | $2.00/M | $0.02 |
| Input (cached) | 5,988,000 | $0.20/M | $1.20 |
| Output | 200,000 | $12.00/M | $2.40 |
| $3.62 |
The input line drops by 90% exactly as advertised: $12.00 to $1.22. The bill drops by 75%, from $14.40 to $3.62, because output was never eligible and output is the expensive column.
That gap between 90% and 75% widens fast. Change the workload to a short question in and a long generated document out — say 800 input tokens and 3,000 output — and caching moves almost nothing, because there is barely any input to discount.
The four conditions
1. Byte-identical, and a prefix. The match runs from the first character. Change one word of the system prompt, add a tool, reorder the few-shot examples, and everything from that point on is a cache miss. The classic self-inflicted wound is interpolating a timestamp, a request id or a user’s name near the top of the prompt — that invalidates the entire cache on every single call while looking, in the code, like a harmless template variable.
2. There is usually a minimum length. Providers do not cache short prefixes; below roughly a thousand tokens the bookkeeping is not worth it and the request is billed at the normal rate. A 400-token system prompt gets no discount no matter how often you send it.
3. Writes are not always free, and the cache expires. OpenAI’s caching is automatic with no write surcharge. Anthropic bills a cache write above the base input rate, with the multiplier depending on the lifetime you ask for. That is fine when a prefix is reused many times, and it inverts when it is not: if your requests arrive further apart than the cache lifetime, every call pays a write and none ever gets a hit, which is strictly more expensive than not caching. Sparse traffic is the one case where prompt caching is a mistake.
4. Only input is cached. Nothing about the discount touches output. If output dominates your spend, the lever you want is a shorter response, a cheaper model, or both.
Two provider quirks that beat caching
DeepSeek’s off-peak window. DeepSeek publishes a time-of-day discount: 01:00–04:00 and 06:00–10:00 UTC bill at half the standard rates. For batch work whose schedule you control, that is a 50% cut on every column including output — which is a larger lever than caching input on a job that only reads each document once.
xAI’s 200K cliff. On the Grok models, crossing 200,000 prompt tokens bills the whole request at double the input, cached and output rates. Trimming a 210,000-token prompt down to 190,000 does not save 10% — it halves the request. Before optimising caching on Grok, check whether you are sitting just over the line. Paste the prompt into the token counter and look at the number.
Designing a prompt that caches well
- Order by volatility. System prompt, then tool schemas, then few-shot examples, then retrieved context, then the user turn. Most stable first, always.
- Never interpolate anything volatile into the prefix. No timestamps, request ids, session ids or random seeds above the user turn. If the model needs the current date, put it in the last message, not the first.
- Keep history append-only. Editing or summarising an earlier turn invalidates the cache from that turn onward. Truncating from the front of a conversation is the most expensive possible way to manage context length.
- Pin your prompt version. An A/B test that alternates two system prompts halves your hit rate. Route each variant to its own worker if you can.
- Measure the prefix once. You cannot reason about any of this without knowing how many tokens the stable part actually is. Paste it into the token counter — for GPT models that is an exact BPE count, computed in your browser.
The summary: cached input is one of the few genuinely large, genuinely easy wins in LLM cost work — on a high-volume endpoint with a long, stable prefix and short answers. On sparse traffic, a short prefix, or an output-heavy workload, it does much less than the rate card implies, and on a provider that charges for writes it can cost you money. Work out which of those you have before you spend a sprint on it.
Put it to work: count tokens, measure a document with the PDF token counter, or compare rates on the LLM pricing page. All posts are on the blog index.