Prompt caching reuses the computed state of a prompt prefix across API calls. Instead of processing your system prompt, tool definitions, and conversation history from scratch on every request, the provider stores the key-value tensors from the first call and reads them back on subsequent ones. The read costs up to 90% less than a fresh input. The write costs slightly more than standard input on Anthropic and DeepSeek, and costs nothing extra on OpenAI.
Most developers never touch it. It requires zero code changes on OpenAI, one JSON field on Anthropic. You can cut your input token bill roughly in half this afternoon.
To understand the fundamentals first, read what AI tokens are and how they work.
How prompt caching works
Every LLM API call goes through a “prefill” phase where the model reads your entire prompt and computes attention across all tokens. This is expensive because the computational cost scales quadratically with context length. For a 10K-token prompt, that means roughly 100 million pairwise comparisons, every single call.
Prompt caching skips the prefill for the part of your prompt that hasn’t changed since the last call. Here is the flow:
- First request: The model processes your entire prompt from scratch. The system computes key-value (KV) tensors, the intermediate state of the attention layers, for all tokens in your prompt’s prefix. It caches these KV tensors on the GPU server that handled your request.
- Same prefix, second request: The system checks whether a matching prefix exists in the cache. If it does, the model skips the prefill for the cached portion and only processes the new tokens after the prefix. You pay the cache-read rate on the cached portion, which is 10% of the standard input rate on Anthropic, and up to 90% off on OpenAI.
- Different prefix or cache expired: The system processes the full prompt from scratch and writes a new cache entry.
The key word is “prefix.” The cached content must be a contiguous block at the beginning of your prompt, identical in every request. Static content at the start (system instructions, tool definitions, document context) gets cached. Variable content at the end (the user’s specific question) doesn’t.
Caching also cuts latency. OpenAI reports up to 80% faster time-to-first-token on cached reads compared to fresh prompts. For latency-sensitive applications like voice assistants or real-time chat, this matters as much as the cost savings.
The context window determines how much you can cache. LLM context windows set the ceiling: a 200K window on Claude Sonnet means you can cache extremely large prefixes provided they meet the minimum token threshold.
Provider comparison: how each one works
| Feature | Anthropic | OpenAI | DeepSeek | Gemini |
|---|---|---|---|---|
| Enable method | cache_control: ephemeral (automatic or explicit) | Automatic, no code needed | Context caching API | Context caching API |
| Min tokens | 4,096 (Opus 4.7), 2,048 (Sonnet 4.6), 4,096 (Haiku 4.5) | 1,024 | Specific minimums | Specific minimums |
| Cache read cost | 10% of base input (90% off) | Up to 90% off | $0.0028/M (98% off for DeepSeek) | 90% off |
| Cache write cost | 1.25x base (5m TTL), 2x base (1h TTL) | Free | Extra cost | Extra cost |
| Default TTL | 5 minutes | 5-10 minutes | Varies | Varies |
| Extended TTL | 1 hour (at 2x cost) | 24 hours (GPT-5.5, GPT-5.4) | Varies | Varies |
| Max breakpoints | 4 (explicit), +1 automatic | Not applicable | Not applicable | Not applicable |
| Pre-warming | max_tokens: 0 | Not available (automatic) | Not available | Not available |
Anthropic
Anthropic gives you fine-grained control with two caching modes. Automatic caching adds cache_control: {type: "ephemeral"} at the top level of your request body. The system automatically places the cache breakpoint at the last cacheable block and moves it forward as conversations grow. This is the right choice for multi-turn chats.
Explicit caching places cache_control directly on individual content blocks like your system prompt or tool definitions. You can set up to four separate breakpoints, each tracking a different prefix. This matters when you have sections that change at different rates: your system prompt might be stable for weeks, your document context changes daily, and your conversation grows by the minute.
The cache-read rate on Anthropic is 10% of the base input price. For Claude Opus 4.7 at $5 per million input tokens, cached reads cost $0.50 per million, a 90% discount. For Claude Sonnet 4.6 at $3 per million, cached reads cost $0.30 per million. Cache writes cost 25% more than standard input for the 5-minute TTL, or 2x for the 1-hour TTL.
Anthropic also exposes a 20-block lookback window. If your breakpoint is on block 35, the system checks 20 positions backward (blocks 35 through 16) for a matching cache entry. If your prior write was at block 15, it’s out of reach. Add a second breakpoint closer to the earlier position to maintain coverage in long conversations.
OpenAI
OpenAI’s caching is the simplest to use: you do nothing. The platform automatically caches prompt prefixes across requests to the same server. Any prompt with 1,024 tokens or more is eligible. The system handles routing, hashing, and eviction behind the scenes.
OpenAI charges nothing extra for writing to the cache. Cache reads are discounted up to 90% off the standard input rate. There is no separate cache-write pricing tier, no breakpoints to manage, no minimum thresholds to worry about beyond 1,024 tokens. This makes OpenAI’s caching the default choice for teams that want the savings without the engineering overhead.
OpenAI offers two retention policies. In-memory caching, available on most models, keeps prefixes alive for 5 to 10 minutes of inactivity, up to one hour maximum. Extended prompt caching on GPT-5.5, GPT-5.4, and newer models retains prefixes for up to 24 hours by offloading KV tensors to GPU-local storage. The prompt_cache_retention parameter lets you choose which policy to use.
OpenAI also provides a prompt_cache_key parameter. You pass a consistent string, and the system routes requests with the same key and prefix to the same server, improving cache hit rates. This is useful when you have many concurrent users with identical system prompts.
DeepSeek
DeepSeek offers the steepest relative discount. The standard input rate for DeepSeek V4-Flash is $0.14 per million tokens. The cache-read rate is $0.0028 per million, a 98% discount. That is a 50x cost reduction on cache hits, the most aggressive of any provider. DeepSeek uses a context caching API with its own minimum threshold requirements.
Gemini
Google’s Gemini models support context caching with standard 90% discounts on cached reads. Gemini 2.5 Pro at $1.25 per million input drops to $0.125 on cache reads. Gemini 2.5 Flash drops from $0.30 to $0.03. Gemini 2.5 Flash-Lite drops from $0.10 to $0.01. Caching requires explicit configuration through the Gemini API and comes with write costs and minimum token thresholds.
The full pricing tables across all models are in the AI token pricing guide if you need the complete breakdown.
What you actually save: concrete numbers
Here is a Python estimator that shows the cost difference between cached and uncached input for a given scenario:
def prompt_caching_savings(provider, model, prefix_tokens, new_tokens_per_call, calls_per_day):
# Pricing per 1M tokens (May 2026) - standard input and cache read
rates = {
"anthropic-opus": (5.00, 0.50),
"anthropic-sonnet": (3.00, 0.30),
"anthropic-haiku": (1.00, 0.10),
"openai-gpt55": (5.00, 0.50),
"openai-gpt54": (2.50, 0.25),
"openai-gpt54-mini": (0.75, 0.075),
"deepseek-flash": (0.14, 0.0028),
"gemini-flash": (0.30, 0.03),
}
input_rate, cache_read_rate = rates[model]
# Without caching: every token is full-price input
uncached_per_call = (prefix_tokens + new_tokens_per_call) / 1_000_000 * input_rate
uncached_monthly = uncached_per_call * calls_per_day * 30
# With caching: prefix at cache-read rate, new tokens at full input
cached_per_call = (prefix_tokens / 1_000_000 * cache_read_rate) + (new_tokens_per_call / 1_000_000 * input_rate)
cached_monthly = cached_per_call * calls_per_day * 30
return {
"uncached_monthly": round(uncached_monthly, 2),
"cached_monthly": round(cached_monthly, 2),
"monthly_savings": round(uncached_monthly - cached_monthly, 2),
"savings_percent": round((1 - cached_monthly / uncached_monthly) * 100, 1),
}
# Support chatbot: 2K-token system prompt, 200 token new message, 5,000 calls/day
result = prompt_caching_savings("anthropic", "anthropic-sonnet", 2000, 200, 5000)
print(f"Sonnet chatbot - Uncached: ${result['uncached_monthly']}/mo, Cached: ${result['cached_monthly']}/mo")
print(f" Savings: ${result['monthly_savings']}/mo ({result['savings_percent']}%)")
# Agent with 8K tools and instructions, 500 token task, 2,000 calls/day
result = prompt_caching_savings("anthropic", "anthropic-opus", 8000, 500, 2000)
print(f"Opus agent - Uncached: ${result['uncached_monthly']}/mo, Cached: ${result['cached_monthly']}/mo")
print(f" Savings: ${result['monthly_savings']}/mo ({result['savings_percent']}%)")
# High-volume classification: 1.5K prompt, 100 tokens, 50K calls/day on mini
result = prompt_caching_savings("openai", "openai-gpt54-mini", 1500, 100, 50000)
print(f"Mini classifier - Uncached: ${result['uncached_monthly']}/mo, Cached: ${result['cached_monthly']}/mo")
print(f" Savings: ${result['monthly_savings']}/mo ({result['savings_percent']}%)")Output:
Sonnet chatbot - Uncached: $990.0/mo, Cached: $180.0/mo
Savings: $810.0/mo (81.8%)
Opus agent - Uncached: $2550.0/mo, Cached: $390.0/mo
Savings: $2160.0/mo (84.7%)
Mini classifier - Uncached: $1800.0/mo, Cached: $281.25/mo
Savings: $1518.75/mo (84.4%)The Opus agent scenario shows the most dramatic savings: $2,160 per month from enabling one feature. The system prompt and tool definitions (8,000 tokens) get prefill’d fresh on every call without caching. With caching, those 8,000 tokens cost $0.50 per million instead of $5 per million. The math is straightforward.
Here are four real-world scenarios with full breakdowns:
| Scenario | Prefix tokens | New tokens/call | Calls/day | Model | Monthly without | Monthly with | Savings |
|---|---|---|---|---|---|---|---|
| Support chatbot | 2,000 | 200 | 5,000 | Claude Sonnet 4.6 | $990 | $180 | $810 (82%) |
| Coding agent | 8,000 | 500 | 2,000 | Claude Opus 4.7 | $2,550 | $390 | $2,160 (85%) |
| Doc Q&A | 15,000 | 300 | 1,000 | GPT-5.4 | $1,148 | $135 | $1,013 (88%) |
| Text classifier | 1,500 | 100 | 50,000 | GPT-5.4-mini | $1,800 | $281 | $1,519 (84%) |
The token waste article covers optimization tactics beyond caching, but prompt caching alone often cuts input costs by 80-90% for the scenarios above.
Caching strategies by use case
Multi-turn conversations
This is the highest-volume use case and the easiest to cache. Every message in a conversation reuses the same system prompt and accumulates history that grows but never changes backward.
On Anthropic, use automatic caching. Add cache_control: {type: "ephemeral"} at the top level of your request. The breakpoint automatically moves to the last cacheable block on each turn, so the first 10 messages are cached reads by turn 11. You never need to update the breakpoint.
On OpenAI, no configuration is needed. The system handles it automatically. The cache hit rate improves when multiple users share the same system prompt prefix, which is why the prompt_cache_key parameter helps for multi-tenant applications.
On both providers, during the first turn of each new conversation, the cache is always a miss. You pay standard input rates on that first turn. From turn two forward, the cache hits and you pay the discount rate. This is why pre-warming (see below) matters for latency-critical user-facing applications.
Agent tool loops
Autonomous agents calling tools in loops present a unique caching challenge. Every tool call generates a new API request, but the system prompt and tool definitions stay identical across all calls within a session.
On Anthropic, use explicit cache breakpoints. Place one cache_control on your system prompt block and another on your tools definition block. The model re-reads both from cache on every tool iteration. A 5,000-token tool and system block on Claude Opus costs $0.025 per fresh call. With caching, it costs $0.0025 per call. Over a 50-iteration agent loop, that is $1.25 versus $0.125 in cache-read costs for the prefix alone.
AI agents that run long autonomous loops burn through thousands of API calls per session. Prompt caching is the single biggest lever you have for reducing agent operating costs.
Document Q&A
When users ask questions about a fixed document, cache the document in the prefix and vary only the question at the end. The 15,000-token contract or manual is the expensive part. The 50-token question at the end is nearly free.
On Anthropic, place cache_control at the end of your document block, before the user’s question. On OpenAI, just include the document early in the prompt structure and caching handles it automatically.
For document collections larger than the context window, RAG still wins. Use prompt caching for the system prompt and retrieved chunks combined, not for the entire document corpus. A well-tuned RAG pipeline retrieves 5-10 relevant chunks per query, and caching that combination across similar queries saves money.
Batch processing
When you process many similar items through the same prompt template, cache the instructions and examples prefix. OpenAI’s Batch API gives a 50% discount on top of the caching discount, but at the cost of delayed processing. For synchronous batch runs where latency matters, caching alone often matches the Batch API savings on the input side.
Common mistakes that kill your cache
1. Breakpoint on changing content
The most common Anthropic mistake: putting cache_control on a block that contains a timestamp, a unique user message, or any per-request data. The breakpoint must mark the end of static, unchanging content. If the block at the breakpoint changes, the prefix hash changes, and the cache misses on every call.
Take a prompt with a large static context followed by a timestamp block. If you set cache_control on the timestamp, every request has a different hash. The system looks back 20 blocks for a match but never finds one because writes only happen at breakpoints, and the content at your breakpoint is always different. Zero cache hits, zero savings.
Fix: move the breakpoint to the last block that stays identical across requests, the end of your static prefix, not the varying suffix.
2. Not meeting minimum token thresholds
Caching fails silently when your prefix is below the minimum token count for your model. On Anthropic with Claude Sonnet 4.6, the minimum is 2,048 tokens. A request with 1,800 tokens marked for caching simply will not cache, and no error is returned. Check cache_creation_input_tokens and cache_read_input_tokens in the usage object. If both are zero, the prefix was too short.
On OpenAI, the minimum is 1,024 tokens. cached_tokens will show as zero for prompts under this threshold. If your prefix is slightly below the threshold, expanding it to meet the minimum is often worthwhile because the savings per call exceed the cost of a slightly longer prefix.
3. Cache invalidation from configuration changes
Several seemingly minor API settings invalidate the cache entirely. On Anthropic, changing tool_choice, adding or removing images anywhere in the prompt, toggling web search or citations, changing the speed setting, or modifying tool definitions all invalidate the cache. On both Anthropic and OpenAI, changes to the thinking or reasoning configuration also invalidate the cache.
This matters because autonomous agents often toggle these settings across iterations. An agent that enables web search for one query invalidates its entire tool and system cache for the next query. Thinking tokens have their own caching rules; on Claude Opus 4.5 and Sonnet 4.6, thinking blocks are preserved through cache reads, but on earlier models, they are stripped and invalidate the messages cache.
The rule: lock your tool configuration, image set, and thinking mode before the first call in a session. Do not change them between calls within a cached session unless you accept the cache invalidation cost.
4. Losing cache hits in long conversations
Anthropic’s 20-block lookback window means a cache write from turn 1 will be out of range by turn 21. If each turn adds 2 blocks (one user, one assistant), your cache write at turn 1’s breakpoint is unreachable by turn 11. Add a second explicit breakpoint at a mid-conversation position early to accumulate a separate cache write closer to the growing tail of the conversation.
Cache pre-warming
Anthropic supports cache pre-warming: sending a request with max_tokens: 0 to prefill the prompt and write the cache entry without generating any output. The model processes your system prompt or tool definitions, saves the KV tensors to the cache, and returns immediately with zero output tokens. The first real request from a user hits a warm cache and gets the latency benefit immediately instead of paying the cache-miss penalty on turn one.
OpenAI does not expose a pre-warming API because its caching is fully automatic. The first request with a given prefix is always a miss, and subsequent requests get the cache hit. For latency-critical applications on OpenAI, a warmup request with a placeholder user message can trigger the initial cache write before real traffic arrives.
FAQ
Does prompt caching change the model’s output?
No. Cache reads produce identical outputs to fresh processing. The only difference is latency and cost. The model does not know whether its input came from a cache or was freshly computed.
Can I use prompt caching with thinking and reasoning tokens?
Yes, with limitations. OpenAI caches the prompt prefix regardless of whether reasoning is enabled. Anthropic caches thinking blocks alongside other content when they appear in previous assistant turns, but thinking blocks cannot be directly marked with cache_control. Cache invalidation rules vary by model: on Claude Opus 4.5 and Sonnet 4.6, thinking blocks are preserved by default and the cache remains valid. On earlier models, adding non-tool-result user content strips thinking blocks and invalidates the messages cache.
How long does the cache last?
Anthropic: 5 minutes by default, refreshable on each cache hit at no extra cost. 1-hour TTL available at 2x the input price. OpenAI: 5 to 10 minutes for in-memory caching, up to 24 hours with extended prompt caching on GPT-5.5 and GPT-5.4. DeepSeek and Gemini: refer to their current docs for TTL specifics.
Do I pay extra for writing to the cache?
Anthropic: yes. Cache writes cost 1.25x the base input rate for the 5-minute TTL, 2x for the 1-hour TTL. OpenAI: no. Cache writes are free. DeepSeek and Gemini: yes, writes incur extra cost. The write premium on Anthropic is small compared to the read savings. A single write at $6.25 per million tokens on Opus 4.7 pays for itself after one cache-read of the same prefix at $0.50 per million.
What happens if I change something in the cached prefix?
The cache is invalidated and a new entry is written on the next request. Anthropic’s cache hierarchy means changes to tool definitions invalidate everything. Changes to the system block invalidate the system and messages caches. Changes to messages only invalidate the messages cache.
Which provider gives the best caching savings?
DeepSeek has the steepest relative discount at 98% off, but their base rates are already extremely low ($0.14/M input). In absolute dollars saved per request, Anthropic’s 90% discount on Opus 4.7 at $5/M base input delivers the highest dollar savings per cached token. OpenAI’s free writes and zero-configuration setup make it the easiest to adopt.
Can I manually clear the cache?
No. Caches auto-evict based on TTL. Anthropic and OpenAI do not expose manual cache clearing. A prefix that has not been seen recently is automatically removed.
All pricing as of May 2026. Prompt caching rates and TTL policies change frequently. Check your provider’s documentation before implementing.
Related topics
- What are AI tokens? A complete guide to tokenization, costs, and context windows. Understand token fundamentals first.
- LLM context windows explained: limits, costs, and developer workarounds. How context size sets the ceiling for what you can cache.
- AI token pricing guide: What LLMs actually cost per model. Full pricing comparison across all providers.
- Thinking tokens explained: What reasoning models actually cost you. How thinking tokens interact with prompt caching and what the rules are.
- What is an AI agent?. Why autonomous agents especially benefit from cached tool definitions and system prompts.
- How to stop wasting tokens in Claude Code: 7 data-backed fixes. Optimization tactics that compound with caching savings.