MLOps

Bedrock prompt caching: cut input token costs by 90%

Bedrock prompt caching explained: cachePoint checkpoints, per-model minimums, TTLs, pricing (+25% writes, -90% reads) and six patterns for production agents.

Bedrock prompt caching: cut input token costs by 90%

Bedrock prompt caching: cut input token costs by 90%

On 15 September 2026 AWS published an engineering guide to Amazon Bedrock prompt caching: reusing the stable part of a prompt (system prompt, documents, tool definitions) across requests instead of having the model process it again every time. The headline numbers are blunt: up to 90% off the price of cached input tokens, and a time-to-first-token that improves as the cached prefix grows.

If you run agents or a RAG pipeline in production, this is not a nice-to-have. The bill for an agent loop is dominated by input tokens, because every turn sends the model the same context again: the same instructions, the same tools, the same documents. Bedrock prompt caching targets exactly that redundancy.

The context

A language model has no memory between calls. Each request carries the whole context, and the provider bills every input token as if it were new. An assistant answering ten questions about the same 10,000-token PDF processes, and pays for, those 10,000 tokens ten times.

Prompt caching keeps the intermediate computation of a prompt prefix on the provider's side so it is not recomputed. Anthropic introduced it on its own API in 2024; Bedrock exposes it uniformly across several model families through the Converse API. The AWS post, written by solutions architects, documents the mechanism, the pricing and six usage patterns, with code exercised on Claude Sonnet 4.5.

How the cache works: checkpoints, minimums, TTL

The cachePoint block

In the Converse API you mark the end of the cacheable zone with a cachePoint content block. Everything before the checkpoint is cached; everything after it stays dynamic:

content = [
    {"text": "<stable document>"},
    {"cachePoint": {"type": "default"}},
    {"text": "<user question>"},
]

The syntax is identical for Anthropic Claude and Amazon Nova, which matters when several models sit behind one gateway. A single cachePoint can cover several preceding content blocks; you do not need one per document.

Model-specific minimums

A checkpoint is only honoured if the prefix reaches a model-specific minimum: 1,024 tokens for Claude Sonnet 4.5 and 4.6, 4,096 tokens for Opus models. Below that, the request goes through without caching and without an error, which is the trap: the application believes it is caching and caches nothing. The cacheWriteInputTokens and cacheReadInputTokens fields in the response's usage object are the only reliable way to check.

Time to live

The default TTL is 5 minutes; selected models accept 1 hour ({"cachePoint": {"type": "default", "ttl": "1h"}}). One request can mix several TTLs under one rule: checkpoints must be ordered from the longest TTL to the shortest. A company knowledge base that rarely changes goes first with 1 hour; session context follows with 5 minutes.

The pricing, and what it implies

Writing to the cache is not free. AWS states three rules:

  • Cache write: 25% more than a standard input token.
  • Cache read: 90% less than a standard input token.
  • Cache write with a 1-hour TTL: 100% more (double).

The worked example in the post is the one to remember: a 10,000-token document sent with ten different questions costs one write, then nine reads at -90%, for a net saving of roughly 75% on that context's input tokens. The arithmetic flips if the cache expires between calls: every request after expiry triggers a fresh write at 125% of the price, and the saving evaporates. Prompt caching pays when a context is reused soon and often; it costs when it is reused rarely.

Two scope constraints to know before forecasting savings: the cache is limited to one AWS account and one Region, and cross-Region inference profiles (the post uses the global profile of Claude Sonnet 4.5) can increase write frequency, because a request does not always land in the Region where the cache was built.

Six patterns, from RAG to multi-tenant

The post applies the mechanism to six cases:

  1. Message content: the document behind a RAG answer, cached once for every follow-up question.
  2. System prompt: a persona or answer policy that is identical on every call.
  3. Tool definitions: in an agent, the tool catalogue is often the largest and most stable part of the prompt; caching it lowers the cost of every loop iteration.
  4. Mixed TTLs: enterprise reference material at 1 hour, session context at 5 minutes, in the same request.
  5. Tenant isolation: a prefix derived from a SHA-256 hash of the tenant identifier, so a cache never serves another customer.
  6. LangChain: ChatBedrockConverse.create_cache_point() inserts the checkpoint in a message array or a ChatPromptTemplate.

Latency, carefully

On time-to-first-token AWS is measured: the gain is real and grows with the size of the cached prefix, but on small documents (roughly 2,000 to 5,000 tokens) it may not be statistically significant over a small number of iterations. In other words, do not sell Bedrock prompt caching on latency for a 1,500-token system prompt; sell it on cost, and on latency only when the prefix runs to tens of thousands of tokens.

What this changes for AI teams

Bedrock prompt caching is an architecture decision, not a flag to switch on. Three practical consequences.

Layer your prompts. For the cache to hit, everything stable must precede everything that changes, and the prefix must clear the model's minimum. A prompt that drops today's date or a session id in the middle of the system prompt breaks the prefix and loses the cache. Block ordering becomes a reviewed coding rule, with a test asserting that cacheReadInputTokens is non-zero on the second request.

Instrument the hit rate. Without observability you cannot tell whether you are paying 125% writes for nothing. Both counters in the usage object belong in your traces and dashboards, per agent and per tenant: read rate, write-to-read ratio, effective cost per turn. This is exactly the kind of metric we add during an agent rollout, next to latency and cost per request.

Pick the TTL from the usage curve. An internal assistant queried all day on the same knowledge base justifies the 1-hour TTL despite its 200% write. A public chatbot whose sessions last three minutes gains nothing from it: 5 minutes is enough, and the context shared across users (system prompt, tools) must be separated from per-session context so it can be cached independently. Adopt the hash-based multi-tenant pattern as soon as one deployment serves several customers; it is a security matter as much as a cost one.

Reading the numbers on a real agent loop

Take an agent with a 6,000-token system prompt and tool catalogue, answering a user over eight turns within a few minutes. Without caching, the eight turns pay 48,000 input tokens for that static block alone. With one cachePoint after the catalogue, the first turn pays 6,000 tokens at 125% and the next seven pay 6,000 tokens at 10%: about 11,700 token-equivalents instead of 48,000, a 75% cut on the static part, before the conversation history is even considered. Add the per-turn history after the checkpoint and the saving stays intact, because the checkpoint sits before the dynamic part. Now shorten the session to two turns spaced ten minutes apart, and the same configuration pays two writes at 125% and no read at all: a 25% surcharge. Same code, opposite outcome. The only way to know which case you are in is to look at the two usage counters over a week of real traffic, then set the TTL and the block order accordingly.

Finally, prompt caching changes the old trade-off between a short prompt and a rich one. A detailed tool catalogue or an 8,000-token style guide becomes affordable once the cache is warm, and the real variable is no longer the size of the prefix but how often it is reused.

Key takeaways

  • Bedrock prompt caching reuses a prompt's stable prefix through a cachePoint block in the Converse API, identical for Claude and Nova.
  • Minimums per checkpoint: 1,024 tokens (Claude Sonnet 4.5/4.6), 4,096 tokens (Opus). TTL 5 minutes by default, 1 hour on selected models, ordered longest to shortest.
  • Pricing: writes +25%, reads -90%, 1-hour writes +100%. A document reused ten times saves about 75% of its input tokens.
  • Latency gains are real but prefix-size dependent; hard to measure below 5,000 tokens.
  • Cache scoped to one account and one Region; cross-Region profiles raise write frequency; isolate tenants by hashing.

Industrialising AI agents? SeedVision offers 3-5 day AI audits and 15-30 day production rollouts. See the packages or book a 30-min call.

Source: Optimizing cost and latency with Amazon Bedrock prompt caching (AWS Machine Learning Blog, 15 September 2026) and the Bedrock prompt caching documentation.

Cover photo: Photo by Albert Stoynov on Unsplash.