Contextwaise
Prompt caching

Context optimization that never breaks your prompt cache.

Providers bill a repeated prompt start at a fraction of the price, but only while it is byte-identical. Contextwaise trims new context once and never edits what the model already saw.

See it live
Docs: keep your prompt cache hitting across a conversation
What the provider bills, turn by turn
turn 3
systemturn 1turn 2turn 3
turn 4
systemturn 1turn 2turn 3turn 4
edited
systemturn 1turn 2turn 3turn 4
cache read, ~0.1x price (system = system prompt + tools)new, full priceeditedcache lost after it
How prompt caching works

A discount on the part that did not change.

LLM APIs are stateless: every turn resends the whole conversation. Providers cache what they have already processed and bill the repeated start of a prompt at a fraction of the input price: Anthropic charges about a tenth for cache reads and a quarter more for writes (Anthropic docs), OpenAI and Gemini apply their discounts automatically.

The catch is in one word: prefix. The cache is matched byte by byte from the start of the prompt. At the first byte that differs, everything after it is billed at full price again, and written to the cache again.

What breaks it

Most context compression edits the past.

Compression that runs on the whole conversation before every call rewrites what earlier turns contained: a chunk kept at turn 3 is dropped at turn 4 because the question changed, a tool result is summarized after the fact, old turns are cleared to make room. Each edit moves the first differing byte to the middle of the history, and the cache is lost from there on. On a long conversation that can cost more than the trimming saved.

On newer Claude models there is a second cost. The model's saved reasoning is tied to the exact conversation that produced it. Editing an earlier turn invalidates it, and some requests with edited history are rejected outright.

Cache killerWhat happensContextwaise
Re-trimming earlier turnsHistory changes, cache lost from that pointNever: decisions are made once and remembered
Re-ranking contextItems move, the prefix changesKept items stay in your order
Switching models mid-conversationCaches are per model, all lostRouting decides once per conversation
Timestamps in the system promptThe very start changes on every callNot ours to fix, but our checklist flags it
The rule

A filter at the door, not an editor of the past.

Contextwaise optimizes each piece of context the first time it enters the conversation: the chunks retrieved for this turn, the tool result that just came back. Your app stores the optimized version and resends it unchanged. From the provider's point of view, the history only ever grows at the end, which is exactly what its cache is built for.

If your code rebuilds the context from the original sources on every turn and you cannot change that, send a conversation id with the API: every decision in that conversation is remembered, and a resent item gets the same decision again, so the history the model sees still never changes.

Your side

A short checklist for a warm cache.

Contextwaise keeps its own output stable. These keep the rest of your prompt stable too.

01
Freeze the system prompt

No dates, user names or modes in it. Put changing information at the end of the conversation instead.

02
Sort and keep your tools

Tool definitions sit at the start of the prompt. Adding, removing or reordering one resets the whole cache.

03
Append, never edit

Store what you sent and resend it unchanged. Pass only new content through Contextwaise.

04
Mark the cache on Claude

Anthropic needs a cache_control marker, automatic or per block; OpenAI and Gemini cache automatically. See how Anthropic prompt caching works.

05
Mind the gaps

Caches expire after minutes of inactivity. Long pauses between calls mean paying to write the cache again.

Cache-safe by design.

Trim new content once

Each piece of context is optimized when it first enters the conversation. What was sent is never re-optimized.

Same input, same output

A tool result's view is decided once and replayed byte for byte by tool call id. Cleanup is deterministic.

Your order, kept

Kept items come back in the order you sent them, never re-ranked, so turn 3 reads the same on turn 30.

Conversation memory

Pass a conversation id and every decision is remembered, so an item sent again gets the decision it got the first time.

One model per conversation

Model routing decides once per conversation and only advises a switch when it pays back the lost cache.

No history edits

On newer Claude models, edited history can invalidate saved reasoning or be rejected. Contextwaise never edits history.

Questions about prompt caching

Does it break the prompt cache?

No. Contextwaise never changes a message that was already sent: it only decides what new context to send, and kept items return in your order. With a conversation id it remembers its decisions, so content sent again is decided the same way. The agent SDK shrinks each tool result once and replays that view byte for byte.

Does the agent SDK keep the cache?

Yes. A tool result is shrunk once, when the tool returns, and that view is stored in the conversation. If the framework asks for it again, the same view is replayed byte for byte.

Is caching a replacement for context optimization?

No. Caching discounts the repeated prefix; it does nothing for the new content each turn adds, and the first read is always at full price. Cut variable context first, then cache the stable prefix.

What about provider-run tools like web search?

Tools the provider runs itself never pass through your code, so nothing can trim them. Cap them with the provider's own settings, such as the maximum number of searches or fetched tokens.

Cut your LLM costs
without cutting quality.

One layer between retrieval and inference removes the context your model never needed, in milliseconds. Fewer input tokens, lower cost, a prompt cache that keeps hitting, and every removal logged with its reason.

See it live
Keep your model, retriever and prompts. Remove one call to roll back.