The context optimization API for production LLM apps.
Keep your retrieval, keep your model. Add one call that returns the smaller, higher-signal context and tells you what it saved.
API reference: the optimize endpoint// Before your LLM call: send the context you would have sent
const candidates = [...docs, ...toolOutputs]; // { text, source }
const res = await fetch(`${process.env.CONTEXTWAISE_URL}/api/v1/optimize`, {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.CONTEXTWAISE_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
query,
candidates,
agent: "support-bot", // per-agent budgets and stats
model: "claude-sonnet-5-5", // prices the savings
}),
});
// On any error (503 scoring_unavailable included), send what you had.
const { context, report } = res.ok ? await res.json() : { context: candidates };
const answer = await llm.generate({ query, context });
console.log(report?.tokens.savedPct, report?.latencyMs); // 70.7 510Decided before the expensive call.
Deciding has to be far cheaper and faster than reading, or it is not worth doing. Each context item becomes one question for a small decision model, all asked in parallel. Your model reads only what survives.
Between retrieval and inference. Nowhere else.
The API is one call placed after you gather context and before you call your model. Your retriever, prompts and model stay exactly as they are. Contextwaise never calls your LLM and never sits in its billing path, so there is no model markup and no new point of failure for generation.
Rolling back is removing the call: pass your original context to the model again and the stack works as it did before.
A few inputs, two outputs.
The common case is a query and the context items you would have sent anyway. The response is the context to send and a report of what changed. Kept items come back in the order you sent them, never re-ranked, so a conversation reads the same on every turn.
| Field | Direction | Meaning |
|---|---|---|
query | Request | The current user question or task, up to 8,000 characters. Every context item is scored against it. |
candidates | Request | The context items: plain strings or { id, text, source }. Up to 500 items of 60,000 characters each. |
agent | Request, optional | Names the agent so budgets, the learned threshold and stats apply per agent. |
budget | Request, optional | Maximum input tokens of context, from 200 to 200,000. Defaults to the agent's budget, else 6,000. See token budgets. |
aggressiveness | Request, optional | conservative, balanced or aggressive for this request. Defaults to the project setting. |
model | Request, optional | The model you send to, so savings are priced at its input rate. |
conversation | Request, optional | Remembers every decision in a conversation, so an item sent again gets the same decision. See prompt caching. |
context | Response | The kept items, in the order you sent them, ready to pass to your model. |
report | Response | Tokens and cost before and after, per-item decisions with reasons, the threshold applied, latency and timings per step. |
The same call fits every context source.
Because every source becomes a list of context items, the integration looks the same whatever produced them. One rule holds for all of them: optimize new content once, when it first enters the conversation, and store what you sent. For chat history, a conversation id keeps every decision fixed.
Retrieve a wider top-k than you do today, optimize, then generate. Wider retrieval improves recall; the optimizer keeps the cost down.
Use the agent SDK, or optimize each new tool result before you append it to the messages.
Pass raw tool responses as context items: test runs, logs, JSON and HTML lose their noise before relevance is judged.
Pass older turns and long-term memory as context items with a conversation id. Irrelevant old turns are dropped once and stay dropped, so the history never changes.
Pass fetched pages or passages and send the model only the ones that answer the question.
One call. Your stack stays the same.
Send a query and the context items you would have sent. Receive the context to send, in your order, and a per-item decision log.
Plain HTTPS and JSON. For agents, the agent SDK shrinks tool results in TypeScript and Python.
Vector search results, web pages, MCP tool output and document sections all pass as context items.
The API never calls your LLM. It returns context you pass to OpenAI, Anthropic, Gemini or a local model.
Every item is scored in parallel by a small decision model. Our example requests took 323 to 510 ms end to end. Each response reports its own latency, step by step.
Tokens and cost before and after, the threshold applied, timings and the removal reasons, ready to log.
Questions about api
Does Contextwaise call my LLM?
No. The API returns optimized context. You pass it to OpenAI, Anthropic, Gemini, an open model or your own model exactly as you do today.
Which languages are supported?
Any language that speaks HTTPS and JSON. For agents, the agent SDK comes for TypeScript (contextwaise on npm) and Python (contextwaise on PyPI), both published at launch, with adapters for the Vercel AI SDK and the Anthropic and OpenAI SDKs.
What counts as a context item?
Anything you would put in the prompt as context: a retrieved chunk, a web page or passage, a tool response or a document section.
How much latency does it add?
Our example requests took 323 to 510 ms end to end. All items are scored in parallel, and the quality check and the request log run after the response. Every response reports latencyMs and per-step timings.
Will it break my prompt cache?
No, if you optimize new content once and store what you sent. See prompt caching.
What if the API is down?
Nothing is cut on a guess. If scoring is unavailable, the API returns 503 scoring_unavailable with Retry-After: 60: send your original context to the model. Input over the limits (more than 500 items, an item over 60,000 characters, a query over 8,000 characters) returns 422 invalid_request.
How do I remove it?
Delete the call and pass your original context to the model. Nothing else in your stack depends on Contextwaise.
Cut your LLM costs
without cutting quality.
One layer between retrieval and inference removes the context your model never needed, in milliseconds. Fewer input tokens, lower cost, a prompt cache that keeps hitting, and every removal logged with its reason.