Contextwaise integration guide for coding agents (Claude Code, Codex, Cursor)

Integrate Contextwaise into an app that calls LLMs. Contextwaise takes the new context you would send to a model (RAG chunks, tool output, web results, retrieved memory), keeps the part relevant to the query, and returns it with a report of tokens and cost saved. Use when asked to "set up Contextwaise", "reduce LLM input tokens", or "optimize context before the model call".

# Contextwaise integration

You are wiring Contextwaise into the user's codebase. Contextwaise sits between
context building and the LLM call:

    retrieval / tools / web  ->  Contextwaise /optimize  ->  append to the conversation  ->  your LLM

It never calls the user's model and never changes prompts, models or retrieval.
It only decides which context items are worth sending.

## The one rule: optimize new content once, never the history

LLM APIs are stateless: every turn resends the whole conversation. Providers
(Anthropic, OpenAI, Google) bill the repeated start of a request at a fraction of
the price (prompt caching), but only while it is byte-identical to last time.
And on newer Claude models (Fable 5.2 and 5.1, Opus 5.5, Sonnet 5.5), editing an earlier
turn invalidates the model's saved reasoning; newer accounts get a 400 error.

So Contextwaise is a filter at the door, not an editor of the past:

- Optimize each piece of context **once, when it first enters the conversation**
  (the chunks retrieved for this turn, the tool result just returned).
- Store the **optimized** version in the conversation history, and resend that
  history unchanged on later turns. Never pass earlier turns back through Contextwaise.
- Never pass the conversation's own messages (user questions, assistant answers)
  as candidates.
- Tools the provider runs itself (Anthropic `web_search` / `web_fetch`, OpenAI
  web search, Gemini grounding) never pass through your code, so Contextwaise can't
  trim them. Cap them with the provider's own settings instead (`max_uses`,
  `max_content_tokens`, `allowed_domains`).

If the code rebuilds context from the original sources on every turn and can't be
changed, pass a stable `conversation` id (see API): Contextwaise then remembers
every decision in that conversation, and an item it has seen before gets the same
decision again, so the history the model sees never changes.

## Steps

1. **Account and API key.** Ask the user for a key, and give them the links:
   - No account yet: sign up at https://contextwaise.com/app/login (Google or email). The Free plan
     includes 1M context tokens, once, with no card.
   - Then create a key at https://contextwaise.com/app/settings#keys. It starts with `sw_live_` and is
     shown once. The new-key panel shows both env lines below, ready to copy.
   Don't wait for the key to write the code: without it the helper returns the
   original context and the SDK cleans tool results locally. Never hardcode the key
   and never commit it.
2. **Environment.** Add to the project's env file and its example/template:
       CONTEXTWAISE_URL=https://contextwaise.com/app
       CONTEXTWAISE_API_KEY=sw_live_...
   Set both or neither: the SDK throws at startup when the key is set without
   CONTEXTWAISE_URL.
3. **Find the call sites.** Search for every place that builds context for an
   LLM call: vector store / RAG results, tool or MCP output, web search
   results, files pasted into prompts, long-term memory lookups. In agent loops,
   the spot is where a tool result is appended to the messages. List them to the
   user before editing. Leave the conversation history itself alone (see the rule above).
4. **Add one helper** (see "Helper" below) in the project's existing lib/utils
   location, in the project's language. One helper, reused by every call site.
5. **Wrap each call site.** Pass the user's question as `query` and each context
   new piece as a candidate with a `source` label (`rag`, `tools`, `web`,
   `memory`, ...). Give each call site a stable `agent` name (`support-triage`,
   `code-review`, ...) so the dashboard and per-agent budgets can tell them apart.
   Pass the `model` id the call site sends to (the same string it passes to its
   LLM SDK) so savings are priced for that model.
   Send the returned `context` to the model instead of the original items, and
   store that same `context` in the conversation history. Kept items come back
   in the order you sent them.
6. **Fail open.** If Contextwaise errors or times out, send the original items.
   Contextwaise must never be the reason an LLM call fails. This is safe because
   only new content goes through Contextwaise: nothing already sent changes.
7. **Verify.** Run one real request, print before/after tokens from
   `report.tokens`, and tell the user to open https://contextwaise.com/app/requests to see every
   kept and removed item with the reason.

Keep the diff minimal. Do not refactor unrelated code, change prompts, swap
models, or add dependencies (plain HTTP is enough).

## API

POST https://contextwaise.com/app/api/v1/optimize
Headers: `Authorization: Bearer $CONTEXTWAISE_API_KEY`, `Content-Type: application/json`

Request body:

    {
      "query": "Why was my invoice charged twice?",       // required, the user question, max 8000 chars
      "candidates": [                                      // required, max 500 items, max 60000 chars each
        { "id": "doc-12", "text": "...", "source": "rag" }, // id and source optional
        "a plain string also works"
      ],
      "agent": "support-triage",                           // optional, groups requests and picks the agent budget
      "model": "claude-sonnet-5",                          // optional, the model you send to; prices the savings (see https://contextwaise.com/app/settings#models)
      "budget": 6000,                                      // optional token budget (200 to 200000); default from the dashboard
      "aggressiveness": "balanced",                        // optional: conservative | balanced | aggressive
      "conversation": "chat_8f2k",                         // optional, only if you resend old items (see "The one rule"); letters, digits, _ - . : (max 128)
      "models": ["claude-haiku-4-5", "claude-sonnet-5-5", "claude-opus-5-5"],  // optional, 2 to 10 models you approve: turns on model routing (shadow mode)
      "system": "You are the support assistant for...",   // optional, your model's instructions: helps routing judge the task
      "outputTokens": 500,                                 // optional, your typical answer length in tokens (default 500): sharpens switch advice
      "promptTokens": 48000                                // optional, your full prompt size incl. history (e.g. last usage.input_tokens): sharpens switch advice
    }

Response 200:

    {
      "id": "req_...",
      "context": [{ "id": "doc-12", "text": "...", "source": "rag" }],  // send this to the model and store it in the history; same order you sent
      "report": {
        "tokens": { "before": 9120, "after": 2310, "saved": 6810, "savedPct": 74.7 },
        "cost": { "before": 0.027, "after": 0.007, "scorer": 0.0004, "avoided": 0.0196 },
        "model": "claude-sonnet-5", "inputPricePerMillion": 3,     // model is null when priced with the project's custom price
        "items": [{ "id": "doc-12", "source": "rag", "tokens": 120, "score": 2.8, "kept": true, "reason": "relevant" }],
        "guard": null,                                     // the quality check runs after the response; see it in the dashboard
        "budget": 6000, "aggressiveness": "balanced", "threshold": 1.4,
        "conversation": null,                              // with a conversation id: { "id", "reused", "protected" }
        "route": null,                                     // with models: see "Model routing" below
        "latencyMs": 180,
        "timings": { "auth": 3, "project": 2, "score": 170, "select": 1 }   // ms per step
      }
    }

Item reasons: `relevant`, `low_relevance`, `duplicate`, `over_budget`.

With a `conversation` id: items seen earlier in the conversation are marked
`"reused": true` and keep their earlier decision (they are not scored again, and
don't count against `budget`, which then applies to the new items). New items that
repeat something already sent are dropped as `duplicate`. `report.conversation.protected`
is false if decisions couldn't be stored; that request isn't guaranteed to match earlier ones.
Relevance threshold: `threshold` is the cut-off applied on the 0 to 3 scale. It
starts at the aggressiveness level and adapts per agent: after each response a
quality check tells whether the context kept was enough, and the next requests
cut less or more accordingly.

Contextwaise never cuts on a guess: if the scoring model can't be reached after
retries, the request fails with 503 scoring_unavailable. Send the original context.

## Model routing (shadow mode)

List the models you approve in `models` (any mix of providers; each must be in the
catalog or priced in Settings > Models), and send the model you call now in
`model`. Contextwaise judges how hard the request is (`simple`, `standard` or
`advanced`) from `query` and `system`, then tells you whether to switch:

    "route": {
      "switch": false,                       // follow this, not the tier alone
      "model": "claude-opus-5-5",            // the model to use: the recommended one if switch, else yours
      "recommended": "claude-haiku-4-5", "currentModel": "claude-opus-5-5",
      "tier": "simple", "confidence": 0.91,
      "reason": "long_context",
      "message": "Keep Claude Opus 5.5: your prompt is 202K tokens. Claude Haiku 4.5 would read it once without cache (~$0.21), paid back only after ~6 turns.",
      "estimate": { "stayPerTurn": 0.06, "switchPerTurn": 0.025, "switchCostOnce": 0.21, "paybackTurns": 6 }
    }

Reasons: `cheaper_fits` (switch: a cheaper approved model does this as well),
`needs_bigger` (switch up: quality), `long_context` (keep: switching would lose
the prompt cache on a long prompt for too long), `already_best`, `unsure` (keep:
couldn't judge), `no_current_model` (send `model`).

It is advice only: keep calling your usual model and watch the "Model routing" card
in the dashboard for what following it would save.

- Switching costs once: the new model reads the whole prompt without cache. A
  switch is advised only when cheaper turns repay that within 3 turns of a
  conversation, or within the request itself without a `conversation` id.
- With a `conversation` id the tier is decided once and kept for the whole
  conversation. Never switch models mid-conversation on your own: it loses the
  prompt cache and, on newer Claude models, the reasoning so far.
- Estimates use each model's input, output and cache prices, `outputTokens`
  (default 500) and `promptTokens` (default: what Contextwaise sees).

An unknown `model` is not an error: the request is priced with the project's
default model, and `report.model` shows what was used. Provider prefixes and
date suffixes are accepted (`anthropic/claude-sonnet-5`, `gpt-5-2025-08-07`).

Errors (`{ "error": { "code", "message" } }`):
- 400 invalid_json / invalid_body: the body must be a JSON object
- 401 invalid_api_key: bad or revoked key; ask the user to check CONTEXTWAISE_API_KEY
- 422 invalid_request: fix the body, the message says which field
- 429 rate_limited: over the plan's requests per minute (Free 60, Pro 1,000, Scale 3,000); send the original context, retry after Retry-After seconds
- 429 quota_exceeded: the Free plan's 1M tokens are used up; send the original context and tell the user to upgrade at https://contextwaise.com/app/billing
- 503 scoring_unavailable: Contextwaise is having trouble; send the original context and retry after Retry-After seconds
- 500 internal_error: send the original context and retry later

## Agent tool results: the contextwaise SDK

If the project runs an agent loop (tools whose results go back to the model, e.g.
Vercel AI SDK `generateText`/`streamText` with `tools`, an `ai.Agent` from the
Vercel AI SDK for Python, or any loop over tool calls), also add the SDK. It exists for TypeScript and Python. It
shrinks every tool result the moment the tool returns, on the user's machine: noise
removed (passing tests, repeated log lines, empty JSON fields, HTML markup, long file
listings), then the parts relevant to what the agent is doing, with any omitted part
one `expand` call away. A result is only cut when that saves at least 1,000 tokens, 30% of
it, and 10% of the conversation's size (a wrong cut costs an `expand`, one more model call
resending the conversation); smaller ones pass whole, cleaned. So in an agent with a big system
prompt, results under a few thousand tokens are never cut: that is expected, not a bug. Each
result is decided once, so prompt caching keeps working.

    npm install contextwaise

    import { createContextwaise } from "contextwaise";
    import { withContextwaise } from "contextwaise/ai-sdk";

    // Reads CONTEXTWAISE_API_KEY (the same sw_live_ key) and CONTEXTWAISE_URL (https://contextwaise.com/app); a key without the URL throws.
    const cw = createContextwaise({ agent: "support-triage", budgets: { read_file: 1500 } });
    const { model, tools } = withContextwaise(cw, { model, tools });   // pass these to generateText/streamText

On the Anthropic or OpenAI SDK (the project runs its own loop), add the expand tool and
replace the code that runs tools and builds their results with one call per response:

    import { expandTool, toolResults } from "contextwaise/anthropic";      // tools: [...tools, expandTool]
    messages.push({ role: "user", content: await toolResults(cw, response, runTool, { request: userRequest }) });

    import { expandTool, expandToolResponses, toolMessages, functionOutputs } from "contextwaise/openai";
    messages.push(...await toolMessages(cw, message, runTool, { request: userRequest, promptTokens: completion.usage?.prompt_tokens }));    // Chat Completions
    input.push(...await functionOutputs(cw, response, runTool, { request: userRequest }));   // Responses API

`runTool(name, args)` runs the project's tool and returns its raw output; a thrown error
reaches the model as an error result. `expand` calls are answered by Contextwaise.

- `budgets`: per-tool token budgets. Give file-reading tools a larger one: files up to
  that size reach the model whole (the agent may edit them).
- Works with `ai` 6 and 7. Never upgrade or change the project's existing dependencies
  for Contextwaise.
- Which tools to wrap: tools that return data (HTTP, MCP, integrations, database, RAG
  search, sandbox/shell output). Tool schemas and lists of actions (an MCP server
  describing its tools, tool definitions, JSON Schema) are detected and always passed
  whole. Pass `full: (tool, args) => …` to `withContextwaise` for any other output the
  model must see exactly, e.g. a file it will edit with an exact-match tool. If the
  project already marks outputs that must never be truncated, reuse that marker for
  `full`. Leave small internal control tools (delegation, skills, memory writes) unwrapped.
- Platforms hosting many agents (users build their own): use one client and name each
  run's agent, `withContextwaise(cw, { model, tools, agent: agent.name })`, so each agent
  shows under its own name, not the platform's. The Anthropic and OpenAI helpers take
  `agent` in their options too.
- Errors never reach the agent (relevance falls back to "pass whole"). To see them, log
  the client's `api_error` events once: `createContextwaise({ onEvent: (e) => e.type ===
  "api_error" && log(e.message) })`. One says when the free tokens are used up.
- With the key, relevance is judged by the Contextwaise API and results appear in
  https://contextwaise.com/app/requests?view=tools (sizes and tool names only), each with why it got its
  view (cut, small enough, saving too small, schema, …). Without it, everything runs locally.
- Verify with one real tool call that returns over ~5K tokens, then check it on the
  Requests page. Don't create test agents or fake tools that report to the user's dashboard.
- In scripts or serverless functions, `await cw.flush()` before exiting.

### Keep the prompt cache (agents)

The tool list and every earlier tool result sit in the cached start of the prompt. The
SDK keeps its own part stable; these mistakes in the integration break it on every turn:

- **Same tools, same order, every turn.** Wrap the tools once per run and pass the
  result as is: `expand` is appended last and existing tools keep their position. Don't
  wrap on some turns and not others, don't add `expand` only after a cut, and don't
  drop it from steps that limit the active tools (keep it whenever a wrapped tool is
  active). A changed tool list re-bills the whole prompt at full price.
- **Store the view, not the raw output.** The history the model sees must hold what
  Contextwaise returned (`response.messages` on the Vercel AI SDK, `view.text` in your
  own loop). Keep the raw output for the app (UI, logs, artifacts), never in the
  model's history. Apps that rebuild history from raw results each turn are fine on
  the Vercel AI SDK: `toModelOutput` runs again and the ledger replays the same view.
- **One ledger for one conversation.** Views are replayed by tool call id from the
  ledger (in memory by default). If a conversation's turns can land on different
  processes or survive a restart, plug in a shared `ledger` (Redis, database) so a
  past result is never re-decided.
- **Don't shrink past results again, or change their view.** Only results the tool
  just returned go through Contextwaise. Changing `budgets`, `full` or `maxTokens`
  only affects new calls, as long as the ledger still has the old ones.
- **Keep Contextwaise output out of the prompt start.** Never put reports, savings or
  reasons into the system prompt or the tool descriptions.
- **Don't switch models mid-conversation** to save tokens: the new model reads the
  whole prompt without cache (see Model routing).

Python (3.10+), in any agent loop:

On the Vercel AI SDK for Python (the `ai` package, Python 3.12+), swap `ai.Agent` for
`ContextwaiseAgent`: same tools, same `agent.run(model, messages)`, plus the expand tool.

    pip install "contextwaise[ai]"

    from contextwaise import AsyncContextwaise
    from contextwaise.ai_sdk import ContextwaiseAgent

    cw = AsyncContextwaise(agent="support-triage", budgets={"read_file": 1500})
    agent = ContextwaiseAgent(cw, tools=tools)   # tools: the project's @ai.tool functions
    async with agent.run(model, messages) as stream: ...

Any other loop:

    pip install contextwaise

    from contextwaise import EXPAND_TOOL, Contextwaise

    # Reads CONTEXTWAISE_API_KEY and CONTEXTWAISE_URL (https://contextwaise.com/app).
    with Contextwaise(agent="support-triage", budgets={"read_file": 1500}) as cw:
        # For each tool call, after the tool returns:
        cw.note_intent(call.id, user_request + " " + text_before_the_call)
        view = cw.shrink(call_id=call.id, tool=call.name, args=call.args, output=output)
        # Many agents on one client: cw.shrink(..., agent=agent_name); ContextwaiseAgent(cw, tools=..., agent=agent_name)
        # send view.text to the model as the tool result (keep `output` for the app)

- Add EXPAND_TOOL to the model's tools (name, description, JSON Schema in
  `parameters`; `input_schema` for Anthropic), and answer its calls with
  `cw.expand(**args, context_tokens=<input tokens of the model call that asked>)`
  (from the provider's usage), so the dashboard counts what the expand cost.
- Chat Completions: pass `prompt_tokens=completion.usage.prompt_tokens` to
  `tool_messages` (a message doesn't carry usage).
- On the Anthropic or OpenAI SDK, the adapters do the loop body: `tool_results`
  (`contextwaise.anthropic`), `tool_messages` / `function_outputs`
  (`contextwaise.openai`), each with its own EXPAND_TOOL and an async `a…` variant.
- asyncio: `AsyncContextwaise`, same methods, awaited (`async with`).
- Leave the `with` block (or call `cw.close()`) before a script exits, so reports are sent.

## Helper (TypeScript, adapt to the project's language)

    type Candidate = { id?: string; text: string; source?: string };

    /**
     * Returns the new context worth sending. Pass only content new to the conversation.
     * Falls back to the original items on any failure.
     */
    export async function optimizeContext(query: string, candidates: Candidate[], agent: string, model?: string): Promise<Candidate[]> {
      if (!process.env.CONTEXTWAISE_API_KEY || candidates.length === 0) return candidates;
      try {
        const res = await fetch(`${process.env.CONTEXTWAISE_URL ?? "https://contextwaise.com/app"}/api/v1/optimize`, {
          method: "POST",
          headers: { Authorization: `Bearer ${process.env.CONTEXTWAISE_API_KEY}`, "Content-Type": "application/json" },
          body: JSON.stringify({ query, candidates, agent, model }),
          signal: AbortSignal.timeout(5000),
        });
        if (!res.ok) return candidates;
        const { context } = await res.json();
        return context;
      } catch {
        return candidates;
      }
    }

Call site (RAG, once per turn):

    const chunks = await vectorStore.search(question);
    const context = await optimizeContext(question, chunks.map((c) => ({ id: c.id, text: c.text, source: "rag" })), "support-triage", MODEL);
    messages.push({ role: "user", content: render(question, context) });   // stored as sent; never re-optimized later

Call site (agent loop, when a tool returns):

    const output = await runTool(call);
    const [kept] = await optimizeContext(task, [{ id: call.id, text: output, source: "tools" }], "code-review", MODEL);
    messages.push(toolResult(call.id, kept?.text ?? "(no relevant output)"));   // appended once, resent unchanged

## Done when

- Every LLM call site with retrieved or tool context goes through the helper,
  once, for new content only. The conversation history is never passed through it.
- The app still works with CONTEXTWAISE_API_KEY unset (helper returns the input).
- One real request shows up at https://contextwaise.com/app/requests.

## Tell the user at the end

- What changed: the call sites wrapped, the helper or SDK added, the env lines.
- The before/after tokens of the real request.
- Where to see results: https://contextwaise.com/app (savings overview), https://contextwaise.com/app/requests (every
  request, each item kept or removed and why; agent tool results under the
  "Contextwaise SDK" tab), https://contextwaise.com/app/settings (defaults, budgets, model prices).
- The Free plan is 1M context tokens, once. When it runs out the app keeps working
  with its original context; upgrade at https://contextwaise.com/app/billing.
- If the key was ever pasted in chat or a shared file, create a new one and revoke the old.