Going to production

A fail-open helper#

Put the call in one helper and use it at every call site. If Contextwaise is slow, unreachable or returns an error, the helper returns your original candidates and the request continues as if Contextwaise weren't there.

type Candidate = { id?: string; text: string; source?: string };

/**
 * Returns the context worth sending. Pass only content new to the conversation.
 * On any failure it returns the original candidates: Contextwaise never blocks an answer.
 */
export async function optimizeContext(
  query: string,
  candidates: Candidate[],
  agent: string,
  model?: string,
): Promise<Candidate[]> {
  if (!process.env.CONTEXTWAISE_API_KEY || candidates.length === 0) return candidates;
  try {
    const res = await fetch(`${process.env.CONTEXTWAISE_URL ?? "https://app.contextwaise.com"}/api/v1/optimize`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.CONTEXTWAISE_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ query, candidates, agent, model }),
      signal: AbortSignal.timeout(5000),
    });
    if (!res.ok) return candidates;
    const { context } = await res.json();
    return context;
  } catch {
    return candidates;
  }
}

import os, requests

def optimize_context(query, candidates, agent, model=None):
    """Returns the context worth sending; the original candidates on any failure."""
    key = os.environ.get("CONTEXTWAISE_API_KEY")
    if not key or not candidates:
        return candidates
    try:
        res = requests.post(
            os.environ.get("CONTEXTWAISE_URL", "https://app.contextwaise.com") + "/api/v1/optimize",
            headers={"Authorization": f"Bearer {key}"},
            json={"query": query, "candidates": candidates, "agent": agent, "model": model},
            timeout=5,
        )
        return res.json()["context"] if res.ok else candidates
    except requests.RequestException:
        return candidates

Failing open is safe because only new content goes through Contextwaise: nothing already sent to your model changes.

Timeouts#

Most requests answer in well under a second; scoring is the bulk of it. A 5-second timeout is a good default: it covers slow moments without letting a user wait on context filtering. Shorter timeouts are fine for latency-critical paths, the helper simply falls back more often.

API keys#

• Create one key per environment (production, staging) in Settings → API keys, so you can revoke one without touching the others.
• Keep keys server-side, in environment variables or your secret manager. Never ship them to browsers or mobile apps.
• A revoked key fails with 401; with the helper above, requests then simply go out unoptimized.

Name your agents#

Pass a stable agent per call site, like support-assistant or code-review. It is how the dashboard groups requests, how per-agent budgets apply, and how the quality check learns each agent's threshold separately. Use the same name in every environment.

Also pass model, the model id the call site sends to, so savings are priced at that model's rate.

Budgets and aggressiveness#

• Start with balanced, the default, and let the quality check tune each agent.
• Use conservative for agents where missing context is costly.
• Set budgets per agent in the dashboard's Budgets page rather than in code, so you can adjust without a deploy.

What Contextwaise stores#

For each API request, the dashboard keeps what you need to understand it, not your full context:

• the query (first 500 characters), the agent, model and settings applied;
• for each candidate: its id, source, size, score, decision, and a preview of its first 280 characters;
• token, cost and timing figures.

The full text of your candidates is processed to score them, and is not stored. With the Agent SDK, tool results still over budget after cleanup are sent in sections to be scored, and are not stored either: the dashboard keeps only tool names and sizes.

Monitor and tune#

• Overview: tokens and cost saved, by channel and agent, and the time Contextwaise adds (median and p95).
• Requests: every request with each item's decision. Look here first if an answer seems to miss context.
• Budgets: how much of its budget each agent uses, and how often it hits it.

Count 503 scoring_unavailable responses in your logs: your requests still go through with their original context, but save nothing while they last.

Launch checklist#

• Every call site goes through one fail-open helper, with a timeout.
• Only new content is optimized; the conversation history is never passed back through it.
• The app works with CONTEXTWAISE_API_KEY unset (the helper returns the input).
• Each call site has a stable agent name and sends its model.
• Keys are per environment, server-side only.
• One real request is visible in Requests, with the decisions you expect.

Endpoint used above: https://app.contextwaise.com/api/v1/optimize