Guides

RAG and retrieved context

Retrieve generously, send precisely. Contextwaise lets you keep a wide top-k for recall while the model only reads what answers the question.

The idea#

Retrieval is tuned for recall: you fetch more chunks than you need so the right one is never missed. Without a filter, every one of them is billed and read by the model, and the irrelevant ones can distract it. Contextwaise sits right after retrieval and keeps the chunks that matter for this question.

Integrate in your retrieval step#

Call Contextwaise once per question, after retrieval and before you build the prompt. The optimizeContext helper from Going to production handles authentication, timeouts and failures:

rag.ts

const chunks = await vectorStore.search(question, { topK: 20 });

const context = await optimizeContext(
  question,
  chunks.map((c) => ({ id: c.id, text: c.text, source: "rag" })),
  "support-assistant",
  MODEL,
);

const answer = await llm.generate({
  model: MODEL,
  system: SYSTEM_PROMPT,
  prompt: render(question, context), // context: only the chunks worth sending
});

Tip

Keep your vector store's top-k as it is, or raise it. Contextwaise removes what isn't needed, so wider retrieval costs less than it used to.

Shaping candidates#

One chunk per candidate. Contextwaise decides item by item, so finer chunks give finer cuts.
Stable ids. Use your document or chunk id. It comes back in context and report.items, and shows in the dashboard.
A source label. rag, web, memory, tools… The dashboard's Sources page breaks savings down by it.
Raw text is fine. HTML pages and JSON payloads are cleaned automatically; you don't need to pre-process them.

Choosing a budget#

budget caps the tokens of context that can come back. When many chunks are relevant, the strongest are kept first until the budget is used. Set a default in Settings → Optimization defaults, a per-agent budget in the Budgets page, or pass budget on a request.

A good starting point is the context size your prompt was designed for. The Agents page shows, per agent, how much of the budget requests actually use and how often they hit it.

Several sources at once#

Send everything that competes for the same prompt in one request. Duplicates are caught across sources, and the budget is shared fairly by relevance:

TypeScript

const candidates = [
  ...docs.map((d) => ({ id: d.id, text: d.text, source: "rag" })),
  ...results.map((r) => ({ id: r.url, text: r.content, source: "web" })),
  ...memories.map((m) => ({ id: m.id, text: m.text, source: "memory" })),
];
const context = await optimizeContext(question, candidates, "research-assistant", MODEL);

Check the result#

Open Requests in the dashboard: each request lists every chunk with its score and the reason it was kept or removed. If you see useful chunks cut as low_relevance, use "aggressiveness": "conservative" for that agent; the quality check also loosens agents that cut too deep.