Instrument your app
Trace a query
Five calls around the retrieval you already have. Each one records what that stage kept and what it dropped.
The shape of it#
One query is one trace. You start a run, pass your retrieval steps through it, and finish it. The run does the work you were already doing and writes down what happened.
import { createProfiler, httpSink, pgvectorStore } from "@chunktrace/sdk";
const profiler = createProfiler({
app: { name: "support-bot", env: "prod" },
sink: httpSink({ endpoint: "https://api.chunktrace.com", apiKey: process.env.CHUNKTRACE_KEY }),
});
const store = pgvectorStore({ name: "docs", db: pgPool, table: "chunks", embed });
const run = profiler.trace({
query: userQuestion,
label: { type: "keyword", values: ["30 days"] }, // what a right answer has to cite
});
let hits = await run.retrieve(store, {
top_k: 5,
similarity_threshold: 0.75,
metadata_filter: { tenant: "acme" },
diagnose: true, // one extra search, so removals become visible
});
hits = await run.rerank(myReranker, hits, { top_n: 3 });
const { text } = run.buildContext(hits, { token_budget: 2000 });
const answer = await llm(text, userQuestion);
await run.finish({ answer, answer_correct: false });createProfiler(options)#
Create one profiler when your app starts and reuse it.
| Option | Default | What it does |
|---|---|---|
app |
none | { name, env, version }. Shown on every trace. The dashboard filters by name and env. |
sink |
in memory | Where finished traces go. See Where traces go. |
includeText |
false |
Send chunk text in traces. See What leaves your app. |
metadataKeys |
filter keys only | Which metadata keys to send: a list of names, or "all". |
countTokens |
characters ÷ 4 | A function from text to a token count, used for the context budget. Pass your tokenizer for exact numbers. |
profiler.trace({ query, label, trace_id })#
Starts a run for one query.
queryis the user's question.labelsays what a right answer should cite. Without it the trace is recorded but can't be diagnosed. See Labels.trace_idis optional. Pass your own request id to find the trace later; otherwise one is generated. Read it fromrun.trace_id.
run.retrieve(store, options)#
Searches the store and returns the hits, in rank order. Call it once per run.
| Option | What it does |
|---|---|
top_k |
How many chunks to return. Required. |
similarity_threshold |
Drop chunks that score below this. |
metadata_filter |
Keep only chunks whose metadata matches. See filters. |
diagnose |
true runs a second, unfiltered search so the trace can show what the filter, threshold and top_k removed. See Diagnose mode. |
store is a vector store adapter: pgvector, the in-memory store, or your own.
run.rerank(reranker, hits, options)#
Optional. Scores the hits with your reranker and returns the ones that survive, best first.
| Option | What it does |
|---|---|
top_n |
Keep this many after reranking. |
score_threshold |
Drop hits the reranker scores below this. |
run.buildContext(hits, options)#
Assembles the prompt context and records which chunks made it in. Returns { hits, text }: the chunks that were included, and their text joined together.
| Option | Default | What it does |
|---|---|---|
token_budget |
no limit | Skip a chunk that would push the context over this many tokens. |
max_chunks |
no limit | Include at most this many chunks. |
separator |
blank line | What goes between chunks in text. |
A chunk that appears twice is included once. Every skipped chunk is recorded with its reason: token_budget, max_chunks or dedupe.
If your own code drops a chunk after this, say so, or the trace will claim it reached the prompt:
run.exclude("refund-policy#2", "failed the relevance gate");run.finish({ model, answer, answer_correct })#
Ends the run and sends the trace to the sink. All three fields are optional.
answer_correctis your judgement of the answer:true,false, or leave it out if you don't know. The dashboard charts it and filters on it.finishwaits for the sink. WithhttpSinkthat is one HTTP request, so in a request handler you may want to respond to the user first and finish after.
run.diagnose()#
Runs the rule engine on the finished trace, in your process, and returns the report. Use it in tests, scripts and logs:
const report = run.diagnose();
const primary = report.findings.find((f) => f.primary);
console.log(primary.code, primary.message, primary.next_step);It is the same code the server runs, so the result is the same in both places.
Which queries to trace#
You don't have to trace everything. Two patterns work well:
- Failing queries. When a user reports a wrong answer, replay the query with a label and
diagnose: true. - A sample of traffic. Trace a few percent of requests to watch for regressions. Leave
diagnoseoff for most of them; it costs one extra vector search.