Free
$0
For experimenting
- 250 diagnoses a month
- 7 days of history
- No card, no time limit
New
Docs, and a five-minute quickstart
ChunkTrace replays one query trace through index, filter, threshold, top_k, rerank and context, and names the first stage that dropped the chunk the answer needed. Deterministic rules, no model in the analysis path, every finding proven from the trace.
Findings
A finding is not a guess. It names the chunk, the stage, and the exact field values from the trace that make it true: the rank, the score, the top_k, the tokens left in the budget. Then it says what to change next.
NO_LABEL and SHADOW_MISSING say what the trace can't prove instead of guessing.UNEXPLAINED_EXCLUSION catches the chunk that should have been returned and wasn't: usually ANN nondeterminism or a filter the SDK didn't capture.
SDK
createProfiler() wraps your vector store and reranker. retrieve,
rerank, buildContext and finish record the trace;
diagnose() returns the report, or a sink ships it to the API.
diagnose: true on a failing query, or a sample of traffic, and the SDK runs one extra unfiltered search so removals become visible. One vector search, no inference.includeText: true. Label matching runs client-side and ships only matches_label.metadataKeys says otherwise.import { createProfiler, httpSink, pgvectorStore } from "@chunktrace/sdk";
const profiler = createProfiler({
app: { name: "support-bot" },
sink: httpSink({ endpoint: "https://api.chunktrace.com", apiKey: process.env.CHUNKTRACE_KEY! }),
});
const run = profiler.trace({
query: userQuestion,
label: { type: "keyword", values: ["30 days"] }, // what the answer should cite
});
let hits = await run.retrieve(store, {
top_k: 5,
similarity_threshold: 0.75,
metadata_filter: { tenant: "acme" },
diagnose: true, // one extra unfiltered query so removals become visible
});
hits = await run.rerank(myReranker, hits, { top_n: 3 });
const { text } = run.buildContext(hits, { token_budget: 2000 });
const answer = await llm(text, userQuestion);
await run.finish({ answer, answer_correct: false });
console.log(run.diagnose().findings.find(f => f.primary)?.message);
// "Chunk refund-policy#2 ranked 6 among eligible results; top_k=5 cut it."
Rule engine
diagnose() is a pure function in @chunktrace/core. The CLI, the SDK and
the API worker call the same code, so a verdict is identical on a laptop and in production.
ENGINE_VERSION is stored with every diagnosis, so you can tell which rule set produced it.
Ten finding codes
$ npx chunktrace diagnose traces/demo-04-topk.json
Trace demo-04-topk
Query: "How many days do I have to request a refund?"
Label: {"type":"keyword","values":["30 days"]}
Timeline
retrieve index=support-docs top_k=1 filter={"tenant":"acme"} -> 1 candidates (shadow: 13 unfiltered, k=50)
context included 1, excluded 0, used 17
generate answer_correct=false
Shadow diff: filter removed 1, threshold removed 0, top_k cut 11
Findings
PRIMARY [TOPK_CUT] Chunk refund-policy#2 ranked 2 among eligible results; top_k=1 cut it.
next: Raise top_k, or add a reranker so a larger candidate pool can be narrowed by relevance.
HTTP API
Post traces one at a time or in batches of a hundred. Each is checked against the trace schema, stored,
and diagnosed moments later. The SDK's httpSink does this for you; anything that can send
JSON can do it too.
200 duplicate, never a second diagnosis.$ curl -X POST https://api.chunktrace.com/v0/traces \
-H "authorization: Bearer $CHUNKTRACE_KEY" \
-H "content-type: application/json" \
--data @traces/demo-04-topk.json
202 {"trace_id":"demo-04-topk","status":"accepted"}
$ curl https://api.chunktrace.com/v0/traces/demo-04-topk \
-H "authorization: Bearer $CHUNKTRACE_KEY"
{"trace_id":"demo-04-topk","status":"diagnosed",
"primary_code":"TOPK_CUT","engine_version":1,
"trace":{…},"diagnosis":{"findings":[…],"summary":{…}}}
How a diagnosis works
The rule engine never reads anything but the trace. Validation is strict on shape and lenient on content: a trace that validates is always diagnosable.
NO_LABEL and stop. Without expected evidence the trace can be described, not diagnosed.SHADOW_MISSING; filter, threshold and top_k effects can't be observed.evidence.Also in the box
pgvectorStore against any table with an embedding column; memoryStore with a hashed bag-of-words embedder for tests and the sample app.fileSink, httpSink and multiSink. The file sink needs no server at all; the CLI reads what it writes.$in, $ne, numeric ranges, AND.GET /v0/traces pages newest first, with filters on code, app, env, answer, time range and query text. GET /v0/stats counts by status, code and answer.chunktrace diagnose trace.json prints the same report in a terminal. No account and no network.
Not yet: one retrieval per trace (multi-query needs a retrievals[] array), and no re-diagnose job when ENGINE_VERSION moves. Both are on the list.
Pricing
A diagnosis is one trace you send. Every plan has the whole product: the rule engine, the dashboard, the SDK, the API and the CLI. Plans differ in how many traces a month, and how far back you can read them.
$0
For experimenting
$29/month
Indie developers
Founding price: $19 a month for your first 12 months, then $29. For the first 20 accounts.
Upgrade in Settings$99/month
Small AI teams
Custom
Larger organizations
How the limits work: the count starts again on the first of each month (UTC). At the limit, new traces are refused with a clear error until then; nothing you have sent is deleted, and a trace sent twice is counted once. Prices are in US dollars.
Get started
Create an account, make a key, and run one deliberately broken query. The SDK ships an in-memory store, so there is no vector database to set up.
# 1. Create an account, then Settings → Create key (full access)
$ export CHUNKTRACE_KEY=ct_…
# 2. Install the SDK from npm
$ npm install @chunktrace/sdk
# 3. Run the quickstart script: one query, with top_k set too low
$ node first-trace.mjs
Chunk refund-policy#2 ranked 2 among eligible results; top_k=1 cut it.