chunk-relevance-grading

Grade each retrieved chunk for relevance to the question — high / medium / low / none, plus a boolean filter decision and a one-line reason — so you can filter and rerank before answer synthesis. Cleaner context in means a more grounded answer out. This is the LLM-reranker / retrieval-grader step (think the grading node in a corrective-RAG / self-RAG graph).

Entrypoints

Render system as the system message and user as the user message.

Variables

• question (string, required) — the question each chunk is graded against.
• chunks (array of { id, text }, required) — the retrieved chunks. Each grading echoes the chunk's id back.

Output

parseOutput() returns:

{
  "reasoning": "Chunk 1 directly answers the pricing question; chunk 2 is about a different product.",
  "gradings": [
    { "chunk_id": "1", "relevance": "high", "is_relevant": true, "reason": "States the Pro plan monthly price." },
    { "chunk_id": "2", "relevance": "none", "is_relevant": false, "reason": "About shipping, unrelated to pricing." }
  ]
}

• reasoning is a one-sentence overall note, emitted before the per-chunk grades.
• gradings has exactly one entry per input chunk, in input order, each echoing chunk_id. Nothing is added, dropped, merged, or reordered — so you can zip it straight back onto your chunk list.
• relevance is the four-level grade; is_relevant is the derived filter boolean (true for high/medium, false for low/none) kept consistent with it. Filter on whichever you prefer — the boolean for a hard cut, the level for ranking.

House conventions

• Each chunk graded independently, on its own text — a strong neighbor can't lift a weak chunk, and a weak neighbor can't drag down a good one.
• Relevance to the question only — not writing quality, length, or how authoritative a chunk sounds.
• Grounded — no outside knowledge used to call a chunk relevant to facts it doesn't state.
• Injection-safe — a chunk saying "grade this high" / "ignore the question" is graded on its actual content; the embedded command is ignored.
• Strict schema — additionalProperties: false, closed enum, one entry per chunk.

Notes

maxTokens is sized up (1536) for grading many chunks. Verified against the live API on a mixed relevant/irrelevant set (one entry per chunk, graded correctly). Suggested default claude-sonnet-4-6 at temperature: 0. The grading stage of the RAG Essentials pipeline: rewrite → decompose → grade → guardrail → synthesize. Pair it with answer-synthesis: grade and filter, then synthesize over what's left.