error-explanation

Published

Public

Turn an error message, stack trace, and optional code context into a plain-language explanation, the single most likely cause, one to three concrete fixes (each with a code_change), and an uncalibrated confidence. Names the concrete mechanism, not just the exception class; grounds every claim in the provided inputs. Eval-gated on a cause-keyword check plus a usefulness judge.

error-explanation

Turn an error message, stack trace, and optional code context into a grounded diagnosis:

{
  "explanation": "The code indexed a list past its end. `items[i]` runs with i == len(items) on the last iteration, so it reads one element beyond the list.",
  "likely_cause": "An off-by-one loop bound: the range goes to len(items) inclusive instead of exclusive.",
  "fixes": [
    { "description": "Stop the loop one short of the length.", "code_change": "for i in range(len(items)):  # was range(len(items) + 1)" }
  ],
  "confidence": 0.9
}

Why this shape

explanation names the mechanism, not the exception class — "read one element past the end", not "IndexError means index error".
likely_cause is a single sentence so an eval can check that the real cause keyword is present: output.likely_cause.contains(case.expected_cause_keyword).
fixes[] are concrete — each carries a code_change, so the output is actionable, not just diagnostic.
confidence is uncalibrated and drops when the stack or code context is missing.

Eval

Gated by @sufleur/error-explanation-cases. Each case pairs an error/stack/context with an expected_cause_keyword. Assertions:

cause keyword present — the correct root-cause term appears in the explanation or likely cause (deterministic);
a fix is offered — output.fixes.size() >= 1;
usefulness — an LLM judge (error-explanation-grader) scores whether the explanation is correct and the fixes are actionable (judge.usefulness.score > 0.7).

Passing threshold 0.85. Recommended + eval model: claude-sonnet-4-6.

@sufleur/