docstring-grader
Published
Public

Judge prompt for docstring-generation evals: given code, a requested style (google/numpy/jsdoc/rustdoc), and the produced docstring, scores whether the docstring follows that style's section names and syntax — format adherence, not prose quality — as a 0-1 score.

docstring-grader

The LLM judge behind the docstring-generation eval. Given the code, the requested style, and the produced docstring, it scores style adherence — does the docstring use the right section markers and syntax for google / numpy / jsdoc / rustdoc — returning { reasoning, score } with score in [0, 1].

Judges format, not wording. The deterministic half of the eval (documented-parameter count) is pure CEL; this judge covers whether the output is actually in the requested style.

Referenced as @sufleur/[email protected]; the eval asserts judge.style.score > 0.7.

@sufleur/