Repository navigation
feat: add opt-in Jev audits and citation repair - #31
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
[1]with[2], which received 0.97 support.RAG_JEV_MODE=off.observereports judgments without changing the answer;repairchanges citations only after a fresh check passes the experimental 0.90 support threshold.Changes
jev-1.13.0, strict response validation, isolated cited-excerpt sets, sanitized errors, and an optional API key kept out of settings representations./api/queryand the UI, including privacy disclosures and repair provenance. Constrain mobile grid tracks so wide PDF tables scroll within passages.Verification
uv run --frozen nox— lint, strict type checking, and 93 tests passed; coverage 85.59%. Provider tests use HTTP mocks and require no API keys or downloaded models.The live public Jev replay agreed with 7/8 expected labels, with zero unsupported acceptances and zero supported withholds. The disagreement remains documented rather than relabeled. The public-document and seeded-negative checks are small development experiments, not threshold calibration or evidence of general correctness. Jev can support individual claims while an answer remains incomplete; this change does not rewrite or remove unsupported claims.
Details: implementation and measured tests.
Screenshots / UX
Hand-authored synthetic UI previews, not live model output. Full live desktop/mobile captures remain local with the ignored test outputs.
Before citation repair (observer):
After a verified replacement:
Checklist
TYPESAFE_API_KEYand an app restart.