release: promote dev to main

.env.example

@@ -13,7 +13,19 @@ RAG_EVAL_MODEL=qwen3:8b
 RAG_OPENAI_BASE_URL=http://localhost:1234/v1
 RAG_OPENAI_API_KEY=
 
-# Retrieval and storage.
+# Optional hosted citation audit. Off keeps all inference local.
+# Observer mode sends generated claims and their cited excerpts to TypeSafe.
+# Repair mode also checks other retrieved excerpts and may replace weak citations.
+# Modes: off, observe, repair. Claim text is preserved in repair mode.
+RAG_JEV_MODE=off
+TYPESAFE_API_KEY=
+RAG_JEV_MODEL=jev-1.13.0
+# Total budget per question part, not per individual claim.
+RAG_JEV_TIMEOUT=5
+# Experimental policy threshold; calibrate against reviewed claims.
+RAG_JEV_THRESHOLD=0.90
+
+# Retrieval and storage. Original uploads are retained below RAG_DATA_DIR/sources.
 RAG_DATA_DIR=data/qdrant
 RAG_COLLECTION=documents
 RAG_CHUNK_SIZE=900

CONTRIBUTING.md

@@ -39,6 +39,11 @@ Use `feat` for user-visible capability, `fix` for a defect, and `!` or a `BREAKI
 - `uv run nox -s tests` runs pytest with an 80% coverage floor.
 - Mark fast isolated tests with `unit`; use `integration` when local infrastructure is exercised.
 - Provider tests must use fakes or HTTP mocks. CI must never require API keys or downloaded models.
+- For browser changes, run the app and verify upload, document selection, question submission,
+  citation navigation, and PDF page viewing at desktop and mobile widths. Check the browser console
+  and confirm keyboard focus remains visible. Use public or synthetic documents for screenshots.
+- Test Jev UI states with deterministic fixtures or mocks. A live TypeSafe check is an optional
+  manual verification and must use public or synthetic evidence; CI must keep Jev disabled.
 - For retrieval, chunking, parsing, prompt, or model changes, run the matching private golden dataset with `uv run local-rag-eval evaluation/private/cases.jsonl` and report metric changes in the pull request. Ragas (`uv run --extra evaluation local-rag-eval-ragas ...`) is an optional diagnostic, not a release gate.
 
 ## Design guardrails

README.md

@@ -5,10 +5,12 @@
 [![Ollama](https://img.shields.io/badge/Ollama-local-111111)](https://ollama.com/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-65dbcc.svg)](LICENSE)
 
-A private-by-default retrieval-augmented generation application built with Python, Ollama, and embedded Qdrant. Upload local documents, retrieve semantically and lexically relevant passages, and generate grounded answers with visible citations—without a paid API or cloud database.
+A private-by-default retrieval-augmented generation application built with Python, Ollama, and embedded Qdrant. Upload local documents, retrieve semantically and lexically relevant passages, and generate grounded answers with visible citations—without a paid API or cloud database. An optional TypeSafe Jev integration can audit claim support and repair weak citations.
 
 **[Explore the interactive architecture](https://sanjaysrinivas.github.io/context-foundry/)** · [Read the source HTML](docs/index.html) · [OpenAPI after startup](http://127.0.0.1:8000/docs)
 
+![Context Foundry workspace answering from the NIST AI Risk Management Framework with a Jev support check and cited PDF source](docs/images/context-foundry-workspace.png)
+
 ## Why this project
 
 This repository exposes the mechanics that RAG frameworks often hide:
@@ -28,10 +30,13 @@ flowchart LR
     A --> I[Load + chunk]
     I --> EP[Embedding provider]
     EP --> Q[(Local Qdrant)]
+    A --> S[(Retained source files)]
     A --> EP
     Q --> R[Hybrid top-k evidence]
     R --> CP[Chat provider]
     CP --> A
+    A -. opt-in claim and evidence check .-> J[TypeSafe Jev]
+    J -. assessment .-> A
     A --> U
 ```
 
@@ -83,9 +88,53 @@ uv run --env-file .env local-rag
 
 On macOS or Linux, replace `Copy-Item` with `cp`. Open <http://127.0.0.1:8000>, upload a `.pdf`, `.md`, or `.txt` file, select the documents to search, and ask a question. Re-uploading a filename replaces its old chunks instead of leaving stale copies.
 
-The browser interface is organized as an evidence desk: manage and select sources in the library, ask from the question workspace, then inspect the answer's numbered citation ledger with page references and hybrid relevance scores.
+The browser workspace keeps your document library, answer, and source passages together. Add
+multiple files by browsing or dropping them into the library, select which documents to search,
+and ask a question. Click a numbered citation to read its passage beside the answer. For PDFs,
+open the original page, move between pages, or download the source. Failed uploads remain ready
+to retry, and a stopped or failed question leaves the previous answer available.
+
+The application stores vectors beneath `RAG_DATA_DIR` (`data/qdrant` by default) and original
+uploads beneath `RAG_DATA_DIR/sources/<collection-hash>`. Replacing, deleting, or clearing a
+document also removes its retained original; files outside the app are unaffected. Documents
+indexed before source retention was added remain searchable; add them again to enable original-page
+previews. Both `.env` and `data/` are ignored by Git.
+
+## Optional Jev experiment
+
+Jev can check whether each generated claim follows from its actual cited excerpts. It runs after
+structured generation, independently of the chat and embedding providers. Enable the observer in
+your local `.env`:
 
-The application stores vectors beneath `data/qdrant`. Both `.env` and `data/` are ignored by Git.
+```dotenv
+RAG_JEV_MODE=observe
+TYPESAFE_API_KEY=your-local-key
+```
+
+Restart the app with `uv run --env-file .env local-rag`. The workspace shows the active Jev mode
+before a question is submitted. After an answer, **Answer support check** summarizes the audit;
+each claim receives a plain-language status, the three probabilities remain available, and the
+technical record stays collapsed until requested. At the default experimental threshold of 0.90,
+Python suggests `keep` for high support, `withhold` for high contradiction or insufficient
+evidence, and `review` otherwise. These suggestions leave the answer and citation ledger
+unchanged. Provider failures show **Check unavailable** and still return the original answer.
+
+For opt-in citation repair, set `RAG_JEV_MODE=repair` and restart. When a claim's original check
+favors insufficient evidence, the app checks alternative passages already retrieved for that
+question part, one at a time in retrieval order. It replaces the citation with the first passage
+whose fresh support check meets the threshold. Claim text and the retrieved ledger stay intact.
+Existing support, contradiction, and unavailable results are retained. If no replacement passes
+or the shared audit budget expires, the original citation remains. Details retain the original
+assessment and every attempted replacement. The workspace marks a successful replacement as
+**Citation repaired** and shows the before-and-after source.
+
+Observation sends generated claims and their cited excerpt text to TypeSafe's hosted API.
+Repair also sends alternative retrieved excerpts when testing replacements. Each candidate is
+checked in isolation; passages from other question parts, filenames, and retained original files
+are excluded.
+The default remains `off`, with no Jev network calls. Review your documents before enabling it.
+See the [design study](docs/jev-integration-study.md), [API notes](docs/jev-api-research.md), and
+[measured tests and replay instructions](docs/jev-testing.md).
 
 ## API
 
@@ -96,7 +145,12 @@ curl -X POST http://127.0.0.1:8000/api/documents \
   -F "file=@notes.pdf"
 ```
 
-List indexed documents with `GET /api/documents`. The response includes each content-addressed `document_id`, raw-file `source_sha256`, filename, page count, and chunk count. Clear and re-index documents created before this field was introduced.
+List indexed documents with `GET /api/documents`. The response includes each content-addressed
+`document_id`, raw-file `source_sha256`, filename, full page count (including blank pages), chunk
+count, and `original_available`. Download a retained source with
+`GET /api/documents/{document_id}/original`. For PDFs,
+`GET /api/documents/{document_id}/pages/{page_number}` returns a locally rendered PNG of the
+original page; page numbers start at 1. Source downloads and previews use `Cache-Control: no-store`.
 
 Ask a grounded question:
 
@@ -122,7 +176,9 @@ The response keeps generation and retrieval separately inspectable:
       "score": 0.82,
       "source_sha256": "0123456789abcdef..."
     }
-  ]
+  ],
+  "audit_mode": "off",
+  "audits": []
 }
 ```
 
@@ -132,6 +188,15 @@ output and retrieved passages is disabled before rendering. Fenced flow diagrams
 PDFs are presented to both the reader and grounded generator as readable steps while their
 original citation `text` remains unchanged.
 
+With Jev enabled, `audit_mode` is `observe` or `repair` and `audits` contains one record per claim.
+Its citation IDs refer to the numbered response ledger. Records include the validated Choice
+distribution, suggested action, model, threshold, elapsed time, excerpt hashes, and request usage.
+Claims sharing the same cited set share a request ID and its token counts; deduplicate request IDs
+when summing usage. Repair records add `original`, `repair_attempts`, and `repair_policy_version`;
+deduplicate request IDs across those nested checks and the final record too. No-claim abstentions
+produce an empty audit list. `/health` also exposes `jev_mode`, `max_upload_mb`, and whether both
+configured model endpoints are local.
+
 Use `POST /api/retrieve` with the same request body to inspect retrieval without generation. Delete one document with `DELETE /api/documents/{document_id}`, or clear the collection with `DELETE /api/documents`. A full clear and re-index is required after changing embedding models because vectors from different embedding spaces cannot be mixed.
 
 ## Swap providers independently
@@ -173,12 +238,17 @@ To swap embeddings instead, change `RAG_EMBEDDING_PROVIDER` and `RAG_EMBEDDING_M
 | `RAG_SCORE_THRESHOLD` | `0.15` | minimum score for the dense retrieval leg; BM25 ranks independently |
 | `RAG_MAX_UPLOAD_MB` | `10` | upload boundary |
 | `RAG_REQUEST_TIMEOUT` | `120` | model request timeout in seconds |
+| `RAG_JEV_MODE` | `off` | `observe` for hosted checks; `repair` also verifies replacement citations |
+| `TYPESAFE_API_KEY` | empty | secret used only for Jev requests |
+| `RAG_JEV_MODEL` | `jev-1.13.0` | pinned Jev model |
+| `RAG_JEV_TIMEOUT` | `5` | total audit budget per question part in seconds; at most three parts |
+| `RAG_JEV_THRESHOLD` | `0.90` | experimental action threshold; greater than 0.5 and at most 1 |
 
 ## Project layout
 
 ```text
 src/local_rag/
-├── api.py          # HTTP endpoints, browser UI, lifecycle
+├── api.py          # HTTP endpoints and application lifecycle
 ├── config.py       # validated environment configuration
 ├── documents.py    # PDF/text loading and chunking
 ├── evaluation.py   # golden-dataset runner and deterministic metrics
@@ -187,7 +257,7 @@ src/local_rag/
 ├── providers.py    # chat/embedding protocols and adapters
 ├── service.py      # ingestion and question-answering pipeline
 ├── store.py        # vector-store protocol and local Qdrant
-└── web/index.html  # dependency-free UI
+└── web/            # dependency-free HTML, CSS, and JavaScript UI
 evaluation/         # public schema/example; private cases and results ignored
 docs/
 ├── index.html      # publishable architecture document
@@ -210,10 +280,12 @@ Development follows `feature/* → dev → main`. `dev` is the default integrati
 ## Privacy and security
 
 - In the default configuration, source text is sent only to Ollama on localhost and stored only in local Qdrant.
+- Original uploads are retained beneath `RAG_DATA_DIR` for local download and PDF page previews until the document is replaced, deleted, or the collection is cleared.
 - Retrieved text is treated as untrusted data; the system prompt instructs the model to ignore instructions embedded in documents.
 - Answers must cover each requested part from cited context, explicitly identify unsupported parts, and avoid filling gaps with model background knowledge.
 - Upload extension and size are validated, and uploaded filenames are never used as filesystem destinations.
 - Choosing a remote compatible endpoint changes the privacy boundary. Review that provider before sending documents.
+- Enabling Jev sends claims and excerpts to TypeSafe; repair mode may also send alternative retrieved passages. Retained originals and filenames are not sent. Its judgments do not establish factual correctness, answer completeness, or injection safety.
 - This demo has no authentication and binds to `127.0.0.1`. Do not expose it directly to a network.
 
 See [SECURITY.md](SECURITY.md) for reporting and deployment guidance.
@@ -228,7 +300,10 @@ See [SECURITY.md](SECURITY.md) for reporting and deployment guidance.
 | GitHub Actions and Pages | $0 within the public-repository allowances |
 | Cloud infrastructure | not used |
 
-Pulumi, AWS, hosted model APIs, authentication, background workers, neural reranking, and multi-user tenancy are deferred. They add cost or operational weight without improving this local portfolio baseline. The [architecture](https://sanjaysrinivas.github.io/context-foundry/) lists the concrete triggers for each upgrade.
+The optional Jev observer uses a paid hosted API; it is outside the zero-cost local default.
+Pulumi, AWS, authentication, background workers, neural reranking, and multi-user tenancy are
+deferred. The [architecture](https://sanjaysrinivas.github.io/context-foundry/) lists the concrete
+triggers for each upgrade.
 
 ## Evaluation and limitations
 

SECURITY.md

@@ -16,6 +16,14 @@ Browser responses set a restrictive same-origin content policy, deny framing and
 
 Generated answers are rendered as Markdown with raw HTML disabled. The browser only inserts the server-rendered `answer_html`; API consumers that render the plain `answer` field must apply equivalent restrictions instead of trusting model output as HTML.
 
+Original uploads are retained beneath `RAG_DATA_DIR/sources/<collection-hash>` so the local UI can
+download them and render PDF pages. Stored filenames are metadata and never become filesystem
+destinations. Replacing or deleting a document removes its retained source; clearing the collection
+removes all retained sources for that collection. Source downloads and page previews are served
+with `Cache-Control: no-store`, but any user or process with access to the data directory can read
+the files, so protect that directory and include it in the same backup and deletion policy as the
+vector store.
+
 Before any shared deployment, add and verify:
 
 - authenticated access and authorization;
@@ -28,5 +36,18 @@ Before any shared deployment, add and verify:
 
 The default Ollama configuration keeps inference local. Selecting an OpenAI-compatible remote URL sends prompts or document chunks to that endpoint; its privacy and retention policy then applies.
 
+`RAG_JEV_MODE=observe` independently enables TypeSafe's hosted Jev API. It sends generated claims
+and the text of their actual cited excerpts, including sensitive content if it occurs in those
+excerpts. Observation excludes uncited passages from the request state. `RAG_JEV_MODE=repair`
+also submits alternative passages already retrieved for the same question part, individually,
+when looking for replacement citations. Claim text and the retrieved ledger are preserved;
+citations change only after a fresh support check passes the threshold. Filenames, credentials,
+retained original files, and passages retrieved only for other question parts remain excluded from
+each check.
+Keep `TYPESAFE_API_KEY` in the Git-ignored `.env` or environment. Audit records expose sanitized
+status errors rather than remote error bodies. They are rendered as plain text in the browser.
+Observation does not filter answers. Citation repair does not rewrite claims or establish that
+the answer is true, complete, or relevant; Jev judgments are not a security boundary.
+
 Retrieved documents are untrusted input. The application includes prompt-injection guidance, but model instructions are not a security boundary. Never grant model output authority to execute commands or perform external actions without separate validation and authorization.
 

docs/evaluation.md

@@ -177,6 +177,28 @@ regressions reproducible but means acceptable paraphrases must be represented by
 phrases or reviewed manually. The gates are initial portfolio targets, not universal production
 guarantees.
 
+## Optional Jev claim/citation replay
+
+The Jev observer has a separate replay runner and eight public synthetic development fixtures in
+`evaluation/jev-smoke.jsonl`. It checks a claim against its cited passages directly, without
+Ollama, ingestion, or retrieval. Uncited decoys test evidence isolation. This diagnostic does not
+alter the approved-gold release metrics or promote synthetic labels to gold.
+
+~~~powershell
+uv run --frozen --env-file .env python -m local_rag.jev_evaluation --output evaluation/results/jev-smoke.json
+~~~
+
+The command sends the fixture claims and cited text to TypeSafe and requires `TYPESAFE_API_KEY`.
+It retains distributions and provenance, and exits with status 1 if any expected label disagrees
+or an audit is unavailable. A disagreement is evidence to inspect, not a reason to rewrite the
+expected label. The default 0.90 action threshold is experimental; reported confidence is not an
+independent probability of correctness. See [live results and limitations](jev-testing.md).
+
+For an enforcement experiment, first add source-reviewed claim/citation labels to a frozen,
+representative holdout and compare unsupported acceptance, supported withholding, review and
+outage rates, latency, and cost. The existing ledger-based citation metric alone does not measure
+whether each displayed claim follows from its particular citations.
+
 ## Change protocol
 
 1. Freeze the corpus, source hashes, evaluation cases, model tags, and retrieval configuration.