diff --git a/apps/elf-eval/src/bin/external_memory_pattern_radar/render.rs b/apps/elf-eval/src/bin/external_memory_pattern_radar/render.rs index 2823962c..141c576f 100644 --- a/apps/elf-eval/src/bin/external_memory_pattern_radar/render.rs +++ b/apps/elf-eval/src/bin/external_memory_pattern_radar/render.rs @@ -23,11 +23,11 @@ pub(super) fn render_summary(cursor: &RadarCursor) -> Result { out.push_str("source_refs: []\n"); out.push_str("code_refs:\n"); out.push_str(" - apps/elf-eval/fixtures/external_memory_pattern_radar/cursor.json\n"); - out.push_str(" - apps/elf-eval/src/bin/external_memory_pattern_radar.rs\n"); + out.push_str(" - apps/elf-eval/src/bin/external_memory_pattern_radar/main.rs\n"); out.push_str("related: []\n"); out.push_str("drift_watch:\n"); out.push_str(" - apps/elf-eval/fixtures/external_memory_pattern_radar/cursor.json\n"); - out.push_str(" - apps/elf-eval/src/bin/external_memory_pattern_radar.rs\n"); + out.push_str(" - apps/elf-eval/src/bin/external_memory_pattern_radar/main.rs\n"); out.push_str("---\n\n"); out.push_str("# External Memory Pattern Radar Summary\n\n"); out.push_str("Goal: Preserve the latest weekly ELF external memory pattern radar outcome.\n"); diff --git a/docs/evidence/external_memory_pattern_radar_latest.md b/docs/evidence/external_memory_pattern_radar_latest.md index cad1348c..4b6c64a8 100644 --- a/docs/evidence/external_memory_pattern_radar_latest.md +++ b/docs/evidence/external_memory_pattern_radar_latest.md @@ -14,7 +14,7 @@ tags: source_refs: [] code_refs: - apps/elf-eval/fixtures/external_memory_pattern_radar/cursor.json - - apps/elf-eval/src/bin/external_memory_pattern_radar.rs + - apps/elf-eval/src/bin/external_memory_pattern_radar/main.rs related: [] drift_watch: - docs/evidence/external_memory_pattern_radar_latest.md diff --git a/docs/log.md b/docs/log.md index 285d64ef..8c0886c4 100644 --- a/docs/log.md +++ b/docs/log.md @@ -6,6 +6,14 @@ Read this when: You need to understand why documentation structure changed. Not this document: Detailed subsystem history, raw research state, or plan execution logs. +## 2026-09-21 + +- Recorded legacy workspace disposition, historical benchmark claim corrections, + and the bounded repair queue. Fixed the radar generator's obsolete entrypoint + reference and its checked-in report. Removed the misplaced application-level + `--locked` argument from radar tasks; Cargo remains locked. Product behavior + is unchanged. + ## 2026-09-20 - Added an exploratory ELF value review using September agent-platform sources and diff --git a/docs/research/2026-09-21-legacy-workspace-disposition.md b/docs/research/2026-09-21-legacy-workspace-disposition.md new file mode 100644 index 00000000..efa7f32f --- /dev/null +++ b/docs/research/2026-09-21-legacy-workspace-disposition.md @@ -0,0 +1,100 @@ +--- +type: Research Contract +title: "Legacy Workspace Disposition" +description: "Retire obsolete workspaces while retaining evidence and bounded repair candidates." +resource: docs/research/2026-09-21-legacy-workspace-disposition.md +status: proposed +authority: exploratory +owner: maintenance +last_verified: 2026-09-21 +code_refs: + - apps/elf-mcp/src/app/server/support.rs + - packages/elf-service/src/context_pack/assembly.rs + - scripts/benchmark_contract/evaluation.py + - apps/elf-eval/src/bin/external_memory_pattern_radar/render.rs +related: + - docs/research/2026-09-20-elf-value-review.md +--- +# Legacy Workspace Disposition + +Purpose: Preserve the useful results of retired July and August work without +restoring obsolete product contracts or benchmark implementations. +Read this when: Recovering an old workspace or choosing a follow-up repair. +Not this document: Proof of product safety, a current competitive ranking, or +approval to change public APIs. + +## Disposition + +| Former branch | Disposition | Retained value | +| --- | --- | --- | +| `xv/benchmark-competitors` | Delivered in PR #381; retire checkout. | Original measured run artifacts and logs. | +| `xv/benchmark-thin-runner` | Retire transitional runner. | Historical disposition inventory and committed source. | +| `xv/product-salvage` | Retain commits `2f12e0ff` and `4840feac` as repair candidates. | Response bounds, source checks, lifecycle propagation, and process cleanup. Do not cherry-pick the combined API changes. | +| `xv/benchmark-contract-refresh` | Retire the experimental implementation. | Historical corrections, source snapshots, and selected test scenarios. | +| `xy/sag-competitor-benchmark-refresh` | Retire the old competitor runner. | Historical run artifacts and evidence of evaluation defects. | +| `xy/final-memory-knowledge-plan` | Supersede the July execution proposal in PR #353. | Source-backed memory principles and valid documentation repairs. | + +Archives must include ignored artifacts, untracked files, original tracked files, +and Git history. The local archive manifest records original HEADs, branch names, +file hashes, and any stash commit used to preserve index and working-tree state. +Git bundles are verified before checkout removal. Archives remain local: they can +contain private experiment data and must not be uploaded with a pull request. + +## Changes that must not return as cleanup + +The contract-refresh experiment changes document read/search/excerpt routes to +404 responses and removes public search-session and recall-debug routes. It also +restricts model-facing context to notes and documents. Those changes alter public +contracts; archive them instead of silently adopting them. + +Its source resolver requires the assertion to equal the source quote and accepts +only a specific v2 source-binding shape. This can reject summaries and existing +source formats. Reuse source revision and scope checks only after a compatible +design and migration analysis. + +The SAG mem0 adapter builds output claims from `expected_answer.must_include` and +its expected evidence links. This does not establish independent answer quality. +Its report status fallback can label out-of-scope-only results as pass while +listing that state separately. Do not restore this scoring path. The current +benchmark evaluator separately checks execution, coverage, and score eligibility. + +## Historical claim corrections + +Retain the July 13 correction to the July 8 benchmark presentation. The old recall +denominator mixed comparable and non-comparable work. The correction itself says +that exact original report bytes were unavailable; its numeric examples are +historical assertions, not independently recomputed measurements. Do not repeat +those values as a current comparison or use post-hoc intersections as a ranking. + +Retain the July 19 correction withdrawing current comparative authority from the +June 20 and June 27 reports. Fixture-derived results and dated closeout claims do +not establish present superiority. Preserve original dated reports; link this +disposition when interpreting them. References in those old corrections to a v2 +experimental contract do not make that contract authoritative today. + +## Bounded repair queue + +| Priority | Owner | Candidate and acceptance evidence | +| --- | --- | --- | +| P1 | MCP transport | Bound response accumulation. Cover declared and chunked oversized bodies. Select a supported limit; the old 1 MiB constant is a proposal. | +| P1 | MCP transport | Prevent raw upstream error bodies from entering model context. Preserve useful typed error codes and correlation IDs; test text, HTML, malformed JSON, nested JSON, and chunk failures. | +| P1 | Context Pack | Reject empty or schema-only source identities without requiring every assertion to be a verbatim quote. Preserve supported reference variants with focused tests. | +| P1 | Service/API | Trace lifecycle state through search, document readback, and Context Pack. Check correction, deletion, scope, and restart behavior against the current contract before porting old tests. | +| P2 | Local harness | Review child-process reaping from `4840feac`; prove cleanup without changing production lifecycle behavior. | + +These are retained candidates, not completed fixes or confirmed exploitable +vulnerabilities. Old test files depend on an incompatible model-delivery contract; +port their independent scenarios, not their output schema or route removals. + +## Roadmap replacement + +The July R0-R9 plan preserves useful provenance, correction, privacy, and cost +principles. Its broad model-ladder, graph, knowledge-workspace, and dreaming +expansion sequence is not the current execution queue. Use the September value +review to design a task-benefit pilot and one cross-agent correction workflow. +That review remains exploratory and does not itself authorize a new product API. + +The old PR also contains valid path and metadata repairs. The radar renderer and +its checked-in report now reference the existing directory entrypoint. Other +historical documentation changes require current path checks; old verdicts must +not be presented as fresh validation. diff --git a/docs/research/index.md b/docs/research/index.md index 803a0dac..4a8cbbde 100644 --- a/docs/research/index.md +++ b/docs/research/index.md @@ -22,3 +22,4 @@ For legacy research JSON disposition, read `docs/evidence/2026-06-18-research-artifact-disposition.md`. - [ELF Value Review: September 2026](2026-09-20-elf-value-review.md): current native-memory pressure, differentiated value hypothesis, and proposed task-level pilot. +- [Legacy Workspace Disposition](2026-09-21-legacy-workspace-disposition.md): retired experiments, historical claim boundaries, and bounded repair candidates. diff --git a/makefiles/research.toml b/makefiles/research.toml index dc7bc55c..a3dbe00d 100644 --- a/makefiles/research.toml +++ b/makefiles/research.toml @@ -10,7 +10,6 @@ args = [ "external_memory_pattern_radar", "--", "run", - "--locked", "--cursor", "apps/elf-eval/fixtures/external_memory_pattern_radar/cursor.json", "--summary", @@ -34,7 +33,6 @@ args = [ "external_memory_pattern_radar", "--", "run", - "--locked", "--cursor", "apps/elf-eval/fixtures/external_memory_pattern_radar/cursor.json", "--out-cursor", @@ -75,7 +73,6 @@ args = [ "external_memory_pattern_radar", "--", "run", - "--locked", "--mode", "offline", "--cursor",