python-math #43: docs(delphi): s7 goal state + journal — mode collapse executed, battery 20/20

delphi/docs/CLJ-PARITY-FIXES-JOURNAL.md

@@ -4213,3 +4213,52 @@ Also purged post-suite: scripts/poller_equiv.py CLI --engine-mode plumbing
 docker-compose delphi-math-poller env line, example.env comment,
 CLOJURE_QUIRKS preamble + MATH_POLLER_DESIGN updated to collapse-era
 wording (historical spec docs left as records).
+
+### s7 wind-down — What's Next
+
+Pushed: PRs #2665-#2671 (collapse series) + #2664 (Phase 0 battery
+speedup). python-ci dispatched on spr/edge/7f42df81 (run 30286254481);
+collapse-series review subagent launched; Copilot requested once on
+#2659/#2663 — ALL to be checked at next orientation. Parks live on jj
+bookmarks improvements/item-{2,4,5,8} (pushed to origin). Remaining
+phases: 3 (clarity refactor 14c/14b), 4 (goldens + double battery pass +
+equiv gate), 5 (EC2 measurement). See GOAL_STATE.md for numbered actions.
+
+## Session 7 (cont.): Phase 3 clarity refactor SHIPPED — battery bit-identity ×2 trees
+
+- **14b**: TestBlobInjectionStats (tests/test_repness_unit.py) — Clojure
+  blob group memberships (unfolded via the blob's own base-clusters) + the
+  dataset votes injected into the PRODUCTION stats path; every blob repness
+  entry compared per (gid, tid) on n-success/n-trials/p-success/p-test/
+  repness/repness-test/repful-for. GREEN on vw + biodiversity. Gotcha:
+  test-conversation matrices carry STRING pids/tids vs the blob's ints
+  (map on the way in); Clojure emits repness-test ROUNDED (~7 sig digits)
+  → that one field compares at rtol 2e-6, the rest at 1e-9.
+- **14c**: compute_group_comment_stats_df →
+  _group_comment_vote_counts (plumbing) + _comment_stats_from_counts
+  (recipe). Pure code motion. **Battery on the refactored tree: 20/20
+  MATCH (full py re-replay, 18m46s)** — bit-identity PROVEN. Full suite
+  1155 passed (+2 blob pins, -1 deduped parametrize, one env-gated skip).
+- Shipped as PR #2673 (spr/edge/acff8fbe); python-ci dispatched (run
+  30288678922); review subagent launched. jj gotcha hit: `jj split`
+  opens an editor (use JJ_EDITOR=true) and gives BOTH halves the original
+  description INCLUDING the spr commit-id trailer, and the spr bookmark
+  follows the working copy — rewrite the second half's description fresh
+  and `jj bookmark set <spr-branch> -r <first-half> --allow-backwards`.
+- Collapse-series review agent verdict: CLEAN (no high-confidence
+  findings; verified every branch reduction = the legacy arm, fingerprint
+  freezing exact, no test-expectation drift). Its 3 sub-threshold
+  cleanups applied (runbook env line, deduped parametrize, docstrings).
+  Phase 0 PR #2664 python-ci: SUCCESS.
+
+### Phase 4 diagnostic (goldens)
+
+`scripts/regression_comparer.py` on the collapsed tree: NO golden
+snapshots exist for the public datasets (vw/biodiversity — never
+recorded in this worktree); the 5 private-dataset goldens live under
+real_data/.local/*/golden_snapshot.json. That's why the suite stayed
+green through the collapse — golden comparisons skip without snapshots.
+Phase 4 re-record therefore = private goldens (--include-local) +
+optionally recording public ones; VERIFY against the battery's certified
+clj recordings BEFORE recording (never blind). Recorder:
+scripts/regression_recorder.py.

delphi/docs/GOAL_STATE.md

@@ -2,50 +2,53 @@ STATUS: IN PROGRESS
 
 # GOAL_STATE — checkpoint for GOAL_CUTOVER_READY.md (cap ~50 lines)
 
-Predecessor GOAL_R1_PARITY.md: ACHIEVED 2026-07-24 (evidence pointers in its
-final journal entries; battery ×4 clean pairs, live equiv PASS ×2 datasets).
-
-## Where we are (2026-07-27, session 6 wind-down)
-
-- Copilot triage COMPLETE: 25 threads — 16 fixes applied (TDD; 647-test gate
-  green), 2 quirk-rejections cited, 3 declines, 1 deferral (#2644). Battery
-  ×2 re-ran on the triaged tree (s6 logs — check verdicts at orientation if
-  this session ended before they landed). Issues #2660/#2661/#2662 opened
-  (Q10/Q12/Q13+Q18 — "fixed by the Python push").
-- Julien rulings captured in POST_CUTOVER_IMPROVEMENTS.md +
-  CUTOVER_RUNBOOK.md: bans DROPPED entirely; equiv = release gate not CI;
-  sharding parked (data: p99=5 active convs/min vs ~100 ticks/min EC2-
-  measured serial capacity); clarity refactor moved PRE-cutover; large-conv
-  EC2 tick measurement is a flip precondition; run_delphi.py is
-  PRODUCTION-called (POST /api/v3/delphi/jobs → job_poller FULL_PIPELINE).
-- Battery timing DATA (journal s6): first pass after an engine change ≈36
-  min (py re-replay, clj cached); second consecutive pass ≈21 SECONDS;
-  full clj re-record ≈91 min. Phase 0 tooling attacks the 36-min pass.
+## Where we are (2026-07-27, session 7 wind-down)
+
+- Phase 0 DONE + A/B-proven: engine-scoped py cache hash + 6-way parallel
+  battery (PR #2664, python-ci green, review clean). Costs now: no edit
+  ~22s, harness-only edit ~2m, engine edit ~19m (long pole
+  pakistan:uniform8 ~18m — <8m target unreachable without intra-entry
+  parallelism; journal s7).
+- Phase 1 DONE: engine_mode inventory journaled (s7).
+- Phase 2 DONE — MODE COLLAPSE EXECUTED: 7 commits = PRs #2665-#2671.
+  DONE-gate grep = 0 hits over delphi/polismath/. Ban filtering DELETED
+  outright. Parks minted: jj bookmarks improvements/item-{2,4,5,8}
+  (verbatim reverse patches off the C7 commit; NOT buildable — they
+  carry old flag refs; re-landing = keep the improved side only).
+  Battery on the collapsed tree: 20/20 MATCH full py re-replay (19m21s)
+  + cached pass 22s. Full suite 1155 passed / 22 skipped / 44 xfailed
+  / 2 XPASS (D9/D10 vw-cold_start now match Clojure — parity improved).
+- Phase 3 DONE: 14b blob-injection pins + 14c two-phase split (PR
+  #2673); battery 20/20 on the refactored tree (bit-identity); collapse
+  review agent verdict CLEAN, its 3 cleanups applied. #2664 CI green.
 
 ## Next actions
 
-1. s6 battery CONFIRMED clean: 20/20 MATCH ×2 on the triaged tree
-   (battery_s6_pass{1,2}.log; only 3 clj re-records fired — the restart
-   entries carry no comments CSV). Commit split DONE: triage fixes = PR
-   #2663 (spr/edge/efb914d7), docs = #2659. python-ci dispatched on
-   #2663's head (run 30276844539) — CHECK at orientation. Thread
-   replies/resolves: CONFIRMED complete — all 20 open threads across 11
-   PRs replied (citing #2663) and resolved; 25/25 Copilot comments closed.
-2. Phase 0 of GOAL_CUTOVER_READY.md: battery tooling speedup (hash scoping
-   + parallel entries; A/B-prove — timing data in journal s6: 36-min
-   first pass / 21-s cached pass). THEN Phase 1 inventory (grep all
-   engine_mode branch sites; classify DELETE/PARK/KEEP; journal it).
-3. Review the new PRs per protocol when review-ready: #2659 (docs), #2663
-   (triage batch — request Copilot once; our agent already covered the
-   content via the triage itself).
-4. Parked questions needing input: none — all rulings recorded.
+1. Orientation: check CI runs 30286254481 (collapse tip) + 30288678922
+   (#2673 14b/c) + the #2673 review agent result; Copilot comments on
+   #2659/#2663. Triage per protocol (a "fix" undoing legacy semantics =
+   post-cutover queue item). Phase 3 is DONE (PR #2673; battery 20/20 on
+   the refactored tree = bit-identity; journal s7 cont.).
+2. Phase 4: goldens. Diagnostic done (journal s7 cont.): public datasets
+   have NO goldens (comparisons skip); the 5 private goldens are at
+   real_data/.local/*/golden_snapshot.json. Verify current private-golden
+   drift with `uv run python scripts/regression_comparer.py --include-local`
+   (needs /private-data setup), cross-check intended new values against
+   the battery clj recordings, THEN re-record via
+   scripts/regression_recorder.py; full suite --include-local green.
+3. Phase 4 gates on the final tree: TWO consecutive clean battery passes
+   + poller_equiv.py full-run PASS (non-vacuous) on vw AND pc-meta-02
+   (pgproxy 127.0.0.1:15432 up: `docker start polis-dev-postgres-1
+   pgproxy`; docker compose services for the clj container).
+4. Phase 5: EC2 large-conv tick measurement (bench AWS profile, 33k
+   ptpts × 783 cmts shape) → number + serial-OK verdict into
+   CUTOVER_RUNBOOK.md risk register.
+5. Only after all of 1-6 of the DONE list hold on the final tree: flip
+   line 1 here to STATUS: DONE.
 
 ## Pointers
 
 - Contract: GOAL_CUTOVER_READY.md. Roadmap: POST_CUTOVER_IMPROVEMENTS.md.
-  Runbook: CUTOVER_RUNBOOK.md. Quirks: CLOJURE_QUIRKS.md Q1-Q19.
-- Battery: scripts/certify.py run (20 entries). Equiv release gate:
-  scripts/poller_equiv.py full-run (needs pgproxy 127.0.0.1:15432 up:
-  `docker start polis-dev-postgres-1 pgproxy`).
-- Suite baseline: 1134 passed / 22 skipped / 46 xfailed (2026-07-24) + s6
-  additions; gate delegation protocol in GOAL_R1_PARITY.md.
+- Battery: cd delphi && uv run python scripts/certify.py run (workers=6).
+- Suite baseline: 1155/22/44 + 2 xpassed (s7 collapse tree).
+- s7 journal: Phase 0/1/2 + battery evidence (CLJ-PARITY-FIXES-JOURNAL.md).