From 9dc3027f2890fd4f650a9f8bb625fdea0b839acf Mon Sep 17 00:00:00 2001 From: Julien Cornebise Date: Mon, 27 Jul 2026 22:15:15 +0200 Subject: [PATCH] =?UTF-8?q?python-math=20#45:=20test(delphi):=20FLI-cold?= =?UTF-8?q?=5Fstart=20repful=20comparison=20=E2=86=92=20xfail=20(prod-blob?= =?UTF-8?q?=20Q12)=20+=20s7=20Phase=204=20journal?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## What Phase 4 of `GOAL_CUTOVER_READY.md` (the standing goal document for the Python math cutover): golden snapshots re-recorded at the collapse tree — the source tree after the mode collapse removed the engine-mode switch — following the verify-then-record protocol. Journal session 7 (continued) shows the drift is exactly the two expected legacy families, and the recording engine was oracle-certified 20/20 on the same datasets. ## Results - Comparer: 7/7 PASS. - `--include-local` suite (private datasets) green with ONE annotation: the FLI-cold_start repful comparison now xfails. The production blob for that dataset carries Clojure's unseeded cold-start PCA — quirk Q12 (#2661: the Clojure cold tick's PCA start vector is unseeded-random, so even two Clojure runs differ) — and the collapse made legacy kmeans the only cold-tick path, so the Python selections no longer intersect that random draw. The battery's pinned-cold-start FLI certification supersedes this comparison. - Battery: TWO consecutive 20/20 MATCH passes; the divergence ledger holds 81 entries, 0 open. commit-id:075d2f77 --- delphi/docs/CLJ-PARITY-FIXES-JOURNAL.md | 26 +++++++++++++++++++++++++ delphi/tests/test_discrepancy_fixes.py | 18 +++++++++++++++-- 2 files changed, 42 insertions(+), 2 deletions(-) diff --git a/delphi/docs/CLJ-PARITY-FIXES-JOURNAL.md b/delphi/docs/CLJ-PARITY-FIXES-JOURNAL.md index 7167730ee..7b09f29c2 100644 --- a/delphi/docs/CLJ-PARITY-FIXES-JOURNAL.md +++ b/delphi/docs/CLJ-PARITY-FIXES-JOURNAL.md @@ -4262,3 +4262,29 @@ Phase 4 re-record therefore = private goldens (--include-local) + optionally recording public ones; VERIFY against the battery's certified clj recordings BEFORE recording (never blind). Recorder: scripts/regression_recorder.py. + +### Phase 4 — goldens: verify-then-re-record (s7 cont.) + +Drift check (`regression_comparer.py --include-local`): 7/7 datasets fail +vs the pre-collapse goldens. Read the FLI detail per the never-blind rule +— the drift is EXACTLY the two expected legacy families and nothing else: +(1) blob shape (folded columnar base-clusters, lastModTimestamp/mod-out/ +mod-in emission, legacy consensus/repness shapes — _apply_legacy_blob_shape +now unconditional); (2) PCA warm-start numerics in the rank-deficient +component tail (comps[1]/proj second-axis jitter at tiny magnitudes). + +Verification argument for re-recording (the "certified before recording" +evidence): the engine writing the new goldens is the SAME TREE the battery +just certified bit-exact against the Clojure oracle on these SAME 7 +datasets (20/20 MATCH, two runs — post-collapse and post-refactor). The +observed drift families match the collapse's documented semantics 1:1; +no third family observed. Goldens are LOCAL-ONLY artifacts (zero +git-tracked golden_snapshot.json), so the re-record is evidenced by the +comparer passing + this entry, not by committed files. Public datasets +(vw/biodiversity) had NO goldens at all — being recorded for the first +time in this worktree. + +PR #2673 review agent verdict: CLEAN — split proven behavior-preserving +(including an equivalence proof of the counts_df.empty seam), blob test +non-vacuous (45+42 entries), tolerances justified against Clojure's +`(float repness-test)` cast at repness.clj:187. diff --git a/delphi/tests/test_discrepancy_fixes.py b/delphi/tests/test_discrepancy_fixes.py index 2a7142f8e..4623f85bc 100644 --- a/delphi/tests/test_discrepancy_fixes.py +++ b/delphi/tests/test_discrepancy_fixes.py @@ -1077,12 +1077,19 @@ def test_repful_classification_boundary(self): def test_repful_matches_clojure_blob(self, request, conv, clojure_blob, dataset_name): """repful-for (Clojure) vs repful (Python) for shared rep comments. - Gates on 9/11 variants since the gid label-swap fix (2026-07-05 + Gates on most variants since the gid label-swap fix (2026-07-05 removal of the group size re-sort). Residual known-bad: two incremental variants with deeper trajectory divergence (pakistan-incremental: Clojure blob PCA computed on a comment subset; vw-incremental: in-conv trajectory divergence) — deferred - to the sequential-parity work. + to the sequential-parity work — plus FLI-cold_start since the mode + collapse (2026-07-27): the PROD blob carries Clojure's UNSEEDED + cold-start PCA (Q12, #2661) and the collapse made the + Clojure-faithful legacy kmeans the only cold-tick path, so the + selection sets no longer intersect that particular random draw. + The battery certifies FLI end-to-end against pinned-cold-start + Clojure recordings (20/20 MATCH), which supersedes this prod-blob + comparison for that variant. """ if request.node.callspec.id in ('vw-incremental', 'pakistan-incremental'): request.applymarker(pytest.mark.xfail( @@ -1091,6 +1098,13 @@ def test_repful_matches_clojure_blob(self, request, conv, clojure_blob, dataset_ reason="residual incremental trajectory divergence (gid " "label swap fixed 2026-07-05; sequential-parity " "work)")) + if request.node.callspec.id == 'FLI-cold_start': + request.applymarker(pytest.mark.xfail( + strict=False, + reason="prod blob Q12 unseeded cold-start PCA (#2661) vs " + "the collapse's legacy-kmeans cold tick — zero " + "shared selections; battery certifies FLI against " + "pinned-cold-start recordings instead")) clojure_repness = clojure_blob.get('repness', {}) if not clojure_repness: pytest.skip("No repness in Clojure blob")