Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions delphi/docs/CLJ-PARITY-FIXES-JOURNAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -4262,3 +4262,29 @@ Phase 4 re-record therefore = private goldens (--include-local) +
optionally recording public ones; VERIFY against the battery's certified
clj recordings BEFORE recording (never blind). Recorder:
scripts/regression_recorder.py.

### Phase 4 — goldens: verify-then-re-record (s7 cont.)

Drift check (`regression_comparer.py --include-local`): 7/7 datasets fail
vs the pre-collapse goldens. Read the FLI detail per the never-blind rule
— the drift is EXACTLY the two expected legacy families and nothing else:
(1) blob shape (folded columnar base-clusters, lastModTimestamp/mod-out/
mod-in emission, legacy consensus/repness shapes — _apply_legacy_blob_shape
now unconditional); (2) PCA warm-start numerics in the rank-deficient
component tail (comps[1]/proj second-axis jitter at tiny magnitudes).

Verification argument for re-recording (the "certified before recording"
evidence): the engine writing the new goldens is the SAME TREE the battery
just certified bit-exact against the Clojure oracle on these SAME 7
datasets (20/20 MATCH, two runs — post-collapse and post-refactor). The
observed drift families match the collapse's documented semantics 1:1;
no third family observed. Goldens are LOCAL-ONLY artifacts (zero
git-tracked golden_snapshot.json), so the re-record is evidenced by the
comparer passing + this entry, not by committed files. Public datasets
(vw/biodiversity) had NO goldens at all — being recorded for the first
time in this worktree.

PR #2673 review agent verdict: CLEAN — split proven behavior-preserving
(including an equivalence proof of the counts_df.empty seam), blob test
non-vacuous (45+42 entries), tolerances justified against Clojure's
`(float repness-test)` cast at repness.clj:187.
18 changes: 16 additions & 2 deletions delphi/tests/test_discrepancy_fixes.py
Original file line number Diff line number Diff line change
Expand Up @@ -1077,12 +1077,19 @@ def test_repful_classification_boundary(self):
def test_repful_matches_clojure_blob(self, request, conv, clojure_blob, dataset_name):
"""repful-for (Clojure) vs repful (Python) for shared rep comments.

Gates on 9/11 variants since the gid label-swap fix (2026-07-05
Gates on most variants since the gid label-swap fix (2026-07-05
removal of the group size re-sort). Residual known-bad: two
incremental variants with deeper trajectory divergence
(pakistan-incremental: Clojure blob PCA computed on a comment
subset; vw-incremental: in-conv trajectory divergence) — deferred
to the sequential-parity work.
to the sequential-parity work — plus FLI-cold_start since the mode
collapse (2026-07-27): the PROD blob carries Clojure's UNSEEDED
cold-start PCA (Q12, #2661) and the collapse made the
Clojure-faithful legacy kmeans the only cold-tick path, so the
selection sets no longer intersect that particular random draw.
The battery certifies FLI end-to-end against pinned-cold-start
Clojure recordings (20/20 MATCH), which supersedes this prod-blob
comparison for that variant.
"""
if request.node.callspec.id in ('vw-incremental', 'pakistan-incremental'):
request.applymarker(pytest.mark.xfail(
Expand All @@ -1091,6 +1098,13 @@ def test_repful_matches_clojure_blob(self, request, conv, clojure_blob, dataset_
reason="residual incremental trajectory divergence (gid "
"label swap fixed 2026-07-05; sequential-parity "
"work)"))
if request.node.callspec.id == 'FLI-cold_start':
request.applymarker(pytest.mark.xfail(
strict=False,
reason="prod blob Q12 unseeded cold-start PCA (#2661) vs "
"the collapse's legacy-kmeans cold tick — zero "
"shared selections; battery certifies FLI against "
"pinned-cold-start recordings instead"))
clojure_repness = clojure_blob.get('repness', {})
if not clojure_repness:
pytest.skip("No repness in Clojure blob")
Expand Down
Loading