python-math #19: feat(delphi): certify.py battery runner + focuser + fingerprint ledger (Spec A, Phase 0) - #2647
Draft
jucor wants to merge 1 commit into
Draft
python-math #19: feat(delphi): certify.py battery runner + focuser + fingerprint ledger (Spec A, Phase 0)#2647jucor wants to merge 1 commit into
jucor wants to merge 1 commit into
Conversation
This was referenced Jul 22, 2026
Draft
Draft
Draft
Draft
jucor
marked this pull request as draft
July 22, 2026 00:52
jucor
changed the title
jucor
changed the title
jucor
changed the title
jucor
changed the title
This was referenced Jul 27, 2026
Draft
python-math #43: docs(delphi): s7 goal state + journal — mode collapse executed, battery 20/20
#2672
Draft
Draft
Draft
5 tasks
This was referenced Jul 28, 2026
Draft
jucor
changed the title
jucor
added a commit
that referenced
this pull request
Jul 28, 2026
…Python math engine to math/
The Clojure implementation leaves the working tree, and the Python math
engine takes its place: `delphi/polismath/` -> `math/polismath/`, now its
own installable package. `delphi/` keeps the umap/narrative service and
the ENTIRE math test estate (tests, real_data datasets + goldens, replay
stores) and depends on the engine as a package.
What moved / changed:
- DELETED: the Clojure tree (math/src, math/test, math/dev, deps.edn,
Dockerfile, ...) — 60 tracked files. It remains in git history as the
certification oracle (`git log -- math/`); math/README.md documents the
`git archive` restore recipe (NOT `git checkout -- math/`, which
clobbers jj-colocated working copies). test-clojure.yml deleted with it.
- MOVED: delphi/polismath -> math/polismath (import name unchanged:
`import polismath`). New math/pyproject.toml (hatchling; deps = the
math-relevant subset of delphi's pins; `run-math-pipeline` console
script moved here; py.typed added). The poller entry
delphi/scripts/math_poller.py became polismath/poller/__main__.py —
service command is now `python -m polismath.poller`.
- PACKAGING: delphi depends on `polismath` via
`[tool.uv.sources] polismath = { path = "../math", editable = true }` —
ONE shared environment (delphi/.venv), `cd delphi && uv sync` as
before. delphi's wheel/sdist no longer bundle polismath; delphi pyright
include drops it (math/pyproject.toml carries its own [tool.pyright]).
- PATH ANCHORS: new polismath/paths.py (REPO_ROOT / DELPHI_ROOT /
REAL_DATA_ROOT / CLJ_TREE_ROOT / PACKAGE_ROOT). Eight modules derived
the delphi root from their own __file__ (`parents[2]`) — after the move
that arithmetic lands on math/ silently. All eight now assign their
historical module-level names (_DELPHI_ROOT, REAL_DATA_ROOT,
_MATH_ROOT, ...) from paths.py, so test monkeypatching is unchanged.
certify's engine-tree cache-key root became an explicit seam
(_ENGINE_TREE_ROOT = paths.PACKAGE_ROOT).
- ORACLE UX: certify's clj cache-key hashing now raises CertifyError
"clj-oracle" with restore instructions when math/dev/replay.clj is
absent (was: bare FileNotFoundError). New test pins it. The
requires_math_tree guards (folded into #2643/#2647) now skip
everywhere the oracle is absent — including local runs on this tree
(5 tests: 4 certify + 1 timing probe).
- DOCKER: delphi/Dockerfile no longer COPYs polismath from its own
context; the engine source arrives via a NAMED ADDITIONAL BUILD
CONTEXT (`COPY --from=mathsrc . /math-src` + `uv pip install --no-deps
/math-src`). All four compose build blocks (docker-compose.yml +
docker-compose.test.yml x delphi/math-python) declare
`additional_contexts: mathsrc: ./math`. requirements.lock unchanged
(polismath's deps are a subset; it installs --no-deps from source).
- CI: python-ci.yml paths gain math/**; its run_math_pipeline.py cp line
repointed. Local suite after the move: 1197 passed / 28 skipped /
44 xfailed / 2 xpassed (baseline 1171/22: +31 shadow-compare tests
from Step #1, +1 oracle-error test, -5 passes now skipping as oracle
guards, +1 env-conditional skip).
- run_delphi.py invokes the pipeline as `python -m
polismath.run_math_pipeline` (was a file path under /app).
SUPPLY-CHAIN NOTE: the name "polismath" is never resolved from a package
index — Docker installs it `--no-deps` from the copied source and uv
resolves it from [tool.uv.sources]; the Dockerfile test stage's
`.[dev]` resolution finds it already installed. If the package is ever
published, register or rename first (the PyPI name is not ours).
MERGE GATE (checklist — after Step #3 has soaked):
- [ ] Verify the PROD math/delphi hosts' compose supports BuildKit named
additional build contexts (docker compose >= 2.17 with BuildKit;
after_install.sh calls /usr/local/bin/docker-compose — if that is
compose v1, upgrade it BEFORE merging this or the on-host image
build fails on `COPY --from=mathsrc`).
- [ ] python-ci green on this PR's branch (workflow_dispatch).
- [ ] Docker image builds verified once (compose build delphi
math-python) on a host/runner with BuildKit.
- [ ] Team OK that certify re-replay now requires the documented oracle
restore (cached pairs unaffected).
KNOWN/ACCEPTED:
- 10 pre-existing pyright reportArgumentType hits in certify.py /
test_certify.py (verified byte-identical code at the pre-cutover tree
— latent noise, not introduced here; not fixed to keep this PR pure
reorg).
- Docs under delphi/docs still say "delphi/polismath" in historical
narratives; forward-looking docs (README, runbook, quirks header)
follow in the wind-down docs pass / post-cutover queue.
ROLLBACK: revert this PR — pure source reorganization; no DB or runtime
coupling (the built image contains the same installed packages either
way). Nothing about the flip (Steps #2-#3) depends on this PR.
Part of the cutover PR series (delphi/docs/HANDOFF_CUTOVER_EXECUTION.md
+ CUTOVER_RUNBOOK.md). NOTHING merges without Julien's explicit go.
commit-id:45664936
jucor
added a commit
that referenced
this pull request
Jul 28, 2026
…ython math engine to math/
The Clojure implementation leaves the working tree, and the Python math
engine takes its place: `delphi/polismath/` -> `math/polismath/`, now its
own installable package. `delphi/` keeps the umap/narrative service and
the ENTIRE math test estate (tests, real_data datasets + goldens, replay
stores) and depends on the engine as a package.
What moved / changed:
- DELETED: the Clojure tree (math/src, math/test, math/dev, deps.edn,
Dockerfile, ...) — 60 tracked files. It remains in git history as the
certification oracle (`git log -- math/`); math/README.md documents the
`git archive` restore recipe (NOT `git checkout -- math/`, which
clobbers jj-colocated working copies). test-clojure.yml deleted with it.
- MOVED: delphi/polismath -> math/polismath (import name unchanged:
`import polismath`). New math/pyproject.toml (hatchling; deps = the
math-relevant subset of delphi's pins; `run-math-pipeline` console
script moved here; py.typed added). The poller entry
delphi/scripts/math_poller.py became polismath/poller/__main__.py —
service command is now `python -m polismath.poller`.
- PACKAGING: delphi depends on `polismath` via
`[tool.uv.sources] polismath = { path = "../math", editable = true }` —
ONE shared environment (delphi/.venv), `cd delphi && uv sync` as
before. delphi's wheel/sdist no longer bundle polismath; delphi pyright
include drops it (math/pyproject.toml carries its own [tool.pyright]).
- PATH ANCHORS: new polismath/paths.py (REPO_ROOT / DELPHI_ROOT /
REAL_DATA_ROOT / CLJ_TREE_ROOT / PACKAGE_ROOT). Eight modules derived
the delphi root from their own __file__ (`parents[2]`) — after the move
that arithmetic lands on math/ silently. All eight now assign their
historical module-level names (_DELPHI_ROOT, REAL_DATA_ROOT,
_MATH_ROOT, ...) from paths.py, so test monkeypatching is unchanged.
certify's engine-tree cache-key root became an explicit seam
(_ENGINE_TREE_ROOT = paths.PACKAGE_ROOT).
- ORACLE UX: certify's clj cache-key hashing now raises CertifyError
"clj-oracle" with restore instructions when math/dev/replay.clj is
absent (was: bare FileNotFoundError). New test pins it. The
requires_math_tree guards (folded into #2643/#2647) now skip
everywhere the oracle is absent — including local runs on this tree
(5 tests: 4 certify + 1 timing probe).
- DOCKER: delphi/Dockerfile no longer COPYs polismath from its own
context; the engine source arrives via a NAMED ADDITIONAL BUILD
CONTEXT (`COPY --from=mathsrc . /math-src` + `uv pip install --no-deps
/math-src`). All four compose build blocks (docker-compose.yml +
docker-compose.test.yml x delphi/math-python) declare
`additional_contexts: mathsrc: ./math`. requirements.lock unchanged
(polismath's deps are a subset; it installs --no-deps from source).
- CI: python-ci.yml paths gain math/**; its run_math_pipeline.py cp line
repointed. Local suite after the move: 1197 passed / 28 skipped /
44 xfailed / 2 xpassed (baseline 1171/22: +31 shadow-compare tests
from Step 1, +1 oracle-error test, -5 passes now skipping as oracle
guards, +1 env-conditional skip).
- run_delphi.py invokes the pipeline as `python -m
polismath.run_math_pipeline` (was a file path under /app).
SUPPLY-CHAIN NOTE: the name "polismath" is never resolved from a package
index — Docker installs it `--no-deps` from the copied source and uv
resolves it from [tool.uv.sources]; the Dockerfile test stage's
`.[dev]` resolution finds it already installed. If the package is ever
published, register or rename first (the PyPI name is not ours).
MERGE GATE (checklist — after Step 3 has soaked):
- [ ] Verify the PROD math/delphi hosts' compose supports BuildKit named
additional build contexts (docker compose >= 2.17 with BuildKit;
after_install.sh calls /usr/local/bin/docker-compose — if that is
compose v1, upgrade it BEFORE merging this or the on-host image
build fails on `COPY --from=mathsrc`).
- [ ] python-ci green on this PR's branch (workflow_dispatch).
- [ ] Docker image builds verified once (compose build delphi
math-python) on a host/runner with BuildKit.
- [ ] Team OK that certify re-replay now requires the documented oracle
restore (cached pairs unaffected).
KNOWN/ACCEPTED:
- 10 pre-existing pyright reportArgumentType hits in certify.py /
test_certify.py (verified byte-identical code at the pre-cutover tree
— latent noise, not introduced here; not fixed to keep this PR pure
reorg).
- Docs under delphi/docs still say "delphi/polismath" in historical
narratives; forward-looking docs (README, runbook, quirks header)
follow in the wind-down docs pass / post-cutover queue.
ROLLBACK: revert this PR — pure source reorganization; no DB or runtime
coupling (the built image contains the same installed packages either
way). Nothing about the flip (Steps #2-#3) depends on this PR.
Part of the cutover PR series (delphi/docs/HANDOFF_CUTOVER_EXECUTION.md
+ CUTOVER_RUNBOOK.md). NOTHING merges without Julien's explicit go.
commit-id:45664936
…fingerprint ledger (Spec A, Phase 0) ## What The backbone of R1 certification — the effort to prove the Python engine reproduces the Clojure engine step-for-step on replayed conversations. Built by a Sonnet agent from SPEC_A: - `polismath/replay/certify.py` + `scripts/certify.py` (click CLI: `run` / `focus`) over `scripts/certify_battery.json`, the starter battery of dataset × vote-schedule entries: vw uniform8 / front-loaded6 / single-cut plus biodiversity uniform8 (vw and biodiversity are public test datasets), all run in clojure-legacy mode (the engine setting that reproduces the old Clojure engine's behavior exactly). - Recording caches: a Clojure recording is reused iff its manifest matches (votes sha256, canonical schedule hash, replay.clj + math/src tree hashes); a Python recording is keyed on the polismath tree hash + engine_mode. Both drivers are invoked as subprocesses via `--schedule <tmp>` — the only way to bake the collision-free `schedule_id` into the recording path without touching `replay_driver.py`. - Hash-first compare (canonical-JSON step hashes after the acceptance projection) plus a content-addressed step-verdict cache; a same-input re-run is a pure cache hit (verified: zero subprocess calls on repeat). - Acceptance projection = the prep-main whitelist (the keys Clojure's blob assembler `prep-main` emits into the JSON result blob) MINUS the subgroup-* trio — quirk Q7 (Clojure computes subgroup results that nothing downstream ever consumes, per `CLOJURE_QUIRKS.md`). The exclusion is printed on EVERY run, never silent. - First-divergence focuser: reports the earliest divergent step and only the divergent key-paths, capped at 40 lines of stdout, with full detail written to `focus-report.json`. Plus a fingerprint ledger, `docs/divergences.json` (normalized path × family × engine_mode) — it ships empty; diagnoses accrue as certification proceeds. - The crosslang bridge is PROMOTED from `tests/replay_harness/clj_crosslang.py` to `polismath/replay/crosslang.py` (now a production API); behavior-identical (84 pre-existing replay-harness tests green before and after), import sites updated (`test_clj_crosslang.py`, the `replay_smoke.sh` heredoc). ## Testing 132 passed, plus 1 gated integration test that ran for real: actual Clojure and Python drivers on vw single-cut, ~11s, then a cache-hit re-run. The agent's real smoke run surfaced ~19 undiagnosed divergent fingerprints on the PRE-port base — the certification loop's starting worklist (the ledger is deliberately left empty; diagnosis is top-level work). commit-id:4a87a6ae
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The backbone of R1 certification — the effort to prove the Python engine reproduces the Clojure engine step-for-step on replayed conversations. Built by a Sonnet agent from SPEC_A:
polismath/replay/certify.py+scripts/certify.py(click CLI:run/focus) overscripts/certify_battery.json, the starter battery of dataset × vote-schedule entries: vw uniform8 / front-loaded6 / single-cut plus biodiversity uniform8 (vw and biodiversity are public test datasets), all run in clojure-legacy mode (the engine setting that reproduces the old Clojure engine's behavior exactly).--schedule <tmp>— the only way to bake the collision-freeschedule_idinto the recording path without touchingreplay_driver.py.prep-mainemits into the JSON result blob) MINUS the subgroup-* trio — quirk Q7 (Clojure computes subgroup results that nothing downstream ever consumes, perCLOJURE_QUIRKS.md). The exclusion is printed on EVERY run, never silent.focus-report.json. Plus a fingerprint ledger,docs/divergences.json(normalized path × family × engine_mode) — it ships empty; diagnoses accrue as certification proceeds.tests/replay_harness/clj_crosslang.pytopolismath/replay/crosslang.py(now a production API); behavior-identical (84 pre-existing replay-harness tests green before and after), import sites updated (test_clj_crosslang.py, thereplay_smoke.shheredoc).Testing
132 passed, plus 1 gated integration test that ran for real: actual Clojure and Python drivers on vw single-cut, ~11s, then a cache-hit re-run. The agent's real smoke run surfaced ~19 undiagnosed divergent fingerprints on the PRE-port base — the certification loop's starting worklist (the ledger is deliberately left empty; diagnosis is top-level work).
commit-id:4a87a6ae
Stack: