diff --git a/delphi/docs/CLJ-PARITY-FIXES-JOURNAL.md b/delphi/docs/CLJ-PARITY-FIXES-JOURNAL.md index 8845b71e3..680d0b4ee 100644 --- a/delphi/docs/CLJ-PARITY-FIXES-JOURNAL.md +++ b/delphi/docs/CLJ-PARITY-FIXES-JOURNAL.md @@ -4470,3 +4470,16 @@ and bounded-distinct semantics (NaN rows, -0.0==0.0). - Bench FULL prod shape (33422x783x2.0M, scratch/vectorized_full.json): cold 28.15s, warm 26.66s — vs the ~31 min warm tick measured s6/s7 (~70x). The largest prod conversation now ticks in under 30s. + +### Rename (Julien ruling, s7): the python poller is engine, not delphi + +Compose service delphi-math-poller → **math-python**, profile delphi-math +→ **math-python**, env var DELPHI_MATH_ENV → **MATH_PYTHON_ENV** (default +math_env value 'delphi' → 'python'; free rename — no rows exist yet +anywhere). Living docs updated (runbook step 1, design §4, example.env); +journal history left as written. Deploy reality recorded in the runbook: +prod starts services BY NAME via scripts/after_install.sh role dispatch +(SERVICE_FROM_FILE: server|math|delphi; profiles gate dev only), prod +tracks branch `stable` — so the shadow wiring is one edit to the math +role's compose-up line, deliberately NOT made yet (Julien weighing +shadow vs clean replace). diff --git a/delphi/docs/CUTOVER_RUNBOOK.md b/delphi/docs/CUTOVER_RUNBOOK.md index 0ab51a878..7aa209435 100644 --- a/delphi/docs/CUTOVER_RUNBOOK.md +++ b/delphi/docs/CUTOVER_RUNBOOK.md @@ -82,17 +82,22 @@ equivalence evidence), CLOJURE_QUIRKS.md (Q1-Q19). ## Step 1 — shadow in prod (same day) -Infrastructure already in compose (#2625): service `delphi-math-poller`, -profile `delphi-math`, `MATH_ENV=${DELPHI_MATH_ENV:-delphi}` — distinct +Infrastructure already in compose (#2625; renamed s7 per Julien — the +math poller is engine, not delphi/UMAP): service `math-python`, +profile `math-python`, `MATH_ENV=${MATH_PYTHON_ENV:-python}` — distinct from Clojure's env, rows invisible to the server (UNIQUE(zid, math_env)). ``` -docker compose --profile delphi-math up -d delphi-math-poller -# env: DELPHI_MATH_ENV=delphi (engine has one path since the mode collapse) +docker compose --profile math-python up -d math-python +# env: MATH_PYTHON_ENV=python (engine has one path since the mode collapse) +# PROD NOTE (deploy-script reality, s7): prod instances start services BY +# NAME from scripts/after_install.sh per-role dispatch (profiles are a +# dev-only gate) — shadow on the math role = add `math-python` to its +# `docker-compose up -d math` line; prod tracks branch `stable`. ``` Verify within minutes: -- math_main rows appearing under math_env='delphi' with advancing +- math_main rows appearing under math_env='python' with advancing caching_tick; - no errorconv dumps / parked zids in the poller log; - spot-compare a few active zids' blobs vs the clojure rows (the certify diff --git a/delphi/docs/MATH_POLLER_DESIGN.md b/delphi/docs/MATH_POLLER_DESIGN.md index d94a459fe..ca055e8bb 100644 --- a/delphi/docs/MATH_POLLER_DESIGN.md +++ b/delphi/docs/MATH_POLLER_DESIGN.md @@ -79,8 +79,9 @@ math_env string written), `VOTE_POLLING_INTERVAL` (ms, default 1000), ## 4. Cutover phases -1. **Shadow (this PR):** new compose service `delphi-math-poller` (profile-gated, - `--profile delphi-math`) running next to the Clojure `math` service, writing under a +1. **Shadow (this PR):** new compose service `math-python` (profile-gated, + `--profile math-python`; renamed s7 from delphi-math-poller) running + next to the Clojure `math` service, writing under a DIFFERENT `math_env` (e.g. `MATH_ENV=delphi` while Clojure writes `prod`/`dev`). `UNIQUE(zid, math_env)` makes the rows invisible to the prod server. No consumer change, zero production risk. diff --git a/docker-compose.yml b/docker-compose.yml index 2acca4cc9..a4dbfa8a9 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -147,12 +147,12 @@ services: # Python math poller — the eventual replacement for the Clojure `math` # container. Profile-gated so it only runs when explicitly requested - # (`--profile delphi-math`). Reuses the delphi build target and overrides the + # (`--profile math-python`). Reuses the delphi build target and overrides the # command to run the poller CLI. Defaults to SHADOW mode: writes under a # DISTINCT math_env (`delphi`) so its rows are invisible to the prod server # (UNIQUE(zid, math_env)) — zero production risk while parity is validated. # See delphi/docs/MATH_POLLER_DESIGN.md §4. - delphi-math-poller: + math-python: image: 050917022930.dkr.ecr.us-east-1.amazonaws.com/polis/delphi:latest build: context: ./delphi @@ -166,7 +166,7 @@ services: - POSTGRES_CONNECT_TIMEOUT=${POSTGRES_CONNECT_TIMEOUT:-30} - LOG_LEVEL=${DELPHI_LOG_LEVEL:-INFO} # Shadow-mode math_env (distinct from the Clojure math service's MATH_ENV). - - MATH_ENV=${DELPHI_MATH_ENV:-delphi} + - MATH_ENV=${MATH_PYTHON_ENV:-python} # Poll cadences (ms) and boot window (days). - POLL_VOTE_INTERVAL_MS=${POLL_VOTE_INTERVAL_MS:-1000} - POLL_MOD_INTERVAL_MS=${POLL_MOD_INTERVAL_MS:-1000} @@ -193,7 +193,7 @@ services: memory: ${DELPHI_POLLER_CONTAINER_MEMORY:-16g} restart: unless-stopped profiles: - - delphi-math + - math-python postgres: restart: always diff --git a/example.env b/example.env index 59c7e9209..548ac0a21 100644 --- a/example.env +++ b/example.env @@ -53,15 +53,15 @@ LOCAL_SERVICES_DOCKER=true # Leave empty for autodetection on AWS deployment. See delphi/DELPHI_AUTOSCALING_SETUP.md for configuring instance size in production. INSTANCE_SIZE=dev -###### PYTHON MATH POLLER (delphi-math-poller, --profile delphi-math) ###### +###### PYTHON MATH POLLER (math-python, --profile math-python) ###### # The Python replacement for the Clojure `math` container. Runs in SHADOW mode by # default: writes math_main/math_bidtopid/math_ptptstats under a DISTINCT math_env # so its rows stay invisible to the prod server (UNIQUE(zid, math_env)) while # parity is validated. See delphi/docs/MATH_POLLER_DESIGN.md. # # math_env the poller writes under. Keep distinct from the Clojure MATH_ENV while -# shadowing; set equal to the server's MATH_ENV to cut over. Default: delphi -# DELPHI_MATH_ENV=delphi +# shadowing; set equal to the server's MATH_ENV to cut over. Default: python +# MATH_PYTHON_ENV=python # Watermark boot window: start polling from N days ago. Default 10 # POLL_FROM_DAYS_AGO=10 # Poll cadences in ms. Defaults 1000/1000