Skip to content

fix(indexer): bound the startup discovery-adapter sync — resume-capable checkpointed pages (#218)#219

Merged
tayy-i merged 1 commit into
mainfrom
fix/218-bounded-startup-discovery-sync
Jul 21, 2026
Merged

fix(indexer): bound the startup discovery-adapter sync — resume-capable checkpointed pages (#218)#219
tayy-i merged 1 commit into
mainfrom
fix/218-bounded-startup-discovery-sync

Conversation

@tayy-i

@tayy-i tayy-i commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Closes #218 — the live deploy blocker: the post-drain startup discovery sync ran the unbounded full-source ENSv1 derivation (whole-span assignment map with raw-log clones + in-memory reconcile), peaking at ~55–85GB on the Base pass and kernel-OOM-looping the indexer every ~15 minutes at post-catch-up two-chain scale. Regression vector was #141 putting the pre-existing unbounded path on every auto boot; the bounded #168/#173 machinery existed but the startup entrypoint bypassed it.

Change

  • Both families sharing the unbounded shape converted — ENSv1 subregistry discovery (focused + broad startup) and ENSv1 unwrapped authority (broad startup). Reverse-claim and ENSv2 paths don't share the shape and are unchanged; inline sync and replay catch-up untouched.
  • Startup-scoped checkpoint context: keyed by chain, deployment profile, adapter, cursor startup_adapter_owned_raw_log_state, scope startup_adapter_sync, fixed range start 0, latched target — collision-proof against replay's full_closure scope, and fenced by discovery_admission_epochs.
  • Bounded memory: the subregistry path is O(page) — 10,000-log page-local staging, streamed reconciliation (existing limits incl. BIGNAME_INDEXER_DISCOVERY_FULL_RECONCILE_MAX_DEACTIVATIONS now apply), checkpoint-paged event emission; unwrapped-authority events stay checkpoint-private until identity materialization then publish in 20,000-event transactions. (Per the adversarial review: the authority path is O(name-state) — ~single-digit GB/chain, byte-identical to the already-shipped replay-lane residency, not a new accumulation — versus the ~55–85GB O(span) class this PR removes; the dominant O(matched-logs) event term is what moved to DB staging.)
  • Resume, not restart: a killed boot resumes from its checkpoint; successful startup cleans the checkpoints up; authority changes invalidate retained state safely.
  • No more silent minutes: progress INFO on the first page and every 25 pages (the 6–9 minute silence was a diagnosis obstacle).

Verification

Equivalence: exact edge/event parity with the former full pass. Kill/resume audit: page counters [1, 2, 3] — no rescan from zero. Page-bounded staging-row assertions; cleanup, higher-target resume, authority-drift, empty-source, event-visibility coverage. Adapters 304 / indexer 514 / manifests 109 / CI-equivalent e2e 90 — all passed; fmt/clippy/file-size clean (two allowlisted files shrank). Codex $verify-loop: definitive CLEAN.

Deploy note: the live indexer currently runs the documented raw-only stopgap; this fix rides the next image, after which the stopgap comes off and full adapter derivation + replay catch-up resume.

🤖 Generated with Claude Code

https://claude.ai/code/session_01C9RTnbCyhqEUSPc7xCK76p

Route startup ENSv1 discovery-derived full-source passes through distinct resumable checkpoints and keep authority page events private until identity materialization.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01C9RTnbCyhqEUSPc7xCK76p
@tayy-i

tayy-i commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator Author

Adversarial review gate (two lenses on 8d05751) — no P1; one P2 (an honesty correction, applied to the PR body), ops-monitoring P3s recorded, residual lanes filed as #220.

Fable pass — the load-bearing boundedness claim verified with quantified constants. Subregistry: genuinely O(page) (fresh 10k-log map per page, latch-bounded index-supported scans, the #173 streamed reconcile with its SQL-COUNT coarse cap enforced before any in-memory load, 20k/5k emit paging; the one cross-page accumulator is the migrated-node guard, bounded by the 2020 eth registry migration — tens of MB). Authority: O(name-state), ~single-digit GB/chain — identical to the already-shipped replay-lane residency, no new accumulation; the O(matched-logs) event term (the OOM's dominant component) moved to DB staging. Equivalence: the latch uses the same canonicality set over the same scan population as the old pass; boot sequencing makes mid-pass writes impossible; mid-span corpus changes between boots are caught by the trigger-maintained revision journal's reset check (not just target monotonicity); the fixed-point repeat makes the startup pass at least as complete as the old one. Lifecycle: epoch fence threaded through every checkpoint writer (surgical jsonb_set in the publish loop) with reset-on-drift tests via the real sync_repository path; startup/replay lane isolation scope-bound per query with PK+FK+CASCADE; the private-event publish is atomic per 20k page (upsert + staged-delete + counter in one tx — kill/resume can neither double-publish nor orphan). The equivalence test is a genuine two-database cross-path comparison with deep JSON equality; kill/resume proven by a DB-trigger page audit. 11/11 new + 2 modified tests re-run green.

Gemini pass adjudicated: cross-page accumulation — confirmed-and-quantified (the guard is tens of MB; the real residency is the authority name-state, now correctly framed); resume re-latch — refuted (fresh-boot-equivalent semantics + revision-journal reset); missing epoch in the resume context — refuted (the latched epoch lives in state_payload and survives every writer, test-proven); non-atomic flush/checkpoint — refuted (idempotent identity upserts + atomic publish pages, per the documented crash-window semantics); cleanup leak — refuted-as-designed (partial retention is the resume feature; cleanup is scope-bound with a completed-status guard).

Ops watch items for the first fixed boot (recorded in #220's tail): fixed-point double scan + ~25–50GB transient checkpoint-staging growth across both chains on a 94%-full disk, then the cascade-delete/autovacuum tail — disk monitored during the roll. Deactivation caps cannot trip spuriously on the raw-only-frozen edge state (candidates ≈ 0).

Merging on CI green + connector triage; then image rebuild, stopgap off, full deploy resume.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

startup discovery-adapter sync runs the unbounded full-source ENSv1 derivation on every auto boot — kernel OOM at two-chain scale (#141 regression)

2 participants