Skip to content

Decouple finalization from head ingestion#1

Merged
kalabukdima merged 1 commit into
masterfrom
net-408-decouple-finalization
Jun 18, 2026
Merged

Decouple finalization from head ingestion#1
kalabukdima merged 1 commit into
masterfrom
net-408-decouple-finalization

Conversation

@kalabukdima

@kalabukdima kalabukdima commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

Rust port of the same fix as subsquid/squid-sdk#499

Problem

The EVM hot-block data service spiked block-lag every ~6.4 min, locked to block_number % 32. Root cause: when Ethereum finalizes an epoch (~32 blocks at once), finalize_stream's prober confirmed finality and emitted a burst of finalized-only batches (blocks: vec![]) through the single stream consumer, serializing ahead of head ingestion and stalling the head-poll loop.

Fix

Take finalization off the data-loading path (crates/evm-source/src/source.rs, finalize_stream):

  • The probe_rx arm of the select! now only records the confirmed head in current_finalized — it no longer yields a {blocks: vec![], finalized_head} batch.
  • The head-batch arm already attaches current_finalized to each head batch, so the finalized head rides the next head block.

The stream now yields only head batches; the finalized head is delivered on the next one. Finality lags the head by minutes, so a sub-block delivery delay costs nothing. Probe batch size stays at 5.

This also removes a metric artifact: sqd_hotblocks_last_block_lag_ms was re-stamped on finalized-only batches with the head's stale age, producing 10–20 s gauge readings. With no finalized-only batches, the gauge is stamped only on head advances.

Validation

The TypeScript port of this change was benchmarked live (Dwellir Ethereum, 3 epochs): the % 32 epoch-lock disappeared (residues 12/13 went from 5–8 s spikes to 0 blocks >5000 ms), max blockAge dropped 13550→6000 ms, and finality advanced in correct single +32 jumps. The Rust consumer applies finalized_head and runs trigger_update/compact per batch identically, so the behavior matches. A local Rust run against the same endpoint confirms it.

🤖 Generated with Claude Code

@kalabukdima kalabukdima marked this pull request as ready for review June 18, 2026 13:05
@kalabukdima kalabukdima merged commit e9b9e2c into master Jun 18, 2026
6 checks passed
@kalabukdima kalabukdima deleted the net-408-decouple-finalization branch June 18, 2026 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant