Skip to content

Commit 7387ebf

Browse files
masihalexander-seicursoragent
authored
Stop log filters failing forever once their cursor is pruned (#4591)
A log filter whose cursor is below the earliest available block fails every `eth_getFilterChanges` with "requested fromBlock N is before earliest available block M" and never recovers. That happens for `fromBlock: "earliest"` (block 0, so even on a node that never pruned), for a numeric `fromBlock` older than the node's earliest block, and for a filter that pruning overtakes between polls. `eth_subscribe("logs")` errors and ends the same way. The cursor is handed to the strict bounds check as is, and since a failed poll does not move it, every later poll fails on the same height. `eth_newFilter` and `eth_subscribe` now reject a numeric `fromBlock` above 0 that is already below the earliest available block, with the same error `eth_getLogs` gives for that range, so the client learns at creation that the history it asked for is gone. A `fromBlock` of 0, which is also what `earliest` resolves to, asks for whatever the node has and polls from the earliest block. When pruning passes the cursor of a live filter, the next poll fails once with "logs of blocks X to Y were pruned before the filter read them" and moves the cursor to the earliest block, so the following poll carries on from there; a subscription notifies that error and ends. The evmrpc guide describes the new behaviour. Fixes PLT-1520 --------- Co-authored-by: alexander-sei <alexanderh@seinetwork.io> Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 0dc0b4a commit 7387ebf

5 files changed

Lines changed: 214 additions & 10 deletions

File tree

‎evmrpc/AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ EVM RPCs prefixed by `eth_` and `debug_` on Sei generally follows [Ethereum's sp
4242
- `eth_newPendingTransactionFilter`
4343
- `eth_syncing`
4444
- `eth_getProof` — deprecated rather than permanently incompatible; the message directs callers who need proofs to the Sei team.
45-
- **Log filters poll from a cursor.** `eth_newFilter` fixes where its polls start, once: a numeric `fromBlock` as given, otherwise the block after the head at creation; a numeric `toBlock` below that start is rejected. Each `eth_getFilterChanges` scans from that cursor to the head, or to a numeric `toBlock` below it; a tag `toBlock` such as `latest` follows the head and never ends the filter. A poll covers at most `max_blocks_for_log` blocks, and fewer when its logs exceed the log caps; later polls return the rest. A block whose logs alone exceed the caps fails the poll that reaches it with an error naming the block, and the filter or subscription continues after it. Unlike geth, a numeric `fromBlock` in the past makes the first poll return that history. `eth_getFilterLogs` resolves tags when called and never moves the cursor. `eth_subscribe("logs")` polls through the same cursor logic, and treats a `fromBlock` of 0 with a tag `toBlock`, which ethclient sends for an open-ended query, as an omitted `fromBlock`. A failed subscription poll notifies `{"error": message}` in place of logs, adding the skipped block's `blockNumber` when the subscription continues past an oversized block; any other failure ends the subscription.
45+
- **Log filters poll from a cursor.** `eth_newFilter` fixes where its polls start, once: a numeric `fromBlock` (or `earliest`, which is block 0) as given, otherwise the block after the head at creation; a numeric `toBlock` below that start is rejected. Each `eth_getFilterChanges` scans from that cursor to the head, or to a numeric `toBlock` below it; a tag `toBlock` such as `latest` follows the head and never ends the filter. A numeric `fromBlock` above 0 that is below the earliest available block is rejected, while a `fromBlock` of 0 polls from that block. When pruning passes the cursor of a filter, its next poll fails with an error naming the pruned blocks, and the filter continues from the earliest block. A poll covers at most `max_blocks_for_log` blocks, and fewer when its logs exceed the log caps; later polls return the rest. A block whose logs alone exceed the caps fails the poll that reaches it with an error naming the block, and the filter or subscription continues after it. Unlike geth, a numeric `fromBlock` in the past makes the first poll return that history. `eth_getFilterLogs` resolves tags when called and never moves the cursor. `eth_subscribe("logs")` polls through the same cursor logic, and treats a `fromBlock` of 0 with a tag `toBlock`, which ethclient sends for an open-ended query, as an omitted `fromBlock`. A failed subscription poll notifies `{"error": message}` in place of logs, adding the skipped block's `blockNumber` when the subscription continues past an oversized block; any other failure, pruned blocks included, ends the subscription.
4646

4747
## `sei_` prefixed endpoints
4848
The legacy `sei_` namespace contains address and Cosmos transaction helpers.

‎evmrpc/filter.go‎

Lines changed: 46 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1037,7 +1037,8 @@ func (f *LogFetcher) GetLogsByFilters(ctx context.Context, crit filters.FilterCr
10371037

10381038
// pollStart returns the cursor a new log filter starts polling from: crit's fromBlock when it
10391039
// is a number, and otherwise the block after the current head. A blockHash crit starts at 0. It
1040-
// rejects a numeric toBlock below that start.
1040+
// rejects a numeric fromBlock above 0 that is below the earliest available block, and a numeric
1041+
// toBlock below the start.
10411042
func (f *LogFetcher) pollStart(ctx context.Context, crit filters.FilterCriteria) (int64, error) {
10421043
if crit.BlockHash != nil {
10431044
return 0, nil
@@ -1049,18 +1050,39 @@ func (f *LogFetcher) pollStart(ctx context.Context, crit filters.FilterCriteria)
10491050
start := head + 1
10501051
if crit.FromBlock != nil && crit.FromBlock.Sign() >= 0 {
10511052
start = crit.FromBlock.Int64()
1053+
if err := f.ensureStartRetained(ctx, start); err != nil {
1054+
return 0, err
1055+
}
10521056
}
10531057
if crit.ToBlock != nil && crit.ToBlock.Sign() >= 0 && crit.ToBlock.Int64() < start {
10541058
return 0, fmt.Errorf("requested toBlock %d is before the filter's start block %d", crit.ToBlock.Int64(), start)
10551059
}
10561060
return start, nil
10571061
}
10581062

1063+
// ensureStartRetained rejects a filter start above 0 that is below the earliest available block.
1064+
func (f *LogFetcher) ensureStartRetained(ctx context.Context, start int64) error {
1065+
if start == 0 {
1066+
return nil
1067+
}
1068+
earliest, err := f.earliestHeight(ctx)
1069+
if err != nil {
1070+
return err
1071+
}
1072+
if start < earliest {
1073+
return fmt.Errorf("requested fromBlock %d is before earliest available block %d", start, earliest)
1074+
}
1075+
return nil
1076+
}
1077+
10591078
// pollLogs returns the logs crit matches from cursor on, and the cursor the next poll starts
1060-
// from, also when it returns an error. A poll scans up to the head, or to crit's toBlock when it
1061-
// is a number below the head, and scans fewer blocks when that range exceeds the block or log
1062-
// caps. A block whose logs alone exceed the log caps fails the poll that reaches it, and the
1063-
// returned cursor is past it. A blockHash crit's block is returned by the poll at cursor 0.
1079+
// from, also when it returns an error. A poll scans from cursor up to the head, or to crit's
1080+
// toBlock when it is a number below the head, and scans fewer blocks when that range exceeds the
1081+
// block or log caps. A cursor of 0 scans from the earliest available block. A cursor above 0 that
1082+
// is below that block fails the poll with an error naming the pruned blocks, and the returned
1083+
// cursor is the earliest block. A block whose logs alone exceed the log caps fails the poll that
1084+
// reaches it, and the returned cursor is past it. A blockHash crit's block is returned by the
1085+
// poll at cursor 0.
10641086
func (f *LogFetcher) pollLogs(ctx context.Context, crit filters.FilterCriteria, cursor int64) ([]*ethtypes.Log, int64, error) {
10651087
if logsExhausted(crit, cursor) {
10661088
return []*ethtypes.Log{}, cursor, nil
@@ -1080,6 +1102,15 @@ func (f *LogFetcher) pollLogs(ctx context.Context, crit filters.FilterCriteria,
10801102
if err != nil {
10811103
return nil, cursor, err
10821104
}
1105+
if earliest, err := f.earliestHeight(ctx); err == nil && cursor < earliest {
1106+
// Pruned blocks cannot be served on any later poll either, so the filter moves on to the
1107+
// earliest block. A filter from block 0 asked for whatever the node has, so only a filter
1108+
// that lost blocks it was due is told.
1109+
if cursor > 0 {
1110+
return nil, earliest, prunedLogsError(crit, cursor, earliest)
1111+
}
1112+
cursor = earliest
1113+
}
10831114
from, to, ok := pollWindow(crit.ToBlock, cursor, head, f.filterConfig.maxBlock)
10841115
if !ok {
10851116
return []*ethtypes.Log{}, cursor, nil
@@ -1123,6 +1154,16 @@ func logsExhausted(crit filters.FilterCriteria, cursor int64) bool {
11231154
return crit.ToBlock != nil && crit.ToBlock.Sign() >= 0 && cursor > crit.ToBlock.Int64()
11241155
}
11251156

1157+
// prunedLogsError reports the blocks from cursor up to earliest, or up to crit's numeric toBlock
1158+
// when it is lower, whose logs were pruned before a filter polled them.
1159+
func prunedLogsError(crit filters.FilterCriteria, cursor, earliest int64) error {
1160+
last := earliest - 1
1161+
if crit.ToBlock != nil && crit.ToBlock.Sign() >= 0 {
1162+
last = min(last, crit.ToBlock.Int64())
1163+
}
1164+
return fmt.Errorf("logs of blocks %d to %d were pruned before the filter read them", cursor, last)
1165+
}
1166+
11261167
// exceedsLogCaps reports whether err is a query failing on the log count or byte caps.
11271168
func exceedsLogCaps(err error) bool {
11281169
return errors.Is(err, receipt.ErrTooManyLogs) || errors.Is(err, receipt.ErrTooManyLogBytes)

‎evmrpc/filter_bounds_test.go‎

Lines changed: 50 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -119,12 +119,15 @@ func TestComputeBlockBounds(t *testing.T) {
119119
}
120120

121121
// TestPollStart pins where a new log filter's polls start: a numeric fromBlock is kept, an
122-
// omitted or tag fromBlock starts after the head at creation, and a numeric toBlock below the
123-
// start is rejected.
122+
// omitted or tag fromBlock starts after the head at creation, and a numeric fromBlock above 0
123+
// below the earliest block or a numeric toBlock below the start is rejected.
124124
func TestPollStart(t *testing.T) {
125125
t.Parallel()
126126

127-
const head = int64(100)
127+
const (
128+
earliest = int64(30)
129+
head = int64(100)
130+
)
128131
blockHash := common.HexToHash("0x01")
129132
tests := []struct {
130133
name string
@@ -140,6 +143,17 @@ func TestPollStart(t *testing.T) {
140143
{name: "tag toBlock leaves the start after the head", crit: filters.FilterCriteria{ToBlock: latestTag}, want: head + 1},
141144
{name: "earliest fromBlock starts at block 0", crit: filters.FilterCriteria{FromBlock: earliestTag}, want: 0},
142145
{name: "past fromBlock starts there", crit: filters.FilterCriteria{FromBlock: big.NewInt(40)}, want: 40},
146+
{name: "fromBlock at the earliest block starts there", crit: filters.FilterCriteria{FromBlock: big.NewInt(earliest)}, want: earliest},
147+
{
148+
name: "fromBlock below the earliest block is rejected",
149+
crit: filters.FilterCriteria{FromBlock: big.NewInt(earliest - 1)},
150+
errContains: "requested fromBlock 29 is before earliest available block 30",
151+
},
152+
{
153+
name: "pruned fromBlock with a tag toBlock is rejected",
154+
crit: filters.FilterCriteria{FromBlock: big.NewInt(1), ToBlock: latestTag},
155+
errContains: "requested fromBlock 1 is before earliest available block 30",
156+
},
143157
{name: "fromBlock at the head starts there", crit: filters.FilterCriteria{FromBlock: big.NewInt(head)}, want: head},
144158
{name: "future fromBlock starts there", crit: filters.FilterCriteria{FromBlock: big.NewInt(150)}, want: 150},
145159
{name: "numeric fromBlock with tag toBlock starts at fromBlock", crit: filters.FilterCriteria{FromBlock: big.NewInt(40), ToBlock: latestTag}, want: 40},
@@ -163,7 +177,7 @@ func TestPollStart(t *testing.T) {
163177
{name: "blockHash filter starts at zero", crit: filters.FilterCriteria{BlockHash: &blockHash}, want: 0},
164178
}
165179

166-
fetcher := &LogFetcher{watermarks: newHeightTestWatermarks(newHeightTestClient(0, 1, head), head)}
180+
fetcher := &LogFetcher{watermarks: newHeightTestWatermarks(newHeightTestClient(0, earliest, head), head)}
167181
for _, tc := range tests {
168182
t.Run(tc.name, func(t *testing.T) {
169183
t.Parallel()
@@ -178,6 +192,38 @@ func TestPollStart(t *testing.T) {
178192
}
179193
}
180194

195+
// TestPollLogsReportsPrunedBlocks pins that a poll whose cursor pruning has passed fails with an
196+
// error naming the pruned blocks, up to a lower numeric toBlock, and moves the cursor to the
197+
// earliest block.
198+
func TestPollLogsReportsPrunedBlocks(t *testing.T) {
199+
t.Parallel()
200+
201+
const (
202+
earliest = int64(30)
203+
head = int64(100)
204+
)
205+
tests := []struct {
206+
name string
207+
crit filters.FilterCriteria
208+
errContains string
209+
}{
210+
{name: "open filter", errContains: "logs of blocks 10 to 29 were pruned before the filter read them"},
211+
{name: "tag toBlock", crit: filters.FilterCriteria{ToBlock: latestTag}, errContains: "logs of blocks 10 to 29 were pruned"},
212+
{name: "toBlock below the earliest block", crit: filters.FilterCriteria{ToBlock: big.NewInt(20)}, errContains: "logs of blocks 10 to 20 were pruned"},
213+
}
214+
215+
fetcher := &LogFetcher{watermarks: newHeightTestWatermarks(newHeightTestClient(0, earliest, head), head)}
216+
for _, tc := range tests {
217+
t.Run(tc.name, func(t *testing.T) {
218+
t.Parallel()
219+
logs, cursor, err := fetcher.pollLogs(t.Context(), tc.crit, 10)
220+
require.ErrorContains(t, err, tc.errContains)
221+
require.Nil(t, logs)
222+
require.Equal(t, earliest, cursor)
223+
})
224+
}
225+
}
226+
181227
// TestPollWindow pins the blocks a log filter's poll scans: from its cursor to the head or a
182228
// lower numeric toBlock, with tags following the head, and at most maxBlock blocks.
183229
func TestPollWindow(t *testing.T) {

‎evmrpc/filter_test.go‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -872,6 +872,51 @@ func TestFilterGetFilterChangesAfterFromBlockIsPruned(t *testing.T) {
872872
require.Equal(t, want, filterChanges(t, filterID))
873873
}
874874

875+
// TestFilterGetFilterChangesReportsPrunedBlocks asserts that eth_newFilter
876+
// rejects a numeric fromBlock above 0 below the earliest available block, that
877+
// a filter from block 0 polls from the earliest block, and that a filter whose
878+
// cursor pruning passes reports the pruned blocks once and then polls from the
879+
// earliest block.
880+
// This test is intentionally not parallel because it moves the latest and
881+
// earliest heights of the mock client shared by the package's HTTP server.
882+
func TestFilterGetFilterChangesReportsPrunedBlocks(t *testing.T) {
883+
t.Cleanup(func() {
884+
filterClient.latestOverride = 0
885+
filterClient.earliestOverride = 0
886+
})
887+
filterClient.latestOverride = 1
888+
lagging := newLogFilter(t, map[string]interface{}{"fromBlock": "0x1"})
889+
require.Empty(t, filterChanges(t, lagging))
890+
891+
// Block 2 holds the mock chain's logs, so a filter that polls from the
892+
// earliest block returns them.
893+
filterClient.earliestOverride = 2
894+
filterClient.latestOverride = 3
895+
earliestTag := newLogFilter(t, map[string]interface{}{"fromBlock": "earliest"})
896+
want := logsInRange(t, 2, 3)
897+
require.NotEmpty(t, want)
898+
require.Equal(t, want, filterChanges(t, earliestTag))
899+
for _, crit := range []map[string]interface{}{
900+
{"fromBlock": "0x1"},
901+
{"fromBlock": "0x1", "toBlock": "latest"},
902+
{"fromBlock": "0x1", "toBlock": "0x1"},
903+
} {
904+
resObj := sendRequestGood(t, evmrpc.NewFilterMethod, crit)
905+
errObj, ok := resObj["error"].(map[string]interface{})
906+
require.True(t, ok, "newFilter %v should fail, got: %v", crit, resObj)
907+
require.Equal(t, "requested fromBlock 1 is before earliest available block 2", errObj["message"])
908+
}
909+
910+
// Pruning passes the cursor of the filter last polled at head 1.
911+
filterClient.earliestOverride = 4
912+
filterClient.latestOverride = MockHeight8
913+
resObj := sendRequestGood(t, evmrpc.GetFilterChangesMethod, lagging)
914+
errObj, ok := resObj["error"].(map[string]interface{})
915+
require.True(t, ok, "expected an error object, got: %v", resObj)
916+
require.Equal(t, "logs of blocks 2 to 3 were pruned before the filter read them", errObj["message"])
917+
require.Equal(t, logsInRange(t, 4, MockHeight8), filterChanges(t, lagging))
918+
}
919+
875920
// TestFilterGetFilterChangesMovesPastAnOversizedBlock asserts that a poll
876921
// failing on a block whose logs alone exceed the log caps moves the filter past
877922
// that block, so the next poll does not fail on it again.
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
package evmrpc
2+
3+
import (
4+
"context"
5+
"encoding/json"
6+
"sync/atomic"
7+
"testing"
8+
"time"
9+
10+
"github.com/ethereum/go-ethereum/rpc"
11+
"github.com/sei-protocol/sei-chain/sei-tendermint/rpc/coretypes"
12+
"github.com/stretchr/testify/require"
13+
)
14+
15+
// pruningTestHeights are the earliest and latest block heights a pruningTestClient reports.
16+
type pruningTestHeights struct {
17+
earliest, latest int64
18+
}
19+
20+
// pruningTestClient is a heightTestClient whose earliest and latest heights move together, so a
21+
// poll never sees one without the other.
22+
type pruningTestClient struct {
23+
*heightTestClient
24+
heights atomic.Pointer[pruningTestHeights]
25+
}
26+
27+
func (c *pruningTestClient) Status(context.Context) (*coretypes.ResultStatus, error) {
28+
heights := c.heights.Load()
29+
return &coretypes.ResultStatus{
30+
SyncInfo: coretypes.SyncInfo{
31+
LatestBlockHeight: heights.latest,
32+
EarliestBlockHeight: heights.earliest,
33+
},
34+
}, nil
35+
}
36+
37+
// TestSubscribeLogsEndsWhenPruningPassesItsCursor asserts that a logs subscription whose cursor
38+
// pruning passes notifies an error naming the pruned blocks and ends.
39+
func TestSubscribeLogsEndsWhenPruningPassesItsCursor(t *testing.T) {
40+
t.Parallel()
41+
tmClient := &pruningTestClient{heightTestClient: newHeightTestClient(0, 1, 100)}
42+
tmClient.heights.Store(&pruningTestHeights{earliest: 1, latest: 100})
43+
api := &SubscriptionAPI{
44+
logFetcher: &LogFetcher{filterConfig: &FilterConfig{}, watermarks: newHeightTestWatermarks(tmClient, 1000)},
45+
subscriptonConfig: &SubscriptionConfig{logLimit: 1},
46+
}
47+
server := rpc.NewServer()
48+
require.NoError(t, server.RegisterName("eth", api))
49+
client := rpc.DialInProc(server)
50+
t.Cleanup(func() {
51+
client.Close()
52+
server.Stop()
53+
})
54+
55+
// The subscription starts past the head, so a poll before pruning scans nothing.
56+
notifications := make(chan json.RawMessage, 4)
57+
sub, err := client.EthSubscribe(t.Context(), notifications, "logs", map[string]interface{}{"fromBlock": "0x6e"})
58+
require.NoError(t, err)
59+
t.Cleanup(sub.Unsubscribe)
60+
tmClient.heights.Store(&pruningTestHeights{earliest: 120, latest: 130})
61+
62+
select {
63+
case raw := <-notifications:
64+
var notification map[string]string
65+
require.NoError(t, json.Unmarshal(raw, &notification), "notification %s", raw)
66+
require.Equal(t, map[string]string{"error": "logs of blocks 110 to 119 were pruned before the filter read them"}, notification)
67+
case <-time.After(2 * SleepInterval):
68+
t.Fatal("no notification for the pruned blocks")
69+
}
70+
require.Eventually(t, func() bool { return api.logSubsCount.Load() == 0 }, 2*SleepInterval, 10*time.Millisecond,
71+
"the subscription should end after notifying the pruned blocks")
72+
}

0 commit comments

Comments
 (0)