Repository navigation
Commit 792ff53
Apply compiled KV repair files at a fixed height and generate them from digest inspect lists (#4400)
## Summary
The app can now apply reviewed key/value repairs to module stores at a
fixed height, and `seidb kvrepair-export` generates the repair file from
two `evm-logical-digest` inspect lists. Each repair registers a handler
with the existing `HardForkManager`, so it runs in `BeginBlock` at that
height, with no governance proposal. A repair file is a
consensus-breaking change: every validator must run a binary that holds
the file before its height, and a node without the file stops at the
next block. The directory `app/upgrades/kvrepair/repairs/` is empty, so
this PR registers no handler and does not change behaviour. The file
format and the generation steps are in
`app/upgrades/kvrepair/repairs/README.md`.
The intended use is on a stopped chain. The chain stops at `H`. The
operator lists the keys in each bucket that the digest reports as
different, on a memiavl reserve and on a production node at `H`.
`kvrepair-export` writes a file with `target_repair_height` `H+1` and
`state_height` `H`. The nodes restart with the new binary and continue
in composite mode, with the EVM migration still in progress.
- `sei-db/common/kvrepair/repair.go`: the file format. `Parse` refuses
unknown fields, a field name used twice in one object, a field name that
is not lowercase ASCII (encoding/json matches names without regard to
case), data after the JSON value, bad hex, a missing `new`, a `null`
old, and an entry with both `old` and `old_absent`. `Validate` refuses
an empty name or chain ID, a `target_repair_height` that is not
positive, a `state_height` that is not positive or not below
`target_repair_height`, an empty entry list, an unknown store, an empty
key, and a key twice in one repair. An entry without an old value is
allowed only when `state_height` is `target_repair_height - 1`. `Encode`
writes one entry per line, so a review diff shows one line for each key.
- `sei-db/common/kvrepair/values.go`: `ValuesEqual` and `ReadsAsAbsent`.
In the `evm` store, an all-zero storage slot, nonce, code hash, or
balance, and an empty code value, read as absent. Every other value
compares exactly, so `""` differs from an absent key, and code made of
zero bytes is real code.
- `app/upgrades/kvrepair/kvrepair.go`: `Register` loads every embedded
file, validates it against the app's store keys, refuses a name used
twice and a key that two repairs touch at one height, and registers one
handler for each repair. The hard fork manager skips repairs for other
chains. An invalid file panics at start.
- `app/upgrades/kvrepair/kvrepair.go` (`Handler.ExecuteHandler`): for
each entry, the handler checks the current value, writes `new` (or
deletes the key), and reads it back. An entry whose key already holds
`new` skips its old value check and is still written, so a node with
correct state commits the same changeset as a repaired node. Any other
entry must find its old value (`old`, or no value with `old_absent`).
These comparisons use `ValuesEqual`, because a FlatKV account row
returns a zero nonce for an account whose nonce was deleted while
another field remains, where memiavl returns no value. A failed old
value check or read-back returns an error, and the hard fork manager
then panics in `BeginBlock`, so the node stops at that height and does
not commit the block.
- `app/app.go`: calls `kvrepair.Register` after the hard fork manager is
created.
- `sei-db/tools/cmd/seidb/operations/kvrepair_export.go`: `seidb
kvrepair-export --reserve --prod --chain-id --name
--target-repair-height` reads two inspect list reports and writes a
repair file. It refuses a reserve that is not memiavl or whose mode is
not `semantic` or `semantic-replay` (a translator report shares the
migration mapping), a production report that is not composite or FlatKV,
two reports that differ in height, bucket, offset, or prefix, a report
that is not a list or that `--list-limit` cut off, an unknown bucket, a
non-EVM physical key, a key listed twice, a target repair height that is
not above the report height, and reports with no differences. For each
key whose rows differ, `new` is the reserve value (`null` when it reads
as absent) and `old` is the production value (`old_absent` when it reads
as absent), so every entry has an old value. An all-zero row that only
one side holds is deleted. An account row is split into the balance,
nonce, and code hash fields that differ. Before it writes, the command
encodes, parses, and validates the file as a node would.
- `sei-db/tools/cmd/seidb/main.go`: registers `kvrepair-export`.
- `sei-db/tools/cmd/seidb/operations/evm_logical_digest.go`: the inspect
usage example gains the `--key-offset 4` that its key prefix needs.
`.dockerignore` removes `*.md`, so the directory is embedded with
`//go:embed all:` and holds a `.gitkeep`.
## Test plan
- `sei-db/common/kvrepair/repair_test.go`: every invalid form listed
above, including a second object or trailing garbage, a field twice in
an entry or at the top, a field name in another case or with a rune that
folds to a lowercase letter, and an entry without an old value across a
gap. A repair with a gap loads when every entry has an old value. `""`
and `"0x"` load as empty values, distinct from `null`. `Encode` writes
one entry per line, omits an empty source, and round-trips through
`Parse`.
- `sei-db/common/kvrepair/values_test.go`: zero and absent compare as
equal for each EVM field kind. Empty code reads as absent and zero-byte
code does not. Misc keys, malformed storage keys, and non-EVM stores
compare exactly.
- `app/upgrades/kvrepair/kvrepair_test.go`: the embedded directory
loads, and the loader refuses a name used twice and two repairs on one
key at one height.
- The handler sets and deletes keys at the target height. An entry that
already holds `new` is written, and its old value check is skipped, as a
store trace shows.
- The handler fails and leaves the store unchanged for a current value
that differs from `old`, a present key with `old_absent`, and a missing
key with `old`. With no old value, the entry writes over any value.
- A zero storage slot counts as holding a delete, and a zero nonce
matches `old_absent`. A different non-zero value still fails. Misc and
non-EVM keys compare exactly.
- `sei-db/tools/cmd/seidb/operations/kvrepair_export_test.go`: storage
sets, deletes, and creates. A zero row that only production holds is
deleted. Code and misc keys map to their store keys. An account entry
writes only the changed field. An account that only the reserve holds
gets entries for its non-zero fields, and an all-zero account row that
only production holds is deleted field by field. The command refuses
each bad input listed above, and the command writes a file that loads.
An end-to-end test inspects a real memiavl reserve and a real FlatKV
store, exports every bucket, applies the entries to the FlatKV store,
and inspects again: the new reports list no differences.
- `sei-db/state_db/sc/composite/store_repair_migration_test.go`: one
block during the EVM migration writes keys that are already in FlatKV,
keys still in memiavl, keys that the same block's migration batch moves,
and an account whose nonce is still in memiavl while its balance goes to
FlatKV. Each key reads back the written value after the block and after
the migration completes, and the FlatKV LtHash verifies.
- Manual, stopped chain, on a harbor chain with 4 composite validators
and a memIAVL-only reserve (fixture branch
`yiren/a8-kvrepair-fixture-v2`, not for merge). A fixture seeds EVM
state at 30, starts the migration at 60, and damages state on the
validators at 300 while the migration is in progress: two storage slots
set, two deleted, one added, a contract's code replaced, one nonce
changed, and one nonce set to zero. The chain stoped with
`stopped-height` and stopped at 803 with the migration in progress.
`kvrepair-export` on inspect lists from the reserve and a validator at
803 wrote 11 entries in 4 files (account, code, storage, misc),
including the zero-storage cleanup checkpoint (`0x1e`), which the
storage damage also moved. The migration markers were left out. With the
files compiled in and the repair at 804:
- Every validator logged `repaired=11 already_correct=0`, and the
reserve logged `repaired=0 already_correct=11`.
- The `changeset` differed between a validator and the reserve at 471
heights from 300 to 803, and matched at every height from 804 to 990.
Two validators had identical hash logs from 1 to 990.
- The migration continued from its boundary and completed at about 1070.
- At 1200, the EVM logical digest of a validator (FlatKV) and the
reserve (memIAVL) matched in every bucket, count and digest.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>1 parent e8ddc72 commit 792ff53
14 files changed
Lines changed: 2007 additions & 5 deletions
File tree
- app
- upgrades/kvrepair
- repairs
- sei-db
- common/kvrepair
- state_db/sc/composite
- tools/cmd/seidb
- operations
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
102 | 102 | | |
103 | 103 | | |
104 | 104 | | |
| 105 | + | |
105 | 106 | | |
106 | 107 | | |
107 | 108 | | |
| |||
989 | 990 | | |
990 | 991 | | |
991 | 992 | | |
| 993 | + | |
992 | 994 | | |
993 | 995 | | |
994 | 996 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
0 commit comments