Repository navigation
Commit a25c7ef
feat(flatkv): combine the post-evm flatkv migration phases into one MigrateCosmos (#4561)
## Summary
Phase 2 and phase 3 of the memiavl -> flatkv migration are merged into a
single transition, `MigrateCosmos`. It moves every non-`evm/` module,
`bank/` included, from memiavl to flatkv in one pass. The migration
chain becomes:
`memiavl_only -> migrate_evm -> evm_migrated -> migrate_cosmos ->
flatkv_only`
Migration version 2 is now the `flatkv_only` terminal state. The
`migrate_all_but_bank`, `all_migrated_but_bank` and `migrate_bank` modes
are removed, and version 3 is unknown. In v6.7, version 2 meant
all-but-bank. Reusing it is safe because no node in sync with a network
can hold that layout: v6.7 starts at runtime only `memiavl_only ->
migrate_evm`, so an auto node never entered `migrate_all_but_bank`. A
node pinned to it would have migrated alone and failed AppHash
verification at its first migrating block. Keeping `flatkv_only` at
version 3 was tried and reverted. Only a node that finished all-but-bank
alone can hold v6.7's version 2, and a diverging node halts at the next
block, so the gap would guard a state that cannot exist.
The existing `NumKeysToMigratePerBlock` gov param paces the new
migration, so phase 1's governance flow is reused as is. A new
chain-state param, `MigrateCosmosEnabled`, gates the step past EVM. The
v6.8 upgrade handler and `InitChainer` both set it, so upgraded chains
and chains started from genesis on this binary can both run the
migration.
Rollout: governance has already set `NumKeysToMigratePerBlock` back to
0, so v6.8 opens the gate while the migration stays paused.
`MigrateCosmos` starts at the first block after a gov proposal raises
the rate, the same flow v6.7 used for `MigrateEVM`. With a rate above 0
at the upgrade, it would start at the upgrade height + 1, because
`BeginBlock` reads the gate before the upgrade handler runs.
- `sei-db/state_db/sc/types/write_mode.go`: `MigrateCosmos` replaces the
three old modes. `ValidateTransition` admits exactly four edges:
`MemiavlOnly -> MigrateEVM`, `MigrateEVM -> EVMMigrated`, `EVMMigrated
-> MigrateCosmos`, `MigrateCosmos -> FlatKVOnly`.
- `sei-db/state_db/sc/migration/migration_versions.go`: versions are `0`
memiavl only, `1` EVM migrated, `2` flatkv only.
- `sei-db/state_db/sc/migration/derive_write_mode.go`: version 1 with an
in-flight boundary derives `MigrateCosmos`. Version 2 derives
`FlatKVOnly`. A boundary at version 2 is rejected as corrupt metadata,
and any other version is rejected as unknown.
`IsModeComplete(MigrateCosmos)` is true once version 2 is persisted.
- `sei-db/state_db/sc/migration/router_builder.go`:
`buildMigrateCosmosRouter` routes `evm/` straight to flatkv and every
other module through one `MigrationManager` (start version 1, target
version 2) whose iterator covers all non-`evm/` stores.
- `sei-db/state_db/sc/composite/store.go`: AppHash and export gating key
on `MigrateCosmos` completion. Memiavl's per-store infos leave the
commit info at the commit that persists version 2, while the live mode
is still `MigrateCosmos`. That keeps a live node and a restarted node
(which derives `FlatKVOnly`) byte-identical. The exclusion latch resets
on rollback below completion. `IsWriteModeComplete` exposes completion
to the root store.
- `sei-cosmos/storev2/rootmulti/store.go`:
`SetMigrationBatchSize(batchSize, allowMigrateCosmos bool)` runs as
named steps. `readyForNextMigration` decides whether a migration is due:
`migrate_evm` from `memiavl_only`, and, with the gate open,
`migrate_cosmos` from `evm_migrated` or from a `migrate_evm` store whose
EVM migration is complete. `mayAdvanceWriteMode` then skips a pinned
store and logs the skip once per batch size. Because readiness is
checked first, a pinned `migrate_evm` node does not report a divergence
while the network is still migrating EVM. `startNextMigration` advances
`migrate_evm -> evm_migrated -> migrate_cosmos` in the same call. A
store that cannot report migration completion returns an error rather
than skipping silently. The transition is level-triggered, so a node
that crashes before the next commit starts it again on the next block.
- `app/migration/params.go`: adds `MigrateCosmosEnabled` (bool) to the
`migration` subspace. An absent value means false, so a pre-v6.8 chain
cannot start the migration. Governance can set it back to false, which
prevents a new start but does not stop a MigrateCosmos in progress.
- `app/abci.go`: `applyMigrationBatchSize` reads `MigrateCosmosEnabled`
from chain state each block and passes it with the batch size. Every
node reads the same value at the same height, so the transition is
consensus-deterministic.
- `app/app.go`, `app/upgrades.go`: `InitChainer` and the v6.8 upgrade
handler call `enableMigrateCosmos`, which sets the param to true. Both
writes happen only at the v6.8 upgrade height and at genesis. Before
v6.8 the param is absent, so blocks replay as on main.
- `sei-db/config/sc_config.go`, `sei-db/config/toml.go`: valid
`sc-write-mode` values are now `memiavl_only`, `migrate_evm`,
`evm_migrated`, `migrate_cosmos`, `flatkv_only`, `test_only_dual_write`,
`auto`.
- `sei-db/state_db/sc/migration/README.md`, `OPERATIONS.md`: the state
machine and docs describe the V0 -> V1 -> V2 chain. The diagrams for the
removed phases are deleted. `OPERATIONS.md` states that one rate
increase runs both migrations back to back, and that clearing
`MigrateCosmosEnabled` does not stop a MigrateCosmos in progress.
## Test plan
- `app/abci_test.go`
- `TestMigrateCosmosEnabledAtGenesis`: a chain started from genesis has
`MigrateCosmosEnabled = true`.
- `TestMigrateCosmosGate`: drives real blocks at the maximum rate. With
the param true, the store reaches `migrate_cosmos`. With the param
false, it stays in `migrate_evm`; opening the gate then starts
`migrate_cosmos` on the first block that reads it, which shows the EVM
migration was complete and only the gate held it.
- `app/upgrade_v68_test.go`
- `TestV68EnablesMigrateCosmos`: clears the param, applies v6.8, and
asserts the param is true.
- `TestV68PrunesTokenfactoryParams`: removes `MigrateCosmosEnabled`
before the snapshot, and requires v6.8 to delete the tokenfactory
subspace, write `MigrateCosmosEnabled = true`, and change nothing else
in the params store.
- `app/upgrade_v68_offline_target_test.go`:
`requireV68OfflineMigrateCosmosEnabled` requires the v6.7 source
snapshot to hold no `MigrateCosmosEnabled` and the migrated database to
hold it set to true. The harness diff checks only recorded keys, so the
added key is checked directly.
- `sei-cosmos/storev2/rootmulti/set_write_mode_test.go`
- `TestRootMultiMigrateCosmosGateClosedAfterEVMComplete`: with the EVM
migration confirmed complete, a closed gate leaves the store in
`migrate_evm`.
- `TestRootMultiMigrateCosmosGateOpenStartsFromLiveAndRestartedState`:
an open gate starts `MigrateCosmos` both from a live `migrate_evm` store
and from a store restarted into `evm_migrated`.
- `TestRootMultiMigrateCosmosGateOpenRestartBeforeFirstCommitReKicks`: a
`MigrateCosmos` start that no commit persisted derives `evm_migrated`
after a restart, and the next open-gate call starts it again.
- `TestRootMultiMigrateCosmosGateOpenPinnedEVMMigratedSkips`: a store
pinned to `evm_migrated` is not advanced by an open gate and logs the
skip.
- `TestRootMultiMigrateCosmosGateOpenPinnedMigrateEVMInFlightIsQuiet`: a
store pinned to `migrate_evm` with its EVM migration in flight logs
nothing. It fails against the previous ordering.
- `sei-db/state_db/sc/migration/migration_transitions_test.go`
- `TestMigrateCosmos`: V1 -> V2 across all real modules, with a restart
mid-migration that resumes from the persisted boundary. After
completion, every module's keys are in flatkv, memiavl is empty, and
version 2 is persisted with no boundary.
- `sei-db/state_db/sc/migration/migration_steady_state_test.go`: the
`FlatKVOnly` steady-state router routes every module to flatkv.
- `sei-db/state_db/sc/migration/derive_write_mode_test.go`: version and
boundary combinations derive the right mode. A version-2 boundary and
unknown versions, 3 included, are rejected. `IsModeComplete` uses the
new target versions.
- `sei-db/state_db/sc/types/write_mode_test.go`: the four legal
transitions, migration-mode classification, and string round-trips.
- `sei-db/state_db/sc/composite/store_migration_test.go`
- Memiavl leaves the commit info exactly at the version-2 commit, and
the live and restarted commit infos stay byte-identical.
- `TestComposite_MigrateCosmos_RollbackAcrossCompletionBoundary`,
`TestComposite_Auto_RollbackAcrossMigrateCosmosCompletionAfterRestart`:
rollback below completion re-includes memiavl and derives the mode
again.
- `TestComposite_PinnedFlatKVOnly_RefusesRollbackIntoMigrateCosmos`: a
pinned `flatkv_only` store refuses a rollback below completion.
- `sei-db/state_db/sc/composite/store_auto_test.go`, `random_test.go`:
an auto store walks the full chain to version 2.
`TestRandomMigration_MigrateCosmos` runs randomized workloads through
the merged transition.
- `sei-db/config/toml_test.go`: the rendered template lists the new
valid modes.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>1 parent 3f2dd73 commit a25c7ef
34 files changed
Lines changed: 898 additions & 823 deletions
File tree
- app
- migration
- sei-cosmos/storev2/rootmulti
- sei-db
- config
- state_db/sc
- composite
- migration
- resources
- types
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
65 | 66 | | |
66 | 67 | | |
67 | 68 | | |
68 | 69 | | |
69 | 70 | | |
| 71 | + | |
70 | 72 | | |
71 | 73 | | |
72 | 74 | | |
| |||
77 | 79 | | |
78 | 80 | | |
79 | 81 | | |
| 82 | + | |
80 | 83 | | |
81 | 84 | | |
82 | 85 | | |
| |||
86 | 89 | | |
87 | 90 | | |
88 | 91 | | |
89 | | - | |
| 92 | + | |
90 | 93 | | |
91 | 94 | | |
92 | 95 | | |
| |||
99 | 102 | | |
100 | 103 | | |
101 | 104 | | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
102 | 115 | | |
103 | 116 | | |
104 | 117 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
| |||
194 | 195 | | |
195 | 196 | | |
196 | 197 | | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
197 | 273 | | |
198 | 274 | | |
199 | 275 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1110 | 1110 | | |
1111 | 1111 | | |
1112 | 1112 | | |
| 1113 | + | |
1113 | 1114 | | |
1114 | 1115 | | |
1115 | 1116 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
40 | 47 | | |
41 | 48 | | |
42 | 49 | | |
43 | 50 | | |
| 51 | + | |
44 | 52 | | |
45 | 53 | | |
46 | 54 | | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
47 | 63 | | |
48 | 64 | | |
49 | 65 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
17 | 18 | | |
18 | 19 | | |
19 | 20 | | |
| |||
149 | 150 | | |
150 | 151 | | |
151 | 152 | | |
152 | | - | |
153 | | - | |
154 | | - | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
155 | 157 | | |
156 | 158 | | |
157 | 159 | | |
| |||
163 | 165 | | |
164 | 166 | | |
165 | 167 | | |
| 168 | + | |
166 | 169 | | |
167 | 170 | | |
168 | 171 | | |
| |||
480 | 483 | | |
481 | 484 | | |
482 | 485 | | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
483 | 498 | | |
484 | 499 | | |
485 | 500 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
661 | 662 | | |
662 | 663 | | |
663 | 664 | | |
664 | | - | |
665 | | - | |
666 | | - | |
667 | | - | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
668 | 669 | | |
669 | 670 | | |
670 | 671 | | |
| |||
676 | 677 | | |
677 | 678 | | |
678 | 679 | | |
| 680 | + | |
| 681 | + | |
679 | 682 | | |
680 | 683 | | |
681 | 684 | | |
682 | 685 | | |
| 686 | + | |
683 | 687 | | |
684 | 688 | | |
685 | 689 | | |
686 | 690 | | |
687 | | - | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
688 | 707 | | |
689 | 708 | | |
690 | 709 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
173 | 173 | | |
174 | 174 | | |
175 | 175 | | |
| 176 | + | |
176 | 177 | | |
177 | 178 | | |
178 | 179 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
132 | 132 | | |
133 | 133 | | |
134 | 134 | | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
135 | 141 | | |
136 | 142 | | |
137 | 143 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
109 | | - | |
| 109 | + | |
110 | 110 | | |
111 | 111 | | |
112 | 112 | | |
| |||
153 | 153 | | |
154 | 154 | | |
155 | 155 | | |
156 | | - | |
| 156 | + | |
157 | 157 | | |
158 | 158 | | |
159 | 159 | | |
| |||
173 | 173 | | |
174 | 174 | | |
175 | 175 | | |
176 | | - | |
| 176 | + | |
177 | 177 | | |
178 | 178 | | |
179 | 179 | | |
| |||
199 | 199 | | |
200 | 200 | | |
201 | 201 | | |
202 | | - | |
| 202 | + | |
203 | 203 | | |
204 | 204 | | |
205 | 205 | | |
| |||
411 | 411 | | |
412 | 412 | | |
413 | 413 | | |
414 | | - | |
| 414 | + | |
415 | 415 | | |
416 | 416 | | |
417 | 417 | | |
| |||
0 commit comments