Repository navigation
Commit c07cde4
authored
Load consensus round state before starting peer routines (#4542)
The consensus reactor caches the round state in `NewReactor`, before the
consensus state has been loaded from the store, so the cached state has
height 0 and a nil vote set. It is only refreshed on the first tick of
`updateRoundStateRoutine`. On a node that switches to consensus straight
away (a frozen-height node, or one started with `waitSync=false`),
`processPeerUpdates` can replay already-connected peers before that
tick. A new peer also reports height 0, so `queryMaj23Routine` sees
matching heights and calls `Prevotes` on the nil vote set, which panics
and kills the node at boot. Reproduced locally on main in 9 of 150 runs
with `GOMAXPROCS=1`.
The reactor now stores the loaded round state once readiness is
signalled, just before it starts `processPeerUpdates`, so no peer
routine can see the constructor's empty state. The refresh interval
becomes a reactor field so the new test can stop the periodic refresh
from hiding the race. The test covers both a reactor that is ready from
boot and one that switches to consensus after starting; without the fix
both cases panic with the same stack as CI on every run, and with it
both pass.
Flaked in:
https://github.com/sei-protocol/sei-chain/actions/runs/38278017470/job/1149097836191 parent b625eff commit c07cde4
3 files changed
Lines changed: 65 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
492 | 492 | | |
493 | 493 | | |
494 | 494 | | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
495 | 513 | | |
496 | 514 | | |
497 | 515 | | |
| |||
534 | 552 | | |
535 | 553 | | |
536 | 554 | | |
537 | | - | |
538 | | - | |
539 | | - | |
540 | | - | |
541 | | - | |
542 | 555 | | |
543 | 556 | | |
544 | 557 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
102 | 102 | | |
103 | 103 | | |
104 | 104 | | |
| 105 | + | |
| 106 | + | |
105 | 107 | | |
106 | 108 | | |
107 | 109 | | |
| |||
143 | 145 | | |
144 | 146 | | |
145 | 147 | | |
146 | | - | |
| 148 | + | |
| 149 | + | |
147 | 150 | | |
148 | 151 | | |
149 | 152 | | |
| |||
182 | 185 | | |
183 | 186 | | |
184 | 187 | | |
| 188 | + | |
| 189 | + | |
185 | 190 | | |
186 | 191 | | |
187 | 192 | | |
| |||
295 | 300 | | |
296 | 301 | | |
297 | 302 | | |
298 | | - | |
| 303 | + | |
299 | 304 | | |
300 | 305 | | |
301 | 306 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
233 | 233 | | |
234 | 234 | | |
235 | 235 | | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
236 | 276 | | |
237 | 277 | | |
238 | 278 | | |
| |||
0 commit comments