fix: stale auth state poisoning

[aromancev]

Currently, a stale auth state can poison history preventing local corrections when diff states are enabled.

Failure mode:

1. Server recorded history with state X for ticks 1 and 2 and sent it to the client for tick 1 (no need to send 2 because no diff from 1).
2. Client recorded tick 1 as auth and carried forward tick 2 (also as auth) with state X.
3. Server received client input for tick 1 and resimulated local state to Y for ticks 1 and 2. It then calculated the diff and sent state Y for tick 1 to the client (no need to send 2 because no diff from 1).
4. Client received the correct state Y for tick 1 and started resimulating from tick 1 but unable to correct state for 2 because it's marked as auth.
5. Carry-forward for state Y for tick 2 does not happen on the client because some state already exists. Corrected auth state Y for tick 2 will never be sent from the server because there is no diff between 1 and 2. The situation only resolves at the next full state send, which produces visible artifacts, such as HP jumping back and forth.

This change truncates all local history on receiving auth state from the past. Tested with out of order delivery and did not discover any new failure modes.

This is just one edge case with diff + auth state history. It is probably less risky and easier to maintain if it's truncated to avoid playing whack-a-mole with more edge cases.

[elementbound]

Added some suggestions, excellent stuff otherwise!

[aromancev]

Please don't merge it yet. I found a possible misbehavior caused by this change. I'd like to verify it is safe first :)

[aromancev]

Update

The first iteration of the fix introduced another regression. Position jittering due to continuous "fighting" of authoritative states and predicted local simulation.

The main reason is that clients received incomplete snapshots and tried to extrapolate player position from it:

• When not all ticks have arrived yet after a resimulation.
• When the transform property is missing from the snapshot.
• When snapshot was not sent at all because there was no diff from the previous one on the authority.
• Maybe more cases I haven't unearthed yet.

I tried to plug them all but I don't think it's the right solution anymore.

I initially wanted to avoid wasting more memory by a band-aid solution of truncating history and letting local resim reconstruct from there. It doesn't work well.

Proper solution

Remember the snapshot that was sent to each peer and calculate diffs against that instead of self-owned state.

How much more memory and CPU will this need? YES!
But the bonus is that we probably save a lot of bandwidth because we won't send the same exact snapshot more than once after a resim.

There is probably room for optimization there but I think this solution closes a LOT of awkward edge cases instead of trying to work around them. For example, I think it fixes another issue I noticed but haven't attempted to debug yet: under heavy latency (tested via Clumsy, starting at around 250ms TTR), a peer sometimes completely hangs for a fraction of a second. My suspicion is that diff calculation flakes out and doesn't send anything so we're waiting for the next full state.