Skip to content

envelope-v2: write v2 envelopes with typed commands#4700

Open
muhamadazmy wants to merge 3 commits into
mainfrom
pr4700
Open

envelope-v2: write v2 envelopes with typed commands#4700
muhamadazmy wants to merge 3 commits into
mainfrom
pr4700

Conversation

@muhamadazmy

@muhamadazmy muhamadazmy commented May 7, 2026

Copy link
Copy Markdown
Contributor

Switch the worker's write path from the v1 Envelope/Command enum to v2
typed commands and Envelope.

SelfProposer::{self_propose, self_propose_many, append_with_notification}
are now generic over C: Command + HasRecordKeys and derive the partition
key from the command via record_keys() — the explicit partition_key
parameter threaded through LeadershipState, the Actuator trait, and every
RPC handler is gone.

All worker RPC paths (append_invocation, append_invocation_response,
append_signal, cancel/kill/purge_invocation, purge_journal,
resume_invocation, restart_as_new_invocation, get_invocation_output) and
all leader-side ActionEffect proposals (Invoker, Shuffle, Timer, Cleaner,
UpsertSchema, UpsertRuleBook, scheduler decisions, version barriers,
AnnounceLeader) now construct typed commands::Command values directly,
removing the Command::
enum wrapping and the BytesMut arena previously
used to pre-encode scheduler decisions.

With v2 envelopes now being written, drop the try_v1_first branch in
PartitionProcessor::decode_record — v2 is the fast path and v1 stays as
the TypedValueMismatch fallback for replay of older records. Tests under
leadership and rpc are updated to mock the new generic Actuator signatures
and decode via Envelope::into_typed::commands::*().


Stack created with Sapling. Best reviewed with ReviewStack.

@github-actions

github-actions Bot commented May 7, 2026

Copy link
Copy Markdown

Test Results

  8 files  ±0    8 suites  ±0   5m 0s ⏱️ +2s
 61 tests ±0   61 ✅ ±0  0 💤 ±0  0 ❌ ±0 
268 runs  ±0  268 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 52d31f3. ± Comparison against base commit de779f3.

♻️ This comment has been updated with latest results.

@muhamadazmy
muhamadazmy force-pushed the pr4700 branch 15 times, most recently from 9e6bfe2 to c99555d Compare May 10, 2026 16:14
@muhamadazmy muhamadazmy changed the title wip: write v2 envelope wal-protocol: rename Record→Command and unify payload types May 10, 2026
@muhamadazmy muhamadazmy changed the title wal-protocol: rename Record→Command and unify payload types worker: write v2 envelopes with typed commands May 10, 2026
@muhamadazmy
muhamadazmy force-pushed the pr4700 branch 6 times, most recently from e68e277 to 94c8dfb Compare May 11, 2026 09:43
@muhamadazmy
muhamadazmy requested a review from AhmedSoliman May 11, 2026 10:16
@muhamadazmy
muhamadazmy force-pushed the pr4700 branch 2 times, most recently from df8cd5c to c908ab4 Compare May 12, 2026 13:12
@muhamadazmy
muhamadazmy force-pushed the pr4700 branch 4 times, most recently from 2e13782 to 8032ac8 Compare May 13, 2026 07:23
@muhamadazmy muhamadazmy changed the title worker: write v2 envelopes with typed commands envelope-v2: write v2 envelopes with typed commands Jun 29, 2026
@muhamadazmy
muhamadazmy force-pushed the pr4700 branch 4 times, most recently from 9551d11 to 58dafbd Compare June 30, 2026 08:58
@muhamadazmy
muhamadazmy marked this pull request as ready for review June 30, 2026 08:58
@muhamadazmy
muhamadazmy force-pushed the pr4700 branch 2 times, most recently from 0724585 to 7284b36 Compare June 30, 2026 14:03
Switch the code paths that uses IngestionClient to start ingesting
v2 Envelope instead of v1
Switch the worker's write path from the v1 Envelope/Command enum to v2
typed commands and Envelope<Raw>.

SelfProposer::{self_propose, self_propose_many, append_with_notification}
are now generic over `C: Command + HasRecordKeys` and derive the partition
key from the command via `record_keys()` — the explicit `partition_key`
parameter threaded through LeadershipState, the Actuator trait, and every
RPC handler is gone.

All worker RPC paths (append_invocation, append_invocation_response,
append_signal, cancel/kill/purge_invocation, purge_journal,
resume_invocation, restart_as_new_invocation, get_invocation_output) and
all leader-side ActionEffect proposals (Invoker, Shuffle, Timer, Cleaner,
UpsertSchema, UpsertRuleBook, scheduler decisions, version barriers,
AnnounceLeader) now construct typed commands::*Command values directly,
removing the Command::* enum wrapping and the BytesMut arena previously
used to pre-encode scheduler decisions.

With v2 envelopes now being written, drop the `try_v1_first` branch in
PartitionProcessor::decode_record — v2 is the fast path and v1 stays as
the TypedValueMismatch fallback for replay of older records. Tests under
leadership and rpc are updated to mock the new generic Actuator signatures
and decode via Envelope<Raw>::into_typed::<commands::*>().
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant