unify(tbtc/signer): adopt RFC-21 Annex A coordinator-seed derivation + cross-language conformance vectors

[mswilkison]

Stacked on #4005 (base: extraction/frost-signer-mirror-2026-05-26). Implements item 3 of the review feedback (duplicated, divergent protocol constants) — Rust half; pairs with the Go-side PR #4030 stacked on #3866.

Problem

Flagged in #4026: the engine validated attempt contexts using int64_be(MessageDigest[0..8]) with the 1-based wire attempt number (the legacy signingAttemptSeed convention), while the Go RFC-21 layer derives fold(SHA256(KeyGroup ‖ SessionID ‖ MessageDigest)) with 0-based attempt numbers. At Phase-7 wiring, every Go-derived attempt context would fail the engine's strict-mode validate_attempt_context — a deterministic, network-wide liveness failure invisible to either side's property tests.

What changed

• roast_attempt_shuffle_seed(key_group, session_id, message_digest_hex) implements the normative RFC-21 Annex A derivation (see spec(frost/roast): normative coordinator-seed derivation (RFC-21 Annex A) + cross-language conformance vectors #4030). The key-group handle — this engine's hex-encoded serialized group verifying key — feeds the hash as an opaque UTF-8 string, exactly matching keep-core's attempt.DeriveAttemptSeed + foldAttemptSeed composition, including the strict 32-byte digest requirement.
• validate_attempt_context now takes the session's key group (threaded from dkg.key_group at StartSignRound and the session's DkgResult at FinalizeSignRound) and composes the shuffle source with the 0-based RFC-21 attempt number. The FFI wire encoding stays 1-based (attempt_number >= 1 still enforced; wire = AttemptNumber + 1); the engine subtracts one before composition, per the annex.
• testdata/coordinator_seed_vectors.json — byte-identical copy of the canonical file generated from the Go implementation. coordinator_seed_derivation_matches_cross_language_vectors pins, for all ten vectors: the folded seed (including negative values, so an unsigned port cannot pass), the selected coordinator (including the n=100 production-shape set), the 0-/1-based wire mapping, and end-to-end strict-mode validate_attempt_context acceptance of a context built from the wire encoding. Either language drifting now fails its own unit suite.
• docs/roast-coordinator-seed-derivation.md mirrors the normative annex for signer-side readers, with the regen/copy procedure.
• The coordinator-mismatch test derives the provably-wrong coordinator instead of hardcoding member 1 (which, under the new seed, happened to become the correct selection — exactly the class of silent assumption these vectors exist to catch).

Notes

• Mixed-version note: engines on the old derivation reject contexts produced under the new one (and vice versa) — strict-mode attempt contexts are not yet produced by the Go layer in any deployment, so this is pre-wiring cleanup with no live-fleet impact.
• The attempt-context vector suite (roast-attempt-context-v1.json) is unaffected: it pins fingerprint/attempt-id domains with the coordinator as an input.
• Port back to the tBTC monorepo signer alongside the next extraction sync.

Tests

Full suite: 245 passed, 0 failed; clippy and rustfmt clean. New conformance test exercises all ten cross-language vectors.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 8f1f323f-a259-4f7e-b8dc-75ed00195056

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch unify/roast-coordinator-seed-rust-2026-06-11

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[mswilkison]

Codex review finding confirmed and fixed in 710e59a: the original change unified the seed derivation function but the engine call site fed it the wrong input — hash_hex(message_bytes) (the engine's SHA256 transcript digest) instead of the padded raw message that Go's messageDigestFromBigInt produces. Since the Go transitional executor passes the 32-byte signing digest as the FFI message, the engine was effectively seeding from SHA256(digest) while the Go RFC-21 layer seeds from digest — strict-mode Phase-7 contexts would still have selected divergent coordinators.

Why the conformance vectors didn't catch it: the end-to-end leg called validate_attempt_context directly with the vector digest, bypassing start_sign_round's digest computation. Fixes:

• rfc21_message_digest mirrors messageDigestFromBigInt exactly (leading-zero insignificance via the big.Int round-trip, big-endian left-pad to 32, >32 significant bytes rejected) and now feeds the shuffle seed; the SHA256 transcript digest feeds only attempt_id/round_id, as before.
• start_sign_round_accepts_go_derived_attempt_context_in_strict_mode reimplements the Go derivation inline (independent of the engine helpers) and proves acceptance through the real strict-mode StartSignRound call path — the test shape that would have caught this.
• The conformance test now treats the vector digest as the raw message (the exact production relationship) and exercises the two-digest split end to end.
• Attempt-context proptests constrained to 32-byte messages, matching the bound the Go bridge enforces.
• Annex A on spec(frost/roast): normative coordinator-seed derivation (RFC-21 Annex A) + cross-language conformance vectors #4030 amended in lockstep to state the MessageDigest input unambiguously (padded raw message, never a transcript hash).

Full suite: 246 passed, clippy/rustfmt clean.