test(salt): remediate the full-run mutation baseline (issue #144)#145
Open
vincent-k2026 wants to merge 9 commits into
Open
test(salt): remediate the full-run mutation baseline (issue #144)#145vincent-k2026 wants to merge 9 commits into
vincent-k2026 wants to merge 9 commits into
Conversation
…line Every entry corresponds to a survivor or timeout from the first full-crate cargo-mutants run (issue #144), classified by reading the mutated source: equivalents carry a proof sketch (disjoint bitfields, unreachable boundaries, performance-only hints, algebraically absorbed resize triggers, number theory around 256^level mod 255), dead entries cover the test-bucket-resize cfg variants that the canonical mutation feature set never compiles, and the timeout entries are observed non-terminating mutants that hang rather than pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Delete the unused vendored DeterministicHasher::new_with_keys and its PI constant; pin the integer write_* outputs (expected values computed externally and cross-checked against all existing pinned hashes), so the update/large_update mixing paths stop being dead code. - Pin that the metadata read-cache preserves non-default metadata: the test keys are chosen so a wrongly cached default provably probes a different slot for both a re-nonced and a resized bucket. - Pin SaltProof::create canonical form for strictly descending keys and the SaltWitness::plain_value_fast unsupported-operation contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The full mutation baseline showed the expansion/contraction machinery in update_bucket_subtrees is only exercised by the test-bucket-resize configuration, which the canonical mutation feature set does not compile. Cover it under default features: - a manual capacity cycle (nonce-only rehash, expansion to a three-level subtree, nonce-only rehash while expanded, contraction back) validated not just by root/rebuild equality but by comparing every persisted node against a store built from scratch - stale intermediate commitments are invisible to the root but not to this; - exact chunk boundaries for create_node_aligned_chunks (run-to-end duplicate runs must not emit a trailing empty range; the stride must advance additively); - a StrictStore TrieReader without default-commitment fallbacks, making skipped or misaddressed expansion cache-warming fatal instead of silently absorbed; - a witness built for a block whose replay itself expands the bucket (SAME_BUCKET_TEST_KEYS pushes past the load factor during creation), asserting stateless replay equality with the full-store execution; - a proof over the expanded first data bucket, pinning that it uses its real subtree level rather than the metadata-bucket level. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mutation testing - PASSNothing to test: no viable mutants were generated. |
Performance Benchmark ComparisonCompared Detailed Comparison
|
The first CI run showed the cycled root differing from the virgin root while every per-phase root == rebuild held. Restructure the final assertions so the failure names its layer: state entries first (modulo the explicitly written default metadata entry), then the exact final metadata value, then per-node commitment comparison against a from-scratch store, and only then the root equality. Also apply the three rustfmt diffs from the Lints job. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both CI failures were the same root cause: under test-bucket-resize (2 buckets, 1% load factor) the plain inserts already resize buckets to 512, so the cycle's minimum-capacity baseline does not hold. The default-feature pass runs the test green - including the final root-equality, which also confirms an explicitly written default metadata entry commits identically to an absent one. The resize configuration exercises the same machinery through its own load-factor-driven tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a1a3ebdcd1
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
- Gate: add a `timeout` suppression category with asymmetric matching. equivalent/dead entries prove no behavior change and keep covering both survivors and flaky timeouts; timeout entries only prove non-termination and can no longer excuse a mutant that ran to completion and survived. - Gate: support optional `line = <n>` pins on line suppressions so entries whose mutant text appears at several sites in one file cover only the reviewed site; orphans hygiene fails when a pinned line drifts. - Suppressions: recategorize the 13 observed-non-terminating entries as `timeout`; pin the default_commitment level-0 entry, the five dead test-bucket-resize bucket_id entries, and the four single-site create_node_aligned_chunks timeout entries to their reviewed lines. - Replace both par_batch_size suppressions with a killing test: the `10 * threads -> 10 / threads` mutant zeroes the batch count and panics in div_ceil on any worker with more than ten Rayon threads, so it was never equivalent. The new test runs in an explicit 16-thread pool. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 540fb8e517
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Troublor
reviewed
Jul 9, 2026
flyq
approved these changes
Jul 10, 2026
Address review feedback on PR #145: - state.rs: add a debug_assert_ne pinning that the capacity-case key probes different slots at 256 vs 512, so test_cached_metadata_keeps_non_default_meta can't silently become a tautology. (Troublor) - state.rs: add same_cap_same_nonce_rehash_is_observably_net_empty, pinning the invariant the two shi_upsert (`>`/`/`) suppressions rely on so drift goes red instead of being silently absorbed. (Troublor) - suppressions.toml: document that the two leftmost_node `-` sites share one source line (line-pin impossible) and that the non-equivalent width-site mutant is killed by leftmost_node_calculation, so the loose suppression only ever covers the equivalent pow_result survivor. (Troublor / codex #9,#10,#15) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lents Verified with a scoped cargo-mutants run (17 mutants) that the remaining survivors flagged in review were real gaps, not equivalents: - is_default -> false (codex #11): value() serves from the same cache the metadata path writes, so caching Some(default) for a default bucket makes a later raw value(metadata_key) return a value the store never stored. Added default_bucket_metadata_caches_as_absent; removed the suppression. - shi_upsert > -> >= and / -> % (codex #12 / Troublor): at/below the load factor compute_resize_capacity keeps the capacity, so committed state is unchanged, but a spurious shi_rehash still marks the bucket in the public rehashed_buckets set. Added shi_upsert_does_not_rehash_at_the_load_factor_threshold (via update(), which doesn't clear the set); removed both suppressions. The genuinely-equivalent survivors are kept and pinned to their surviving line, with the mutation run confirming their non-equivalent twins are caught: - fallback write > -> >= pinned to line 120 (len==16); > 8 and while > 16 caught. - hash_with_nonce + -> * pinned to line 71; the 74-75 slice sites caught. Each kill verified by applying the mutant and observing the new test fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The two new kill tests assume the production 80% load factor; the test-bucket-resize feature (used by the coverage job) swaps it for a 1% env-driven threshold, so a 204-key fill resizes and the bucket is no longer default — both tests failed under that feature. Gate them with #[cfg(not(feature = "test-bucket-resize"))], matching the other threshold-sensitive tests (state.rs:1211/1464/1478/2699). The mutation gate runs with --features parallel, so both still run there and kill their mutants. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes out the action list from the first full-crate mutation baseline (#144): every one of the 54 survivors and 20 timeouts is now either killed by a test, deleted as dead code, or registered as a reviewed suppression with a proof sketch.
Suppressions (41 new entries) — equivalents with justifications (disjoint bitfields where
|≡^, structurally unreachable boundaries,Vec::with_capacity/parallel-batch performance hints, resize triggers algebraically absorbed by same-capacity rehashes,256^level ≡ 1 (mod 255)number theory), thetest-bucket-resizecfg variants the mutation feature set never compiles (with an explicit body-ambiguity caveat), and observed non-terminating timeouts.Killing tests
fallback.rs: deleted the unused vendorednew_with_keys+PI, and pinned the integerwrite_*outputs (expected values computed with an external model cross-checked against all 18 existing pinned hashes) soupdate/large_updatestop being dead code.state.rs: the metadata read-cache must preserve non-default metadata — test keys chosen so a wrongly cached default provably probes the wrong slot for both a re-nonced and a resized bucket.prover.rs: canonical proof construction for strictly descending keys (contract pin; the survivors themselves are equivalent sincecreate_sub_trieaggregates into order-insensitive BTree structures — a correction to Full-crate mutation baseline: 54 survivors + 20 timeouts, triaged #144's initial classification) and a proof over the expanded first data bucket, pinning its real subtree level.salt_witness.rs:plain_value_fastunsupported-operation contract.trie.rs: exact chunk boundaries (run-to-end duplicate runs, additive stride); a manual capacity cycle (nonce-only → expand to 3 levels → nonce-only expanded → contract) validated by comparing every persisted node against a from-scratch store, which sees stale intermediate commitments the root cannot; a StrictStore TrieReader without default fallbacks that turns skipped/misaddressed expansion cache-warming into hard failures.witness.rs: a witness for a block whose replay itself expands the bucket (260SAME_BUCKET_TEST_KEYScross the load factor during creation), asserting stateless replay equality with full-store execution — the expansion path finally runs under the canonical feature set.Verification
mutation_gate.py exclude-re; TOML parses (43 entries total).write_*pins generated from a Python reimplementation of the hasher validated against all existing pinned values (hash,hash_with_nonce, length classes).After this lands, the next nightly full run (which also re-covers the shard lost to a runner eviction) should come back clean, making the PR gate eligible to become a required check.
Refs #144.
🤖 Generated with Claude Code