Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions .github/workflows/spec-mutation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: Spec Mutation
Comment thread
vincent-k2026 marked this conversation as resolved.
Outdated

on:
workflow_dispatch:
inputs:
packs:
description: "Rule packs to run (space-separated: trie canon endian)"
required: false
default: "trie canon endian"

permissions:
contents: read

jobs:
spec-mutation:
name: universalmutator packs
runs-on: ubuntu-24.04
timeout-minutes: 330
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false

- name: Install Rust
uses: actions-rust-lang/setup-rust-toolchain@v1
Comment thread
vincent-k2026 marked this conversation as resolved.
Outdated

- name: Install cargo-nextest
uses: taiki-e/install-action@nextest

- name: Install universalmutator
run: pip install universalmutator==1.14.1
Comment thread
vincent-k2026 marked this conversation as resolved.
Outdated

- name: Warm test binaries
run: cargo nextest run -p salt --no-default-features --features parallel --no-run

- name: Run spec mutation
env:
PACKS: ${{ inputs.packs }}
run: scripts/spec_mutation.sh $PACKS

- name: Publish step summary
if: always()
run: |
if [ -f target/spec-mutants/report.md ]; then
cat target/spec-mutants/report.md >> "$GITHUB_STEP_SUMMARY"
fi

- name: Upload results
if: always()
uses: actions/upload-artifact@v4
with:
name: spec-mutation-results
path: |
target/spec-mutants/report.md
target/spec-mutants/survivors.txt
target/spec-mutants/manifest.txt
if-no-files-found: ignore
60 changes: 60 additions & 0 deletions mutants/operators/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Domain-specific mutation operators

Regex mutation packs run through
[universalmutator](https://github.com/agroce/universalmutator), complementing
the `cargo-mutants` gate (issue #141). `cargo-mutants` ships a fixed set of
structural operators and has no plugin API, so it never generates the
mutations that matter most for a state-commitment system. Each pack targets a
specific soundness or determinism invariant; each rules file documents its
family. The selection filter for adding rules: *"what soundness or
determinism invariant would a surviving mutant here violate?"* Generic
arithmetic/relational/logical operators are deliberately excluded — the
`cargo-mutants` gate already owns those.

| Pack | Rules | Scope | Invariant guarded |
| --- | --- | --- | --- |
| `trie` | `trie.rules` | `trie/node_utils.rs`, `trie/trie.rs`, `constant.rs`, `types.rs`, `proof/shape.rs` | Level-base and bit-width arithmetic of the two-tier trie (`STARTING_NODE_ID`, 8/24/40 splits, 256-way fanout, ceil rounding) |
| `canon` | `canon.rules` | `proof/prover.rs`, `proof/salt_witness.rs` | Canonical form: sort/dedup on the proof path must not be removable or reversible |
| `endian` | `endian.rules` | `state/ahash/convert.rs`, `state/hasher.rs`, `types.rs` | Byte-order determinism of the consensus hash and 12-byte metadata codec (see issue #127) |

## Running

```bash
pip install universalmutator

# Everything (generation + kill analysis; needs cargo + nextest):
scripts/spec_mutation.sh

# One pack, or generation only (no Rust toolchain needed):
scripts/spec_mutation.sh trie
GEN_ONLY=1 scripts/spec_mutation.sh canon endian
```

In CI: the `Spec Mutation` workflow (`workflow_dispatch`) runs the packs and
uploads `report.md` + `survivors.txt`. Expect roughly 60 mutants for all
three packs and a few hours of runtime (each mutant rebuilds `salt` and runs
the suite; killed mutants stop at the first failing test).

Mutants are generated only for production lines (everything before the first
top-level `#[cfg(test)]`), so they never land in test code.

## Triage policy

Every **survivor** is either:

1. a missing test — add a killing test (preferred), or
2. an equivalent mutant — document the justification in the rules file next
to the rule (and consider tightening the rule's regex).

**Invalid** (non-compiling) mutants are fine in moderation; if a rule mostly
produces invalid mutants, tighten its pattern.

## Known gaps

- Multi-line `sort_unstable_by!(...)` macro invocations (e.g. in
`trie/trie.rs`) are not matched by the line-based deletion rules.
- The `endian` pack finds nothing in `convert.rs` until the explicit
little-endian reads land (PR #142); after that it guards them.
- Families C (field-element chunk widths), D (serialization codec config)
and F (hash diffusion ops) from issue #141 are staged for a follow-up
after the first survivor triage of A/B/E.
24 changes: 24 additions & 0 deletions mutants/operators/canon.rules
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Family E - canonicalization removal (issue #141).
#
# The proof path sorts and deduplicates to reach canonical form. Deleting
# those calls (or flipping comparator direction) is a soundness mutation:
# duplicates or unordered elements should be rejected or change the root.
# Statement deletion is not in cargo-mutants' repertoire.
#
# Known gap: multi-line sort_unstable_by!/by_key! macro invocations are not
# matched by these line-based rules (e.g. trie.rs update_leaf_nodes).

# Delete single-line sort/dedup statements, keeping the indentation.
^(\s*).*\.sort\(\); ==> \1
^(\s*).*\.sort_unstable\(\); ==> \1
^(\s*).*\.sort_unstable_by\(.*\); ==> \1
^(\s*).*\.sort_unstable_by_key\(.*\); ==> \1
^(\s*).*\.dedup\(\); ==> \1
^(\s*).*\.dedup_by\(.*\); ==> \1
^(\s*)sort_unstable!\([A-Za-z_0-9]+\); ==> \1

# Flip comparator direction inside sort closures. Name-agnostic: matches any
# `<ident>.cmp(<ident>)` / `.cmp(&<ident>)` and swaps the operands, so a
# comparator written with `left`/`right`, `x`/`y`, etc. is covered too — not
# just the `a`/`b` the current call sites happen to use (Troublor, PR #143).
([A-Za-z_][A-Za-z_0-9]*)\.cmp\((&?)([A-Za-z_][A-Za-z_0-9]*)\) ==> \3.cmp(\2\1)
13 changes: 13 additions & 0 deletions mutants/operators/endian.rules
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Family B - endianness / byte-order (issue #141).
#
# Explicit byte packing in the hasher and codecs; flipping endianness
# silently breaks determinism and cross-platform reproducibility (see
# issue #127), and no generic operator flips it.

# Anchored to the method-call site (leading `.`, trailing word boundary) so a
# rule can only rewrite the stdlib method, never a substring of an identifier
# like `serialize_to_le_bytes` or `raw_from_be_bytes` (flyq, PR #143).
\.to_le_bytes\b ==> .to_be_bytes
\.to_be_bytes\b ==> .to_le_bytes
\.from_le_bytes\b ==> .from_be_bytes
\.from_be_bytes\b ==> .from_le_bytes
Comment thread
vincent-k2026 marked this conversation as resolved.
Outdated
34 changes: 34 additions & 0 deletions mutants/operators/trie.rules
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Family A - trie-structure operators (issue #141).
#
# The SALT trie's correctness reduces to "right level base and right
# bit-width": pure integer arithmetic over the STARTING_NODE_ID table and
# the 8/24/40 bit-width split. A wrong constant produces a plausible,
# compiling, wrong structure that generic operators never generate.
# universalmutator rule format: <python-regex> ==> <replacement>.

# A1. Level-index off-by-one in the STARTING_NODE_ID base table.
STARTING_NODE_ID\[([a-z_\.]+) \+ 1\] ==> STARTING_NODE_ID[\1]
STARTING_NODE_ID\[([a-z_\.]+) - 1\] ==> STARTING_NODE_ID[\1]
MAIN_TRIE_LEVELS - 1 ==> MAIN_TRIE_LEVELS
MAX_SUBTREE_LEVELS - 1 ==> MAX_SUBTREE_LEVELS - 2

# A2. Bit-width constant confusion (swap one named width for another real one).
<< BUCKET_SLOT_BITS ==> << BUCKET_ID_BITS
Comment thread
vincent-k2026 marked this conversation as resolved.
Outdated
>> MIN_BUCKET_SIZE_BITS ==> >> BUCKET_SLOT_BITS
<< BUCKET_SLOT_BITS ==> << (BUCKET_SLOT_BITS - 1)

# A3. Fanout shift direction / amount (the 256-way branching).
\(parent_relative_position << TRIE_WIDTH_BITS\) ==> (parent_relative_position >> TRIE_WIDTH_BITS)
relative_position >> TRIE_WIDTH_BITS ==> relative_position << TRIE_WIDTH_BITS
<< TRIE_WIDTH_BITS ==> << (TRIE_WIDTH_BITS + 1)

# A4. Segment granularity (256 slots per leaf).
>> MIN_BUCKET_SIZE_BITS ==> >> (MIN_BUCKET_SIZE_BITS + 1)
# Trailing word boundary: `TRIE_WIDTH` is a prefix of `TRIE_WIDTH_BITS`, so
# without `\b` this rewrites `% TRIE_WIDTH_BITS` into garbage (flyq/Troublor, PR #143).
% TRIE_WIDTH\b ==> % (TRIE_WIDTH - 1)

# A5. Ceil-division rounding in the subtree-root climb (subtree_root_level).
Comment thread
vincent-k2026 marked this conversation as resolved.
Outdated
\(capacity \+ MIN_BUCKET_SIZE as u64 - 1\) ==> (capacity + MIN_BUCKET_SIZE as u64)
capacity > MIN_BUCKET_SIZE as u64 ==> capacity >= MIN_BUCKET_SIZE as u64
level -= 1 ==> level += 1
Loading
Loading