docs(qa-guide): §3 routes mutation validation to mutation-probe, not a hand loop

.github/workflows/rust.yml

@@ -443,7 +443,7 @@ jobs:
       - uses: rainlanguage/rainix/.github/actions/prompt-cap@main
         with:
           paths: "**/*prompt*"
-          cap: 153919
+          cap: 154969
   # The plugin prompts, capped separately from the cron runners above. Same tax,
   # different payer: a `/nr` or `/ncc` dispatch reads its agent file whole into a
   # fresh window, so every byte is charged once per turn of that agent's run, and

QA-GUIDE.md

@@ -47,11 +47,28 @@ Every behavior the diff claims to fix or add gets a test that:
 sides of your change has pinned nothing (cyclo.site#398: three deploy-gate
 fixes, 15KB test file untouched, every test green before AND after).
 
-## 3. Mutation-validate the new tests
+## 3. Mutation-validate the new tests — with the bundled tool
 
-For each new test, apply ONE targeted mutation to the line it claims to cover
-(negate the guard, flip the comparison, drop the call), confirm the test fails,
-restore. A test that survives its own mutation is decoration.
+The route is the `adversarial-mutation-test` skill scoped to your change, whose
+probe step authors ONE targeted mutation per behavior (its catalog) into a
+`mutants.toml` and runs the bundled bin:
+
+```sh
+nix run github:rainlanguage/adversarial-mutation-test#mutation-probe -- mutants.toml
+```
+
+`mutation-probe --help` is the manual: file format, verdicts, exit codes. Do
+NOT hand-roll an edit-run-restore loop — the bin ENFORCES what a hand loop can
+only assert. A red, silent or zero-test baseline aborts before any probe; the
+suite's own tally proves it RAN, so a crash or compile error is NO-RUN and
+never a pass; each target must occur EXACTLY once in its file; and every
+restore is verified byte-exact before the next mutant. Two duties stay yours:
+COMMIT before the first probe, and keep targets out of test code — a mutant in
+the oracle co-mutates the expectation and voids the result.
+
+SURVIVED is a real gap: strengthen the test in place or add one, re-probe
+(`--only`) until KILLED, and never edit a test to pass under a mutation. A test
+that survives its own mutation is decoration.
 
 ## 4. Oracle discipline
 
@@ -101,7 +118,9 @@ don't hold.
 
 All four lines are required. A line your change cannot have takes `n/a` **with
 the reason** (a docs-only diff has no mutations to apply); an absent line is not
-an option.
+an option. `Mutations applied` is TRANSCRIBED from the probe's verdicts (§3) —
+the mutant, and the test that KILLED it — never recalled; a mutation nothing
+scored is not evidence.
 
 That is enforced where the PR is opened, not only where it is judged. You open
 PRs with the `open_pr` MCP tool, which reads the body file and REFUSES (exit 3)

campaign-prompt.txt

@@ -51,7 +51,7 @@ Each run:
        • PRE-EXISTING `main` RED (the failing check ALSO fails on the base branch, not introduced by this PR) is FIXABLE and is NOT a "leave it": a branch CAN carry ANY fix, INCLUDING the root-cause fix for a red `main`. Do NOT paper over it — a merge-main-only commit or an empty-commit retrigger CANNOT green a genuinely-broken test, and re-doing them every run is the churn this rule exists to kill (a PR with 4+ merge-main / empty-retrigger commits still red is the smell). Instead: READ the failing log, diagnose the ACTUAL failure on `main`, and FIX IT ON THE BRANCH (mutation-validated per (ii) if it touches a test — a real fix, NEVER a weakening) so the check goes green on the PR. Because the same breakage sits on `main`, ALSO open a dedicated minimal fix-`main` PR (one issue per PR) carrying that same root-cause fix — one such fix typically greens the whole cohort of PRs branched off that red `main`.
        • CANNOT-RESOLVE → FLICK TO DESIGN, never re-churn. If after a genuine diagnosis the red cannot be greened on the branch because doing so needs a HUMAN DECISION — the failing assertion encodes an ambiguous spec / design question, or the only available "fix" is a judgment call the producer must not make unilaterally (weakening an assertion, choosing between incompatible behaviours, a cross-repo contract change) — do NOT leave it to re-attempt every run. FLICK IT TO DESIGN via the transition: `pr-review-report flag-design <owner/repo> <n> "<the SPECIFIC question the human must answer to unblock it>"` — the reason IS the question. A design-flagged PR is PARKED (its `ai:design` label surfaces it in `human-queue` and is the cross-run marker; skip it like any human-gated state, never re-attempt).
       Never claim a red fixed without reading its log.
-   (ii) Clone fresh into `{{WORK_DIR}}/<repo>-pr<n>`, `gh pr checkout <n>` (if that dir already exists from a prior run, RE-SYNC to the PR branch with checkout-force + clean — `git -C <dir> fetch origin && git -C <dir> checkout -f -B <pr-branch> origin/<pr-branch> && git -C <dir> clean -fdx` — NEVER `git reset --hard`, which is forbidden and hook-blocked). REPRODUCE the red locally on the UNCHANGED checkout FIRST: if the failing check is already GREEN locally before you change anything (you cannot reproduce it), it is CI-only — do NOT push a no-op code "fix"; handle it per (i) (empty-commit re-trigger if the log shows a transient flake, else LEAVE and note). Only once you reproduce the red: make the MINIMAL fix for exactly that failure. If the fix touches a test/assertion you MUST prove it is a correctness fix, NOT a weakening: mutation-validate per step 4 (break the covered line → the assertion FAILS → restore → PASSES); making the test pass by loosening/deleting an assertion is FORBIDDEN. A lint/format failure → run the repo's formatter. Then build + run the failing check locally to CONFIRM it goes green.
+   (ii) Clone fresh into `{{WORK_DIR}}/<repo>-pr<n>`, `gh pr checkout <n>` (if that dir already exists from a prior run, RE-SYNC to the PR branch with checkout-force + clean — `git -C <dir> fetch origin && git -C <dir> checkout -f -B <pr-branch> origin/<pr-branch> && git -C <dir> clean -fdx` — NEVER `git reset --hard`, which is forbidden and hook-blocked). REPRODUCE the red locally on the UNCHANGED checkout FIRST: if the failing check is already GREEN locally before you change anything (you cannot reproduce it), it is CI-only — do NOT push a no-op code "fix"; handle it per (i) (empty-commit re-trigger if the log shows a transient flake, else LEAVE and note). Only once you reproduce the red: make the MINIMAL fix for exactly that failure. If the fix touches a test/assertion you MUST prove it is a correctness fix, NOT a weakening: mutation-validate per step 4 (the skill's `mutation-probe` pass, never a hand loop); making the test pass by loosening/deleting an assertion is FORBIDDEN. A lint/format failure → run the repo's formatter. Then build + run the failing check locally to CONFIRM it goes green.
    (iii) Push the fix as a NEW fast-forward commit on top of the branch tip — with the `mcp__fsm__push` TOOL, `{"clone": "<clone-dir-name>"}` (add `"branch": "<remote-branch>"` only when the local branch is named something else). PUSHING IS A TOOL, never a bare `git push`: the tool's result is this run's TYPED RECORD that the rework happened — it reports the sha it created and the PR whose head that sha is, which is what `work-tokens` counts as a work item, and a `git push` in Bash records nothing any reader can join to a PR. It cannot spell a force-push at all. NEVER force-push in ANY form or spelling: not `--force`, `-f`, `--force-with-lease`, `--force-if-includes`, a leading-`+` refspec (`git push origin +branch`), a `HEAD:branch --force`, NOR any of these routed through `git -C <dir> push …`. If a plain push is rejected as non-fast-forward, STOP and leave the PR for a human — never reach for a force/lease/refspec variant to land it. Never `git reset --hard` (any form, incl. `git -C <dir>`), never rebase-force, never change the PR's scope or its linked issue. Only ever fix PRs authored by `{{ASSIGNEE}}`; never touch another author's PR or branch.
    (iv) MIGRATION — the deploy-shaped case, triggered by EITHER (a) a red prod-pin / `testProdDeploy*` / deploy-pinned-constant check (REPLACES (ii)/(iii) for it: there is no code fix to push), OR (b) a GREEN PR carrying a legacy `REQUIRES redeploy at land` marker in its body or a trusted note. Under the split deploy/abstract release lifecycle NO merge waits on a deploy (#162): deploy repos freeze per-tag records and publish on `rainix-tag-release`, the on-chain deploy is a separate manual workflow dispatch decoupled from publishing, and library repos never deploy — so either trigger firing is evidence of exactly ONE thing: THIS REPO HAS NOT MIGRATED to the split lifecycle. The PR is blocked on that migration, not on any deploy, and you deploy NOTHING, EVER (there is no producer deploy path; deploys are a human's decoupled manual dispatch). Route it: (1) find the repo's lifecycle-migration issue/PR (search the repo's open issues/PRs for the split-lifecycle / release-workflow migration); (2) if none exists, FILE it — `gh issue create` in that repo, assignee {{ASSIGNEE}}, shaped as the audit skill's lifecycle-split finding (the repo keeps a single "current" deployment pin that premerge CI tests against, so every bytecode-changing PR goes red until an out-of-band deploy; the fix is the split lifecycle: per-tag frozen deploy records + `rainix-tag-release` publishing, with the on-chain deploy a decoupled manual dispatch) — this is the ONE sanctioned `gh issue create` and the only exception to step 4's no-new-issues posture; (3) TRANSITION the PR: `pr-review-report flag-blocked-on <owner/repo> <n> "repo not migrated to the split release lifecycle: <the red pin check / the legacy marker>" --blocked-by <owner/repo#n of the migration issue/PR>` — the typed dep makes clearance automatic: the vetter clears the flag the run after the migration lands and the PR re-enters vetting fresh. An APPROVED PR is human-sacred and cannot be flagged — report it as blocked-on-migration in the run summary instead (still do (1)/(2) so the dep exists). DEDUP: if the PR already carries `ai:blocked-on` naming the migration, it is parked — do not re-flag. There is no blocked-deploy state and no producer deploy any more (#162/#221) — a PR still wearing the string is an unmodelled label, so treat it as any other PR and route it by the rules above. Still NEVER merge; landing stays interactive on the human's explicit word.
 3c. BACKFILL MISSING SCREENSHOTS on your OWN open UI PRs (still existing-work — after 3b's red fixes, before step 4 opens anything new): a UI PR with no screenshot is, like a red PR, in-flight work that is NOT review-ready (step 5), so finish it before starting new work. YOUR LIST IS STEP 2a's `worklist` ROWS WHOSE `nextAction` IS `screenshot-3c`, never a path list written here: `worklist`'s `is_ui_path` is the ONE definition of which paths fall under this requirement — raindex's `packages/webapp` and `packages/ui-components`, everything under a `site/` dashboard tree INCLUDING the `health.json` its panels draw, and every `.svelte`/`.css`/`.html` file wherever a repo keeps it (`cyclo.site` keeps its components in `src/lib/components/`, which is where both of #140's PRs lived) — and a second enumeration in this step is a second definition, free to drift from it while reading as though it agrees. The row's `markers.uiTouch` tells you which answer routed it: `yes` a path under the requirement is in the changed list, `unknown` the changed list could not be ruled out — BOTH route here, because `unknown` means nothing ruled UI out, not that there is none. For each, judge FROM THE DIFF whether it changes RENDERED OUTPUT a user sees (per step 5: `.svelte` markup, a visible component/element, user-visible styling, a non-`.svelte` service whose output is directly displayed — chart data / labels / formatters — OR a `site/` dashboard page's visible markup/section). A change that ONLY touches `<script>`/store/load/pure-logic with NO visible effect does NOT need a screenshot — skip it. If it needs one, render + post it via step 5's `render-component` — a row that reached this step has neither a shot nor a waiver on it already, which is exactly what routed it here (`markers.screenshotSettled` is `false`). BACK-OFF: at most ONE screenshot attempt per PR per run; the screenshot comment IS your cross-run marker, and a PR carrying one never routes here again — a trusted comment embedding ANY `pr-screenshots/…png` on the PR settles it, whatever step 5 named the file, so do not go looking for one particular filename. If `render-component` exits 3, that IS the focused attempt and its message is the finding — quote it. ASK THE HUMAN FOR THE HARNESS: `pr-review-report flag-design <owner/repo> <n> "unrenderable-render: <the tool's own exit-3 message, verbatim>"`. Exit 12 is NOT this: that is the box missing node/chromium/fonts, which is an environment failure and ends the run instead. A stack the harness has NEVER been able to render is not a passing outage, it is a standing gap only a person can close, so it is a design question. Its `ai:design` label surfaces it in `human-queue` and is the cross-run marker; skip that PR on future runs (never re-attempt a PR already carrying its screenshot or that state).

pr-review-report-rs/src/main.rs

@@ -59749,6 +59749,84 @@ mod settings_tests {
         );
     }
 
+    /// QA-GUIDE §3 routes mutation validation to the BUNDLED TOOL, and no longer teaches a hand
+    /// loop. Step 4 below has always said to INVOKE the `adversarial-mutation-test` skill rather
+    /// than hand-copy its rules, but §3 read as a complete self-contained recipe — "apply ONE
+    /// targeted mutation … confirm the test fails, restore" — so it won for anyone reading the
+    /// guide alone, and those readers wrote their own harness every run. What a hand loop cannot
+    /// claim is exactly what `mutation-probe` enforces: a green non-empty baseline, proof from the
+    /// suite's own tally that it RAN, an exactly-once target, and a byte-exact restore.
+    ///
+    /// Asserted both ways round, like the send-back rules below: the recipe must be GONE as an
+    /// instruction AND the tool present as the route, because a rule merely deleted is a rule the
+    /// next edit reinvents.
+    #[test]
+    fn the_qa_guide_routes_mutation_validation_to_the_bundled_probe() {
+        let (Some(guide), Some(prompt)) = (
+            repo_root_text("QA-GUIDE.md"),
+            repo_root_text("campaign-prompt.txt"),
+        ) else {
+            return; // not checked out (nix build sandbox) — enforced by the rs-test gate
+        };
+        assert!(
+            guide.contains(
+                "nix run github:rainlanguage/adversarial-mutation-test#mutation-probe -- \
+                 mutants.toml"
+            ),
+            "§3 must hand the reader the runnable probe command, not a method to re-implement"
+        );
+        assert!(
+            guide.contains("`mutation-probe --help` is the manual"),
+            "the file format, verdicts and exit codes must route to the tool's own manual rather \
+             than to a snapshot of them here, which is what goes stale"
+        );
+        // The integrity properties are WHY the tool is the route rather than a preference, so the
+        // guide carries them: a hand loop can assert every one of these and prove none.
+        for property in [
+            "A red, silent or zero-test baseline aborts before any probe",
+            "is NO-RUN and",
+            "must occur EXACTLY once in its file",
+            "verified byte-exact before the next mutant",
+        ] {
+            assert!(
+                guide.contains(property),
+                "§3 must say what the bin enforces that a hand loop cannot: {property:?}"
+            );
+        }
+        // The recipe itself, gone from BOTH files. Each of these is a step an agent hand-rolled
+        // because a mandatory document spelled it out.
+        for taught in [
+            "apply ONE targeted mutation to the line it claims to cover",
+            "confirm the test fails",
+            "break the covered line",
+        ] {
+            assert!(
+                !guide.contains(taught) && !prompt.contains(taught),
+                "the hand-rolled loop is still taught: {taught:?}"
+            );
+        }
+        assert!(
+            guide.contains("hand-roll an edit-run-restore loop"),
+            "the guide must FORBID the hand loop, not merely omit it"
+        );
+        // §8's evidence line is where a hand-rolled matrix gets written up as prose, so it names
+        // where the line's content comes from.
+        assert!(
+            guide.contains("TRANSCRIBED from the probe's verdicts"),
+            "§8's `Mutations applied` line must be transcribed from the probe's own verdicts"
+        );
+        // The producer's route into §3 stays the SKILL, which is how a skill upgrade reaches a run
+        // without an edit here; the red-PR step points at the same probe rather than a loop.
+        assert!(
+            prompt.contains("INVOKE THE ACTUAL SKILL"),
+            "step 4 must keep routing the producer to the skill itself"
+        );
+        assert!(
+            prompt.contains("`mutation-probe` pass, never a hand loop"),
+            "the red-PR test fix must point at the probe, not restate the loop it replaced"
+        );
+    }
+
     /// #290, the PROMPT half. `state-load` exists so a run opens on a typed answer "rather than on
     /// a blob it re-slices with `jq`" — and every producer run opened by taking `--json` (95,370
     /// bytes), redirecting it to a scratch file and paying three main-thread `jq` calls, the first