review: the re-review mode dial (full | scoped | flip-gated | fast)

[jwbron]

The re-review mode dial: the runs-per-PR cost lever from the plan of record's "Re-review economics" section. Stacked on the eval stack (#232-#238) and fold-in batch two (#243-#245); the base should track the top of the batch-two stack as it grows.

Why

Two measured lifecycles of the same seeded PR paid the full 11-agent fan-out on every push, with per-run cost rising as threads accumulate (v1.3.1: $7.19 / $8.77 / $9.14; v1.4.0: $8.31 / $9.87 / $11.29, runs 29044383251 / 29045581242 / 29046928431 in Khan/webapp). The approval run is the most expensive while emitting the least. Both lifecycles also caught a fresh seeded defect on a re-review push that a reconcile-only fast path would have missed, which argues for scoped as the default reduced depth.

What

A per-repo mode line in .github/aw/review/ROUTING:

re-review full|scoped|flip-gated|fast

• full: today's behavior, and the default. No consumer changes behavior until its repo sets a mode line.
• scoped: full roster, staged only the hunks new since the last fully-reviewed fingerprint; comments scoped to those hunks.
• flip-gated: reconcile plus the correctness pass over new hunks; a REQUEST_CHANGES to APPROVE flip is vetoed by that pass's validated blocking findings (the findings gate the flip instead of being discarded).
• fast: reconcile-only.

Three guards, all deterministic (workflows/review/lib/rereview-mode.ts):

1. Ready-for-review anchor: the one full review the cheap modes lean on is taken at ready-for-review, never on a draft skeleton (a draft-taken fingerprint forces one more full review when the PR leaves draft).
2. Flip gate: as above; mechanically this rides the existing verdict function, since a validated blocking finding posts and blocks.
3. Divergence tripwire: every full-depth review stamps a content-hashed hunk signature into its review body as a hidden comment, so it survives cache eviction. Each push is compared against the last fully-reviewed fingerprint; at unreviewed share >= 0.4 the run re-arms to full. Signatures are computed over the generated-stripped diff, so lockfile churn neither enters the fingerprint nor counts as divergence.

Named interactions handled by construction:

• dismiss-stale-approvals: the prior verdict and fingerprint live in the stamp, not the review state, so a dismissed approval still anchors and still reads as an approval.
• COMMENTED-only prior review: the stamp itself (not the verdict state) marks "a full review happened", so a comment-only history reduces normally after its first stamped run instead of re-running full forever.

Eval and measurement

The roster-level evals live in #251 (stacked on this PR): open-PR corpus cases with reconciler dispatch and flip-gate/duplicate scoring, the per-mode pricing sweep (eval/rereview-sweep.ts), and the ported seeded lifecycle. This PR carries the deterministic layer:

• Adversarial lifecycle cases as data (eval/lifecycle/): rewrite-after-approval and sparse-pr-then-payload, each scored as "the tripwire re-armed and the payload got a full review", plus an economy control (fix-push-stays-cheap) scored as "the reduced path actually ran". Replayed deterministically in eval/lifecycle.test.ts under all three reduced modes.
• Pricing wiring for the live A/B: each run's executed depth lands in out/rereview-plan.json; lib/counters.ts aggregates costByRereviewDepth; the review-trial skill gains a ROUTING-override arms pattern (two arms at the same ref differing only in the mode line) and scores adversarial lifecycle pushes.
• This is the playbook's cost-affecting class: mode changes are earned per repo through the live A/B on recall and dollars, the way enable lines are.

Composition with batch two

Based on the top of the batch-two stack (#243 -> #244 -> #245 -> #247) so the mode dial's re-review paths compose with the re-review accountability rendering (#244) rather than race it: the accountability section stays the visible body surface; the fingerprint stamp is appended last as a hidden comment; and the flip gate reads keptBlockingCount, a small additive field this PR adds to the accountability renderer's rereview.json, so "did the reconciler keep any blocking thread" is a number read at verdict time, not a label judgment re-made there. The orchestrator wiring (Step 1 prior-review staging, the Step 3 depth plan and per-depth dispatch/staging, the Step 4 flip rule, the Step 6 stamp, reduced-depth skips in Steps 7/8) is the last commit on the branch.

Supersedes

Khan/frontend#13420 (frontend's consumer-side re-review stopgap, held on the old pin) is superseded by this: the dial lives in the shared workflow and frontend opts in with a single re-review line in its ROUTING file once priced. Frontend is deliberately not touched here.

[changeset-bot]

🦋 Changeset detected

Latest commit: 64f8115

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
review	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[jwbron]

Heads-up: jwbron/review-dispatch-tax (this PR's base) was rebuilt on the refreshed chain per the topology note on #243; its history was rewritten (old tip 8e015ff, new tip 813767c, content unchanged apart from prettier formatting). Reset or rebase local state onto origin/jwbron/review-dispatch-tax before continuing. The prettier/lint CI failures on the old chain are fixed on the new one.

[github-actions]

Review live A/B

Baseline: origin/jwbron/review-dispatch-tax (review.md f7508948b32b); candidate: working tree (review.md e55e2ace5c95).

Metric	Baseline	Candidate	Delta
Must-catch recall	83%	100%	+17%
Verdict agreement	100%	100%	+0%
Noise (unmatched posted)	62%	63%	+1%
Clean false flags	0	0
Judge mean quality	0.91	0.90	-0.01
Cost	$5.22	$5.63
Wall clock	649s	688s
Cases run / skipped	7 / 0	7 / 0
Misses found-but-dropped	1	0

Improvements (candidate caught, baseline missed)

• incident-sql-missing-index:dm-missing-index-1

Adversarial hard gate: PASSED on the candidate arm.

Single-run-stable rows: recall, verdict agreement, regressions, adversarial gate. Judge quality and noise are not: they jitter run-to-run at this corpus size, and a regressed reviewer can score HIGHER on judge quality (fewer, surer comments each read better). Recall against the labeled specs is the load-bearing metric.

[jwbron]

Eval pointer: the real (roster-level) evals for the mode dial live in #251, stacked on this PR.

This PR carries the mechanism plus its deterministic layer only: the depth/tripwire replays in eval/lifecycle/ (rewrite-after-approval, sparse-PR-then-payload, the fix-push economy control), the unit-level guards, and the cost-attribution surfaces (out/rereview-plan.json, costByRereviewDepth).

What actually measures reviewer behavior under each dial setting is in #251:

• open-PR corpus cases (live.rereview): staged prior threads, the stamped prior review, the depth plan; the live producer dispatches the thread-reconciler and sizes the finder roster by depth
• scoring: thread-resolution accuracy, the flip-gate input (kept blocking count), duplicate comments on kept threads, plus fresh-defect recall via the ordinary mustCatchSpecs
• pricing: eval/rereview-sweep.ts runs all four modes in one command (recall, resolution, flips, dups, dollars, with the EXECUTED depth per case since the tripwire can override the dial), and live-ab.ts --re-review-mode prices one mode against a full-depth baseline
• golden-retention-lifecycle-1/2/3 (sanitized port of the measured webapp#40730 seeded lifecycle) and golden-retention-fix-push (the under-tripwire case where the reduced paths actually execute)

Reviewers judging whether the dial is measured before any repo flips a mode line: that evidence is #251's; both merge before any consumer can set re-review, since this PR ships default-full.