diff --git a/.changeset/review-shed-ranking.md b/.changeset/review-shed-ranking.md new file mode 100644 index 00000000..54756b81 --- /dev/null +++ b/.changeset/review-shed-ranking.md @@ -0,0 +1,5 @@ +--- +"review": minor +--- + +Value-ranked shedding and dispatch. The graceful-landing rule's first shed bucket previously said "skip any not-yet-dispatched opt-in reviewers and lenses" with no ordering, so a budget-pressed run shed arbitrarily; the 2026-07-10 production runs showed the consequence (an invocation cap smaller than the roster filled its slots by enablement mechanism, not value, and a matched specialist lens could be shed to afford `conventions`). The shed order is now explicit, lowest value first: conventions, first-principles, holistic, completeness/test-adequacy, and path-triggered specialist lenses last, since a matched lens is the most targeted signal in the run. The same ranking read from the other end (defaults, lenses, targeted opt-ins, generic opt-ins) is now the mandated dispatch order when `maxReviewerInvocations` cannot fit the full roster, with every undispatched reviewer recorded as a planned shed. And reviewer requests (Step 8) are removed from the shed list entirely: pulling a human in matters most on exactly the run whose own coverage is partial. The ranking is a first-cut editorial ordering; replace it with measured per-dimension must-catch contribution from the eval corpus when that data is compiled. diff --git a/workflows/review/review.md b/workflows/review/review.md index ecffa4e6..6d69e176 100644 --- a/workflows/review/review.md +++ b/workflows/review/review.md @@ -570,7 +570,11 @@ either run by default; a reviewer earns its `enable` line through the eval suite not by shipping). Dispatch the default reviewers (`correctness-reviewer`, `skill-auditor`, `thread-reconciler`) **plus** every reviewer named in `enabledReviewers` **plus** every lens named in `lensesToSpawn`, all **in parallel** -(one turn), and wait for all. +(one turn), and wait for all. If `runBudget.maxReviewerInvocations` cannot fit +that whole set, fill the slots by the dispatch ranking (the budget rule below: +Step 3, graceful-landing bucket 1): defaults first, then matched lenses, then +the targeted opt-in dimensions, then the generic ones. Never choose arbitrarily, and record every +reviewer left undispatched as a planned shed (Step 6 note). **One candidate contract.** Every finding-producing reviewer returns `findings[]` in the same shape (a `label` per finding, from the fixed label set in Step 4); a @@ -799,10 +803,23 @@ When any proxy passes roughly three-quarters of its soft target (or the trajecto is clearly expensive), stop starting new work and shed remaining work in this order: -1. Skip any not-yet-dispatched opt-in reviewers and specialist lenses; each becomes - a skipped dimension (Step 6 note). -2. Skip the risks/patterns comment and reviewer requests (Steps 7-8) if they have - not happened yet. +1. Skip not-yet-dispatched opt-in reviewers and specialist lenses in value + order, lowest value first; each becomes a skipped dimension (Step 6 note). + The ranking, from first-shed to last-shed: `conventions`, then + `first-principles`, then `holistic`, then `completeness` and + `test-adequacy`, and only then any path-triggered specialist lens from + `lensesToSpawn`. A matched lens is the most targeted signal in the run (the + router chose it for the specific files this PR touches), so it outranks + every generic dimension; shedding `security-auth` on an auth-path diff to + afford `conventions` is exactly backwards. This same ranking, read from the + other end (defaults, lenses, targeted opt-ins, generic opt-ins), is the + dispatch order when the invocation cap cannot fit the roster (Phase 2). + The interior order is a first-cut editorial ranking; replace it with + measured per-dimension must-catch contribution once the eval corpus + yields that data. +2. Skip the risks/patterns comment (Step 7) if it has not happened yet. + Reviewer requests (Step 8) are **never** shed: pulling a human in matters + most on exactly the run whose own coverage is partial. 3. Last, and never at the soft targets alone: the `claim-validator`. It is the false-positive gate, and its cost scales with the candidate count (which you can already see when deciding), not with the diff, so validating a small