feat(release): canonical immutable GitHub Release mechanism (ADR 0129)

CHANGELOG.md

@@ -10,6 +10,50 @@ and this project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
 
 ## [0.2.0] - Unreleased
 
+### Added
+
+- A canonical, immutable release mechanism: `.github/workflows/release.yml`
+  (`workflow_dispatch` only, explicit `version` input, never triggered by
+  push/schedule/merge), split into a read-only, credential-less `verify` job
+  and a write-scoped `publish` job for least privilege. `verify` checks the
+  dispatched commit is protected `main`'s untampered current tip and that
+  every one of this repository's own known push-triggered checks (Tests,
+  Fuzz, Security's jobs — `RELEASE_EXPECTED_PUSH_CHECKS`) has actually
+  registered as a check-run for that exact commit *and* every check GitHub
+  reports for it is complete with an acceptable conclusion (excluding this
+  release run's own checks) — a dispatch fired moments after a merge, before
+  GitHub has finished registering those push-triggered check-runs at all,
+  is correctly "not ready" rather than a vacuous pass on an empty report —
+  checks the requested version against `pyproject.toml`'s `[project]` table
+  (table-boundary aware, so a same-named `version` key in an unrelated table
+  can never be mistaken for it), resolves any existing `vX.Y.Z` tag via the
+  GitHub commits API (rejecting only one that points at a different commit;
+  a tag at this commit is always a safe idempotent resume, whether or not
+  its Release already exists — see below; a failed tag or Release lookup is
+  read as "absent" only on a confirmed 404 / "release not found" — any other
+  lookup failure, e.g. a rate limit or transient network/5xx error, fails
+  the step closed instead of guessing, so a later dispatch retries and
+  resolves cleanly rather than compounding a wrong assumption), re-runs the
+  full test suite fresh, renders release notes from this file's matching
+  `## [X.Y.Z]` section via the tested `scripts/ci/release_notes.py`, and
+  best-effort looks up a CycloneDX SBOM (a missing SBOM or failed lookup
+  warns, never blocks). `publish` re-verifies `main`'s tip has not advanced
+  and every expected check is still registered and green since `verify`
+  started testing — immediately before it creates anything — then creates
+  the annotated `vX.Y.Z` tag (skipped on a tag resume) and the GitHub
+  Release (skipped on a Release resume, e.g. a prior run whose asset upload
+  failed after the Release itself was already created), always attempting
+  the best-effort SBOM asset attach afterward either way. Gives downstream
+  consumers (`ContextualWisdomLab/keyverse#132`, `bandscope#881`, and the
+  Wardnet consumer-owner handoff, all recorded on
+  `contextual-orchestrator#971`) an immutable pin target
+  (`.../releases/tag/vX.Y.Z` — not the mutable `.../releases/latest` alias)
+  instead of a vendored source SHA. See
+  `docs/planning/adrs/0129-canonical-immutable-release.md` and
+  `docs/RELEASING.md`. No release has been cut yet — landing this mechanism
+  and dispatching the first `v0.2.0` release are deliberately separate
+  actions.
+
 ### Deprecated
 
 - Internal callers now use

README.md

@@ -419,6 +419,8 @@ python tests/test_commercial_evidence_export.py
 python tests/test_commercial_acceptance_check.py
 python tests/test_release_authorization.py
 python tests/test_release_authority_snapshot.py
+python tests/test_release_notes.py
+python tests/test_release_workflow_contract.py
 python tests/test_commercial_buyer_acceptance_workflow.py
 python tests/test_commercial_release_candidate.py
 python tests/test_commercial_gap_register.py

conductor/tracks.md

@@ -6,3 +6,4 @@
 | 002-enterprise-design-foundation | active | Add paper-grounded screen design, user stories, REST API, code/DB conventions, and i18n |
 | 003-reasoning-effort-profiles | active | Issue #568: versioned per-role `reasoning_effort_profile`, equal-budget θ̂ RMSE ablation, snapshot on run/stream/batch, production defaults locked |
 | 004-nim-cost-quality-benchmark | active | Evidence-grade NIM catalog discovery, all-modality capability probes, and the route/conduct/single-worker cost-quality benchmark (docs/nim_benchmark.md) |
+| 005-canonical-immutable-release | active | Deliberate `workflow_dispatch` release mechanism (ADR 0129): protected-main-tip + version + fresh-suite gate, CHANGELOG-derived notes, annotated tag, GitHub Release — so consumers pin a release instead of vendoring source (docs/RELEASING.md) |

docs/RELEASING.md

@@ -0,0 +1,193 @@
+# Releasing
+
+This document is for a human maintainer cutting a real, immutable
+`contextual-orchestrator` release. It is deliberately narrow — see
+[`docs/planning/adrs/0129-canonical-immutable-release.md`](planning/adrs/0129-canonical-immutable-release.md)
+for the full design and its explicit non-goals.
+
+## What a release is, and is not
+
+A release is a git tag `vX.Y.Z` and a GitHub Release built from it. It gives
+downstream consumers (Keyverse, BandScope, Wardnet, and others) an immutable,
+citable pin target they never again need to vendor a mutable source SHA off
+`main` for. Those are two different URLs with two different guarantees:
+
+- `.../releases/tag/vX.Y.Z` is the actual immutable pin — always the same
+  commit, forever. **Consumers should pin this one.**
+- `.../releases/latest` is a **mutable discovery alias** that repoints to
+  whatever the newest release is; it is useful for finding "what's current"
+  in a human workflow, but a consumer pinning to it is *not* protected from
+  behavior changes across future releases and should not use it as a pin
+  target.
+
+A release is **not** the same thing as
+[`/api/v1/commercial_release_candidates/latest`](commercial_release_candidate.md)
+or `contextual_orchestrator/release_authorization.py`. Those answer "is this
+pull request commercially/buyer-sale-ready" for a human procurement audience,
+gated behind admin auth inside the running gateway. This document's release
+mechanism answers a narrower question — "does an immutable, citable artifact
+exist for this exact commit" — and is unaffected by, and does not affect,
+that separate system.
+
+## Preconditions
+
+1. The version to release is already merged to `main`: `pyproject.toml`'s
+   `version` field carries the exact `X.Y.Z` you intend to release, landed
+   through the normal PR process (review, required checks, no exceptions).
+2. `CHANGELOG.md` has a `## [X.Y.Z]` section (an `- Unreleased` or dated
+   suffix is fine) with real, non-empty content describing what changed.
+3. Either no git tag `vX.Y.Z` exists yet, or one does and points at a commit
+   that is an ancestor of `main`'s current tip (the commit you're
+   dispatching, or an earlier one `main` has since advanced past) — in
+   which case the workflow safely resumes using **the tag's own target
+   commit**, never the commit you happen to be dispatching against: if that
+   tag has no GitHub Release published yet, it creates one; if the Release
+   also already exists (e.g. a prior run's asset-upload step failed after
+   `gh release create` itself succeeded), it still attempts the best-effort
+   SBOM asset attach rather than treating the run as nothing left to do —
+   see step 4 below. This is exactly the "tag pushed, then the run failed
+   before creating the Release, and more commits merged to `main` before
+   you retried" scenario — it stays recoverable no matter how far `main`
+   has advanced since the tag was pushed. A tag pointing at a commit that is
+   **not** an ancestor of `main`'s current tip (a genuinely different or
+   stray tag) is rejected outright: a tag is never reused or moved onto a
+   different commit — bump the version again if you need to re-release.
+4. `main` is currently green — its own required checks (Tests, Security,
+   Fuzz, and the org-central Strix/OpenCode/security-scan/OSV/Scorecard
+   checks from `ContextualWisdomLab/.github`) are passing. The release
+   workflow re-verifies both automatically: the commit is genuinely `main`'s
+   untampered tip, and every check GitHub reports for that exact commit
+   (excluding the release run's own) is complete with a successful,
+   skipped, or neutral conclusion — it fails closed otherwise. It also
+   re-runs the full test suite fresh, but it does not re-run CodeQL, Trivy,
+   OSV, Scorecard, or the review bots — those already had to pass before this
+   commit could exist on protected `main` at all.
+   - **If you dispatch moments after a merge lands**, the gate can fail
+     with "expected push-triggered check(s) ... have not registered yet" —
+     GitHub has not finished creating this new tip's Tests/Security/Fuzz
+     check-run entries yet. This is expected and safe: wait a few moments
+     for those workflows to actually start, then re-dispatch. It is
+     distinct from a genuine pending/failed check, which the same gate
+     reports as "not both complete and green" instead.
+
+## Cutting a release
+
+1. Go to **Actions → Release → Run workflow** in the GitHub UI (or
+   `gh workflow run release.yml -f version=X.Y.Z`).
+2. Select branch `main` (the workflow refuses to run against anything else).
+3. Enter the exact version, e.g. `0.2.0` — no leading `v`, must match
+   `pyproject.toml` byte-for-byte.
+4. Dispatch. The workflow is two jobs, least-privilege: `verify` runs with no
+   write permission and no persisted git credential while it executes any
+   repository-controlled code; `publish` holds the write token and does
+   nothing but tag and publish. In order:
+   - **`verify`** (read-only):
+     - first resolves any existing `vX.Y.Z` tag via the commit API and
+       decides `TARGET_SHA` — the exact commit every later gate evaluates
+       against. No tag yet: `TARGET_SHA` is the dispatched commit itself
+       (a fresh publish). A tag that exists and is an ancestor of `main`'s
+       current tip: `TARGET_SHA` is the *tag's own target commit* (a
+       resume — see step 3 above), regardless of how far `main` has since
+       advanced. A tag that exists and is **not** an ancestor of `main`'s
+       current tip fails closed outright (a genuine conflict). A failed tag
+       or Release lookup is treated as "absent" only on a *confirmed* 404 /
+       "release not found"; any other lookup failure (rate limit, auth,
+       network, 5xx) fails this step closed instead of guessing —
+       re-dispatch once the transient failure clears;
+     - checks out `TARGET_SHA` so every subsequent step in this job reads
+       *that* commit's tree, never a possibly-newer `main` tip;
+     - for a **fresh publish only**, fails closed if `TARGET_SHA` is not
+       `main`'s current tip (a race with a concurrent merge) — a resume
+       skips this comparison entirely, since `main` having advanced past
+       the tag's target commit is exactly the situation a resume exists to
+       handle;
+     - fails closed (via the shared `scripts/ci/release_checks_gate.sh`)
+       unless every one of this repository's own known push-triggered
+       checks (Tests' two jobs, Fuzz's two jobs, Security's two jobs — see
+       `RELEASE_EXPECTED_PUSH_CHECKS` in `release.yml`) has actually
+       registered as a check-run for `TARGET_SHA` *and* every check GitHub
+       reports for it is complete with a successful, skipped, or neutral
+       conclusion;
+     - fails closed if the input version does not match `TARGET_SHA`'s
+       `pyproject.toml` `[project]` table;
+     - runs the full test suite fresh, on `TARGET_SHA`'s tree (`uv run
+       --locked --extra api --extra db --extra queue --group dev python -m
+       pytest -q`);
+     - renders release notes from `TARGET_SHA`'s `CHANGELOG.md` matching
+       section (`scripts/ci/release_notes.py`, tested in
+       `tests/test_release_notes.py`);
+     - best-effort looks up and downloads the CycloneDX SBOM from the
+       matching successful `security.yml` run for `TARGET_SHA`, if one
+       exists (a missing SBOM, or a failed lookup, warns — it never blocks
+       the release);
+     - uploads the rendered notes and any SBOM for `publish` to pick up.
+   - **`publish`** (write-scoped, only after `verify` succeeds):
+     - for a **fresh publish only**, re-verifies `main`'s tip has not
+       advanced while `verify` was testing and rendering notes (a resume
+       skips this — the tagged commit is already immutable, so there is
+       nothing for a live main-tip comparison to protect against); either
+       way, re-verifies (via the same shared script) that every check for
+       `TARGET_SHA` is still complete and green — a second, authoritative
+       recheck right before anything is created (see "Known limitations"
+       below for the small residual window a fresh publish still leaves);
+     - creates and pushes an annotated tag `vX.Y.Z` — skipped when resuming
+       a run whose tag already exists;
+     - creates the GitHub Release using the notes `verify` produced —
+       skipped when resuming a run whose Release already exists;
+     - attempts the best-effort SBOM asset attach, whether the Release was
+       just created or already existed — a failure here warns and never
+       blocks (re-dispatch to retry the attach).
+5. Confirm at
+   <https://github.com/ContextualWisdomLab/contextual-orchestrator/releases/latest>.
+
+## After a release
+
+- Bump `pyproject.toml`'s `version` and open a new `## [next-version] -
+  Unreleased` `CHANGELOG.md` section in an ordinary PR, so the repository is
+  never left claiming to already be the version it just released.
+- Downstream consumers with an open handoff on this gap
+  (`ContextualWisdomLab/keyverse#132`, `ContextualWisdomLab/bandscope#881`,
+  the Wardnet consumer-owner handoff on `contextual-orchestrator#971`) can
+  now bump to the published tag instead of a vendored source SHA.
+
+## Known limitations
+
+**This section describes a fresh publish only.** A resume of a tag-only
+interrupted publication evaluates every gate against the tag's own target
+commit, which is already immutable once pushed — there is no live
+`main`-tip comparison to race for a resume, so the window below does not
+apply to it.
+
+**A small, accepted check-then-act window remains before the tag/Release are
+actually created.** `publish`'s recheck of `main`'s tip and of every check
+for that commit is the very first thing it does, back-to-back, before
+anything else — but GitHub exposes no atomic "create this tag only if branch
+`X` is still at commit `Y`" API, so there is no way to make that window
+literally zero. In practice it is small (a same-org artifact download plus
+the `git tag`/`git push` themselves, on the order of seconds), this is a
+manual, maintainer-triggered dispatch rather than a high-frequency automated
+path, and the only realistic outcome if the window is ever actually hit is
+releasing a commit that genuinely *was* `main`'s verified, all-checks-green
+tip moments earlier — not a wrong, unreviewed, or malicious commit, and not
+one that skipped this workflow's own fresh test run. See
+`docs/planning/adrs/0129-canonical-immutable-release.md`'s "Known
+limitations" section for the full reasoning.
+
+If you ever discover a release published a commit that was immediately
+superseded by another merge: **do not** retroactively move, delete, or
+retag the published release (see Rollback below — tags here are immutable
+once published, and this is not the "genuine publishing mistake caught
+immediately" case that section's narrow deletion exception covers). Instead,
+just cut a new patch (or minor) release from the actual intended tip through
+the normal dispatch process above; the superseded release stays as an
+accurate record of what `main`'s tip briefly was.
+
+## Rollback
+
+Releases are immutable — never delete or retag a published release to "fix"
+it. If a released commit turns out to be broken, release a new patch/minor
+version with the fix through the same process above. `gh release delete
+vX.Y.Z` (and its tag) is reserved for a genuine publishing mistake caught
+immediately after dispatch, before any consumer could plausibly have pinned
+it, and should still be treated as an exceptional, logged action, not routine
+practice.