From 0d937362562a6cbd2e80993adc872fee8292becd Mon Sep 17 00:00:00 2001 From: Clay Good Date: Mon, 29 Jun 2026 16:18:27 -0500 Subject: [PATCH 1/8] =?UTF-8?q?docs(openspec):=20propose=20add-unarchive-c?= =?UTF-8?q?ommand=20=E2=80=94=20deterministic=20openspec=20unarchive?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add an OpenSpec change proposing `openspec unarchive [change-name]` and a thin `/opsx:unarchive` skill: the deterministic inverse of `openspec archive`. Core finding driving the design: the archived delta is not self-inverting. ADDED/RENAMED are reversible from the delta; REMOVED/MODIFIED are not (the forward merge discards the pre-image, confirmed in specs-apply.ts and #1246). So archive gains a forward-only reversal snapshot (pre-image + post-merge digest) that makes reversal byte-exact, drift-guarded, and atomic, with graceful degradation for pre-snapshot archives and a --keep-specs escape hatch. Also folds in the CI-determinism recommendation: expose the deterministic merge engine (today applySpecs() has zero callers, reachable only inside archive) as a model-free `openspec sync --check` drift gate — captured as a separable companion, de-risked by this PR's round-trip determinism. Dogfoods OpenSpec: proposal.md, design.md (7 decisions), tasks.md, and spec deltas for cli-unarchive (new), opsx-unarchive-skill (new), cli-archive (reversal snapshot). Validates with `openspec validate --strict`. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../add-unarchive-command/.openspec.yaml | 2 + .../changes/add-unarchive-command/design.md | 90 +++++++++ .../changes/add-unarchive-command/proposal.md | 106 +++++++++++ .../specs/cli-archive/spec.md | 29 +++ .../specs/cli-unarchive/spec.md | 175 ++++++++++++++++++ .../specs/opsx-unarchive-skill/spec.md | 56 ++++++ .../changes/add-unarchive-command/tasks.md | 59 ++++++ 7 files changed, 517 insertions(+) create mode 100644 openspec/changes/add-unarchive-command/.openspec.yaml create mode 100644 openspec/changes/add-unarchive-command/design.md create mode 100644 openspec/changes/add-unarchive-command/proposal.md create mode 100644 openspec/changes/add-unarchive-command/specs/cli-archive/spec.md create mode 100644 openspec/changes/add-unarchive-command/specs/cli-unarchive/spec.md create mode 100644 openspec/changes/add-unarchive-command/specs/opsx-unarchive-skill/spec.md create mode 100644 openspec/changes/add-unarchive-command/tasks.md diff --git a/openspec/changes/add-unarchive-command/.openspec.yaml b/openspec/changes/add-unarchive-command/.openspec.yaml new file mode 100644 index 0000000000..34f9314d22 --- /dev/null +++ b/openspec/changes/add-unarchive-command/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-06-29 diff --git a/openspec/changes/add-unarchive-command/design.md b/openspec/changes/add-unarchive-command/design.md new file mode 100644 index 0000000000..21d9210713 --- /dev/null +++ b/openspec/changes/add-unarchive-command/design.md @@ -0,0 +1,90 @@ +# Design: `openspec unarchive` + +## Context + +`openspec archive` ([src/core/archive.ts](../../../src/core/archive.ts)) is a one-way, non-transactional door: it merges a change's delta specs into `openspec/specs/` via `buildUpdatedSpec` ([src/core/specs-apply.ts:240-307](../../../src/core/specs-apply.ts)) and moves the change folder to `openspec/changes/archive/YYYY-MM-DD-/`. This design covers the inverse command, the one archive-side change required to make the inverse deterministic, and a separable CI companion. + +The governing constraint is the reversibility asymmetry established in the proposal: **ADDED and RENAMED are invertible from the archived delta; REMOVED and MODIFIED are not**, because the forward merge discards the pre-image. Every decision below follows from deciding *where the pre-image comes from*. + +## Goals / Non-Goals + +**Goals** +- A deterministic, code-only `openspec unarchive` that restores `specs/` exactly and moves the folder back. +- Never silently corrupt `specs/`: when a faithful reversal is not possible, refuse with a clear, actionable message. +- Atomic: success or "no files were changed." +- `--keep-specs` as an always-safe escape hatch. +- Backward compatibility with changes archived before this feature. + +**Non-Goals** +- Re-deriving `specs/` from git history as the *primary* mechanism (git is an optional assist only). +- Multi-change / bulk unarchive (mirror of bulk-archive) — future, not here. +- Archive/unarchive lifecycle **hooks** (#704/#682) and lifecycle **timestamps** (#1245) — accommodated, not built. +- Choosing the archive-folder prefix scheme (#409/#787/#1192) — unarchive is tolerant of all of them. + +## Decision 1 — The reversibility guarantee comes from a snapshot at archive time, not from inverting the delta + +**Decision.** Make `openspec archive` write a self-contained **reversal snapshot** into the change folder whenever it rewrites `specs/`. For each affected spec, record the **pre-merge file content** (the exact pre-image, or an explicit "absent" marker when archive *creates* a new spec) and the **post-merge content digest**. `unarchive` reverses by restoring the recorded pre-image; the digest is the drift guard (Decision 3). + +**Why.** The pre-image is the only thing that makes REMOVED/MODIFIED reversible, and it exists only at archive time. Capturing the whole pre-image (rather than a diff or a hash) makes reversal a byte-exact file restore — uniform across ADDED/MODIFIED/REMOVED/RENAMED/created/deleted, with zero re-parsing and zero inference. It is the most robust and the simplest thing that can possibly work. The snapshot lives *inside the change folder* so it moves into `archive/` with everything else, is self-contained, and survives in git without extra plumbing. + +**Alternatives considered.** +- **(A) Reconstruct the pre-image from git** (`git show :specs/...`). Rejected as the foundation: it requires a git repo and a clean pre-archive commit, and the motivating case is exactly when the archive is buried in a multi-file commit (the maintainer's "am I in a bind?" scenario). Kept as an *optional* recovery assist for pre-snapshot archives (Decision 5), never required. +- **(B) Invert the delta only** (no new state). Faithfully reverses ADDED/RENAMED; cannot reverse REMOVED/MODIFIED. Rejected as the foundation because it cannot meet the "deterministic reversal" bar; retained as the graceful-degradation path for pre-snapshot archives (Decision 5). +- **(C) Store a base *hash* per requirement** ([#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)'s proposal). A hash detects drift but cannot *restore* content. The snapshot is the strict generalization: it both detects (digest) and restores (pre-image). If #1246's fingerprint lands first, the snapshot subsumes it. + +**Cost.** A copy of each affected spec's prior bytes per archive — negligible, and only for changes that touch specs. + +## Decision 2 — Resolution is prefix-tolerant and never auto-picks among ambiguous matches + +**Decision.** `unarchive` accepts either the change `` or a full archived directory id. It locates candidates under `changes/archive/` by treating the leading prefix as **opaque up to ``** — stripping a `YYYY-MM-DD-` prefix today, and tolerating `NNN-`/ISO/configurable prefixes ([#409](https://github.com/Fission-AI/OpenSpec/issues/409)/[#787](https://github.com/Fission-AI/OpenSpec/pull/787)/[#1192](https://github.com/Fission-AI/OpenSpec/issues/1192)). When more than one archived entry matches a bare ``: +- **interactive**: list candidates (most-recent first) and prompt — never auto-select; +- **`--json` / non-interactive**: error with the candidate list and require the full archived id. + +**Why.** Multiple archives of one name already happen today (same name archived on two different days), and the proposed sequence scheme makes it routine. Silent selection is the kind of guess that produces the wrong outcome on the rare-but-costly path. Resolution reuses `getArchivedChangesData()` ([#399](https://github.com/Fission-AI/OpenSpec/pull/399)) rather than re-reading the archive dir. + +## Decision 3 — Drift guard + atomicity: refuse rather than clobber, and never leave a partial state + +**Decision.** Before un-merging, compare each affected spec's current content to the **post-merge digest** the snapshot recorded. If any spec has drifted (a later change touched the same requirement — the stacked-archive hazard of [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) and [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md)), unarchive **refuses** the spec reversal and points the user to `--keep-specs`. The whole operation is staged and validated first; on any failure it aborts with *"Abort. No files were changed."* — matching archive's existing contract and the failure surface [#1112](https://github.com/Fission-AI/OpenSpec/issues/1112) describes, and supplying the rollback [#682](https://github.com/Fission-AI/OpenSpec/issues/682) noted archive lacks. + +**Why.** Restoring a pre-image over a spec that a *later* change has since modified would silently delete that later change's contribution — exactly the data loss the project guards against. Detecting drift and stopping is strictly safer than a best-effort merge, and the user always has `--keep-specs` to proceed without spec changes. Atomicity matters because unarchive mutates both `specs/` and the folder location; a half-applied reversal is worse than none. + +**Order of operations (commit phase, after all checks pass):** restore/delete spec files → move folder `archive/` → `changes/`. The move is last so that a move failure (e.g. destination appearing) does not strand rewritten specs without a matching active change; if the move fails after specs are restored, roll the spec writes back from the still-present snapshot. + +## Decision 4 — `--keep-specs`: semantics, and naming vs. archive's `--skip-specs` + +**Decision.** `--keep-specs` moves the folder back and leaves `specs/` **untouched** — always deterministic, always safe. Primary spelling is `--keep-specs` (the request's, and it reads naturally for the inverse: "keep the specs as they are"). Accept `--skip-specs` as a **hidden alias** so muscle memory from archive transfers. When unarchive *cannot* safely reverse specs (drift, or a pre-snapshot REMOVED/MODIFIED) and `--keep-specs` was not given, it refuses and *recommends* `--keep-specs` — it does not silently fall back. + +**Why.** `--skip-specs` on archive means "move the folder, don't merge in" ([#28](https://github.com/Fission-AI/OpenSpec/pull/28)); `--keep-specs` on unarchive means "move the folder, don't un-merge." Same shape, inverse direction. Honoring the requested spelling while aliasing the familiar one is the low-surprise choice. **Open for owner confirmation:** if strict CLI symmetry is preferred, make `--skip-specs` the primary and `--keep-specs` the alias — purely a naming call. + +## Decision 5 — Backward compatibility: graceful degradation for pre-snapshot archives + +**Decision.** Changes archived before this feature have no snapshot. Unarchive then: +1. reverses **ADDED** and **RENAMED** by delta inversion (the self-invertible half), and +2. for any **REMOVED**/**MODIFIED**, **refuses to guess** — it names each requirement it cannot safely restore and directs the user to `--keep-specs` (optionally noting the git-based manual recovery for the pre-image). + +It never silently emits a wrong spec. If git is available and a clean pre-archive commit can be identified, it *may* offer to recover the pre-image from git as an assist — strictly opt-in, never assumed. + +**Why.** The feature must be useful the day it ships, including for already-archived changes, without ever degrading into corruption. Reversing the half that is provably reversible, and stopping honestly on the half that is not, is the correct contract. + +## Decision 6 — Companion: a model-free CI drift gate (separable) + +**Context.** The motivating Discord thread ended on: letting archive's `sync` run through the agent makes merged `specs/` non-deterministic (wording drift), so a CI gate would seem to need a model. + +**Decision / recommendation.** Don't run a model in CI. The delta merge is mechanical and already deterministic in code (`buildUpdatedSpec`); only the agent path drifts. Let the agent author locally, and have CI verify the deterministic output as a plain binary. The enabling gap: `applySpecs()` ([src/core/specs-apply.ts:391](../../../src/core/specs-apply.ts)) — the standalone deterministic apply — has **zero callers**; the engine is reachable only inside `archive`, fused to the folder move. Expose it as `openspec sync [change]` (apply deltas to `specs/` without archiving) with a `--check` mode that exits non-zero when committed `specs/` diverge from the regenerated output — a codegen/IaC-style drift gate. This PR's reversal snapshot makes `archive → unarchive` a byte-exact round-trip, which is the determinism property such a gate relies on and a ready-made test of it. + +**Scope.** Separable. No `cli-sync` spec delta is included here; it is captured so the owner can decide whether it rides this PR or a fast follow-up. The unarchive core does not depend on it. + +## Decision 7 — Move strategy and lifecycle metadata + +- **Move**: reuse `moveDirectory()`/`copyDirRecursive()` ([src/core/archive.ts:96-128](../../../src/core/archive.ts)) for the Windows EPERM/EXDEV fallback ([#605](https://github.com/Fission-AI/OpenSpec/pull/605)). A future switch to `git mv` ([#709](https://github.com/Fission-AI/OpenSpec/issues/709)) would cover both directions through this one helper. +- **Metadata**: today archive persists no `archived` timestamp (only the folder-name prefix), so there is nothing to clear. If [#1245](https://github.com/Fission-AI/OpenSpec/issues/1245) lands an `archived` field, unarchive should null it on restore — noted, not built. + +## Risks / trade-offs + +- **Round-trip is not guaranteed identity in the forward direction.** Unarchive restores the exact pre-archive `specs/` from the snapshot, but a *subsequent re-archive* re-runs the forward merge, which is itself lossy across stacked MODIFIED changes ([#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)) and may reorder ADDED requirements. Unarchive is faithful; re-archiving inherits archive's existing caveats. Documented, not solved here. +- **Snapshot adds a small artifact to archived folders.** Acceptable; it is the price of reversibility and is inert for changes that touch no specs. +- **Pre-snapshot archives get partial reversal.** Mitigated by honest refusal + `--keep-specs`; fully resolved for any change archived after this ships. + +## Migration / rollout + +Purely additive. Archive gains snapshot-writing (forward-only, no behavior/output change). Unarchive is a new command + skill in the expanded profile. Existing archives keep working under Decision 5. diff --git a/openspec/changes/add-unarchive-command/proposal.md b/openspec/changes/add-unarchive-command/proposal.md new file mode 100644 index 0000000000..b2dbe59adf --- /dev/null +++ b/openspec/changes/add-unarchive-command/proposal.md @@ -0,0 +1,106 @@ +## Why + +`openspec archive` does two things at once: it **merges** a change's delta specs into `openspec/specs/` and **moves** the change folder into `openspec/changes/archive/`. There is **no command to undo it.** When a change is archived by mistake — or peer-review feedback lands after the archive has — the user is left with manual `git` surgery the maintainer has to walk people through case by case ([Discord, 2026-06], where he committed to "follow up with a PR to add `/opsx:unarchive` (or `openspec unarchive `)"). + +`git revert` only works when the archive is its own clean commit. The motivating case is the hard one: *"the archive is part of a commit with other changes, so I cannot simply revert. Am I in a bind?"* This proposal adds the inverse command — `openspec unarchive [change-name]` plus a thin `/opsx:unarchive` skill — and closes the one archive-side gap that makes a **deterministic** reversal possible at all. + +## Background: archive is a one-way door, and the delta is not self-inverting + +Verified against `Fission-AI/OpenSpec` at `main` (commit `546224e`, #1248) on 2026-06-29. Two facts define the whole design. + +**1. The forward merge is deterministic; the reverse is not free.** `buildUpdatedSpec` ([src/core/specs-apply.ts:240-307](../../../src/core/specs-apply.ts)) builds a `name → requirement-block` map from the base spec and applies operations in a fixed order — RENAMED → REMOVED → MODIFIED → ADDED. Inverting each operation is asymmetric: + +| Delta op | Forward effect | Invertible from the delta alone? | +|---|---|---| +| **ADDED** | inserts a new requirement block | **Yes** — the block carries its own name; remove it by name. | +| **RENAMED** | `FROM`→`TO`, body preserved | **Yes** — both names are stored; rename `TO`→`FROM`. | +| **REMOVED** | deletes a requirement | **No** — the delta stores only the *name* (`plan.removed: string[]`) plus a prose reason. The deleted body is gone. | +| **MODIFIED** | whole-block replace, keyed by name | **No** — the delta stores only the *new* block, never the prior one. | + +The conventions spec makes the loss explicit: a MODIFIED delta must "include the complete modified requirement (not a diff)" ([openspec/specs/openspec-conventions/spec.md](../../specs/openspec-conventions/spec.md)). So the pre-image — the bytes that were in `specs/` *before* the archive — is **not** retained anywhere after a forward archive. [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) independently documents this exact `nameToBlock.set(key, mod)` whole-block replace as a *forward-direction* data-loss bug ("discarding any sibling scenarios"), which is the same mechanism that makes MODIFIED non-invertible in reverse. + +**2. Archive is non-transactional and has no rollback.** The hook-design discussion states it plainly: *"the current archive operation is not transactional — if a step fails partway, there's no rollback"* ([#682](https://github.com/Fission-AI/OpenSpec/issues/682)). Unarchive is, in effect, the rollback that archive never had — so it must itself be atomic. + +**The consequence:** a reversal that only reads the archived delta can faithfully undo ADDED and RENAMED, but **cannot** restore a REMOVED or MODIFIED requirement. A command that silently "reverses" by deleting the modified requirement (or leaving a hole) would corrupt `specs/` — the precise failure mode OpenSpec already treats as a cardinal sin ([#1246](https://github.com/Fission-AI/OpenSpec/issues/1246); the `prevent-silent-spec-drop` work). True determinism requires capturing the pre-image **at archive time**, when it still exists. + +## What Changes + +The design principle mirrors the one the project is converging on (deterministic CLI, thin agent): **the reversal is a pure, code-only transform the CLI owns; the agent does none of it.** Ordered by leverage: + +1. **REVERSAL SNAPSHOT — make archive reversible (`cli-archive`, MODIFIED).** When `openspec archive` rewrites `specs/`, it also writes a small, self-contained **reversal snapshot** *inside the change folder* (so it moves into the archive with everything else and survives in git): for each affected spec it records the pre-merge file content (the pre-image, or "absent" for a newly-created spec) and the post-merge content digest. This is the one missing primitive. It is forward-only metadata — it changes nothing about how archive merges or moves — and it is what lets unarchive restore `specs/` **byte-for-byte deterministically**, uniformly across ADDED/MODIFIED/REMOVED/RENAMED/created/deleted, with no re-parsing and no inference. (Conceptually adjacent to [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)'s proposed base-fingerprint, generalized from a hash to the full pre-image.) + +2. **THE COMMAND — `openspec unarchive [change-name]` (`cli-unarchive`, NEW).** The deterministic inverse of archive: + - **Resolve** the archived folder. `unarchive` accepts either the change `` or the full archived directory id. The leading prefix is treated as **opaque up to the name** — it strips a `YYYY-MM-DD-` prefix today and tolerates the sequence/`NNN-` and configurable prefixes proposed in [#409](https://github.com/Fission-AI/OpenSpec/issues/409)/[#787](https://github.com/Fission-AI/OpenSpec/pull/787)/[#1192](https://github.com/Fission-AI/OpenSpec/issues/1192). When several archived entries share one name, it **never silently picks**: interactive mode prompts; `--json`/non-interactive requires the full id or errors with the candidate list. Resolution reuses the archive-reading plumbing added in [#399](https://github.com/Fission-AI/OpenSpec/pull/399) (`getArchivedChangesData()`). + - **Reverse the spec merge**, deterministically, from the snapshot: restore each affected spec's recorded pre-image (recreating deleted specs, deleting specs that archive created). Guarded by a **drift check** — if a spec no longer matches the post-merge image the snapshot recorded (e.g. a later change touched the same requirement; the stacked-archive hazard of [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md) and [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)), unarchive **refuses** rather than clobber, and points the user at `--keep-specs`. + - **Move** the folder back from `changes/archive//` to `changes//`, reusing the Windows-safe `moveDirectory()` helper ([src/core/archive.ts:116-128](../../../src/core/archive.ts), the EPERM/EXDEV fallback from [#605](https://github.com/Fission-AI/OpenSpec/pull/605)); error if an active `changes//` already exists (mirrors archive's no-overwrite contract). + - **Atomicity**: stage and validate the whole reversal first; on any failure, *"Abort. No files were changed."* — matching archive's existing abort contract and the failure surface [#1112](https://github.com/Fission-AI/OpenSpec/issues/1112) defines, and finally giving archive the rollback [#682](https://github.com/Fission-AI/OpenSpec/issues/682) noted it lacks. + - **`--keep-specs`**: restore the folder *without* touching `specs/`. Always deterministic and always safe (a pure folder move). It is the literal mirror of archive's `--skip-specs` ([#28](https://github.com/Fission-AI/OpenSpec/pull/28)) and the escape hatch for any case the snapshot can't cover. + - **Graceful degradation for pre-snapshot archives.** Changes archived before this feature have no snapshot. Unarchive then restores ADDED/RENAMED by delta inversion (the self-invertible half), and for any REMOVED/MODIFIED it **refuses to guess** — it reports exactly what it cannot safely reverse and directs the user to `--keep-specs` (optionally assisted by git). It never silently produces a wrong spec. + +3. **THE SKILL — `/opsx:unarchive` (`opsx-unarchive-skill`, NEW).** A thin wrapper that **delegates to the `openspec unarchive` CLI** rather than re-implementing the move-and-un-merge in prose. This is the maintainers' stated direction after the `/opsx:archive` skill drifted from the CLI ([#863](https://github.com/Fission-AI/OpenSpec/issues/863), [#799](https://github.com/Fission-AI/OpenSpec/issues/799), [#656](https://github.com/Fission-AI/OpenSpec/issues/656)): the deterministic reversal lives in TypeScript, the skill only selects the change, confirms, and renders results. It ships in the **expanded** workflow profile, not the deliberately-minimal core ([#913](https://github.com/Fission-AI/OpenSpec/issues/913), [#762](https://github.com/Fission-AI/OpenSpec/issues/762)), and because it calls the CLI it carries no dependency on another skill being installed (the [#913](https://github.com/Fission-AI/OpenSpec/issues/913) failure mode). + +### Companion (separable): a model-free CI drift gate + +This thread answers the question the same Discord conversation ended on — *"this means that inference is needed in the CI… any recommendations here?"* — when the user observed that letting the archive-time `sync` run through the agent makes the merged `specs/` non-deterministic, so a naive CI gate would have to run a model. + +**Recommendation: don't put a model in CI.** The delta merge is mechanical and already deterministic in code; only the agent-driven `/opsx:sync` path introduces wording drift. The clean fix is to let the agent author locally and have CI verify the deterministic output as a plain binary. Today that is impossible: the deterministic engine `applySpecs()` ([src/core/specs-apply.ts:391](../../../src/core/specs-apply.ts)) has **zero callers** — it is reachable only *inside* `openspec archive`, bundled with the folder move. There is no standalone command that applies (or just checks) the merge. + +The fix is small and reuses the engine this PR is already exercising: expose it as `openspec sync [change]` (apply deltas to `specs/` without archiving) with a `--check` mode that exits non-zero when committed `specs/` differ from the regenerated output — a codegen/IaC-style drift gate CI runs without any model. **This PR de-risks that companion directly:** the reversal snapshot makes `archive` → `unarchive` a verifiable, byte-exact round-trip on `specs/`, which is the determinism property the CI gate depends on. + +It is called out as **separable** so the owner can decide whether it lands in this PR or a fast follow-up; the unarchive core does not depend on it. (Scoped here, fully argued in [design.md](./design.md) Decision 6, but no `cli-sync` spec delta is included in this PR.) + +## Capabilities + +### New Capabilities + +- `cli-unarchive`: the `openspec unarchive [change-name]` command — the deterministic inverse of `openspec archive`. It resolves an archived change (prefix-tolerant, never auto-picking among ambiguous matches), moves the folder back to active, and reverses the spec merge from the archive's reversal snapshot under a drift guard, atomically (abort leaves no partial state). `--keep-specs` restores the folder without touching `specs/`; pre-snapshot archives degrade gracefully (reverse the self-invertible half, refuse to guess the rest). +- `opsx-unarchive-skill`: a `/opsx:unarchive` workflow skill that delegates to the `openspec unarchive` CLI for the deterministic work, handling only change selection, confirmation, and result rendering. Ships in the expanded profile; carries no cross-skill dependency. + +### Modified Capabilities + +- `cli-archive`: when archive rewrites `specs/`, it additionally captures a self-contained **reversal snapshot** inside the change folder (per-spec pre-image plus post-merge digest) so the operation becomes deterministically reversible. Forward-only and backward-compatible — it does not change how archive merges, moves, validates, or what it prints; archives created before this feature simply have no snapshot. + +### Companion capability (separable; not specced in this PR) + +- `cli-sync` (recommended follow-up): a deterministic `openspec sync [change]` that applies delta specs to `specs/` without archiving, plus a `--check` drift-gate mode for model-free CI. Enabled and de-risked by this PR's snapshot/round-trip work; included here as a design recommendation so the owner can decide PR-vs-follow-up. + +## Impact + +- `src/core/archive.ts` — at the point specs are rewritten (`buildUpdatedSpec`/`writeUpdatedSpec`, ~414-472), also write the reversal snapshot (pre-image + post-merge digest) into the change folder before the move. No change to merge/move/validate behavior or output. Reuse `moveDirectory()`/`copyDirRecursive()` (116-128). +- `src/core/unarchive.ts` (**new**) — `UnarchiveCommand` mirroring `ArchiveCommand`'s shape (human + `--json` modes, `ResolvedOpenSpecRoot`, blocked-error → diagnostic): resolve archived dir, drift-check, restore pre-images (or delta-invert / refuse for pre-snapshot), move folder back, atomic abort. +- `src/core/specs-apply.ts` — factor the invertible half (delta-inversion for ADDED/RENAMED used by the pre-snapshot fallback) alongside the existing `buildUpdatedSpec`; no change to forward behavior. +- `src/core/list.ts` / `src/core/view.ts` — reuse `getArchivedChangesData()` ([#399](https://github.com/Fission-AI/OpenSpec/pull/399)) to resolve/disambiguate archived candidates; after unarchive the entry leaves `list --archived` and returns to active `list`. +- `src/cli/index.ts` — register `unarchive [change-name]` with `--keep-specs`, `-y/--yes`, `--no-validate`, `--json`, `--store` (mirrors the archive registration at 326-343). +- `src/core/templates/workflows/unarchive-change.ts` (**new**) — `getUnarchiveChangeSkillTemplate()` + `getOpsxUnarchiveCommandTemplate()`, modeled on `archive-change.ts` but delegating to the CLI. +- Registration/wiring — export from `skill-templates.ts`; add `unarchive` to `ALL_WORKFLOWS` ([src/core/profiles.ts:19](../../../src/core/profiles.ts)) and `WORKFLOW_TO_SKILL_DIR` ([src/core/init.ts:65](../../../src/core/init.ts)); **not** added to `CORE_WORKFLOWS`. Command generation is generic (no adapter changes). Add a `/opsx:unarchive` row to `docs/opsx.md`. +- Tests — mirror `test/core/archive.test.ts`: resolve/disambiguate, byte-exact round-trip (`archive` then `unarchive` restores `specs/` exactly), drift refusal, `--keep-specs`, destination-collision abort, pre-snapshot degradation, atomic "no files changed" on failure, Windows `moveDirectory` path, and template-generation snapshots. Per [openspec/config.yaml](../../config.yaml), run on Windows CI. + +## Issues addressed + +All references verified against `Fission-AI/OpenSpec` at `main` (`546224e`) on 2026-06-29. No prior unarchive/rollback/restore issue or PR exists — this is greenfield. + +Closes / delivers: + +- The maintainer's Discord commitment to add `/opsx:unarchive` (or `openspec unarchive `) that "moves the folder back and reverses the spec merge for you," including the hard case where the archive is buried in a multi-file commit so `git revert` is not an option. + +Directly enables / strengthens: + +- [#682](https://github.com/Fission-AI/OpenSpec/issues/682) — archive "is not transactional… there's no rollback." Unarchive is that rollback, and is itself atomic. +- [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) — confirms the MODIFIED whole-block-replace loses the pre-image. The reversal snapshot retains it (generalizing #1246's proposed base-fingerprint), and unarchive's drift guard refuses to clobber a requirement a later change has since touched. +- [#863](https://github.com/Fission-AI/OpenSpec/issues/863), [#799](https://github.com/Fission-AI/OpenSpec/issues/799), [#656](https://github.com/Fission-AI/OpenSpec/issues/656) — "the archive skill re-implements merge/move instead of calling the CLI." `/opsx:unarchive` is CLI-first by construction, the pattern these issues ask for. + +Companion answers (the CI-determinism thread): + +- The Discord "inference in CI" question and the [#863](https://github.com/Fission-AI/OpenSpec/issues/863)/[#656](https://github.com/Fission-AI/OpenSpec/issues/656) "why not the deterministic CLI" cluster → recommend a model-free `openspec sync --check` drift gate (companion; see design Decision 6), de-risked by this PR's round-trip determinism. + +Delineated from adjacent work (coordinate, don't collide): + +- [#409](https://github.com/Fission-AI/OpenSpec/issues/409) / [#787](https://github.com/Fission-AI/OpenSpec/pull/787) / [#1192](https://github.com/Fission-AI/OpenSpec/issues/1192) (archive-folder prefix scheme: date → sequence/configurable) — unarchive's resolver is built prefix-tolerant so it survives whichever scheme lands; it does not pick a scheme. +- [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md) — planning-time archive ordering and "parallel work reintroducing assumptions another change removed." Unarchive's drift guard is the runtime counterpart at the spec layer; the two compose (ordering metadata up front, drift refusal at reversal time). +- [#704](https://github.com/Fission-AI/OpenSpec/issues/704) / [#682](https://github.com/Fission-AI/OpenSpec/issues/682) (archive hooks) — if `pre/post-archive` hooks land, a symmetric `pre/post-unarchive` point is the natural extension; named so it fits, not built here. +- [#709](https://github.com/Fission-AI/OpenSpec/issues/709) (`git mv` for archive) — unarchive reuses the same `moveDirectory()` helper, so a future switch to `git mv` covers both directions at once. + +Related, out of scope (referenced, not closed): + +- [#1245](https://github.com/Fission-AI/OpenSpec/issues/1245) — first-class lifecycle timestamps (`created`/`updated`/`archived`). Today archive does not persist an `archived` timestamp (only the folder-name prefix). If/when #1245 lands, unarchive should clear it on restore; this PR notes the hook but does not add the field. +- [#1112](https://github.com/Fission-AI/OpenSpec/issues/1112) — moving archive-time existence checks into `validate`. Unarchive adopts the same "Abort. No files were changed." contract; broadening `validate` itself is that issue's scope. diff --git a/openspec/changes/add-unarchive-command/specs/cli-archive/spec.md b/openspec/changes/add-unarchive-command/specs/cli-archive/spec.md new file mode 100644 index 0000000000..0895a9893c --- /dev/null +++ b/openspec/changes/add-unarchive-command/specs/cli-archive/spec.md @@ -0,0 +1,29 @@ +## ADDED Requirements + +### Requirement: Reversal Snapshot Capture + +When the archive operation rewrites main specs, it SHALL capture a self-contained reversal snapshot inside the change folder so that the operation can later be reversed deterministically by `openspec unarchive`. The snapshot SHALL be forward-only and SHALL NOT alter how archive merges, moves, validates, or what it outputs. + +#### Scenario: Snapshot captured when specs are updated + +- **WHEN** archiving a change applies delta specs to `openspec/specs/` +- **THEN** the command records, for each affected spec, its pre-merge content and its post-merge content digest in a reversal snapshot stored inside the change folder +- **AND** the snapshot moves into `openspec/changes/archive/-/` together with the rest of the change + +#### Scenario: Created specs marked absent + +- **WHEN** archiving creates a new spec that did not previously exist +- **THEN** the snapshot records the pre-merge state of that spec as absent +- **AND** unarchive can delete it to restore the pre-archive state + +#### Scenario: No snapshot when specs are not updated + +- **WHEN** archiving runs with `--skip-specs`, or the change has no delta specs to apply +- **THEN** no spec content is changed +- **AND** no reversal snapshot is written + +#### Scenario: Archive behavior and output unchanged + +- **WHEN** capturing the reversal snapshot +- **THEN** the merge, move, validation, and confirmation behavior of archive are unchanged +- **AND** the command's human-readable and `--json` output are unchanged diff --git a/openspec/changes/add-unarchive-command/specs/cli-unarchive/spec.md b/openspec/changes/add-unarchive-command/specs/cli-unarchive/spec.md new file mode 100644 index 0000000000..e86a95c707 --- /dev/null +++ b/openspec/changes/add-unarchive-command/specs/cli-unarchive/spec.md @@ -0,0 +1,175 @@ +## ADDED Requirements + +### Requirement: Unarchive Command + +The system SHALL provide an `openspec unarchive [change-name]` command that is the inverse of `openspec archive`: it moves a change folder out of `openspec/changes/archive/-/` back to `openspec/changes//` and reverses the spec merge that archiving applied to `openspec/specs/`. + +#### Scenario: Restore an archived change + +- **WHEN** the user runs `openspec unarchive ` for a change that was archived by this version or later +- **THEN** the command restores `openspec/specs/` to its pre-archive state +- **AND** moves the change folder back to `openspec/changes//` +- **AND** reports the restored change name and location + +#### Scenario: Interactive selection + +- **WHEN** no change-name is provided in an interactive session +- **THEN** the command lists archived changes (most-recently-archived first) and prompts the user to select one +- **AND** it does not auto-select + +#### Scenario: Non-interactive requires a name + +- **WHEN** the command is run with `--json` (or otherwise non-interactively) and no change-name is provided +- **THEN** it fails with a machine-readable diagnostic stating a change name is required +- **AND** exits with a non-zero status code + +### Requirement: Archived Change Resolution + +The command SHALL resolve the target archived directory from either a bare change `` or a full archived directory id, treating the leading prefix as opaque up to the name, and SHALL never silently choose among multiple matches. + +#### Scenario: Resolve a bare name with a single match + +- **WHEN** exactly one archived directory matches `` after stripping its leading prefix +- **THEN** the command resolves to that directory + +#### Scenario: Tolerate differing prefix schemes + +- **WHEN** an archived directory uses a date prefix (`YYYY-MM-DD-`) or a sequence/other prefix (e.g. `NNN-`) +- **THEN** the command resolves `` regardless of which prefix scheme produced the directory + +#### Scenario: Ambiguous bare name in interactive mode + +- **WHEN** more than one archived directory matches a bare `` +- **AND** the session is interactive +- **THEN** the command lists the matching directories (most-recent first) and prompts the user to choose +- **AND** it does not auto-select + +#### Scenario: Ambiguous bare name in non-interactive mode + +- **WHEN** more than one archived directory matches a bare `` +- **AND** the command is run with `--json` or otherwise non-interactively +- **THEN** it fails with a diagnostic listing the candidate directory ids +- **AND** directs the user to re-run with the full archived directory id + +#### Scenario: Full directory id resolves unambiguously + +- **WHEN** the user passes the full archived directory id (including its prefix) +- **THEN** the command resolves to exactly that directory without prompting + +#### Scenario: No matching archive + +- **WHEN** no archived directory matches the provided name or id +- **THEN** the command fails with a clear not-found error and makes no changes + +### Requirement: Deterministic Spec Reversal + +The command SHALL reverse the spec merge deterministically by restoring, for each affected spec, the pre-merge content recorded in the change's reversal snapshot — recreating specs that archiving deleted and deleting specs that archiving created — without re-parsing or inferring requirement content. + +#### Scenario: Restore modified and removed requirements exactly + +- **WHEN** the archived change's snapshot records pre-merge content for an affected spec +- **THEN** the command restores that spec to the recorded pre-merge bytes +- **AND** requirements that were MODIFIED or REMOVED during archiving are restored to their exact prior content + +#### Scenario: Reverse added requirements + +- **WHEN** archiving added requirements to a spec +- **THEN** restoring the recorded pre-merge content removes exactly those added requirements + +#### Scenario: Recreate a spec that archiving created + +- **WHEN** archiving created a new spec (the snapshot marks the pre-image as absent) +- **THEN** the command deletes that spec on reversal, returning `openspec/specs/` to its pre-archive shape + +#### Scenario: Round-trip is byte-exact + +- **WHEN** a change is archived and then unarchived with no intervening edits to the affected specs +- **THEN** `openspec/specs/` is byte-for-byte identical to its state before the archive + +### Requirement: Drift Guard + +The command SHALL verify, before reversing any spec, that each affected spec still matches the post-merge state recorded in the reversal snapshot, and SHALL refuse to reverse a spec that has drifted rather than overwrite later changes. + +#### Scenario: Refuse on drift + +- **WHEN** an affected spec's current content no longer matches the post-merge digest recorded in the snapshot (for example, a later change modified the same requirement) +- **THEN** the command refuses to reverse the spec merge +- **AND** it reports which specs drifted +- **AND** it directs the user to re-run with `--keep-specs` to restore the folder without touching specs + +#### Scenario: Proceed when no drift + +- **WHEN** every affected spec still matches its recorded post-merge state +- **THEN** the command proceeds with the deterministic spec reversal + +### Requirement: Keep Specs Option + +The command SHALL support a `--keep-specs` flag that restores the change folder to active without modifying `openspec/specs/`, and SHALL accept `--skip-specs` as an equivalent alias. + +#### Scenario: Restore folder without touching specs + +- **WHEN** the user runs `openspec unarchive --keep-specs` +- **THEN** the command moves the change folder back to `openspec/changes//` +- **AND** it makes no changes to `openspec/specs/` +- **AND** it does not perform drift or reversal checks on specs + +#### Scenario: Skip-specs alias + +- **WHEN** the user runs `openspec unarchive --skip-specs` +- **THEN** the command behaves identically to `--keep-specs` + +### Requirement: Atomic Operation + +The command SHALL apply the reversal atomically: it stages and validates all changes first, and on any failure it leaves the filesystem unchanged. + +#### Scenario: Failure leaves no partial state + +- **WHEN** any step of the reversal fails (for example, the folder move fails after specs were restored) +- **THEN** the command reports `Abort. No files were changed.` +- **AND** both `openspec/specs/` and the folder location are returned to their pre-command state + +#### Scenario: Destination already exists + +- **WHEN** an active change directory `openspec/changes//` already exists +- **THEN** the command fails without overwriting it +- **AND** it makes no changes to specs + +### Requirement: Backward Compatibility For Pre-Snapshot Archives + +For changes archived before reversal snapshots existed, the command SHALL reverse the operations it can invert from the archived delta alone and SHALL refuse to guess the operations it cannot, never producing an incorrect spec. + +#### Scenario: Reverse the self-invertible operations + +- **WHEN** an archived change has no reversal snapshot +- **AND** its delta contains only ADDED and/or RENAMED requirements +- **THEN** the command reverses them by delta inversion (removing added requirements, renaming renamed requirements back) +- **AND** restores `openspec/specs/` accordingly + +#### Scenario: Refuse to guess irreversible operations + +- **WHEN** an archived change has no reversal snapshot +- **AND** its delta contains MODIFIED or REMOVED requirements (whose pre-image is not recoverable from the delta) +- **THEN** the command does not modify those specs +- **AND** it reports which requirements cannot be safely reversed +- **AND** it directs the user to `--keep-specs` (and optionally to git-based recovery) + +#### Scenario: Keep-specs always available + +- **WHEN** an archived change has no reversal snapshot +- **AND** the user runs `openspec unarchive --keep-specs` +- **THEN** the command restores the folder without touching specs, regardless of which delta operations the change contains + +### Requirement: Error Conditions + +The command SHALL handle error conditions gracefully and consistently with `openspec archive`. + +#### Scenario: Missing archive directory + +- **WHEN** no `openspec/changes/archive/` directory exists +- **THEN** the command fails with a clear message and makes no changes + +#### Scenario: JSON diagnostics + +- **WHEN** the command is run with `--json` and a blocked condition occurs (not found, ambiguous, drift, destination exists, or pre-snapshot irreversibility) +- **THEN** it emits a machine-readable diagnostic with a stable code +- **AND** exits with a non-zero status code diff --git a/openspec/changes/add-unarchive-command/specs/opsx-unarchive-skill/spec.md b/openspec/changes/add-unarchive-command/specs/opsx-unarchive-skill/spec.md new file mode 100644 index 0000000000..742592f506 --- /dev/null +++ b/openspec/changes/add-unarchive-command/specs/opsx-unarchive-skill/spec.md @@ -0,0 +1,56 @@ +## ADDED Requirements + +### Requirement: OPSX Unarchive Skill + +The system SHALL provide an `/opsx:unarchive` skill that restores an archived change to active, delegating the deterministic work to the `openspec unarchive` CLI rather than moving folders or editing specs itself. + +#### Scenario: Unarchive a selected change + +- **WHEN** the agent executes `/opsx:unarchive` with a change name +- **THEN** the skill invokes `openspec unarchive ` to perform the restore +- **AND** it displays the CLI's result (restored location and spec-reversal outcome) + +#### Scenario: Change selection prompt + +- **WHEN** the agent executes `/opsx:unarchive` without specifying a change +- **AND** the change cannot be inferred from conversation context +- **THEN** the skill lists archived changes (via `openspec list --archived --json`) and asks the user to choose +- **AND** it never auto-selects, including when several archives share a name + +### Requirement: CLI Delegation + +The skill SHALL perform all deterministic operations — resolution, spec reversal, drift checking, folder move, and atomicity — by calling the `openspec unarchive` CLI, and SHALL NOT re-implement that logic in prose. + +#### Scenario: No manual file operations + +- **WHEN** the skill restores a change +- **THEN** it does not manually move the change directory +- **AND** it does not manually edit files under `openspec/specs/` +- **AND** it relies on the CLI for the reversal + +#### Scenario: Surface CLI guardrails verbatim + +- **WHEN** the CLI refuses to reverse specs because they have drifted, or because a pre-snapshot change contains irreversible operations +- **THEN** the skill reports that refusal to the user +- **AND** it offers the `--keep-specs` option the CLI recommends rather than attempting its own workaround + +### Requirement: Confirmation And Output + +The skill SHALL confirm the action before running it and present a clear summary of the outcome. + +#### Scenario: Confirm before restoring + +- **WHEN** the skill has resolved which archived change to restore +- **THEN** it shows the user what it will do (target change, whether specs will be reversed or kept) before invoking the CLI +- **AND** it proceeds only after the user confirms + +#### Scenario: Summarize the result + +- **WHEN** the CLI completes +- **THEN** the skill displays the restored change name, its new active location, and whether specs were reversed or kept + +#### Scenario: Report a clean failure + +- **WHEN** the CLI aborts (for example, the destination already exists or the change name is ambiguous) +- **THEN** the skill reports the CLI's diagnostic and the suggested next step +- **AND** it does not attempt a manual fallback diff --git a/openspec/changes/add-unarchive-command/tasks.md b/openspec/changes/add-unarchive-command/tasks.md new file mode 100644 index 0000000000..f6e061f366 --- /dev/null +++ b/openspec/changes/add-unarchive-command/tasks.md @@ -0,0 +1,59 @@ +# Tasks: `openspec unarchive` — deterministic inverse of archive + +> Sequenced so each layer is independently testable: archive-side snapshot (the reversibility primitive) → resolution → deterministic reversal + guards → CLI surface → skill/wiring → docs → e2e. Cross-platform concerns (`path.join`, Windows `moveDirectory`, newline-normalized digests) are called out where relevant, per [openspec/config.yaml](../../config.yaml). The CI drift-gate companion (design Decision 6) is intentionally **not** in these tasks — it is a separable follow-up. + +## 1. Archive-side: reversal snapshot (the reversibility primitive) + +- [ ] 1.1 Define the snapshot format: a self-contained record inside the change folder (e.g. `.openspec/unarchive/` with a manifest + per-spec pre-image files) capturing, for each affected spec, its **pre-merge content** (or an explicit `absent` marker when archive creates the spec) and its **post-merge content digest** (newline-normalized CRLF→LF, scheme-tagged so future canonicalization changes are detectable). Document it in design if the shape shifts during build. +- [ ] 1.2 In `ArchiveCommand` ([src/core/archive.ts](../../../src/core/archive.ts)), at the spec-rewrite point (~414-472), capture each target's pre-image **before** `writeUpdatedSpec`, and the post-merge digest **after**, then persist the snapshot into the change folder **before** `moveDirectory` so it travels into `archive/`. +- [ ] 1.3 Strictly forward-only: assert archive's merge/move/validate behavior and stdout (human + `--json`) are byte-for-byte unchanged when no one reads the snapshot. Add a regression test pinning existing archive output. +- [ ] 1.4 Tests: snapshot present after archiving a change that ADDs/MODIFIEs/REMOVEs/RENAMEs/creates a spec; pre-image matches pre-archive bytes exactly; `absent` marker recorded for created specs; digest stable across CRLF vs LF; `--skip-specs` archive writes **no** snapshot (nothing was merged). + +## 2. Resolve an archived change (prefix-tolerant, never auto-pick) + +- [ ] 2.1 Add a resolver that maps a bare `` or a full archived dir id to a candidate set under `changes/archive/`, treating the leading prefix as **opaque up to ``** (strip `YYYY-MM-DD-`; tolerate `NNN-`/ISO/configurable per #409/#787/#1192). Reuse `getArchivedChangesData()` ([src/core/view.ts](../../../src/core/view.ts) / [src/core/list.ts](../../../src/core/list.ts), #399). +- [ ] 2.2 Ambiguity policy: >1 match → interactive prompt (most-recent first, never auto-select); `--json`/non-interactive → error listing candidates and require the full id. +- [ ] 2.3 Tests: single match resolves; date-prefixed and (simulated) sequence-prefixed dirs both resolve; bare-name with two matches → prompt (interactive) / error+candidates (`--json`); full-id always resolves unambiguously; unknown name → clean not-found error. + +## 3. Deterministic reversal + guards (core) + +- [ ] 3.1 Create `src/core/unarchive.ts` with `UnarchiveCommand` mirroring `ArchiveCommand` structure (human + `--json`, `resolveOpenSpecRoot`, blocked-error → diagnostic, exit codes). +- [ ] 3.2 Snapshot path: restore each affected spec's recorded pre-image (rewrite modified, recreate deleted, delete archive-created specs) — a byte-exact file restore, no re-parsing. +- [ ] 3.3 Drift guard: before restoring, compare each spec's current content to the snapshot's post-merge digest; on drift, **refuse** the spec reversal and direct the user to `--keep-specs`. (Same-scheme digests only; unknown scheme → treat as drift/unknown, refuse.) +- [ ] 3.4 Destination-collision guard: if `changes//` already exists, abort (mirror archive's no-overwrite). +- [ ] 3.5 Atomicity: stage + validate the full reversal first; commit order = restore specs → move folder (`moveDirectory`, reuse Windows EPERM/EXDEV fallback); on any failure, *"Abort. No files were changed."*, rolling back restored specs from the still-present snapshot if the move fails. +- [ ] 3.6 `--keep-specs`: skip all spec work; pure folder move; always succeeds when 3.4 passes. +- [ ] 3.7 Tests: byte-exact round-trip (`archive` then `unarchive` restores `specs/` exactly) for ADD/MODIFY/REMOVE/RENAME/create; drift → refusal (no writes); destination collision → abort; `--keep-specs` leaves `specs/` untouched; atomic abort leaves zero partial state; Windows `moveDirectory` path. + +## 4. Backward compatibility: pre-snapshot archives (graceful degradation) + +- [ ] 4.1 Factor the self-invertible half out of `specs-apply.ts`: delta-inversion for **ADDED** (remove by name) and **RENAMED** (rename `TO`→`FROM`, rewriting the header line). No change to forward `buildUpdatedSpec`. +- [ ] 4.2 When no snapshot exists, reverse ADDED/RENAMED via 4.1; for any REMOVED/MODIFIED, **refuse to guess** — list each requirement that cannot be safely restored and direct to `--keep-specs`. +- [ ] 4.3 (Optional, opt-in) git assist: if a clean pre-archive commit is identifiable, offer to recover the pre-image from git; never automatic. +- [ ] 4.4 Tests: pre-snapshot archive with only ADDED/RENAMED → fully reversed; with REMOVED/MODIFIED → refusal naming the requirements; `--keep-specs` always works regardless of snapshot presence. + +## 5. CLI surface + +- [ ] 5.1 Register `unarchive [change-name]` in [src/cli/index.ts](../../../src/cli/index.ts) mirroring archive (326-343): `--keep-specs` (with hidden `--skip-specs` alias), `-y/--yes`, `--no-validate`, `--json`, `--store`, hidden store-path; action → `new UnarchiveCommand().execute(...)`. +- [ ] 5.2 `--json` is non-interactive: ambiguity/drift/collision become machine-readable diagnostics with non-zero exit (mirror `ArchiveBlockedError`). +- [ ] 5.3 Tests: flag parsing, `--json` shape on success and each blocked path, `--store` resolution. + +## 6. The `/opsx:unarchive` skill (delegates to the CLI) + +- [ ] 6.1 Create `src/core/templates/workflows/unarchive-change.ts` with `getUnarchiveChangeSkillTemplate()` + `getOpsxUnarchiveCommandTemplate()`, modeled on `archive-change.ts` but **invoking `openspec unarchive`** for all deterministic work (selection via `openspec list --archived --json`; confirm; render CLI result). No hand-moving folders, no hand-editing specs (anti-#863). +- [ ] 6.2 Encode guardrails: never auto-select among ambiguous archives; surface the CLI's drift refusal and the `--keep-specs` option verbatim; do not re-implement reversal logic in prose. +- [ ] 6.3 Export from `skill-templates.ts`; add `unarchive` to `ALL_WORKFLOWS` ([src/core/profiles.ts:19](../../../src/core/profiles.ts)) and `WORKFLOW_TO_SKILL_DIR` ([src/core/init.ts:65](../../../src/core/init.ts)); **do not** add to `CORE_WORKFLOWS` (#913/#762). +- [ ] 6.4 Tests: template-generation snapshot; assert the template calls the CLI and contains no manual move/merge instructions. + +## 7. Docs + +- [ ] 7.1 Add a `/opsx:unarchive` row to the command table in `docs/opsx.md` and a short "Undoing an archive" section (covering `--keep-specs` and the drift-refusal behavior). +- [ ] 7.2 Note in docs that the deterministic reversal requires a change archived by this version or later; older archives degrade per design Decision 5. + +## 8. End-to-end verification + +- [ ] 8.1 E2E: scaffold a spec-driven change with one ADDED, one MODIFIED, one REMOVED, one RENAMED requirement; `archive`; `unarchive`; assert `specs/` is byte-identical to pre-archive and the folder is back under `changes//`. +- [ ] 8.2 E2E stacked-archive drift: archive change A (MODIFIES req X), then a second change re-MODIFIES X and is archived; `unarchive A` refuses spec reversal and `unarchive A --keep-specs` succeeds with `specs/` untouched. +- [ ] 8.3 E2E ambiguity: two archived entries for one name → `--json` errors with candidates; full-id resolves. +- [ ] 8.4 Validation: `openspec validate add-unarchive-command --strict` passes; `openspec status --change add-unarchive-command` shows artifacts complete. +- [ ] 8.5 Run the suite on macOS, Linux, and Windows CI. From babf94e007b516baaa99e6e8013d73140eb8e257 Mon Sep 17 00:00:00 2001 From: Clay Good Date: Mon, 29 Jun 2026 16:42:54 -0500 Subject: [PATCH 2/8] docs(openspec): pivot to deterministic spec-merge engine (sync + unarchive) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Re-scope the unarchive proposal around the long-term direction the full Discord thread converged on: the delta->specs merge should be a deterministic, idempotent, REVERSIBLE pure-code engine, exposed as first-class commands. Unarchive becomes one payoff of that engine, alongside a standalone `openspec sync`, a model-free CI/pre-commit drift gate, and crumb-free re-merge. Key insight: one primitive — a per-change applied-delta baseline (pre-image + applied-result digest) — unifies three asks the chat raised separately: deterministic reverse (unarchive), idempotent "no crumbs" re-sync, and hash-based drift detection. Frozen into the archive, it is the unarchive reversal snapshot. Adopted from the discussion: - deterministic, code-only `openspec sync` + `--check` (no model in CI) - `sync --fix` as eslint-style pre-commit auto-fixer - skills (/opsx:sync, /opsx:unarchive) delegate to the CLI; agent never merges (fixes the #1246 "intelligent merge drops scenarios" failure mode) Rejected (documented): removing archive / folding merge into apply — preserves the specs/ = shipped-reality invariant (design Decision 5). Rename change add-unarchive-command -> add-deterministic-sync-and-unarchive. Spec deltas: cli-sync (new), cli-unarchive (new), opsx-unarchive-skill (new), cli-archive (baseline + shared engine), specs-sync-skill (CLI delegation). Validates with `openspec validate --strict`. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../.openspec.yaml | 0 .../design.md | 108 +++++++++++++++ .../proposal.md | 103 ++++++++++++++ .../specs/cli-archive/spec.md | 39 ++++++ .../specs/cli-sync/spec.md | 130 ++++++++++++++++++ .../specs/cli-unarchive/spec.md | 24 ++-- .../specs/opsx-unarchive-skill/spec.md | 0 .../specs/specs-sync-skill/spec.md | 37 +++++ .../tasks.md | 62 +++++++++ .../changes/add-unarchive-command/design.md | 90 ------------ .../changes/add-unarchive-command/proposal.md | 106 -------------- .../specs/cli-archive/spec.md | 29 ---- .../changes/add-unarchive-command/tasks.md | 59 -------- 13 files changed, 491 insertions(+), 296 deletions(-) rename openspec/changes/{add-unarchive-command => add-deterministic-sync-and-unarchive}/.openspec.yaml (100%) create mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/design.md create mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/proposal.md create mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-archive/spec.md create mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md rename openspec/changes/{add-unarchive-command => add-deterministic-sync-and-unarchive}/specs/cli-unarchive/spec.md (86%) rename openspec/changes/{add-unarchive-command => add-deterministic-sync-and-unarchive}/specs/opsx-unarchive-skill/spec.md (100%) create mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/specs/specs-sync-skill/spec.md create mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/tasks.md delete mode 100644 openspec/changes/add-unarchive-command/design.md delete mode 100644 openspec/changes/add-unarchive-command/proposal.md delete mode 100644 openspec/changes/add-unarchive-command/specs/cli-archive/spec.md delete mode 100644 openspec/changes/add-unarchive-command/tasks.md diff --git a/openspec/changes/add-unarchive-command/.openspec.yaml b/openspec/changes/add-deterministic-sync-and-unarchive/.openspec.yaml similarity index 100% rename from openspec/changes/add-unarchive-command/.openspec.yaml rename to openspec/changes/add-deterministic-sync-and-unarchive/.openspec.yaml diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/design.md b/openspec/changes/add-deterministic-sync-and-unarchive/design.md new file mode 100644 index 0000000000..eda0947bda --- /dev/null +++ b/openspec/changes/add-deterministic-sync-and-unarchive/design.md @@ -0,0 +1,108 @@ +# Design: deterministic spec-merge engine — `sync` (forward), `unarchive` (reverse) + +## Context + +OpenSpec's spec merge (delta → `specs/`) is reached two ways today: deterministically but only *inside* `openspec archive` (fused to the folder move), or by an AI agent in `/opsx:sync` that "directly edits main specs." The standalone deterministic apply (`applySpecs()`, [src/core/specs-apply.ts:391](../../../src/core/specs-apply.ts)) has zero callers. The Discord thread (samholmes ↔ Clay, 2026-06) worked from "add unarchive" to a single root cause: **the merge must be deterministic, idempotent, and reversible pure code, exposed as commands** — then unarchive, a code-only CI drift gate, a `sync --fix` pre-commit hook, and crumb-free re-merge all fall out of one primitive. + +This design covers that primitive and the commands built on it. It supersedes the earlier "unarchive + separable CI companion" framing of this PR: the deterministic engine is no longer a companion, it is the spine. + +## Goals / Non-Goals + +**Goals** +- A deterministic merge: same (delta + base) → byte-identical `specs/`, in pure code, on every platform. +- Idempotent: re-running `sync` is a no-op; revising a delta regenerates `specs/` with no crumbs. +- Reversible: `unarchive` restores `specs/` exactly and moves the folder back, atomically. +- Code-only drift detection (`sync --check`) usable by CI and a pre-commit hook — no model, no API keys. +- Skills delegate the merge to the CLI; the agent never performs it. +- Backward compatible with changes archived (and specs synced) before this feature. + +**Non-Goals** +- Removing or replacing the `archive` lifecycle boundary (Decision 5). +- Folding the spec merge into `apply` (Decision 5). +- Wiring a specific hook runner or CI job into this repo (primitives in scope; config staged — Decision 4). +- A scenario-granular "smart" merge (Decision 8 — the conventions mandate whole-requirement deltas; whole-block apply is the correct, deterministic semantics). +- Bulk `sync`/`unarchive`, archive-folder prefix scheme, lifecycle timestamps — out of scope, accommodated. + +## Decision 1 — One primitive: the applied-delta baseline + +**Decision.** When a change's deltas are merged into `specs/` (by `sync` or `archive`), record a per-change **applied-delta baseline**: for each affected spec, the **pre-merge file content** (the pre-image, or an explicit `absent` marker when the spec is created) and a **digest of the applied result** (newline-normalized CRLF→LF, scheme-tagged). The baseline lives with the change; when archived, it travels into the archive folder. + +**Why one primitive.** The three asks in the thread are the same computation viewed three ways: +- **Reverse** (`unarchive`) = restore the pre-image. Deterministic for *every* op, including the REMOVED/MODIFIED ones the delta alone cannot invert. +- **Idempotent re-merge** (`sync` after a delta revision) = `specs_new = apply(delta_new, base)` where `base = current specs with delta_old reversed` (via the pre-image). This is the "no leftover crumbs" guarantee — the prior revision's contribution is removed, not layered over. +- **Drift** (`sync --check`, the hook, CI) = current `specs/` digest ≠ baseline digest. This is samholmes' "track the hash of the changes/ state applied to the spec," made precise. + +Building three mechanisms would invite three drift bugs. Building one — the pre-image + digest — and deriving the rest is why this is the spine. + +**Form.** Store the whole pre-image (not a reverse-diff), so restore is a byte copy with no re-parsing. Cost is a copy of each affected spec's prior bytes per merge — negligible, and only for changes that touch specs. Coordinate the digest convention with [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278)'s artifact-graph digest ledger so the two drift layers (artifact staleness vs. spec-merge drift) speak the same digest dialect. + +## Decision 2 — Deterministic sync is pure parser code; the agent never merges + +**Decision.** Promote `applySpecs()`/`buildUpdatedSpec` into the canonical merge and expose it as `openspec sync`. The merge is a pure function of (delta files + base spec bytes) producing byte-identical output across runs and platforms (stable requirement ordering, newline normalization, deterministic recomposition). `archive` calls the same function. `/opsx:sync` is rewritten to **invoke the CLI**, not edit specs itself. + +**Why.** This is the thread's keystone verbatim: *"Moving it out of the agent prompt and into plain parser code that always produces byte-identical output for the same delta + base."* The agent path isn't just non-deterministic; its stated reason for existing — "intelligent merging (e.g., adding a scenario without copying the entire requirement)" — is precisely how [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) silently drops sibling scenarios. The conventions spec already requires deltas to carry "the complete modified requirement, not a diff," so whole-block apply is the *correct* semantics and the agent's cleverness is the bug. Deterministic code is therefore more faithful, not less (Decision 8). + +**Alternatives.** Keep agent sync and only diff its output in CI — rejected: needs a model in CI, the exact thing the thread set out to avoid. + +## Decision 3 — Idempotency and "no crumbs" + +**Decision.** `sync` is idempotent: running it on an already-synced, unchanged change writes nothing (`--check` is clean). When a delta is revised, `sync` reverses the prior revision's contribution via the baseline pre-image, then applies the new delta, then refreshes the baseline — so `specs/` equals `apply(current delta, original base)` with zero residue from the old revision. + +**Why.** Directly answers *"a revised delta should let sync regenerate the spec output from scratch, idempotently, no leftover crumbs."* Without the baseline, re-applying a revised delta against already-merged `specs/` either conflicts (ADDED now exists) or layers (renamed-then-renamed) — crumbs. The baseline makes regeneration a clean reverse-then-apply. + +**Edge.** If `specs/` drifted from the baseline since the last sync (a later change touched the same requirement), `sync` does not silently reverse-then-apply over someone else's edit — it reports drift and requires `--check`-style acknowledgement, mirroring `unarchive`'s drift guard (Decision 7). Consistency over cleverness. + +## Decision 4 — `--check` and `--fix`: the gate and the auto-fixer (wiring staged) + +**Decision.** `sync --check` is read-only and exits non-zero when the change's deltas are not cleanly appliable to the base, or — when the team commits merged `specs/` during review — when committed `specs/` ≠ the regenerated output. `sync --fix` (and bare `sync`) regenerates `specs/`. This PR ships both CLI modes and documents two integration patterns; it does **not** add a hook runner or CI job to this repo (none exists today). + +- **CI drift gate**: a job runs `openspec sync --check` (or `--all`) and fails the PR on drift — "the same pattern as a codegen or IaC drift gate," as a plain binary. +- **pre-commit hook**: `openspec sync --check` detects drift before `git commit --amend`; `openspec sync --fix` is the eslint-`--fix`-style auto-remediation, after which the amend proceeds with `specs/` canonical. + +**Why staged.** The primitives are the reusable, testable part and belong here. Choosing husky vs. lefthook vs. native hooks, and the CI matrix, are repo-policy calls better made as a focused follow-up than bundled into a foundational engine PR. Keeping them separable also keeps this PR reviewable. + +## Decision 5 — Keep the `archive` boundary; reject folding the merge into `apply` + +**Decision.** The lifecycle boundary stays: `specs/` is written at `sync`/`archive`, and `archive` remains the "fold finished change into shipped specs, then move the folder" step. Reject samholmes' proposal to remove `archive`, keep every change in a dated archive dir for its whole life, and make `apply` do both code and spec application. + +**Why.** The invariant that pays for OpenSpec's value is **`specs/` describes only shipped reality**. Folding the merge into `apply` would (a) pollute the source of truth with proposed-but-unmerged or abandoned changes, and (b) let parallel changes overwrite each other's specs before any lands. The thread's own resolution: the awkwardness of the "final archive step" is not the boundary's fault, it is the *non-determinism* of crossing it. Make the crossing deterministic and reversible (Decisions 1–3) and the boundary becomes cheap in both directions — which is the actual fix. This is why determinism, not deletion, is the spine of this PR. + +**Consequence for `sync` on active changes.** Because of the invariant, the *default* workflow still merges at archive. Standalone `sync` serves: the engine archive uses; `--check` (read-only, never pollutes `specs/`); and the opt-in "generated-artifact" workflow where a team chooses to commit merged `specs/` mid-review and gate it. The tool enables both policies; it does not force early merge. + +## Decision 6 — `unarchive`: resolution, prefix-tolerance, never auto-pick + +**Decision.** `unarchive [change-name]` accepts a bare `` or a full archived directory id, treating the leading prefix as **opaque up to ``** — strips `YYYY-MM-DD-` today, tolerates `NNN-`/ISO/configurable ([#409](https://github.com/Fission-AI/OpenSpec/issues/409)/[#787](https://github.com/Fission-AI/OpenSpec/pull/787)/[#1192](https://github.com/Fission-AI/OpenSpec/issues/1192)). Multiple matches for a bare name → interactive prompt (most-recent first, never auto-select); `--json`/non-interactive → error listing candidates, require the full id. Reuse `getArchivedChangesData()` ([#399](https://github.com/Fission-AI/OpenSpec/pull/399)). + +**Why.** Multiple archives of one name already happen (different dates), and the proposed sequence scheme makes it routine. Silent selection is a guess on the rare, costly path. + +## Decision 7 — Reverse under a drift guard, atomically + +**Decision.** Before reversing any spec, compare its current content to the baseline's applied-result digest. On drift (a later change touched the same requirement — [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246), [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md)), **refuse** and point to `--keep-specs`. Stage and validate the whole reversal first; commit order = restore/delete specs → move folder (`moveDirectory`, Windows EPERM/EXDEV fallback). On any failure: *"Abort. No files were changed."*, rolling restored specs back from the still-present baseline if the move fails. + +**Why.** Restoring a pre-image over a requirement a later change modified would silently delete that change's contribution — the loss OpenSpec guards against. Refusing is strictly safer, and `--keep-specs` always lets the user proceed. Atomicity gives archive the rollback [#682](https://github.com/Fission-AI/OpenSpec/issues/682) noted it lacks, and matches the abort contract [#1112](https://github.com/Fission-AI/OpenSpec/issues/1112) describes. + +## Decision 8 — `--keep-specs` and naming vs. `--skip-specs` + +**Decision.** `--keep-specs` moves the folder back and leaves `specs/` untouched — always deterministic, always safe. Primary spelling `--keep-specs` (the request's; reads naturally for the inverse). Accept `--skip-specs` as a hidden alias for muscle-memory symmetry with archive ([#28](https://github.com/Fission-AI/OpenSpec/pull/28)). When `unarchive` cannot safely reverse specs (drift, or pre-baseline REMOVED/MODIFIED) and `--keep-specs` was not given, it refuses and *recommends* `--keep-specs` — it does not silently fall back. **Open for owner confirmation:** if strict symmetry is preferred, swap which spelling is primary. + +## Decision 9 — Backward compatibility: pre-baseline archives and synced specs + +**Decision.** Changes archived/synced before baselines existed have none. `unarchive` then reverses **ADDED**/**RENAMED** by delta inversion (the self-invertible half) and, for any **REMOVED**/**MODIFIED**, **refuses to guess** — naming each requirement it cannot safely restore and directing to `--keep-specs` (optionally offering opt-in git recovery of the pre-image). `sync` on a pre-baseline change establishes a baseline on its next run. Never silently emit a wrong spec. + +**Why.** The feature must be useful on day one, including for already-archived changes, without ever degrading into corruption. Reverse what is provably reversible; stop honestly on the rest. + +## Decision 10 — Move strategy and lifecycle metadata + +- **Move**: reuse `moveDirectory()`/`copyDirRecursive()` ([src/core/archive.ts:96-128](../../../src/core/archive.ts)) and their Windows fallbacks ([#605](https://github.com/Fission-AI/OpenSpec/pull/605)). A future `git mv` ([#709](https://github.com/Fission-AI/OpenSpec/issues/709)) covers both directions through this one helper. +- **Metadata**: archive persists no `archived` timestamp today (only the folder-name prefix). If [#1245](https://github.com/Fission-AI/OpenSpec/issues/1245) lands one, `unarchive` should null it on restore — noted, not built. + +## Risks / trade-offs + +- **Behavior shift for `/opsx:sync` users.** Replacing agent merge with deterministic merge changes output for anyone who relied on the agent's scenario-level merges. This is intended (it fixes [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)), but it is a real change; call it out in the changeset and docs, and keep the conventions' "complete requirement, not a diff" rule prominent so deltas are authored to merge cleanly. +- **Baseline storage in the change folder.** Adds a small artifact; inert for spec-less changes. Format is scheme-tagged so canonicalization can evolve without silent mis-compares. +- **Forward round-trip is not guaranteed identity.** `unarchive` restores the exact pre-archive `specs/`; a *subsequent* re-archive re-runs the forward merge, which inherits archive's existing cross-change caveats ([#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)). Documented, not solved here. +- **Scope.** This is larger than a lone `unarchive` command. It is intentionally the foundational engine the thread converged on; tasks phase it so the keystone (engine + `sync --check`) is independently shippable before reverse, idempotency, and skill delegation. + +## Migration / rollout + +Additive and phased (see tasks). The engine + baseline land first (no user-visible change to `archive` output). `sync` and `unarchive` are new commands in the expanded profile. `/opsx:sync` delegation ships with a changeset noting the determinism shift. Pre-baseline changes degrade per Decision 9. Hook/CI wiring is a follow-up. diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md b/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md new file mode 100644 index 0000000000..2c08fdae5f --- /dev/null +++ b/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md @@ -0,0 +1,103 @@ +## Why + +The Discord thread that asked for `openspec unarchive` ended somewhere bigger: the spec merge (delta → `specs/`) is **the** load-bearing operation in OpenSpec, and today it is reachable only two ways — buried *inside* `openspec archive`, or performed by an **AI agent** in `/opsx:sync` ("This is an agent-driven operation… you will read delta specs and directly edit main specs"). Both are problems. The agent path is non-deterministic (wording drift, and the "intelligent" scenario-merge is the very mechanism that silently drops scenarios in [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)). The CLI path is deterministic but cannot be run on its own — `applySpecs()` has **zero callers**. + +So you cannot reverse an archive, you cannot gate drift in CI without a model, and you cannot re-merge a revised delta cleanly. One missing primitive sits under all three. This PR builds it: a deterministic, idempotent, **reversible** merge engine, exposed as first-class commands. + +## Background: one engine, three missing directions + +Verified against `Fission-AI/OpenSpec` at `main` (`546224e`, #1248) on 2026-06-29. The merge `buildUpdatedSpec` ([src/core/specs-apply.ts:240-307](../../../src/core/specs-apply.ts)) applies a change's deltas to a base spec in fixed order (RENAMED → REMOVED → MODIFIED → ADDED). That engine is missing three things the workflow needs: + +1. **A standalone forward command.** `openspec sync` does not exist as a binary. The deterministic merge runs only inside `archive` (fused to the folder move) or is re-implemented by the agent in `/opsx:sync`. So the Discord conclusion — *"a deterministic, code-only sync path that CI can run as a plain binary… no model in CI"* — is impossible today. CI can only gate drift by re-running archive or invoking the LLM. + +2. **A reverse direction.** Archiving merges deltas into `specs/` and moves the folder to `changes/archive/`. There is no undo. The maintainer's manual fix (`git revert`, or a hand-scoped `git checkout -- …`) fails in the motivating case — *"the archive is part of a commit with other changes, so I cannot simply revert."* And the reverse is not free: the archived delta is **not self-inverting**. ADDED and RENAMED can be undone from the delta; **REMOVED and MODIFIED cannot** — the forward merge discards the pre-image (a MODIFIED delta carries "the complete modified requirement, not a diff"; [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) documents the same whole-block replace as forward data loss). + +3. **Idempotent regeneration ("no crumbs").** When review feedback revises a delta, re-merging should regenerate `specs/` from scratch with no leftovers from the prior revision — *"a revised delta should let sync regenerate the spec output from scratch, idempotently, no leftover crumbs."* Today re-applying a revised delta against already-merged `specs/` conflicts or double-applies; there is no record of what the prior revision contributed. + +**The unification.** All three need the *same* missing primitive: a per-change **applied-delta baseline** — the pre-image of each affected spec plus a digest of the applied result, recorded whenever a change's deltas are merged. With it: + +- **reverse** is a byte-exact restore of the pre-image (deterministic for every op, including REMOVED/MODIFIED) → `unarchive`; +- **idempotent re-merge** is "reverse the old delta via the baseline, apply the new" → crumb-free `sync`; +- **drift detection** is "current `specs/` digest ≠ baseline digest" (samholmes' "track the hash of the changes/ state applied to the spec") → a code-only gate for CI and a `sync --fix` pre-commit hook (the eslint-`--fix` analogy both arrived at). + +Frozen into the archived folder, that baseline *is* the unarchive reversal snapshot. One primitive, three payoffs. + +## What Changes + +Design principle the whole thread converges on, and the one this project is already adopting elsewhere ([#1278](https://github.com/Fission-AI/OpenSpec/pull/1278), prevent-silent-spec-drop): **the merge is pure parser code that produces byte-identical output for the same delta + base; the agent never performs it.** Ordered by leverage: + +1. **THE ENGINE — deterministic, idempotent, reversible merge core (`cli-sync` NEW, shared by archive).** Promote the existing `applySpecs()` into a first-class, **byte-deterministic** apply that records an applied-delta baseline, and add its inverse. `openspec archive` keeps merging-then-moving but routes its merge through this shared core and writes the baseline before moving (so it travels into the archive). This is the keystone the other three items stand on; it is the "rework it… plain parser code that always produces byte-identical output for the same delta + base" the thread calls for. + +2. **FORWARD COMMAND + DRIFT GATE — `openspec sync [change]` (`cli-sync` NEW).** Apply a change's deltas to `specs/` without archiving, deterministically and idempotently. Three modes: + - default / `--fix`: write `specs/` to the regenerated result (idempotent — re-running is a no-op; revising the delta regenerates with no crumbs); + - `--check`: read-only; exit non-zero if the change's deltas are not cleanly appliable, or (when `specs/` has been synced) if committed `specs/` ≠ the regenerated output. This is the **codegen/IaC-style drift gate CI runs as a plain binary, no model, no API keys** — and the auto-fixer a `pre-commit` hook can call. + +3. **REVERSE COMMAND — `openspec unarchive [change-name]` (`cli-unarchive` NEW).** The deterministic inverse of archive: resolve the archived folder (prefix-tolerant, never auto-picking among ambiguous matches), reverse the spec merge from the baseline under a **drift guard** (refuse rather than clobber a requirement a later change has since touched), move the folder back to `changes//`, **atomically** ("Abort. No files were changed."). `--keep-specs` restores the folder without touching `specs/` (the always-safe escape hatch, mirror of archive's `--skip-specs`). Changes archived before this feature have no baseline and degrade gracefully — reverse the self-invertible half (ADDED/RENAMED), refuse to guess the rest. + +4. **NO MODEL IN THE MERGE — skills delegate to the CLI (`specs-sync-skill` MODIFIED; `opsx-unarchive-skill` NEW).** `/opsx:sync` stops doing agent-driven edits and **invokes `openspec sync`**; the new `/opsx:unarchive` invokes `openspec unarchive`. The deterministic work lives in TypeScript; skills only select, confirm, and render. This is the direction [#863](https://github.com/Fission-AI/OpenSpec/issues/863)/[#799](https://github.com/Fission-AI/OpenSpec/issues/799)/[#656](https://github.com/Fission-AI/OpenSpec/issues/656) ask for, and it removes the [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) "intelligent merge drops scenarios" failure mode by construction. + +### What this deliberately does *not* change + +samholmes proposed removing `archive` entirely — keep every change in its dated archive location for its whole life and fold the spec merge into `apply`. **Rejected, with the maintainer's reasoning, and it shapes the design:** `specs/` only ever describes *shipped* reality. Folding the merge into `apply` would let proposed-but-unmerged (or abandoned) changes pollute the source of truth and let parallel changes step on each other's specs before any land. So the archive boundary stays; what changes is that the merge across it becomes deterministic and reversible. The determinism is what makes the boundary cheap to cross in both directions, which is the real fix for the "awkward final step." (Full treatment in [design.md](./design.md) Decision 5.) + +## Capabilities + +### New Capabilities + +- `cli-sync`: the deterministic, idempotent spec-merge engine exposed as `openspec sync [change]` — `--fix`/default writes `specs/` from the change's deltas (byte-identical for the same delta + base; re-running is a no-op; a revised delta regenerates with no crumbs), `--check` is a read-only, model-free drift gate for CI and pre-commit hooks. Records a per-change applied-delta baseline (pre-image + digest) that also powers reverse and drift detection. The same engine `archive` uses internally. +- `cli-unarchive`: `openspec unarchive [change-name]` — the deterministic inverse of archive. Resolves an archived change (prefix-tolerant, never auto-picking ambiguous matches), reverses the spec merge from the baseline under a drift guard, moves the folder back, atomically. `--keep-specs` restores the folder without touching `specs/`; pre-baseline archives degrade gracefully. +- `opsx-unarchive-skill`: a `/opsx:unarchive` workflow skill that delegates to `openspec unarchive` for all deterministic work (selection, confirmation, rendering only). Expanded profile; no cross-skill dependency. + +### Modified Capabilities + +- `cli-archive`: routes its spec merge through the shared deterministic engine and records the applied-delta baseline inside the change folder before moving it, so archiving becomes deterministically reversible. Forward-only and backward-compatible — no change to how archive merges, moves, validates, or what it prints. +- `specs-sync-skill`: `/opsx:sync` delegates the merge to the deterministic `openspec sync` CLI instead of performing agent-driven edits to `specs/`, making the result byte-deterministic and removing the scenario-dropping "intelligent merge" failure mode. The skill handles selection, confirmation, and output only. + +### Integration surface (primitives in scope; wiring staged) + +- **CI drift gate** and **pre-commit `sync --fix` hook** are the payoff of `cli-sync --check`/`--fix`. This PR delivers the CLI primitives and documents both patterns; wiring a specific hook runner (husky/lefthook — none exists in the repo today) or a CI job is a small, separable follow-up the owner can stage. + +## Impact + +- `src/core/specs-apply.ts` — make `buildUpdatedSpec`/`applySpecs` byte-deterministic and idempotent; add the inverse (delta-inversion for ADDED/RENAMED; pre-image restore for all ops); add baseline read/write. Forward output unchanged for existing callers. +- `src/core/sync.ts` (**new**) — `SyncCommand` (default/`--fix`/`--check`), mirroring `ArchiveCommand`'s human + `--json` shape; writes/refreshes the applied-delta baseline. +- `src/core/unarchive.ts` (**new**) — `UnarchiveCommand`: resolve archived dir, drift-check, restore pre-images (or delta-invert / refuse for pre-baseline), move folder back, atomic abort. Reuse `moveDirectory()`/`copyDirRecursive()` ([src/core/archive.ts:96-128](../../../src/core/archive.ts)). +- `src/core/archive.ts` — route the merge through the shared engine (already deterministic) and persist the baseline before `moveDirectory` (~414-506). No behavior/output change. +- `src/core/change-metadata/` or a sibling baseline store — persist the applied-delta baseline per change (pre-image + digest, newline-normalized, scheme-tagged). Coordinate with [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278)'s digest ledger so the two drift layers share a digest convention. +- `src/core/list.ts` / `src/core/view.ts` — reuse `getArchivedChangesData()` ([#399](https://github.com/Fission-AI/OpenSpec/pull/399)) to resolve/disambiguate archived candidates. +- `src/cli/index.ts` — register `sync [change]` (`--check`, `--fix`, `--json`, `--store`) and `unarchive [change-name]` (`--keep-specs`/`--skip-specs` alias, `-y/--yes`, `--no-validate`, `--json`, `--store`), mirroring archive (326-343). +- `src/core/templates/workflows/sync-specs.ts` — rewrite `/opsx:sync` to invoke `openspec sync` (drop agent-driven edits). `src/core/templates/workflows/unarchive-change.ts` (**new**) — `/opsx:unarchive` delegating to the CLI. Registration: export from `skill-templates.ts`; add `unarchive` to `ALL_WORKFLOWS` ([src/core/profiles.ts:19](../../../src/core/profiles.ts)) and `WORKFLOW_TO_SKILL_DIR` ([src/core/init.ts:65](../../../src/core/init.ts)); **not** added to `CORE_WORKFLOWS`. `docs/opsx.md` gains a `/opsx:unarchive` row and a determinism note for `/opsx:sync`. +- Tests — byte-identical sync determinism (same delta+base → same bytes; CRLF/LF; Windows), idempotency (re-run no-op; revised delta no crumbs), `--check` exit codes, archive→unarchive byte-exact round-trip, drift refusal, `--keep-specs`, destination-collision abort, pre-baseline degradation, atomic "no files changed", skill-template delegation snapshots. Per [openspec/config.yaml](../../config.yaml), run on Windows CI. + +## Issues addressed + +All references verified against `Fission-AI/OpenSpec` at `main` (`546224e`) on 2026-06-29. No prior unarchive/sync-command/rollback issue or PR exists — this is greenfield. + +Delivers (the Discord conclusions): + +- **Deterministic, code-only sync path** *("the right fix is a deterministic, code-only openspec sync/archive path that CI can run as a plain binary… I'll fold this into the unarchive PR")* → `openspec sync` + `--check`. +- **`openspec unarchive` / `/opsx:unarchive`** that "moves the folder back and reverses the spec merge," including the hard case where the archive is buried in a multi-file commit. +- **Idempotent, crumb-free regeneration** and the **`sync --fix` pre-commit hook** *(samholmes' "sync is a lint step… eslint --fix", Clay's "sync acts as the auto-fixer")*. +- **Hash-tracked drift** *(samholmes' "track the hash of the changes/ state applied to the spec")* → the baseline digest. + +Directly fixes / strengthens: + +- [#863](https://github.com/Fission-AI/OpenSpec/issues/863), [#799](https://github.com/Fission-AI/OpenSpec/issues/799), [#656](https://github.com/Fission-AI/OpenSpec/issues/656) — "the archive/sync skill re-implements the merge instead of calling the CLI." Both skills become CLI-first; the merge is one deterministic code path. +- [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) — the agent "intelligent merge" silently drops scenarios. Deterministic whole-block apply (per the conventions spec's "complete modified requirement, not a diff") removes the failure mode, and the baseline retains the pre-image #1246 wants for drift detection. +- [#682](https://github.com/Fission-AI/OpenSpec/issues/682) — archive "is not transactional… there's no rollback." `unarchive` is that rollback, itself atomic. +- [#1112](https://github.com/Fission-AI/OpenSpec/issues/1112) — MODIFIED/REMOVED/RENAMED-from headers absent from base pass `validate` but abort at archive. `sync --check` surfaces the same appliability check earlier, as a gate. + +Delineated from adjacent work (coordinate, don't collide): + +- [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278) (this author's sibling) — artifact-graph drift (proposal→design→tasks staleness) via a content-digest ledger. This PR is the *spec-merge* drift layer (delta→`specs/`). Same digest philosophy, different layer; share the digest/newline-normalization convention. +- [#409](https://github.com/Fission-AI/OpenSpec/issues/409) / [#787](https://github.com/Fission-AI/OpenSpec/pull/787) / [#1192](https://github.com/Fission-AI/OpenSpec/issues/1192) (archive-folder prefix scheme) — unarchive's resolver is prefix-tolerant; it does not pick a scheme. +- [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md) — planning-time archive ordering. Unarchive's drift guard and sync's idempotency are the runtime spec-layer counterpart; the two compose. +- [#704](https://github.com/Fission-AI/OpenSpec/issues/704)/[#682](https://github.com/Fission-AI/OpenSpec/issues/682) (archive hooks), [#709](https://github.com/Fission-AI/OpenSpec/issues/709) (`git mv`) — symmetric `unarchive` hook points and a shared `moveDirectory()` make a future switch cover both directions. + +Considered and rejected (documented in design): + +- **Remove `archive`; fold the merge into `apply`; edit changes in place in dated dirs** (samholmes' proposal). Rejected to preserve the invariant that `specs/` describes only shipped reality (design Decision 5). + +Related, out of scope (referenced, not closed): + +- [#1245](https://github.com/Fission-AI/OpenSpec/issues/1245) — first-class lifecycle timestamps. If/when `archived` is persisted, `unarchive` should clear it; noted, not built. diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-archive/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-archive/spec.md new file mode 100644 index 0000000000..5496cd72ac --- /dev/null +++ b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-archive/spec.md @@ -0,0 +1,39 @@ +## ADDED Requirements + +### Requirement: Applied-Delta Baseline Capture + +When the archive operation rewrites main specs, it SHALL record a self-contained applied-delta baseline inside the change folder so that the operation can later be reversed deterministically by `openspec unarchive`. The baseline SHALL be forward-only and SHALL NOT alter how archive merges, moves, validates, or what it outputs. + +#### Scenario: Baseline captured when specs are updated + +- **WHEN** archiving a change applies delta specs to `openspec/specs/` +- **THEN** the command records, for each affected spec, its pre-merge content and its applied-result digest in an applied-delta baseline stored inside the change folder +- **AND** the baseline moves into `openspec/changes/archive/-/` together with the rest of the change + +#### Scenario: Created specs marked absent + +- **WHEN** archiving creates a new spec that did not previously exist +- **THEN** the baseline records the pre-merge state of that spec as absent +- **AND** unarchive can delete it to restore the pre-archive state + +#### Scenario: No baseline when specs are not updated + +- **WHEN** archiving runs with `--skip-specs`, or the change has no delta specs to apply +- **THEN** no spec content is changed +- **AND** no applied-delta baseline is written + +#### Scenario: Archive behavior and output unchanged + +- **WHEN** capturing the applied-delta baseline +- **THEN** the merge, move, validation, and confirmation behavior of archive are unchanged +- **AND** the command's human-readable and `--json` output are unchanged + +### Requirement: Shared Deterministic Merge Engine + +The archive operation SHALL apply delta specs using the same deterministic merge engine as `openspec sync`, so that archiving and syncing produce identical spec output for the same change. + +#### Scenario: Archive merge matches sync + +- **WHEN** archiving applies a change's deltas to `openspec/specs/` +- **THEN** the resulting spec content is identical to what `openspec sync` produces for the same change +- **AND** the merge is performed in code without AI inference diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md new file mode 100644 index 0000000000..af46973a98 --- /dev/null +++ b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md @@ -0,0 +1,130 @@ +## ADDED Requirements + +### Requirement: Sync Command + +The system SHALL provide an `openspec sync [change-name]` command that applies a change's delta specs to `openspec/specs/` deterministically and in pure code, without archiving the change. + +#### Scenario: Apply deltas to main specs + +- **WHEN** the user runs `openspec sync ` for a change that has delta specs +- **THEN** the command applies the change's ADDED/MODIFIED/REMOVED/RENAMED operations to the corresponding main specs +- **AND** it does not move or archive the change folder +- **AND** it reports, per capability, the counts of requirements added, modified, removed, and renamed + +#### Scenario: No delta specs + +- **WHEN** a change has no delta specs to apply +- **THEN** the command reports that there is nothing to sync and makes no changes + +#### Scenario: Interactive selection + +- **WHEN** no change-name is provided in an interactive session +- **THEN** the command lists changes that have delta specs and prompts the user to choose +- **AND** it does not auto-select + +### Requirement: Deterministic Merge + +The sync merge SHALL be a pure function of the change's delta files and the base spec content, producing byte-identical output for the same inputs on every platform, performed in code without any AI inference. + +#### Scenario: Same inputs produce identical output + +- **WHEN** sync is run more than once on the same delta specs and the same base specs +- **THEN** the resulting `openspec/specs/` content is byte-for-byte identical every time + +#### Scenario: Platform independence + +- **WHEN** sync runs on different operating systems with the same inputs +- **THEN** the resulting spec content is identical regardless of line-ending or path-separator differences in the environment + +#### Scenario: No inference + +- **WHEN** sync applies deltas +- **THEN** it computes the result in code +- **AND** it does not call a language model or otherwise depend on non-deterministic input + +### Requirement: Idempotent Regeneration + +Sync SHALL be idempotent: re-running it on an unchanged change makes no further changes, and re-running it after a delta is revised regenerates the affected specs from scratch with no residue from the prior revision. + +#### Scenario: Re-running is a no-op + +- **WHEN** the user runs `openspec sync ` again with no change to the change's deltas or to the affected specs +- **THEN** no spec files are modified +- **AND** a subsequent `openspec sync --check` reports no drift + +#### Scenario: Revised delta leaves no crumbs + +- **WHEN** a change's delta is revised (for example, a requirement that was added is removed, or a modified requirement's text changes) and sync is run again +- **THEN** the affected specs reflect exactly the current delta applied to the original base +- **AND** no requirement or content from the prior revision of the delta remains + +### Requirement: Applied-Delta Baseline + +When sync applies deltas to `openspec/specs/`, it SHALL record a per-change applied-delta baseline capturing, for each affected spec, its pre-merge content (or an absent marker when the spec is created) and a digest of the applied result, so that the merge can later be reversed and drift can be detected. + +#### Scenario: Baseline recorded on apply + +- **WHEN** sync writes changes to `openspec/specs/` +- **THEN** it records the pre-merge content and applied-result digest for each affected spec in the change's baseline + +#### Scenario: Baseline refreshed on re-sync + +- **WHEN** sync re-applies a revised delta +- **THEN** it refreshes the baseline to reflect the new pre-merge state and applied-result digest + +### Requirement: Drift Check Mode + +The sync command SHALL support a read-only `--check` mode that verifies spec consistency and exits non-zero on a problem, without modifying any files, so it can gate commits and CI as a plain binary. + +#### Scenario: Deltas not cleanly appliable + +- **WHEN** `openspec sync --check` runs and the change's deltas cannot be cleanly applied to the base specs (for example, a MODIFIED, REMOVED, or RENAMED-from header is absent from the base) +- **THEN** the command reports the problem +- **AND** it exits with a non-zero status code +- **AND** it modifies no files + +#### Scenario: Committed specs drifted from regenerated output + +- **WHEN** `openspec sync --check` runs, the change's specs have been synced, and the committed `openspec/specs/` content differs from the regenerated output +- **THEN** the command reports drift +- **AND** it exits with a non-zero status code +- **AND** it modifies no files + +#### Scenario: Check passes + +- **WHEN** `--check` runs and the specs are consistent with the deltas +- **THEN** the command exits zero and modifies no files + +### Requirement: Fix Mode + +The sync command SHALL support a `--fix` mode (the same as the default write behavior) that regenerates `openspec/specs/` from the change's deltas, suitable for use as an auto-fixer in a pre-commit hook. + +#### Scenario: Fix regenerates specs + +- **WHEN** the user runs `openspec sync --fix` +- **THEN** the command writes `openspec/specs/` to the regenerated result +- **AND** a subsequent `--check` reports no drift + +### Requirement: Shared Engine With Archive + +The deterministic merge used by sync SHALL be the same engine used by `openspec archive`, so that syncing and archiving produce identical spec output for the same change. + +#### Scenario: Archive and sync agree + +- **WHEN** a change's deltas are applied by `openspec sync` and, separately, by `openspec archive` +- **THEN** the resulting `openspec/specs/` content is identical + +### Requirement: JSON Output + +The sync command SHALL support `--json` for non-interactive use, emitting machine-readable results and diagnostics. + +#### Scenario: JSON success + +- **WHEN** `openspec sync --json` succeeds +- **THEN** it emits the per-capability counts and the root context as JSON + +#### Scenario: JSON blocked path + +- **WHEN** `openspec sync --json` cannot proceed (no change, deltas not appliable, or drift under `--check`) +- **THEN** it emits a machine-readable diagnostic with a stable code +- **AND** exits with a non-zero status code diff --git a/openspec/changes/add-unarchive-command/specs/cli-unarchive/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-unarchive/spec.md similarity index 86% rename from openspec/changes/add-unarchive-command/specs/cli-unarchive/spec.md rename to openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-unarchive/spec.md index e86a95c707..264bd52b01 100644 --- a/openspec/changes/add-unarchive-command/specs/cli-unarchive/spec.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-unarchive/spec.md @@ -63,11 +63,11 @@ The command SHALL resolve the target archived directory from either a bare chang ### Requirement: Deterministic Spec Reversal -The command SHALL reverse the spec merge deterministically by restoring, for each affected spec, the pre-merge content recorded in the change's reversal snapshot — recreating specs that archiving deleted and deleting specs that archiving created — without re-parsing or inferring requirement content. +The command SHALL reverse the spec merge deterministically by restoring, for each affected spec, the pre-merge content recorded in the change's applied-delta baseline — recreating specs that archiving deleted and deleting specs that archiving created — without re-parsing or inferring requirement content. #### Scenario: Restore modified and removed requirements exactly -- **WHEN** the archived change's snapshot records pre-merge content for an affected spec +- **WHEN** the archived change's baseline records pre-merge content for an affected spec - **THEN** the command restores that spec to the recorded pre-merge bytes - **AND** requirements that were MODIFIED or REMOVED during archiving are restored to their exact prior content @@ -78,7 +78,7 @@ The command SHALL reverse the spec merge deterministically by restoring, for eac #### Scenario: Recreate a spec that archiving created -- **WHEN** archiving created a new spec (the snapshot marks the pre-image as absent) +- **WHEN** archiving created a new spec (the baseline marks the pre-image as absent) - **THEN** the command deletes that spec on reversal, returning `openspec/specs/` to its pre-archive shape #### Scenario: Round-trip is byte-exact @@ -88,18 +88,18 @@ The command SHALL reverse the spec merge deterministically by restoring, for eac ### Requirement: Drift Guard -The command SHALL verify, before reversing any spec, that each affected spec still matches the post-merge state recorded in the reversal snapshot, and SHALL refuse to reverse a spec that has drifted rather than overwrite later changes. +The command SHALL verify, before reversing any spec, that each affected spec still matches the applied-result state recorded in the applied-delta baseline, and SHALL refuse to reverse a spec that has drifted rather than overwrite later changes. #### Scenario: Refuse on drift -- **WHEN** an affected spec's current content no longer matches the post-merge digest recorded in the snapshot (for example, a later change modified the same requirement) +- **WHEN** an affected spec's current content no longer matches the applied-result digest recorded in the baseline (for example, a later change modified the same requirement) - **THEN** the command refuses to reverse the spec merge - **AND** it reports which specs drifted - **AND** it directs the user to re-run with `--keep-specs` to restore the folder without touching specs #### Scenario: Proceed when no drift -- **WHEN** every affected spec still matches its recorded post-merge state +- **WHEN** every affected spec still matches its recorded applied-result state - **THEN** the command proceeds with the deterministic spec reversal ### Requirement: Keep Specs Option @@ -134,20 +134,20 @@ The command SHALL apply the reversal atomically: it stages and validates all cha - **THEN** the command fails without overwriting it - **AND** it makes no changes to specs -### Requirement: Backward Compatibility For Pre-Snapshot Archives +### Requirement: Backward Compatibility For Pre-Baseline Archives -For changes archived before reversal snapshots existed, the command SHALL reverse the operations it can invert from the archived delta alone and SHALL refuse to guess the operations it cannot, never producing an incorrect spec. +For changes archived before applied-delta baselines existed, the command SHALL reverse the operations it can invert from the archived delta alone and SHALL refuse to guess the operations it cannot, never producing an incorrect spec. #### Scenario: Reverse the self-invertible operations -- **WHEN** an archived change has no reversal snapshot +- **WHEN** an archived change has no applied-delta baseline - **AND** its delta contains only ADDED and/or RENAMED requirements - **THEN** the command reverses them by delta inversion (removing added requirements, renaming renamed requirements back) - **AND** restores `openspec/specs/` accordingly #### Scenario: Refuse to guess irreversible operations -- **WHEN** an archived change has no reversal snapshot +- **WHEN** an archived change has no applied-delta baseline - **AND** its delta contains MODIFIED or REMOVED requirements (whose pre-image is not recoverable from the delta) - **THEN** the command does not modify those specs - **AND** it reports which requirements cannot be safely reversed @@ -155,7 +155,7 @@ For changes archived before reversal snapshots existed, the command SHALL revers #### Scenario: Keep-specs always available -- **WHEN** an archived change has no reversal snapshot +- **WHEN** an archived change has no applied-delta baseline - **AND** the user runs `openspec unarchive --keep-specs` - **THEN** the command restores the folder without touching specs, regardless of which delta operations the change contains @@ -170,6 +170,6 @@ The command SHALL handle error conditions gracefully and consistently with `open #### Scenario: JSON diagnostics -- **WHEN** the command is run with `--json` and a blocked condition occurs (not found, ambiguous, drift, destination exists, or pre-snapshot irreversibility) +- **WHEN** the command is run with `--json` and a blocked condition occurs (not found, ambiguous, drift, destination exists, or pre-baseline irreversibility) - **THEN** it emits a machine-readable diagnostic with a stable code - **AND** exits with a non-zero status code diff --git a/openspec/changes/add-unarchive-command/specs/opsx-unarchive-skill/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/opsx-unarchive-skill/spec.md similarity index 100% rename from openspec/changes/add-unarchive-command/specs/opsx-unarchive-skill/spec.md rename to openspec/changes/add-deterministic-sync-and-unarchive/specs/opsx-unarchive-skill/spec.md diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/specs-sync-skill/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/specs-sync-skill/spec.md new file mode 100644 index 0000000000..58f2ba4739 --- /dev/null +++ b/openspec/changes/add-deterministic-sync-and-unarchive/specs/specs-sync-skill/spec.md @@ -0,0 +1,37 @@ +## MODIFIED Requirements + +### Requirement: Specs Sync Skill +The system SHALL provide an `/opsx:sync` skill that syncs delta specs from a change to the main specs by invoking the deterministic `openspec sync` CLI command, rather than editing main specs through agent inference. + +#### Scenario: Sync delta specs to main specs +- **WHEN** agent executes `/opsx:sync` with a change name +- **THEN** the skill invokes `openspec sync ` to apply the change's deltas to `openspec/specs/` +- **AND** it reports the per-capability counts the CLI returns + +#### Scenario: Change selection prompt +- **WHEN** agent executes `/opsx:sync` without specifying a change +- **THEN** the agent prompts user to select from available changes +- **AND** shows changes that have delta specs + +#### Scenario: Idempotent operation +- **WHEN** agent executes `/opsx:sync` multiple times on the same change +- **THEN** the result is the same as running it once +- **AND** no duplicate requirements are created + +### Requirement: Delta Reconciliation Logic +The reconciliation of delta operations into main specs SHALL be performed by the deterministic `openspec sync` engine in code; the skill SHALL NOT add, modify, remove, or rename requirements in `openspec/specs/` by agent inference. + +#### Scenario: Reconciliation delegated to the CLI +- **WHEN** delta operations (ADDED/MODIFIED/REMOVED/RENAMED) must be applied to main specs +- **THEN** the skill relies on `openspec sync` to apply them deterministically +- **AND** the skill does not directly edit files under `openspec/specs/` + +#### Scenario: No scenario-level guessing +- **WHEN** a MODIFIED requirement is applied +- **THEN** the deterministic engine replaces the whole requirement block as written in the delta (per the conventions: a complete requirement, not a diff) +- **AND** the skill does not perform partial, scenario-level merges that could silently drop sibling scenarios + +#### Scenario: Surface engine diagnostics +- **WHEN** `openspec sync` reports that deltas are not cleanly appliable (for example, a MODIFIED/REMOVED/RENAMED-from header is absent from the base) +- **THEN** the skill reports that diagnostic to the user +- **AND** it does not attempt a manual workaround diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md b/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md new file mode 100644 index 0000000000..9726870641 --- /dev/null +++ b/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md @@ -0,0 +1,62 @@ +# Tasks: deterministic spec-merge engine — `sync` (forward) + `unarchive` (reverse) + +> Phased so the keystone ships first and each phase is independently testable and shippable. Phase 1 (engine + baseline) and Phase 2 (`sync` + `--check`) deliver the deterministic, model-free drift path on their own; Phase 3 (`unarchive`) and Phase 4 (idempotency) build on the same baseline; Phase 5 (skills) removes the model from the merge; Phase 6 (integration) is the staged hook/CI follow-up. Cross-platform concerns (`path.join`, Windows `moveDirectory`, newline-normalized digests) are called out per [openspec/config.yaml](../../config.yaml). + +## 1. The engine: deterministic, byte-stable merge + applied-delta baseline + +- [ ] 1.1 Make the merge byte-deterministic: audit `buildUpdatedSpec` ([src/core/specs-apply.ts:240-307](../../../src/core/specs-apply.ts)) for any nondeterministic ordering/whitespace; guarantee stable requirement ordering and newline normalization so the same (delta + base) yields byte-identical output on macOS/Linux/Windows. +- [ ] 1.2 Define the **applied-delta baseline** format (per design Decision 1): per affected spec, the pre-merge content (or `absent` marker) plus a scheme-tagged, newline-normalized digest of the applied result. Store it with the change (e.g. `.openspec/merge-baseline/`); document the location. +- [ ] 1.3 Add baseline read/write helpers (safe read-modify-write, preserving unrelated fields), coordinating the digest convention with [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278)'s ledger. +- [ ] 1.4 Add the inverse merge in `specs-apply.ts`: delta-inversion for ADDED (remove by name) and RENAMED (rename TO→FROM, rewriting the header line); pre-image restore for all ops when a baseline exists. No change to forward behavior for existing callers. +- [ ] 1.5 Tests: same delta+base → byte-identical output across repeated runs and simulated CRLF/LF + Windows paths; baseline round-trips through read/write; inverse of a forward apply restores the base exactly (ADD/MODIFY/REMOVE/RENAME/create). + +## 2. `openspec sync` + the drift gate (the keystone command) + +- [ ] 2.1 Create `src/core/sync.ts` `SyncCommand` mirroring `ArchiveCommand`'s human + `--json` shape (`resolveOpenSpecRoot`, blocked-error → diagnostic, exit codes); default/`--fix` writes `specs/` from the deltas and refreshes the baseline. +- [ ] 2.2 `--check`: read-only; exit non-zero when deltas are not cleanly appliable to the base, or (when `specs/` was synced) when committed `specs/` ≠ regenerated output. Never writes `specs/`. Support `--all`/`--changes` style fan-out for CI (or document the loop). +- [ ] 2.3 Register `sync [change]` in [src/cli/index.ts](../../../src/cli/index.ts) (`--check`, `--fix`, `--json`, `--store`, hidden store-path), mirroring archive (326-343). +- [ ] 2.4 Tests: write produces deterministic `specs/`; `--check` clean vs drifted exit codes; `--check` never mutates; unknown/ambiguous change diagnostics; JSON shape on success and each blocked path. + +## 3. `openspec unarchive` (reverse, built on the baseline) + +- [ ] 3.1 Create `src/core/unarchive.ts` `UnarchiveCommand` (human + `--json`, blocked-error → diagnostic). +- [ ] 3.2 Resolver: bare `` or full archived id; treat prefix as opaque up to `` (strip `YYYY-MM-DD-`; tolerate `NNN-`/ISO/configurable, #409/#787/#1192); reuse `getArchivedChangesData()` (#399). Ambiguity → interactive prompt (most-recent first, never auto-pick) / `--json` error with candidates. +- [ ] 3.3 Reverse from baseline: restore each affected spec's pre-image (rewrite modified, recreate deleted, delete archive-created), under the drift guard (current content vs baseline applied-result digest); on drift, refuse and direct to `--keep-specs`. +- [ ] 3.4 Destination-collision guard (`changes//` exists → abort, no overwrite). Atomicity: stage + validate first; commit order restore specs → move folder (`moveDirectory`, Windows fallback); on failure *"Abort. No files were changed."*, rolling specs back from the baseline if the move fails. +- [ ] 3.5 `--keep-specs` (with hidden `--skip-specs` alias): pure folder move, no spec work, no drift/reversal checks. +- [ ] 3.6 Register `unarchive [change-name]` in `src/cli/index.ts` (`--keep-specs`/`--skip-specs`, `-y/--yes`, `--no-validate`, `--json`, `--store`). +- [ ] 3.7 Tests: byte-exact archive→unarchive round-trip for ADD/MODIFY/REMOVE/RENAME/create; drift refusal (no writes); destination collision abort; `--keep-specs` leaves `specs/` untouched; atomic abort leaves zero partial state; Windows move path; `--json` blocked-path diagnostics. + +## 4. Idempotency & "no crumbs" + +- [ ] 4.1 `sync` on an already-synced, unchanged change writes nothing and `--check` is clean (idempotent no-op). +- [ ] 4.2 Revised-delta re-merge: reverse the prior revision via the baseline, apply the new delta, refresh the baseline — assert `specs/` equals `apply(current delta, original base)` with no residue from the prior revision. +- [ ] 4.3 Drift-on-resync: if `specs/` drifted from the baseline since last sync, do not silently reverse-then-apply over the edit — report drift and require acknowledgement (consistent with unarchive's guard). +- [ ] 4.4 Tests: double-sync no-op; add→sync→revise(add+remove a requirement)→sync yields exactly the new delta's result, no crumbs; drift-on-resync refusal. + +## 5. Skills delegate to the deterministic CLI (no model in the merge) + +- [ ] 5.1 Rewrite `src/core/templates/workflows/sync-specs.ts` so `/opsx:sync` invokes `openspec sync` (drop the "agent-driven… directly edit main specs" instructions); skill does selection/confirmation/output only. +- [ ] 5.2 Create `src/core/templates/workflows/unarchive-change.ts` (`getUnarchiveChangeSkillTemplate()` + `getOpsxUnarchiveCommandTemplate()`) delegating to `openspec unarchive`; surface the CLI's drift refusal and `--keep-specs` option verbatim; never hand-move folders or hand-edit specs. +- [ ] 5.3 Export from `skill-templates.ts`; add `unarchive` to `ALL_WORKFLOWS` ([src/core/profiles.ts:19](../../../src/core/profiles.ts)) and `WORKFLOW_TO_SKILL_DIR` ([src/core/init.ts:65](../../../src/core/init.ts)); **not** to `CORE_WORKFLOWS` (#913/#762). +- [ ] 5.4 Tests: template snapshots assert `/opsx:sync` and `/opsx:unarchive` call the CLI and contain no manual merge/move instructions (anti-#863/#1246 guard). +- [ ] 5.5 Changeset + docs note: `/opsx:sync` is now deterministic; the agent no longer performs scenario-level merges (fixes #1246); author deltas as complete requirements per the conventions. + +## 6. Integration surface (staged): CI gate + pre-commit hook + +- [ ] 6.1 Document the CI drift-gate pattern (`openspec sync --check` as a plain binary, no model/API keys) in `docs/`; provide a copy-paste job snippet. (Wiring the actual workflow file is the owner's call.) +- [ ] 6.2 Document the pre-commit hook pattern (`sync --check` to detect, `sync --fix` to auto-remediate before `git commit --amend`), eslint-`--fix` style. (Choosing husky/lefthook/native is a separable follow-up; none exists in the repo today.) + +## 7. Docs & supersession + +- [ ] 7.1 `docs/opsx.md`: add `/opsx:unarchive`; update `/opsx:sync` to note CLI delegation + determinism; document `openspec sync` and `--check`/`--fix`. +- [ ] 7.2 Note the `specs/` = shipped-reality invariant and why `archive` is retained (design Decision 5), so the "why not just remove archive" question has a documented answer. + +## 8. End-to-end verification + +- [ ] 8.1 E2E determinism: scaffold a change with ADDED/MODIFIED/REMOVED/RENAMED; `openspec sync` twice → byte-identical `specs/`; `sync --check` clean. +- [ ] 8.2 E2E round-trip: `archive` then `unarchive` → `specs/` byte-identical to pre-archive, folder back under `changes//`. +- [ ] 8.3 E2E stacked drift: change A MODIFIES req X and is archived; a second change re-MODIFIES X and is archived; `unarchive A` refuses spec reversal; `unarchive A --keep-specs` succeeds untouched. +- [ ] 8.4 E2E no-crumbs: sync, revise the delta, re-sync → `specs/` reflects only the current delta. +- [ ] 8.5 Validation: `openspec validate add-deterministic-sync-and-unarchive --strict` passes; `openspec status` shows artifacts complete. +- [ ] 8.6 Run the suite on macOS, Linux, and Windows CI. diff --git a/openspec/changes/add-unarchive-command/design.md b/openspec/changes/add-unarchive-command/design.md deleted file mode 100644 index 21d9210713..0000000000 --- a/openspec/changes/add-unarchive-command/design.md +++ /dev/null @@ -1,90 +0,0 @@ -# Design: `openspec unarchive` - -## Context - -`openspec archive` ([src/core/archive.ts](../../../src/core/archive.ts)) is a one-way, non-transactional door: it merges a change's delta specs into `openspec/specs/` via `buildUpdatedSpec` ([src/core/specs-apply.ts:240-307](../../../src/core/specs-apply.ts)) and moves the change folder to `openspec/changes/archive/YYYY-MM-DD-/`. This design covers the inverse command, the one archive-side change required to make the inverse deterministic, and a separable CI companion. - -The governing constraint is the reversibility asymmetry established in the proposal: **ADDED and RENAMED are invertible from the archived delta; REMOVED and MODIFIED are not**, because the forward merge discards the pre-image. Every decision below follows from deciding *where the pre-image comes from*. - -## Goals / Non-Goals - -**Goals** -- A deterministic, code-only `openspec unarchive` that restores `specs/` exactly and moves the folder back. -- Never silently corrupt `specs/`: when a faithful reversal is not possible, refuse with a clear, actionable message. -- Atomic: success or "no files were changed." -- `--keep-specs` as an always-safe escape hatch. -- Backward compatibility with changes archived before this feature. - -**Non-Goals** -- Re-deriving `specs/` from git history as the *primary* mechanism (git is an optional assist only). -- Multi-change / bulk unarchive (mirror of bulk-archive) — future, not here. -- Archive/unarchive lifecycle **hooks** (#704/#682) and lifecycle **timestamps** (#1245) — accommodated, not built. -- Choosing the archive-folder prefix scheme (#409/#787/#1192) — unarchive is tolerant of all of them. - -## Decision 1 — The reversibility guarantee comes from a snapshot at archive time, not from inverting the delta - -**Decision.** Make `openspec archive` write a self-contained **reversal snapshot** into the change folder whenever it rewrites `specs/`. For each affected spec, record the **pre-merge file content** (the exact pre-image, or an explicit "absent" marker when archive *creates* a new spec) and the **post-merge content digest**. `unarchive` reverses by restoring the recorded pre-image; the digest is the drift guard (Decision 3). - -**Why.** The pre-image is the only thing that makes REMOVED/MODIFIED reversible, and it exists only at archive time. Capturing the whole pre-image (rather than a diff or a hash) makes reversal a byte-exact file restore — uniform across ADDED/MODIFIED/REMOVED/RENAMED/created/deleted, with zero re-parsing and zero inference. It is the most robust and the simplest thing that can possibly work. The snapshot lives *inside the change folder* so it moves into `archive/` with everything else, is self-contained, and survives in git without extra plumbing. - -**Alternatives considered.** -- **(A) Reconstruct the pre-image from git** (`git show :specs/...`). Rejected as the foundation: it requires a git repo and a clean pre-archive commit, and the motivating case is exactly when the archive is buried in a multi-file commit (the maintainer's "am I in a bind?" scenario). Kept as an *optional* recovery assist for pre-snapshot archives (Decision 5), never required. -- **(B) Invert the delta only** (no new state). Faithfully reverses ADDED/RENAMED; cannot reverse REMOVED/MODIFIED. Rejected as the foundation because it cannot meet the "deterministic reversal" bar; retained as the graceful-degradation path for pre-snapshot archives (Decision 5). -- **(C) Store a base *hash* per requirement** ([#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)'s proposal). A hash detects drift but cannot *restore* content. The snapshot is the strict generalization: it both detects (digest) and restores (pre-image). If #1246's fingerprint lands first, the snapshot subsumes it. - -**Cost.** A copy of each affected spec's prior bytes per archive — negligible, and only for changes that touch specs. - -## Decision 2 — Resolution is prefix-tolerant and never auto-picks among ambiguous matches - -**Decision.** `unarchive` accepts either the change `` or a full archived directory id. It locates candidates under `changes/archive/` by treating the leading prefix as **opaque up to ``** — stripping a `YYYY-MM-DD-` prefix today, and tolerating `NNN-`/ISO/configurable prefixes ([#409](https://github.com/Fission-AI/OpenSpec/issues/409)/[#787](https://github.com/Fission-AI/OpenSpec/pull/787)/[#1192](https://github.com/Fission-AI/OpenSpec/issues/1192)). When more than one archived entry matches a bare ``: -- **interactive**: list candidates (most-recent first) and prompt — never auto-select; -- **`--json` / non-interactive**: error with the candidate list and require the full archived id. - -**Why.** Multiple archives of one name already happen today (same name archived on two different days), and the proposed sequence scheme makes it routine. Silent selection is the kind of guess that produces the wrong outcome on the rare-but-costly path. Resolution reuses `getArchivedChangesData()` ([#399](https://github.com/Fission-AI/OpenSpec/pull/399)) rather than re-reading the archive dir. - -## Decision 3 — Drift guard + atomicity: refuse rather than clobber, and never leave a partial state - -**Decision.** Before un-merging, compare each affected spec's current content to the **post-merge digest** the snapshot recorded. If any spec has drifted (a later change touched the same requirement — the stacked-archive hazard of [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) and [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md)), unarchive **refuses** the spec reversal and points the user to `--keep-specs`. The whole operation is staged and validated first; on any failure it aborts with *"Abort. No files were changed."* — matching archive's existing contract and the failure surface [#1112](https://github.com/Fission-AI/OpenSpec/issues/1112) describes, and supplying the rollback [#682](https://github.com/Fission-AI/OpenSpec/issues/682) noted archive lacks. - -**Why.** Restoring a pre-image over a spec that a *later* change has since modified would silently delete that later change's contribution — exactly the data loss the project guards against. Detecting drift and stopping is strictly safer than a best-effort merge, and the user always has `--keep-specs` to proceed without spec changes. Atomicity matters because unarchive mutates both `specs/` and the folder location; a half-applied reversal is worse than none. - -**Order of operations (commit phase, after all checks pass):** restore/delete spec files → move folder `archive/` → `changes/`. The move is last so that a move failure (e.g. destination appearing) does not strand rewritten specs without a matching active change; if the move fails after specs are restored, roll the spec writes back from the still-present snapshot. - -## Decision 4 — `--keep-specs`: semantics, and naming vs. archive's `--skip-specs` - -**Decision.** `--keep-specs` moves the folder back and leaves `specs/` **untouched** — always deterministic, always safe. Primary spelling is `--keep-specs` (the request's, and it reads naturally for the inverse: "keep the specs as they are"). Accept `--skip-specs` as a **hidden alias** so muscle memory from archive transfers. When unarchive *cannot* safely reverse specs (drift, or a pre-snapshot REMOVED/MODIFIED) and `--keep-specs` was not given, it refuses and *recommends* `--keep-specs` — it does not silently fall back. - -**Why.** `--skip-specs` on archive means "move the folder, don't merge in" ([#28](https://github.com/Fission-AI/OpenSpec/pull/28)); `--keep-specs` on unarchive means "move the folder, don't un-merge." Same shape, inverse direction. Honoring the requested spelling while aliasing the familiar one is the low-surprise choice. **Open for owner confirmation:** if strict CLI symmetry is preferred, make `--skip-specs` the primary and `--keep-specs` the alias — purely a naming call. - -## Decision 5 — Backward compatibility: graceful degradation for pre-snapshot archives - -**Decision.** Changes archived before this feature have no snapshot. Unarchive then: -1. reverses **ADDED** and **RENAMED** by delta inversion (the self-invertible half), and -2. for any **REMOVED**/**MODIFIED**, **refuses to guess** — it names each requirement it cannot safely restore and directs the user to `--keep-specs` (optionally noting the git-based manual recovery for the pre-image). - -It never silently emits a wrong spec. If git is available and a clean pre-archive commit can be identified, it *may* offer to recover the pre-image from git as an assist — strictly opt-in, never assumed. - -**Why.** The feature must be useful the day it ships, including for already-archived changes, without ever degrading into corruption. Reversing the half that is provably reversible, and stopping honestly on the half that is not, is the correct contract. - -## Decision 6 — Companion: a model-free CI drift gate (separable) - -**Context.** The motivating Discord thread ended on: letting archive's `sync` run through the agent makes merged `specs/` non-deterministic (wording drift), so a CI gate would seem to need a model. - -**Decision / recommendation.** Don't run a model in CI. The delta merge is mechanical and already deterministic in code (`buildUpdatedSpec`); only the agent path drifts. Let the agent author locally, and have CI verify the deterministic output as a plain binary. The enabling gap: `applySpecs()` ([src/core/specs-apply.ts:391](../../../src/core/specs-apply.ts)) — the standalone deterministic apply — has **zero callers**; the engine is reachable only inside `archive`, fused to the folder move. Expose it as `openspec sync [change]` (apply deltas to `specs/` without archiving) with a `--check` mode that exits non-zero when committed `specs/` diverge from the regenerated output — a codegen/IaC-style drift gate. This PR's reversal snapshot makes `archive → unarchive` a byte-exact round-trip, which is the determinism property such a gate relies on and a ready-made test of it. - -**Scope.** Separable. No `cli-sync` spec delta is included here; it is captured so the owner can decide whether it rides this PR or a fast follow-up. The unarchive core does not depend on it. - -## Decision 7 — Move strategy and lifecycle metadata - -- **Move**: reuse `moveDirectory()`/`copyDirRecursive()` ([src/core/archive.ts:96-128](../../../src/core/archive.ts)) for the Windows EPERM/EXDEV fallback ([#605](https://github.com/Fission-AI/OpenSpec/pull/605)). A future switch to `git mv` ([#709](https://github.com/Fission-AI/OpenSpec/issues/709)) would cover both directions through this one helper. -- **Metadata**: today archive persists no `archived` timestamp (only the folder-name prefix), so there is nothing to clear. If [#1245](https://github.com/Fission-AI/OpenSpec/issues/1245) lands an `archived` field, unarchive should null it on restore — noted, not built. - -## Risks / trade-offs - -- **Round-trip is not guaranteed identity in the forward direction.** Unarchive restores the exact pre-archive `specs/` from the snapshot, but a *subsequent re-archive* re-runs the forward merge, which is itself lossy across stacked MODIFIED changes ([#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)) and may reorder ADDED requirements. Unarchive is faithful; re-archiving inherits archive's existing caveats. Documented, not solved here. -- **Snapshot adds a small artifact to archived folders.** Acceptable; it is the price of reversibility and is inert for changes that touch no specs. -- **Pre-snapshot archives get partial reversal.** Mitigated by honest refusal + `--keep-specs`; fully resolved for any change archived after this ships. - -## Migration / rollout - -Purely additive. Archive gains snapshot-writing (forward-only, no behavior/output change). Unarchive is a new command + skill in the expanded profile. Existing archives keep working under Decision 5. diff --git a/openspec/changes/add-unarchive-command/proposal.md b/openspec/changes/add-unarchive-command/proposal.md deleted file mode 100644 index b2dbe59adf..0000000000 --- a/openspec/changes/add-unarchive-command/proposal.md +++ /dev/null @@ -1,106 +0,0 @@ -## Why - -`openspec archive` does two things at once: it **merges** a change's delta specs into `openspec/specs/` and **moves** the change folder into `openspec/changes/archive/`. There is **no command to undo it.** When a change is archived by mistake — or peer-review feedback lands after the archive has — the user is left with manual `git` surgery the maintainer has to walk people through case by case ([Discord, 2026-06], where he committed to "follow up with a PR to add `/opsx:unarchive` (or `openspec unarchive `)"). - -`git revert` only works when the archive is its own clean commit. The motivating case is the hard one: *"the archive is part of a commit with other changes, so I cannot simply revert. Am I in a bind?"* This proposal adds the inverse command — `openspec unarchive [change-name]` plus a thin `/opsx:unarchive` skill — and closes the one archive-side gap that makes a **deterministic** reversal possible at all. - -## Background: archive is a one-way door, and the delta is not self-inverting - -Verified against `Fission-AI/OpenSpec` at `main` (commit `546224e`, #1248) on 2026-06-29. Two facts define the whole design. - -**1. The forward merge is deterministic; the reverse is not free.** `buildUpdatedSpec` ([src/core/specs-apply.ts:240-307](../../../src/core/specs-apply.ts)) builds a `name → requirement-block` map from the base spec and applies operations in a fixed order — RENAMED → REMOVED → MODIFIED → ADDED. Inverting each operation is asymmetric: - -| Delta op | Forward effect | Invertible from the delta alone? | -|---|---|---| -| **ADDED** | inserts a new requirement block | **Yes** — the block carries its own name; remove it by name. | -| **RENAMED** | `FROM`→`TO`, body preserved | **Yes** — both names are stored; rename `TO`→`FROM`. | -| **REMOVED** | deletes a requirement | **No** — the delta stores only the *name* (`plan.removed: string[]`) plus a prose reason. The deleted body is gone. | -| **MODIFIED** | whole-block replace, keyed by name | **No** — the delta stores only the *new* block, never the prior one. | - -The conventions spec makes the loss explicit: a MODIFIED delta must "include the complete modified requirement (not a diff)" ([openspec/specs/openspec-conventions/spec.md](../../specs/openspec-conventions/spec.md)). So the pre-image — the bytes that were in `specs/` *before* the archive — is **not** retained anywhere after a forward archive. [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) independently documents this exact `nameToBlock.set(key, mod)` whole-block replace as a *forward-direction* data-loss bug ("discarding any sibling scenarios"), which is the same mechanism that makes MODIFIED non-invertible in reverse. - -**2. Archive is non-transactional and has no rollback.** The hook-design discussion states it plainly: *"the current archive operation is not transactional — if a step fails partway, there's no rollback"* ([#682](https://github.com/Fission-AI/OpenSpec/issues/682)). Unarchive is, in effect, the rollback that archive never had — so it must itself be atomic. - -**The consequence:** a reversal that only reads the archived delta can faithfully undo ADDED and RENAMED, but **cannot** restore a REMOVED or MODIFIED requirement. A command that silently "reverses" by deleting the modified requirement (or leaving a hole) would corrupt `specs/` — the precise failure mode OpenSpec already treats as a cardinal sin ([#1246](https://github.com/Fission-AI/OpenSpec/issues/1246); the `prevent-silent-spec-drop` work). True determinism requires capturing the pre-image **at archive time**, when it still exists. - -## What Changes - -The design principle mirrors the one the project is converging on (deterministic CLI, thin agent): **the reversal is a pure, code-only transform the CLI owns; the agent does none of it.** Ordered by leverage: - -1. **REVERSAL SNAPSHOT — make archive reversible (`cli-archive`, MODIFIED).** When `openspec archive` rewrites `specs/`, it also writes a small, self-contained **reversal snapshot** *inside the change folder* (so it moves into the archive with everything else and survives in git): for each affected spec it records the pre-merge file content (the pre-image, or "absent" for a newly-created spec) and the post-merge content digest. This is the one missing primitive. It is forward-only metadata — it changes nothing about how archive merges or moves — and it is what lets unarchive restore `specs/` **byte-for-byte deterministically**, uniformly across ADDED/MODIFIED/REMOVED/RENAMED/created/deleted, with no re-parsing and no inference. (Conceptually adjacent to [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)'s proposed base-fingerprint, generalized from a hash to the full pre-image.) - -2. **THE COMMAND — `openspec unarchive [change-name]` (`cli-unarchive`, NEW).** The deterministic inverse of archive: - - **Resolve** the archived folder. `unarchive` accepts either the change `` or the full archived directory id. The leading prefix is treated as **opaque up to the name** — it strips a `YYYY-MM-DD-` prefix today and tolerates the sequence/`NNN-` and configurable prefixes proposed in [#409](https://github.com/Fission-AI/OpenSpec/issues/409)/[#787](https://github.com/Fission-AI/OpenSpec/pull/787)/[#1192](https://github.com/Fission-AI/OpenSpec/issues/1192). When several archived entries share one name, it **never silently picks**: interactive mode prompts; `--json`/non-interactive requires the full id or errors with the candidate list. Resolution reuses the archive-reading plumbing added in [#399](https://github.com/Fission-AI/OpenSpec/pull/399) (`getArchivedChangesData()`). - - **Reverse the spec merge**, deterministically, from the snapshot: restore each affected spec's recorded pre-image (recreating deleted specs, deleting specs that archive created). Guarded by a **drift check** — if a spec no longer matches the post-merge image the snapshot recorded (e.g. a later change touched the same requirement; the stacked-archive hazard of [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md) and [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)), unarchive **refuses** rather than clobber, and points the user at `--keep-specs`. - - **Move** the folder back from `changes/archive//` to `changes//`, reusing the Windows-safe `moveDirectory()` helper ([src/core/archive.ts:116-128](../../../src/core/archive.ts), the EPERM/EXDEV fallback from [#605](https://github.com/Fission-AI/OpenSpec/pull/605)); error if an active `changes//` already exists (mirrors archive's no-overwrite contract). - - **Atomicity**: stage and validate the whole reversal first; on any failure, *"Abort. No files were changed."* — matching archive's existing abort contract and the failure surface [#1112](https://github.com/Fission-AI/OpenSpec/issues/1112) defines, and finally giving archive the rollback [#682](https://github.com/Fission-AI/OpenSpec/issues/682) noted it lacks. - - **`--keep-specs`**: restore the folder *without* touching `specs/`. Always deterministic and always safe (a pure folder move). It is the literal mirror of archive's `--skip-specs` ([#28](https://github.com/Fission-AI/OpenSpec/pull/28)) and the escape hatch for any case the snapshot can't cover. - - **Graceful degradation for pre-snapshot archives.** Changes archived before this feature have no snapshot. Unarchive then restores ADDED/RENAMED by delta inversion (the self-invertible half), and for any REMOVED/MODIFIED it **refuses to guess** — it reports exactly what it cannot safely reverse and directs the user to `--keep-specs` (optionally assisted by git). It never silently produces a wrong spec. - -3. **THE SKILL — `/opsx:unarchive` (`opsx-unarchive-skill`, NEW).** A thin wrapper that **delegates to the `openspec unarchive` CLI** rather than re-implementing the move-and-un-merge in prose. This is the maintainers' stated direction after the `/opsx:archive` skill drifted from the CLI ([#863](https://github.com/Fission-AI/OpenSpec/issues/863), [#799](https://github.com/Fission-AI/OpenSpec/issues/799), [#656](https://github.com/Fission-AI/OpenSpec/issues/656)): the deterministic reversal lives in TypeScript, the skill only selects the change, confirms, and renders results. It ships in the **expanded** workflow profile, not the deliberately-minimal core ([#913](https://github.com/Fission-AI/OpenSpec/issues/913), [#762](https://github.com/Fission-AI/OpenSpec/issues/762)), and because it calls the CLI it carries no dependency on another skill being installed (the [#913](https://github.com/Fission-AI/OpenSpec/issues/913) failure mode). - -### Companion (separable): a model-free CI drift gate - -This thread answers the question the same Discord conversation ended on — *"this means that inference is needed in the CI… any recommendations here?"* — when the user observed that letting the archive-time `sync` run through the agent makes the merged `specs/` non-deterministic, so a naive CI gate would have to run a model. - -**Recommendation: don't put a model in CI.** The delta merge is mechanical and already deterministic in code; only the agent-driven `/opsx:sync` path introduces wording drift. The clean fix is to let the agent author locally and have CI verify the deterministic output as a plain binary. Today that is impossible: the deterministic engine `applySpecs()` ([src/core/specs-apply.ts:391](../../../src/core/specs-apply.ts)) has **zero callers** — it is reachable only *inside* `openspec archive`, bundled with the folder move. There is no standalone command that applies (or just checks) the merge. - -The fix is small and reuses the engine this PR is already exercising: expose it as `openspec sync [change]` (apply deltas to `specs/` without archiving) with a `--check` mode that exits non-zero when committed `specs/` differ from the regenerated output — a codegen/IaC-style drift gate CI runs without any model. **This PR de-risks that companion directly:** the reversal snapshot makes `archive` → `unarchive` a verifiable, byte-exact round-trip on `specs/`, which is the determinism property the CI gate depends on. - -It is called out as **separable** so the owner can decide whether it lands in this PR or a fast follow-up; the unarchive core does not depend on it. (Scoped here, fully argued in [design.md](./design.md) Decision 6, but no `cli-sync` spec delta is included in this PR.) - -## Capabilities - -### New Capabilities - -- `cli-unarchive`: the `openspec unarchive [change-name]` command — the deterministic inverse of `openspec archive`. It resolves an archived change (prefix-tolerant, never auto-picking among ambiguous matches), moves the folder back to active, and reverses the spec merge from the archive's reversal snapshot under a drift guard, atomically (abort leaves no partial state). `--keep-specs` restores the folder without touching `specs/`; pre-snapshot archives degrade gracefully (reverse the self-invertible half, refuse to guess the rest). -- `opsx-unarchive-skill`: a `/opsx:unarchive` workflow skill that delegates to the `openspec unarchive` CLI for the deterministic work, handling only change selection, confirmation, and result rendering. Ships in the expanded profile; carries no cross-skill dependency. - -### Modified Capabilities - -- `cli-archive`: when archive rewrites `specs/`, it additionally captures a self-contained **reversal snapshot** inside the change folder (per-spec pre-image plus post-merge digest) so the operation becomes deterministically reversible. Forward-only and backward-compatible — it does not change how archive merges, moves, validates, or what it prints; archives created before this feature simply have no snapshot. - -### Companion capability (separable; not specced in this PR) - -- `cli-sync` (recommended follow-up): a deterministic `openspec sync [change]` that applies delta specs to `specs/` without archiving, plus a `--check` drift-gate mode for model-free CI. Enabled and de-risked by this PR's snapshot/round-trip work; included here as a design recommendation so the owner can decide PR-vs-follow-up. - -## Impact - -- `src/core/archive.ts` — at the point specs are rewritten (`buildUpdatedSpec`/`writeUpdatedSpec`, ~414-472), also write the reversal snapshot (pre-image + post-merge digest) into the change folder before the move. No change to merge/move/validate behavior or output. Reuse `moveDirectory()`/`copyDirRecursive()` (116-128). -- `src/core/unarchive.ts` (**new**) — `UnarchiveCommand` mirroring `ArchiveCommand`'s shape (human + `--json` modes, `ResolvedOpenSpecRoot`, blocked-error → diagnostic): resolve archived dir, drift-check, restore pre-images (or delta-invert / refuse for pre-snapshot), move folder back, atomic abort. -- `src/core/specs-apply.ts` — factor the invertible half (delta-inversion for ADDED/RENAMED used by the pre-snapshot fallback) alongside the existing `buildUpdatedSpec`; no change to forward behavior. -- `src/core/list.ts` / `src/core/view.ts` — reuse `getArchivedChangesData()` ([#399](https://github.com/Fission-AI/OpenSpec/pull/399)) to resolve/disambiguate archived candidates; after unarchive the entry leaves `list --archived` and returns to active `list`. -- `src/cli/index.ts` — register `unarchive [change-name]` with `--keep-specs`, `-y/--yes`, `--no-validate`, `--json`, `--store` (mirrors the archive registration at 326-343). -- `src/core/templates/workflows/unarchive-change.ts` (**new**) — `getUnarchiveChangeSkillTemplate()` + `getOpsxUnarchiveCommandTemplate()`, modeled on `archive-change.ts` but delegating to the CLI. -- Registration/wiring — export from `skill-templates.ts`; add `unarchive` to `ALL_WORKFLOWS` ([src/core/profiles.ts:19](../../../src/core/profiles.ts)) and `WORKFLOW_TO_SKILL_DIR` ([src/core/init.ts:65](../../../src/core/init.ts)); **not** added to `CORE_WORKFLOWS`. Command generation is generic (no adapter changes). Add a `/opsx:unarchive` row to `docs/opsx.md`. -- Tests — mirror `test/core/archive.test.ts`: resolve/disambiguate, byte-exact round-trip (`archive` then `unarchive` restores `specs/` exactly), drift refusal, `--keep-specs`, destination-collision abort, pre-snapshot degradation, atomic "no files changed" on failure, Windows `moveDirectory` path, and template-generation snapshots. Per [openspec/config.yaml](../../config.yaml), run on Windows CI. - -## Issues addressed - -All references verified against `Fission-AI/OpenSpec` at `main` (`546224e`) on 2026-06-29. No prior unarchive/rollback/restore issue or PR exists — this is greenfield. - -Closes / delivers: - -- The maintainer's Discord commitment to add `/opsx:unarchive` (or `openspec unarchive `) that "moves the folder back and reverses the spec merge for you," including the hard case where the archive is buried in a multi-file commit so `git revert` is not an option. - -Directly enables / strengthens: - -- [#682](https://github.com/Fission-AI/OpenSpec/issues/682) — archive "is not transactional… there's no rollback." Unarchive is that rollback, and is itself atomic. -- [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) — confirms the MODIFIED whole-block-replace loses the pre-image. The reversal snapshot retains it (generalizing #1246's proposed base-fingerprint), and unarchive's drift guard refuses to clobber a requirement a later change has since touched. -- [#863](https://github.com/Fission-AI/OpenSpec/issues/863), [#799](https://github.com/Fission-AI/OpenSpec/issues/799), [#656](https://github.com/Fission-AI/OpenSpec/issues/656) — "the archive skill re-implements merge/move instead of calling the CLI." `/opsx:unarchive` is CLI-first by construction, the pattern these issues ask for. - -Companion answers (the CI-determinism thread): - -- The Discord "inference in CI" question and the [#863](https://github.com/Fission-AI/OpenSpec/issues/863)/[#656](https://github.com/Fission-AI/OpenSpec/issues/656) "why not the deterministic CLI" cluster → recommend a model-free `openspec sync --check` drift gate (companion; see design Decision 6), de-risked by this PR's round-trip determinism. - -Delineated from adjacent work (coordinate, don't collide): - -- [#409](https://github.com/Fission-AI/OpenSpec/issues/409) / [#787](https://github.com/Fission-AI/OpenSpec/pull/787) / [#1192](https://github.com/Fission-AI/OpenSpec/issues/1192) (archive-folder prefix scheme: date → sequence/configurable) — unarchive's resolver is built prefix-tolerant so it survives whichever scheme lands; it does not pick a scheme. -- [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md) — planning-time archive ordering and "parallel work reintroducing assumptions another change removed." Unarchive's drift guard is the runtime counterpart at the spec layer; the two compose (ordering metadata up front, drift refusal at reversal time). -- [#704](https://github.com/Fission-AI/OpenSpec/issues/704) / [#682](https://github.com/Fission-AI/OpenSpec/issues/682) (archive hooks) — if `pre/post-archive` hooks land, a symmetric `pre/post-unarchive` point is the natural extension; named so it fits, not built here. -- [#709](https://github.com/Fission-AI/OpenSpec/issues/709) (`git mv` for archive) — unarchive reuses the same `moveDirectory()` helper, so a future switch to `git mv` covers both directions at once. - -Related, out of scope (referenced, not closed): - -- [#1245](https://github.com/Fission-AI/OpenSpec/issues/1245) — first-class lifecycle timestamps (`created`/`updated`/`archived`). Today archive does not persist an `archived` timestamp (only the folder-name prefix). If/when #1245 lands, unarchive should clear it on restore; this PR notes the hook but does not add the field. -- [#1112](https://github.com/Fission-AI/OpenSpec/issues/1112) — moving archive-time existence checks into `validate`. Unarchive adopts the same "Abort. No files were changed." contract; broadening `validate` itself is that issue's scope. diff --git a/openspec/changes/add-unarchive-command/specs/cli-archive/spec.md b/openspec/changes/add-unarchive-command/specs/cli-archive/spec.md deleted file mode 100644 index 0895a9893c..0000000000 --- a/openspec/changes/add-unarchive-command/specs/cli-archive/spec.md +++ /dev/null @@ -1,29 +0,0 @@ -## ADDED Requirements - -### Requirement: Reversal Snapshot Capture - -When the archive operation rewrites main specs, it SHALL capture a self-contained reversal snapshot inside the change folder so that the operation can later be reversed deterministically by `openspec unarchive`. The snapshot SHALL be forward-only and SHALL NOT alter how archive merges, moves, validates, or what it outputs. - -#### Scenario: Snapshot captured when specs are updated - -- **WHEN** archiving a change applies delta specs to `openspec/specs/` -- **THEN** the command records, for each affected spec, its pre-merge content and its post-merge content digest in a reversal snapshot stored inside the change folder -- **AND** the snapshot moves into `openspec/changes/archive/-/` together with the rest of the change - -#### Scenario: Created specs marked absent - -- **WHEN** archiving creates a new spec that did not previously exist -- **THEN** the snapshot records the pre-merge state of that spec as absent -- **AND** unarchive can delete it to restore the pre-archive state - -#### Scenario: No snapshot when specs are not updated - -- **WHEN** archiving runs with `--skip-specs`, or the change has no delta specs to apply -- **THEN** no spec content is changed -- **AND** no reversal snapshot is written - -#### Scenario: Archive behavior and output unchanged - -- **WHEN** capturing the reversal snapshot -- **THEN** the merge, move, validation, and confirmation behavior of archive are unchanged -- **AND** the command's human-readable and `--json` output are unchanged diff --git a/openspec/changes/add-unarchive-command/tasks.md b/openspec/changes/add-unarchive-command/tasks.md deleted file mode 100644 index f6e061f366..0000000000 --- a/openspec/changes/add-unarchive-command/tasks.md +++ /dev/null @@ -1,59 +0,0 @@ -# Tasks: `openspec unarchive` — deterministic inverse of archive - -> Sequenced so each layer is independently testable: archive-side snapshot (the reversibility primitive) → resolution → deterministic reversal + guards → CLI surface → skill/wiring → docs → e2e. Cross-platform concerns (`path.join`, Windows `moveDirectory`, newline-normalized digests) are called out where relevant, per [openspec/config.yaml](../../config.yaml). The CI drift-gate companion (design Decision 6) is intentionally **not** in these tasks — it is a separable follow-up. - -## 1. Archive-side: reversal snapshot (the reversibility primitive) - -- [ ] 1.1 Define the snapshot format: a self-contained record inside the change folder (e.g. `.openspec/unarchive/` with a manifest + per-spec pre-image files) capturing, for each affected spec, its **pre-merge content** (or an explicit `absent` marker when archive creates the spec) and its **post-merge content digest** (newline-normalized CRLF→LF, scheme-tagged so future canonicalization changes are detectable). Document it in design if the shape shifts during build. -- [ ] 1.2 In `ArchiveCommand` ([src/core/archive.ts](../../../src/core/archive.ts)), at the spec-rewrite point (~414-472), capture each target's pre-image **before** `writeUpdatedSpec`, and the post-merge digest **after**, then persist the snapshot into the change folder **before** `moveDirectory` so it travels into `archive/`. -- [ ] 1.3 Strictly forward-only: assert archive's merge/move/validate behavior and stdout (human + `--json`) are byte-for-byte unchanged when no one reads the snapshot. Add a regression test pinning existing archive output. -- [ ] 1.4 Tests: snapshot present after archiving a change that ADDs/MODIFIEs/REMOVEs/RENAMEs/creates a spec; pre-image matches pre-archive bytes exactly; `absent` marker recorded for created specs; digest stable across CRLF vs LF; `--skip-specs` archive writes **no** snapshot (nothing was merged). - -## 2. Resolve an archived change (prefix-tolerant, never auto-pick) - -- [ ] 2.1 Add a resolver that maps a bare `` or a full archived dir id to a candidate set under `changes/archive/`, treating the leading prefix as **opaque up to ``** (strip `YYYY-MM-DD-`; tolerate `NNN-`/ISO/configurable per #409/#787/#1192). Reuse `getArchivedChangesData()` ([src/core/view.ts](../../../src/core/view.ts) / [src/core/list.ts](../../../src/core/list.ts), #399). -- [ ] 2.2 Ambiguity policy: >1 match → interactive prompt (most-recent first, never auto-select); `--json`/non-interactive → error listing candidates and require the full id. -- [ ] 2.3 Tests: single match resolves; date-prefixed and (simulated) sequence-prefixed dirs both resolve; bare-name with two matches → prompt (interactive) / error+candidates (`--json`); full-id always resolves unambiguously; unknown name → clean not-found error. - -## 3. Deterministic reversal + guards (core) - -- [ ] 3.1 Create `src/core/unarchive.ts` with `UnarchiveCommand` mirroring `ArchiveCommand` structure (human + `--json`, `resolveOpenSpecRoot`, blocked-error → diagnostic, exit codes). -- [ ] 3.2 Snapshot path: restore each affected spec's recorded pre-image (rewrite modified, recreate deleted, delete archive-created specs) — a byte-exact file restore, no re-parsing. -- [ ] 3.3 Drift guard: before restoring, compare each spec's current content to the snapshot's post-merge digest; on drift, **refuse** the spec reversal and direct the user to `--keep-specs`. (Same-scheme digests only; unknown scheme → treat as drift/unknown, refuse.) -- [ ] 3.4 Destination-collision guard: if `changes//` already exists, abort (mirror archive's no-overwrite). -- [ ] 3.5 Atomicity: stage + validate the full reversal first; commit order = restore specs → move folder (`moveDirectory`, reuse Windows EPERM/EXDEV fallback); on any failure, *"Abort. No files were changed."*, rolling back restored specs from the still-present snapshot if the move fails. -- [ ] 3.6 `--keep-specs`: skip all spec work; pure folder move; always succeeds when 3.4 passes. -- [ ] 3.7 Tests: byte-exact round-trip (`archive` then `unarchive` restores `specs/` exactly) for ADD/MODIFY/REMOVE/RENAME/create; drift → refusal (no writes); destination collision → abort; `--keep-specs` leaves `specs/` untouched; atomic abort leaves zero partial state; Windows `moveDirectory` path. - -## 4. Backward compatibility: pre-snapshot archives (graceful degradation) - -- [ ] 4.1 Factor the self-invertible half out of `specs-apply.ts`: delta-inversion for **ADDED** (remove by name) and **RENAMED** (rename `TO`→`FROM`, rewriting the header line). No change to forward `buildUpdatedSpec`. -- [ ] 4.2 When no snapshot exists, reverse ADDED/RENAMED via 4.1; for any REMOVED/MODIFIED, **refuse to guess** — list each requirement that cannot be safely restored and direct to `--keep-specs`. -- [ ] 4.3 (Optional, opt-in) git assist: if a clean pre-archive commit is identifiable, offer to recover the pre-image from git; never automatic. -- [ ] 4.4 Tests: pre-snapshot archive with only ADDED/RENAMED → fully reversed; with REMOVED/MODIFIED → refusal naming the requirements; `--keep-specs` always works regardless of snapshot presence. - -## 5. CLI surface - -- [ ] 5.1 Register `unarchive [change-name]` in [src/cli/index.ts](../../../src/cli/index.ts) mirroring archive (326-343): `--keep-specs` (with hidden `--skip-specs` alias), `-y/--yes`, `--no-validate`, `--json`, `--store`, hidden store-path; action → `new UnarchiveCommand().execute(...)`. -- [ ] 5.2 `--json` is non-interactive: ambiguity/drift/collision become machine-readable diagnostics with non-zero exit (mirror `ArchiveBlockedError`). -- [ ] 5.3 Tests: flag parsing, `--json` shape on success and each blocked path, `--store` resolution. - -## 6. The `/opsx:unarchive` skill (delegates to the CLI) - -- [ ] 6.1 Create `src/core/templates/workflows/unarchive-change.ts` with `getUnarchiveChangeSkillTemplate()` + `getOpsxUnarchiveCommandTemplate()`, modeled on `archive-change.ts` but **invoking `openspec unarchive`** for all deterministic work (selection via `openspec list --archived --json`; confirm; render CLI result). No hand-moving folders, no hand-editing specs (anti-#863). -- [ ] 6.2 Encode guardrails: never auto-select among ambiguous archives; surface the CLI's drift refusal and the `--keep-specs` option verbatim; do not re-implement reversal logic in prose. -- [ ] 6.3 Export from `skill-templates.ts`; add `unarchive` to `ALL_WORKFLOWS` ([src/core/profiles.ts:19](../../../src/core/profiles.ts)) and `WORKFLOW_TO_SKILL_DIR` ([src/core/init.ts:65](../../../src/core/init.ts)); **do not** add to `CORE_WORKFLOWS` (#913/#762). -- [ ] 6.4 Tests: template-generation snapshot; assert the template calls the CLI and contains no manual move/merge instructions. - -## 7. Docs - -- [ ] 7.1 Add a `/opsx:unarchive` row to the command table in `docs/opsx.md` and a short "Undoing an archive" section (covering `--keep-specs` and the drift-refusal behavior). -- [ ] 7.2 Note in docs that the deterministic reversal requires a change archived by this version or later; older archives degrade per design Decision 5. - -## 8. End-to-end verification - -- [ ] 8.1 E2E: scaffold a spec-driven change with one ADDED, one MODIFIED, one REMOVED, one RENAMED requirement; `archive`; `unarchive`; assert `specs/` is byte-identical to pre-archive and the folder is back under `changes//`. -- [ ] 8.2 E2E stacked-archive drift: archive change A (MODIFIES req X), then a second change re-MODIFIES X and is archived; `unarchive A` refuses spec reversal and `unarchive A --keep-specs` succeeds with `specs/` untouched. -- [ ] 8.3 E2E ambiguity: two archived entries for one name → `--json` errors with candidates; full-id resolves. -- [ ] 8.4 Validation: `openspec validate add-unarchive-command --strict` passes; `openspec status --change add-unarchive-command` shows artifacts complete. -- [ ] 8.5 Run the suite on macOS, Linux, and Windows CI. From 137b6c02bec9a2941e1103c2c8e8b8c6c5a22aed Mon Sep 17 00:00:00 2001 From: Clay Good Date: Mon, 29 Jun 2026 16:51:20 -0500 Subject: [PATCH 3/8] docs(openspec): add early-conflict + provenance; engage edit-in-place; scrub names MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Continue folding the design discussion into PR #1279, within OpenSpec's scope. Adopt (in scope): - Early conflict resolution — sync --check surfaces delta-vs-base and cross-change conflicts at commit/PR time, not at archive. Gets the "resolve conflicts immediately" benefit without abandoning deltas. - Provenance — the merge records which change/delta produced each spec edit; enables a deterministic delta<->spec correspondence check (orphan edit / unapplied delta -> fail) and a future spec-aware diff driver. - 3 new cli-sync requirements: Change Provenance, Delta-Spec Correspondence, Cross-Change Conflict Detection. New design Decisions 11-12. Engage and reject, with reasoning (documented): - Edit specs/ in place; treat git diff as the delta; semantic deltas in a sidecar log. Rejected: the delta layer is what isolates proposed from shipped and makes parallel changes + single-change reverse possible. Its valid kernel (early conflict resolution, reasoning-with-the-change) is adopted via the items above. (Decision 11) Defer (delineated): the spec-aware git diff driver — provenance data is in scope; the diff-driver presentation is a follow-up (Decision 12). Also: scrub contributor names from all PR content (neutral attributions). Validates with `openspec validate --strict`. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../design.md | 31 ++++++++-- .../proposal.md | 22 +++++-- .../specs/cli-sync/spec.md | 59 ++++++++++++++++++- .../tasks.md | 5 ++ 4 files changed, 106 insertions(+), 11 deletions(-) diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/design.md b/openspec/changes/add-deterministic-sync-and-unarchive/design.md index eda0947bda..1810d63798 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/design.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/design.md @@ -2,7 +2,7 @@ ## Context -OpenSpec's spec merge (delta → `specs/`) is reached two ways today: deterministically but only *inside* `openspec archive` (fused to the folder move), or by an AI agent in `/opsx:sync` that "directly edits main specs." The standalone deterministic apply (`applySpecs()`, [src/core/specs-apply.ts:391](../../../src/core/specs-apply.ts)) has zero callers. The Discord thread (samholmes ↔ Clay, 2026-06) worked from "add unarchive" to a single root cause: **the merge must be deterministic, idempotent, and reversible pure code, exposed as commands** — then unarchive, a code-only CI drift gate, a `sync --fix` pre-commit hook, and crumb-free re-merge all fall out of one primitive. +OpenSpec's spec merge (delta → `specs/`) is reached two ways today: deterministically but only *inside* `openspec archive` (fused to the folder move), or by an AI agent in `/opsx:sync` that "directly edits main specs." The standalone deterministic apply (`applySpecs()`, [src/core/specs-apply.ts:391](../../../src/core/specs-apply.ts)) has zero callers. The design discussion (2026-06) worked from "add unarchive" to a single root cause: **the merge must be deterministic, idempotent, and reversible pure code, exposed as commands** — then unarchive, a code-only CI drift gate, a `sync --fix` pre-commit hook, and crumb-free re-merge all fall out of one primitive. This design covers that primitive and the commands built on it. It supersedes the earlier "unarchive + separable CI companion" framing of this PR: the deterministic engine is no longer a companion, it is the spine. @@ -13,12 +13,15 @@ This design covers that primitive and the commands built on it. It supersedes th - Idempotent: re-running `sync` is a no-op; revising a delta regenerates `specs/` with no crumbs. - Reversible: `unarchive` restores `specs/` exactly and moves the folder back, atomically. - Code-only drift detection (`sync --check`) usable by CI and a pre-commit hook — no model, no API keys. +- Early, deterministic conflict detection (delta-vs-base and cross-change) at commit/PR time, not at archive (Decision 11). +- Per-edit provenance and a delta↔spec correspondence check, within the delta model (Decision 12). - Skills delegate the merge to the CLI; the agent never performs it. - Backward compatible with changes archived (and specs synced) before this feature. **Non-Goals** -- Removing or replacing the `archive` lifecycle boundary (Decision 5). +- Removing or replacing the `archive` lifecycle boundary, or editing `specs/` in place instead of via deltas (Decisions 5, 11). - Folding the spec merge into `apply` (Decision 5). +- Building the spec-aware git diff driver (Decision 12 — provenance data is in scope; the diff driver is a deferred follow-up). - Wiring a specific hook runner or CI job into this repo (primitives in scope; config staged — Decision 4). - A scenario-granular "smart" merge (Decision 8 — the conventions mandate whole-requirement deltas; whole-block apply is the correct, deterministic semantics). - Bulk `sync`/`unarchive`, archive-folder prefix scheme, lifecycle timestamps — out of scope, accommodated. @@ -30,7 +33,7 @@ This design covers that primitive and the commands built on it. It supersedes th **Why one primitive.** The three asks in the thread are the same computation viewed three ways: - **Reverse** (`unarchive`) = restore the pre-image. Deterministic for *every* op, including the REMOVED/MODIFIED ones the delta alone cannot invert. - **Idempotent re-merge** (`sync` after a delta revision) = `specs_new = apply(delta_new, base)` where `base = current specs with delta_old reversed` (via the pre-image). This is the "no leftover crumbs" guarantee — the prior revision's contribution is removed, not layered over. -- **Drift** (`sync --check`, the hook, CI) = current `specs/` digest ≠ baseline digest. This is samholmes' "track the hash of the changes/ state applied to the spec," made precise. +- **Drift** (`sync --check`, the hook, CI) = current `specs/` digest ≠ baseline digest. This is the discussion's "track the hash of the changes/ state applied to the spec," made precise. Building three mechanisms would invite three drift bugs. Building one — the pre-image + digest — and deriving the rest is why this is the spine. @@ -63,7 +66,7 @@ Building three mechanisms would invite three drift bugs. Building one — the pr ## Decision 5 — Keep the `archive` boundary; reject folding the merge into `apply` -**Decision.** The lifecycle boundary stays: `specs/` is written at `sync`/`archive`, and `archive` remains the "fold finished change into shipped specs, then move the folder" step. Reject samholmes' proposal to remove `archive`, keep every change in a dated archive dir for its whole life, and make `apply` do both code and spec application. +**Decision.** The lifecycle boundary stays: `specs/` is written at `sync`/`archive`, and `archive` remains the "fold finished change into shipped specs, then move the folder" step. Reject the proposal to remove `archive`, keep every change in a dated archive dir for its whole life, and make `apply` do both code and spec application. **Why.** The invariant that pays for OpenSpec's value is **`specs/` describes only shipped reality**. Folding the merge into `apply` would (a) pollute the source of truth with proposed-but-unmerged or abandoned changes, and (b) let parallel changes overwrite each other's specs before any lands. The thread's own resolution: the awkwardness of the "final archive step" is not the boundary's fault, it is the *non-determinism* of crossing it. Make the crossing deterministic and reversible (Decisions 1–3) and the boundary becomes cheap in both directions — which is the actual fix. This is why determinism, not deletion, is the spine of this PR. @@ -96,6 +99,26 @@ Building three mechanisms would invite three drift bugs. Building one — the pr - **Move**: reuse `moveDirectory()`/`copyDirRecursive()` ([src/core/archive.ts:96-128](../../../src/core/archive.ts)) and their Windows fallbacks ([#605](https://github.com/Fission-AI/OpenSpec/pull/605)). A future `git mv` ([#709](https://github.com/Fission-AI/OpenSpec/issues/709)) covers both directions through this one helper. - **Metadata**: archive persists no `archived` timestamp today (only the folder-name prefix). If [#1245](https://github.com/Fission-AI/OpenSpec/issues/1245) lands one, `unarchive` should null it on restore — noted, not built. +## Decision 11 — Keep deltas; don't edit specs in place — and get early conflict resolution anyway + +**Context.** The discussion pushed a deeper question than "remove archive": *are we simulating deltas above git when git already stores deltas?* The proposed alternative: edit `specs/` in place (the git diff **is** the delta), drop the delta folder, and keep the "why" in a sidecar reasoning log; `sync` becomes a lint that checks spec-edits against log entries. Its sharpest argument is about **timing**: if in-flight changes are deltas applied at archive, conflicts surface late, at merge; editing the spec first "forces conflict resolution immediately." + +**Decision.** Keep the delta layer; do **not** make in-flight edits directly to `specs/`. But **adopt the timing argument's goal** by moving conflict detection earlier with `sync --check` (Decision 12). + +**Why keep deltas.** The delta layer is exactly what a raw git diff cannot give you: a clean separation between **proposed** and **shipped**. `specs/` always describes reality; multiple in-flight changes stay isolated and independently reviewable until each lands. Edit-in-place collapses that — proposed-but-unmerged or abandoned edits sit in the source of truth, and N parallel changes mutate the same files, so the only place conflicts can be resolved is one big merge at the end. The delta model makes each change a self-contained, reviewable unit and is what makes `unarchive` (reverse one change) and isolation-preserving parallelism *possible at all*. Git stores byte deltas; OpenSpec's deltas are behavioral agreements one level up — they answer *why* and *what-should-be* before code exists, which a diff cannot. + +**The synthesis — adopt the valid kernel.** The timing concern is real and we take it: rather than discovering at archive that a delta no longer applies (or that two changes touched the same requirement), `sync --check` surfaces those conflicts at commit/PR time, deterministically, as a plain binary (Decision 12, "Cross-Change Conflict Detection"). So we get "resolve conflicts immediately" **without** sacrificing proposed-vs-shipped isolation. This is the same shape as Decision 5: adopt the goal, reject the mechanism that would break the invariant. + +## Decision 12 — Provenance, delta↔spec correspondence, and the spec-aware diff driver (deferred) + +**Context.** The discussion also wanted the "why" to travel with the "what": a reasoning log spliced inline so a reviewer sees *what changed and why together* ("git as the delta store plus a spec-aware diff driver that splices the reasoning log inline"), and a consistency lint where a spec change with no log entry — or a log entry with no spec change — fails. + +**Decision.** Record **provenance** as part of the applied-delta baseline: when the engine writes a spec change, it records which change and which delta operation produced it. Expose it (e.g. `openspec sync --explain` / a provenance entry). Use it for a deterministic **delta↔spec correspondence** check in `sync --check`: every committed `specs/` edit for a change must trace to one of its delta operations (no orphan edits), and every delta operation must have landed (no unapplied deltas). The prose "why" is not re-authored — provenance links each spec edit to its change, whose `proposal.md` already holds the rationale. + +**Deferred (delineated follow-up): the spec-aware git diff driver.** A `git diff` driver for spec files that follows the provenance link and splices the change's rationale inline is a genuinely useful, separable feature. This PR delivers the **data** (provenance in the baseline) and the **check** (correspondence); the **presentation** (diff driver, custom textconv/diff attributes) is a follow-up — the same primitives-in-scope / wiring-staged split as the CI hook (Decision 4). Building a diff driver into a sync/unarchive PR would overscope it. + +**Why this is the right slice.** Provenance falls out of the merge for free (the engine already knows exactly what it applied), and correspondence reuses the baseline. Together they answer the discussion's consistency lint *within the delta model* (deltas are the source; specs are generated) rather than inverting it (specs as source, deltas as sidecar log) — which would reintroduce the edit-in-place problems of Decision 11. + ## Risks / trade-offs - **Behavior shift for `/opsx:sync` users.** Replacing agent merge with deterministic merge changes output for anyone who relied on the agent's scenario-level merges. This is intended (it fixes [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)), but it is a real change; call it out in the changeset and docs, and keep the conventions' "complete requirement, not a diff" rule prominent so deltas are authored to merge cleanly. diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md b/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md index 2c08fdae5f..e3066f7ff2 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md @@ -18,7 +18,7 @@ Verified against `Fission-AI/OpenSpec` at `main` (`546224e`, #1248) on 2026-06-2 - **reverse** is a byte-exact restore of the pre-image (deterministic for every op, including REMOVED/MODIFIED) → `unarchive`; - **idempotent re-merge** is "reverse the old delta via the baseline, apply the new" → crumb-free `sync`; -- **drift detection** is "current `specs/` digest ≠ baseline digest" (samholmes' "track the hash of the changes/ state applied to the spec") → a code-only gate for CI and a `sync --fix` pre-commit hook (the eslint-`--fix` analogy both arrived at). +- **drift detection** is "current `specs/` digest ≠ baseline digest" (the discussion's "track the hash of the changes/ state applied to the spec") → a code-only gate for CI and a `sync --fix` pre-commit hook (the eslint-`--fix` analogy the discussion arrived at). Frozen into the archived folder, that baseline *is* the unarchive reversal snapshot. One primitive, three payoffs. @@ -32,19 +32,23 @@ Design principle the whole thread converges on, and the one this project is alre - default / `--fix`: write `specs/` to the regenerated result (idempotent — re-running is a no-op; revising the delta regenerates with no crumbs); - `--check`: read-only; exit non-zero if the change's deltas are not cleanly appliable, or (when `specs/` has been synced) if committed `specs/` ≠ the regenerated output. This is the **codegen/IaC-style drift gate CI runs as a plain binary, no model, no API keys** — and the auto-fixer a `pre-commit` hook can call. + `--check` also surfaces conflicts **early** — at commit/PR time instead of at archive: a delta that no longer applies to the current base, or two active changes targeting the same requirement. This is the discussion's "force conflict resolution immediately rather than at a later merge step," delivered without abandoning the delta model (design Decision 11). And the engine records **provenance** — which change and delta op produced each spec edit — so every line in `specs/` traces to a delta, enabling a deterministic delta↔spec correspondence check (orphan edit / unapplied delta → fail) and a future spec-aware diff driver (design Decision 12). + 3. **REVERSE COMMAND — `openspec unarchive [change-name]` (`cli-unarchive` NEW).** The deterministic inverse of archive: resolve the archived folder (prefix-tolerant, never auto-picking among ambiguous matches), reverse the spec merge from the baseline under a **drift guard** (refuse rather than clobber a requirement a later change has since touched), move the folder back to `changes//`, **atomically** ("Abort. No files were changed."). `--keep-specs` restores the folder without touching `specs/` (the always-safe escape hatch, mirror of archive's `--skip-specs`). Changes archived before this feature have no baseline and degrade gracefully — reverse the self-invertible half (ADDED/RENAMED), refuse to guess the rest. 4. **NO MODEL IN THE MERGE — skills delegate to the CLI (`specs-sync-skill` MODIFIED; `opsx-unarchive-skill` NEW).** `/opsx:sync` stops doing agent-driven edits and **invokes `openspec sync`**; the new `/opsx:unarchive` invokes `openspec unarchive`. The deterministic work lives in TypeScript; skills only select, confirm, and render. This is the direction [#863](https://github.com/Fission-AI/OpenSpec/issues/863)/[#799](https://github.com/Fission-AI/OpenSpec/issues/799)/[#656](https://github.com/Fission-AI/OpenSpec/issues/656) ask for, and it removes the [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) "intelligent merge drops scenarios" failure mode by construction. ### What this deliberately does *not* change -samholmes proposed removing `archive` entirely — keep every change in its dated archive location for its whole life and fold the spec merge into `apply`. **Rejected, with the maintainer's reasoning, and it shapes the design:** `specs/` only ever describes *shipped* reality. Folding the merge into `apply` would let proposed-but-unmerged (or abandoned) changes pollute the source of truth and let parallel changes step on each other's specs before any land. So the archive boundary stays; what changes is that the merge across it becomes deterministic and reversible. The determinism is what makes the boundary cheap to cross in both directions, which is the real fix for the "awkward final step." (Full treatment in [design.md](./design.md) Decision 5.) +The discussion proposed removing `archive` entirely — keep every change in its dated archive location for its whole life and fold the spec merge into `apply`. **Rejected, with the maintainer's reasoning, and it shapes the design:** `specs/` only ever describes *shipped* reality. Folding the merge into `apply` would let proposed-but-unmerged (or abandoned) changes pollute the source of truth and let parallel changes step on each other's specs before any land. So the archive boundary stays; what changes is that the merge across it becomes deterministic and reversible. The determinism is what makes the boundary cheap to cross in both directions, which is the real fix for the "awkward final step." (Full treatment in [design.md](./design.md) Decision 5.) + +The discussion also raised the deeper question — *are deltas just simulating, above git, what git already does?* — and the matching proposal to **edit `specs/` in place** (the git diff *is* the delta) with a sidecar reasoning log. **Also rejected, and likewise constructive:** the delta layer is precisely what a raw git diff cannot give — clean isolation of *proposed* from *shipped*, so `specs/` stays canonical while several in-flight changes remain independently reviewable until each lands (it is also what makes reversing a single change, i.e. `unarchive`, possible). But its sharpest argument — that editing the spec first forces conflict resolution *now* rather than at merge — is valid, so we **take that goal**: `sync --check` moves conflict detection early (design Decision 11), and provenance + correspondence give the "reasoning travels with the change, every edit is accounted for" property *within* the delta model rather than by inverting it (design Decision 12). ## Capabilities ### New Capabilities -- `cli-sync`: the deterministic, idempotent spec-merge engine exposed as `openspec sync [change]` — `--fix`/default writes `specs/` from the change's deltas (byte-identical for the same delta + base; re-running is a no-op; a revised delta regenerates with no crumbs), `--check` is a read-only, model-free drift gate for CI and pre-commit hooks. Records a per-change applied-delta baseline (pre-image + digest) that also powers reverse and drift detection. The same engine `archive` uses internally. +- `cli-sync`: the deterministic, idempotent spec-merge engine exposed as `openspec sync [change]` — `--fix`/default writes `specs/` from the change's deltas (byte-identical for the same delta + base; re-running is a no-op; a revised delta regenerates with no crumbs), `--check` is a read-only, model-free gate for CI and pre-commit hooks that also surfaces conflicts early (delta-vs-base, and two active changes targeting one requirement) and verifies delta↔spec correspondence. Records a per-change applied-delta baseline (pre-image + digest + provenance) that powers reverse, drift detection, and tracing each spec edit to its delta. The same engine `archive` uses internally. - `cli-unarchive`: `openspec unarchive [change-name]` — the deterministic inverse of archive. Resolves an archived change (prefix-tolerant, never auto-picking ambiguous matches), reverses the spec merge from the baseline under a drift guard, moves the folder back, atomically. `--keep-specs` restores the folder without touching `specs/`; pre-baseline archives degrade gracefully. - `opsx-unarchive-skill`: a `/opsx:unarchive` workflow skill that delegates to `openspec unarchive` for all deterministic work (selection, confirmation, rendering only). Expanded profile; no cross-skill dependency. @@ -77,8 +81,8 @@ Delivers (the Discord conclusions): - **Deterministic, code-only sync path** *("the right fix is a deterministic, code-only openspec sync/archive path that CI can run as a plain binary… I'll fold this into the unarchive PR")* → `openspec sync` + `--check`. - **`openspec unarchive` / `/opsx:unarchive`** that "moves the folder back and reverses the spec merge," including the hard case where the archive is buried in a multi-file commit. -- **Idempotent, crumb-free regeneration** and the **`sync --fix` pre-commit hook** *(samholmes' "sync is a lint step… eslint --fix", Clay's "sync acts as the auto-fixer")*. -- **Hash-tracked drift** *(samholmes' "track the hash of the changes/ state applied to the spec")* → the baseline digest. +- **Idempotent, crumb-free regeneration** and the **`sync --fix` pre-commit hook** *(the discussion's "sync is a lint step… eslint --fix" / "sync acts as the auto-fixer")*. +- **Hash-tracked drift** *(the discussion's "track the hash of the changes/ state applied to the spec")* → the baseline digest. Directly fixes / strengthens: @@ -94,9 +98,15 @@ Delineated from adjacent work (coordinate, don't collide): - [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md) — planning-time archive ordering. Unarchive's drift guard and sync's idempotency are the runtime spec-layer counterpart; the two compose. - [#704](https://github.com/Fission-AI/OpenSpec/issues/704)/[#682](https://github.com/Fission-AI/OpenSpec/issues/682) (archive hooks), [#709](https://github.com/Fission-AI/OpenSpec/issues/709) (`git mv`) — symmetric `unarchive` hook points and a shared `moveDirectory()` make a future switch cover both directions. +Deferred follow-ups (enabled by this PR, not built here): + +- **Spec-aware git diff driver** — a diff driver for spec files that follows the recorded provenance to splice each change's rationale inline, so reviewers see *what changed and why together* (the discussion's idea). This PR ships the data (provenance) and the correspondence check; the diff driver is a separable follow-up (design Decision 12). +- **CI drift gate + pre-commit `sync --fix` hook** — the CLI primitives (`--check`/`--fix`) are in scope; wiring a hook runner (none exists in the repo) or a CI job is staged (design Decision 4). + Considered and rejected (documented in design): -- **Remove `archive`; fold the merge into `apply`; edit changes in place in dated dirs** (samholmes' proposal). Rejected to preserve the invariant that `specs/` describes only shipped reality (design Decision 5). +- **Remove `archive`; fold the merge into `apply`** (raised in the discussion). Rejected to preserve the invariant that `specs/` describes only shipped reality (design Decision 5). +- **Edit `specs/` in place; treat the git diff as the delta; keep semantic deltas in a sidecar log** (raised in the discussion). Rejected because the delta layer is what isolates *proposed* from *shipped* and makes parallel changes and single-change reverse possible; its valid kernel (early conflict resolution, reasoning-with-the-change) is adopted via `sync --check` conflict detection and provenance/correspondence instead (design Decisions 11–12). Related, out of scope (referenced, not closed): diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md index af46973a98..a6844f01b0 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md @@ -125,6 +125,63 @@ The sync command SHALL support `--json` for non-interactive use, emitting machin #### Scenario: JSON blocked path -- **WHEN** `openspec sync --json` cannot proceed (no change, deltas not appliable, or drift under `--check`) +- **WHEN** `openspec sync --json` cannot proceed (no change, deltas not appliable, conflict, or drift under `--check`) - **THEN** it emits a machine-readable diagnostic with a stable code - **AND** exits with a non-zero status code + +### Requirement: Change Provenance + +When sync applies a change's deltas to `openspec/specs/`, it SHALL record, for each resulting spec change, which change and which delta operation produced it, so that any spec content can be traced back to its source delta. + +#### Scenario: Provenance recorded on apply + +- **WHEN** sync applies a delta operation (ADDED/MODIFIED/REMOVED/RENAMED) that changes a spec +- **THEN** it records, with the applied-delta baseline, the originating change name and the delta operation that produced that spec change + +#### Scenario: Explainability + +- **WHEN** the user runs sync with an explain option (for example `openspec sync --explain` or `--json`) +- **THEN** the output associates each affected requirement with the change and delta operation that produced it +- **AND** it does not re-author rationale prose (the change's `proposal.md` remains the source of the "why") + +### Requirement: Delta–Spec Correspondence + +In `--check` mode, sync SHALL verify a two-way correspondence between a change's deltas and the committed `openspec/specs/` content: every spec change attributable to the change traces to one of its delta operations, and every delta operation has landed in the specs. + +#### Scenario: Orphan spec edit detected + +- **WHEN** `--check` finds a change to `openspec/specs/` attributable to the change that does not correspond to any of its delta operations +- **THEN** the command reports the unexplained spec change +- **AND** it exits with a non-zero status code and modifies no files + +#### Scenario: Unapplied delta detected + +- **WHEN** `--check` finds a delta operation that has not been reflected in `openspec/specs/` +- **THEN** the command reports the unapplied delta +- **AND** it exits with a non-zero status code and modifies no files + +#### Scenario: Correspondence holds + +- **WHEN** every spec change traces to a delta operation and every delta operation has landed +- **THEN** the correspondence check passes + +### Requirement: Cross-Change Conflict Detection + +Sync SHALL surface conflicts deterministically and early — at commit or PR time rather than only at archive — including when a change's deltas no longer apply to the current base specs and when two active changes target the same requirement. + +#### Scenario: Delta no longer applies to the current base + +- **WHEN** `--check` runs and a change's MODIFIED, REMOVED, or RENAMED-from header is absent from the current base spec (for example, the base moved since the delta was authored) +- **THEN** the command reports the conflict with the specific requirement +- **AND** it exits with a non-zero status code and modifies no files + +#### Scenario: Two active changes target the same requirement + +- **WHEN** `--check` runs across active changes and more than one active change modifies, removes, or renames the same requirement +- **THEN** the command reports the overlapping changes and requirement as a potential conflict to resolve before archiving +- **AND** it exits with a non-zero status code + +#### Scenario: No conflicts + +- **WHEN** a change's deltas apply cleanly to the current base and no other active change targets the same requirements +- **THEN** the conflict check passes diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md b/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md index 9726870641..b02afa6603 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md @@ -16,6 +16,10 @@ - [ ] 2.2 `--check`: read-only; exit non-zero when deltas are not cleanly appliable to the base, or (when `specs/` was synced) when committed `specs/` ≠ regenerated output. Never writes `specs/`. Support `--all`/`--changes` style fan-out for CI (or document the loop). - [ ] 2.3 Register `sync [change]` in [src/cli/index.ts](../../../src/cli/index.ts) (`--check`, `--fix`, `--json`, `--store`, hidden store-path), mirroring archive (326-343). - [ ] 2.4 Tests: write produces deterministic `specs/`; `--check` clean vs drifted exit codes; `--check` never mutates; unknown/ambiguous change diagnostics; JSON shape on success and each blocked path. +- [ ] 2.5 Provenance: record, with the baseline, the originating change + delta operation for each applied spec change; add an `--explain` (and JSON) output that maps each affected requirement to its source delta. Do not re-author rationale (link to the change's `proposal.md`). (design Decision 12) +- [ ] 2.6 Delta↔spec correspondence in `--check`: fail on an orphan spec edit (attributable to the change but matching no delta op) and on an unapplied delta (delta op not reflected in `specs/`); pass when both directions hold. (design Decision 12) +- [ ] 2.7 Cross-change conflict detection in `--check` (design Decision 11): (a) delta-vs-base — a MODIFIED/REMOVED/RENAMED-from header absent from the current base (surfaces #1112 early); (b) cross-change — two active changes targeting the same requirement; report specifics, non-zero exit, no writes. Coordinate the cross-change check with [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md). +- [ ] 2.8 Tests: provenance recorded + `--explain` mapping; orphan-edit and unapplied-delta both fail `--check`; delta-vs-base conflict fails with the right requirement; two active changes on one requirement flagged; clean case passes. ## 3. `openspec unarchive` (reverse, built on the baseline) @@ -46,6 +50,7 @@ - [ ] 6.1 Document the CI drift-gate pattern (`openspec sync --check` as a plain binary, no model/API keys) in `docs/`; provide a copy-paste job snippet. (Wiring the actual workflow file is the owner's call.) - [ ] 6.2 Document the pre-commit hook pattern (`sync --check` to detect, `sync --fix` to auto-remediate before `git commit --amend`), eslint-`--fix` style. (Choosing husky/lefthook/native is a separable follow-up; none exists in the repo today.) +- [ ] 6.3 Document the **spec-aware git diff driver** follow-up (design Decision 12): a diff driver for spec files that follows the recorded provenance to splice each change's rationale inline. Out of scope to build here; capture the provenance shape it would consume so the follow-up is unblocked. ## 7. Docs & supersession From 041387f3d8dca97dc8483bfa3c9230e4f9bc6178 Mon Sep 17 00:00:00 2001 From: Clay Good Date: Mon, 29 Jun 2026 16:58:14 -0500 Subject: [PATCH 4/8] docs(openspec): add incremental checking; engage spec-as-source-of-truth vision MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Continue folding the design discussion into PR #1279, within OpenSpec's scope. Adopt (in scope, new): - Incremental checking — `sync --check` uses the baseline per-spec digests to skip unchanged specs and re-check only what moved ("use hashes to optimize when checks are needed"). Correctness invariant: a skip is allowed only on an exact digest match; mismatch/missing/unknown-scheme forces a full check, so the incremental verdict always equals the full verdict. New cli-sync requirement + design Decision 13 + tasks. Acknowledge and defer: - Spec linter/formatter ("how the spec looks/reads/organized"). This PR makes OUTPUT specs canonical and byte-stable; an authoring-side formatter is a separable follow-up adjacent to openspec-conventions/validate. Engage and reject, with reasoning (documented, Decision 13): - Spec-as-source-of-truth: edit in place + commit only the spec, treating proposal/design/tasks as disposable. Rejected — those artifacts are the reviewable, resumable product (agreement before code); the delta layer keeps proposed vs shipped isolated. Valid kernel (lightweight changes needn't carry every artifact) is already served by schema flexibility, outside this PR. No human names anywhere in PR content (verified). Validates with `openspec validate --strict`. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../design.md | 16 ++++++++++++ .../proposal.md | 6 +++-- .../specs/cli-sync/spec.md | 25 +++++++++++++++++++ .../tasks.md | 3 +++ 4 files changed, 48 insertions(+), 2 deletions(-) diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/design.md b/openspec/changes/add-deterministic-sync-and-unarchive/design.md index 1810d63798..0c90643a64 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/design.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/design.md @@ -15,6 +15,7 @@ This design covers that primitive and the commands built on it. It supersedes th - Code-only drift detection (`sync --check`) usable by CI and a pre-commit hook — no model, no API keys. - Early, deterministic conflict detection (delta-vs-base and cross-change) at commit/PR time, not at archive (Decision 11). - Per-edit provenance and a delta↔spec correspondence check, within the delta model (Decision 12). +- Hash-optimized incremental checking: `--check` re-checks only specs whose digest changed, without changing any verdict (Decision 13). - Skills delegate the merge to the CLI; the agent never performs it. - Backward compatible with changes archived (and specs synced) before this feature. @@ -22,6 +23,7 @@ This design covers that primitive and the commands built on it. It supersedes th - Removing or replacing the `archive` lifecycle boundary, or editing `specs/` in place instead of via deltas (Decisions 5, 11). - Folding the spec merge into `apply` (Decision 5). - Building the spec-aware git diff driver (Decision 12 — provenance data is in scope; the diff driver is a deferred follow-up). +- Building a spec linter/formatter or reducing the committed artifact set (Decision 13 — output specs are made canonical here; an authoring formatter and the artifact-model question are separable). - Wiring a specific hook runner or CI job into this repo (primitives in scope; config staged — Decision 4). - A scenario-granular "smart" merge (Decision 8 — the conventions mandate whole-requirement deltas; whole-block apply is the correct, deterministic semantics). - Bulk `sync`/`unarchive`, archive-folder prefix scheme, lifecycle timestamps — out of scope, accommodated. @@ -119,6 +121,20 @@ Building three mechanisms would invite three drift bugs. Building one — the pr **Why this is the right slice.** Provenance falls out of the merge for free (the engine already knows exactly what it applied), and correspondence reuses the baseline. Together they answer the discussion's consistency lint *within the delta model* (deltas are the source; specs are generated) rather than inverting it (specs as source, deltas as sidecar log) — which would reintroduce the edit-in-place problems of Decision 11. +## Decision 13 — Spec-as-source-of-truth, incremental checking, and the spec linter + +**Context.** The discussion's end-state wish: treat *the spec* as the single source of truth ("new source code"), iterate on it directly, let tools "align the spec to implementation like a linter/formatter would," using "hashes to optimize when checks are needed," and reduce what gets committed to just the spec — treating proposal/design/tasks as "artifacts of an individual's workflow, not the product to commit." The framework's job becomes "how the spec looks and reads and is organized/formatted." + +**Decision.** Adopt the one part that is squarely in this PR's scope — **hash-optimized incremental checking** — and explicitly position the rest (spec linter/formatter; reduced artifact set) as adjacent, not folded in. + +**Adopt: incremental checking.** The applied-delta baseline already carries a per-spec digest. So `sync --check` (and the CI/pre-commit gate) can skip any spec whose content digest is unchanged since it was last reconciled, and re-check only what changed — exactly "use hashes to optimize when checks are needed." The correctness invariant: a skip is allowed *only* on an exact digest match against a recorded baseline; a mismatch, a missing baseline, or an unknown digest scheme forces a full check. So incremental mode is an optimization that can never change a verdict versus a full check — important for a gate. This makes the gate cheap enough to run on every commit even in a repo with hundreds of specs. + +**Reaffirm (with nuance): deltas + planning artifacts stay.** The wish to commit "only the spec" and treat design/tasks as disposable runs against the reason those artifacts exist: in OpenSpec they *are* the reviewable, resumable product — the agreement about behavior *before* code, and the context a fresh session (or a reader six months later) needs. The delta layer is, again, what keeps *proposed* separate from *shipped* and lets parallel changes stay isolated and individually reversible (Decisions 5, 11). The valid kernel — that not every change needs the full artifact set — is real but is already served *outside this PR* by schema flexibility (a change can run a minimal, spec-only schema); this PR neither needs nor changes that. Recorded here so the "just commit the spec" question has a documented answer. + +**Defer (delineated follow-up): the spec linter/formatter.** "How the spec looks, reads, and is organized" — a formatter (prettier-for-specs) and structural/organization lint — is a genuine, separable direction adjacent to `openspec-conventions` and `openspec validate`. This PR already contributes the half that belongs to the merge engine: its output specs are **canonical and byte-stable** (deterministic recomposition, newline normalization), so synced/archived specs have one well-defined form. A formatter for *authoring* input (deltas and hand-edited specs), and richer organization lint, are future work — not built here, to avoid overscoping an engine PR into an authoring-experience PR. + +**Why this split.** Incremental checking is a pure consequence of the digest this PR already records — high value, near-zero added surface. The formatter and the artifact-model questions are larger, separable products; folding them in would blur a focused engine PR. Adopt the kernel, document the boundary. + ## Risks / trade-offs - **Behavior shift for `/opsx:sync` users.** Replacing agent merge with deterministic merge changes output for anyone who relied on the agent's scenario-level merges. This is intended (it fixes [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)), but it is a real change; call it out in the changeset and docs, and keep the conventions' "complete requirement, not a diff" rule prominent so deltas are authored to merge cleanly. diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md b/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md index e3066f7ff2..bcb35d447c 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md @@ -32,7 +32,7 @@ Design principle the whole thread converges on, and the one this project is alre - default / `--fix`: write `specs/` to the regenerated result (idempotent — re-running is a no-op; revising the delta regenerates with no crumbs); - `--check`: read-only; exit non-zero if the change's deltas are not cleanly appliable, or (when `specs/` has been synced) if committed `specs/` ≠ the regenerated output. This is the **codegen/IaC-style drift gate CI runs as a plain binary, no model, no API keys** — and the auto-fixer a `pre-commit` hook can call. - `--check` also surfaces conflicts **early** — at commit/PR time instead of at archive: a delta that no longer applies to the current base, or two active changes targeting the same requirement. This is the discussion's "force conflict resolution immediately rather than at a later merge step," delivered without abandoning the delta model (design Decision 11). And the engine records **provenance** — which change and delta op produced each spec edit — so every line in `specs/` traces to a delta, enabling a deterministic delta↔spec correspondence check (orphan edit / unapplied delta → fail) and a future spec-aware diff driver (design Decision 12). + `--check` also surfaces conflicts **early** — at commit/PR time instead of at archive: a delta that no longer applies to the current base, or two active changes targeting the same requirement. This is the discussion's "force conflict resolution immediately rather than at a later merge step," delivered without abandoning the delta model (design Decision 11). And the engine records **provenance** — which change and delta op produced each spec edit — so every line in `specs/` traces to a delta, enabling a deterministic delta↔spec correspondence check (orphan edit / unapplied delta → fail) and a future spec-aware diff driver (design Decision 12). Because the baseline carries a per-spec digest, `--check` is **incremental** — it skips specs whose content is unchanged and re-checks only what moved (the discussion's "use hashes to optimize when checks are needed"), so the gate stays cheap on every commit even in a large repo (design Decision 13). 3. **REVERSE COMMAND — `openspec unarchive [change-name]` (`cli-unarchive` NEW).** The deterministic inverse of archive: resolve the archived folder (prefix-tolerant, never auto-picking among ambiguous matches), reverse the spec merge from the baseline under a **drift guard** (refuse rather than clobber a requirement a later change has since touched), move the folder back to `changes//`, **atomically** ("Abort. No files were changed."). `--keep-specs` restores the folder without touching `specs/` (the always-safe escape hatch, mirror of archive's `--skip-specs`). Changes archived before this feature have no baseline and degrade gracefully — reverse the self-invertible half (ADDED/RENAMED), refuse to guess the rest. @@ -42,7 +42,7 @@ Design principle the whole thread converges on, and the one this project is alre The discussion proposed removing `archive` entirely — keep every change in its dated archive location for its whole life and fold the spec merge into `apply`. **Rejected, with the maintainer's reasoning, and it shapes the design:** `specs/` only ever describes *shipped* reality. Folding the merge into `apply` would let proposed-but-unmerged (or abandoned) changes pollute the source of truth and let parallel changes step on each other's specs before any land. So the archive boundary stays; what changes is that the merge across it becomes deterministic and reversible. The determinism is what makes the boundary cheap to cross in both directions, which is the real fix for the "awkward final step." (Full treatment in [design.md](./design.md) Decision 5.) -The discussion also raised the deeper question — *are deltas just simulating, above git, what git already does?* — and the matching proposal to **edit `specs/` in place** (the git diff *is* the delta) with a sidecar reasoning log. **Also rejected, and likewise constructive:** the delta layer is precisely what a raw git diff cannot give — clean isolation of *proposed* from *shipped*, so `specs/` stays canonical while several in-flight changes remain independently reviewable until each lands (it is also what makes reversing a single change, i.e. `unarchive`, possible). But its sharpest argument — that editing the spec first forces conflict resolution *now* rather than at merge — is valid, so we **take that goal**: `sync --check` moves conflict detection early (design Decision 11), and provenance + correspondence give the "reasoning travels with the change, every edit is accounted for" property *within* the delta model rather than by inverting it (design Decision 12). +The discussion also raised the deeper question — *are deltas just simulating, above git, what git already does?* — and the matching proposals to **edit `specs/` in place** (the git diff *is* the delta) with a sidecar reasoning log, and to **commit only the spec**, treating proposal/design/tasks as disposable personal workflow. **Also rejected, and likewise constructive:** the delta layer is precisely what a raw git diff cannot give — clean isolation of *proposed* from *shipped*, so `specs/` stays canonical while several in-flight changes remain independently reviewable until each lands (it is also what makes reversing a single change, i.e. `unarchive`, possible); and the planning artifacts *are* the reviewable, resumable product, not crumbs. But the sharpest kernels are valid, so we **take them**: editing-the-spec-first "forces conflict resolution now" → `sync --check` moves conflict detection early (Decision 11); "reasoning travels with the change, every edit accounted for" → provenance + correspondence, within the delta model (Decision 12); "use hashes to optimize when checks are needed" → incremental `--check` (Decision 13). The framework-as-spec-formatter wish is acknowledged and bounded (Decision 13): this PR makes *output* specs canonical and byte-stable, while a full authoring formatter stays a separable follow-up. ## Capabilities @@ -102,11 +102,13 @@ Deferred follow-ups (enabled by this PR, not built here): - **Spec-aware git diff driver** — a diff driver for spec files that follows the recorded provenance to splice each change's rationale inline, so reviewers see *what changed and why together* (the discussion's idea). This PR ships the data (provenance) and the correspondence check; the diff driver is a separable follow-up (design Decision 12). - **CI drift gate + pre-commit `sync --fix` hook** — the CLI primitives (`--check`/`--fix`) are in scope; wiring a hook runner (none exists in the repo) or a CI job is staged (design Decision 4). +- **Spec linter/formatter** — a "prettier-for-specs" formatter and richer organization lint (the discussion's "the framework's goal is how the spec looks and reads and is organized"). This PR makes *output* specs canonical and byte-stable; an authoring-side formatter is adjacent to `openspec-conventions`/`validate` and stays a separable follow-up (design Decision 13). Considered and rejected (documented in design): - **Remove `archive`; fold the merge into `apply`** (raised in the discussion). Rejected to preserve the invariant that `specs/` describes only shipped reality (design Decision 5). - **Edit `specs/` in place; treat the git diff as the delta; keep semantic deltas in a sidecar log** (raised in the discussion). Rejected because the delta layer is what isolates *proposed* from *shipped* and makes parallel changes and single-change reverse possible; its valid kernel (early conflict resolution, reasoning-with-the-change) is adopted via `sync --check` conflict detection and provenance/correspondence instead (design Decisions 11–12). +- **Commit only the spec; treat proposal/design/tasks as disposable personal workflow** (raised in the discussion). Rejected because those artifacts are the reviewable, resumable product — the behavioral agreement before code. The valid kernel (lightweight changes needn't carry every artifact) is already served by schema flexibility, outside this PR's scope (design Decision 13). Related, out of scope (referenced, not closed): diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md index a6844f01b0..6b13f9932b 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md @@ -185,3 +185,28 @@ Sync SHALL surface conflicts deterministically and early — at commit or PR tim - **WHEN** a change's deltas apply cleanly to the current base and no other active change targets the same requirements - **THEN** the conflict check passes + +### Requirement: Incremental Checking + +The check operation MAY use the recorded baseline digests to skip specs whose content is unchanged since they were last reconciled, so that checking cost scales with what changed rather than with repository size. A skip SHALL be permitted only when it cannot change the result versus a full check. + +#### Scenario: Unchanged spec is skipped + +- **WHEN** `--check` runs and a spec's current content digest matches the digest recorded in the baseline +- **THEN** the command may skip re-checking that spec +- **AND** the overall result is identical to checking it fully + +#### Scenario: Changed spec is re-checked + +- **WHEN** a spec's current content digest does not match the recorded baseline digest +- **THEN** the command performs the full check for that spec + +#### Scenario: Missing or unknown baseline forces a full check + +- **WHEN** no baseline digest is recorded for a spec, or the recorded digest uses an unrecognized scheme +- **THEN** the command performs the full check for that spec rather than skipping it + +#### Scenario: Incremental result equals full result + +- **WHEN** the same change is checked incrementally and with all skips disabled +- **THEN** both runs reach the same pass or fail verdict diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md b/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md index b02afa6603..60707d912e 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md @@ -20,6 +20,8 @@ - [ ] 2.6 Delta↔spec correspondence in `--check`: fail on an orphan spec edit (attributable to the change but matching no delta op) and on an unapplied delta (delta op not reflected in `specs/`); pass when both directions hold. (design Decision 12) - [ ] 2.7 Cross-change conflict detection in `--check` (design Decision 11): (a) delta-vs-base — a MODIFIED/REMOVED/RENAMED-from header absent from the current base (surfaces #1112 early); (b) cross-change — two active changes targeting the same requirement; report specifics, non-zero exit, no writes. Coordinate the cross-change check with [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md). - [ ] 2.8 Tests: provenance recorded + `--explain` mapping; orphan-edit and unapplied-delta both fail `--check`; delta-vs-base conflict fails with the right requirement; two active changes on one requirement flagged; clean case passes. +- [ ] 2.9 Incremental checking (design Decision 13): `--check` skips a spec whose current digest matches the recorded baseline; re-checks on mismatch; forces a full check when the baseline is missing or its scheme is unrecognized. Add an escape hatch to disable skips (e.g. `--no-incremental`) for the equivalence test. +- [ ] 2.10 Tests: unchanged spec skipped; changed spec re-checked; missing/unknown-scheme baseline → full check; **incremental verdict == full verdict** on the same input (the correctness invariant); a touched spec in a large fixture is the only one re-checked. ## 3. `openspec unarchive` (reverse, built on the baseline) @@ -51,6 +53,7 @@ - [ ] 6.1 Document the CI drift-gate pattern (`openspec sync --check` as a plain binary, no model/API keys) in `docs/`; provide a copy-paste job snippet. (Wiring the actual workflow file is the owner's call.) - [ ] 6.2 Document the pre-commit hook pattern (`sync --check` to detect, `sync --fix` to auto-remediate before `git commit --amend`), eslint-`--fix` style. (Choosing husky/lefthook/native is a separable follow-up; none exists in the repo today.) - [ ] 6.3 Document the **spec-aware git diff driver** follow-up (design Decision 12): a diff driver for spec files that follows the recorded provenance to splice each change's rationale inline. Out of scope to build here; capture the provenance shape it would consume so the follow-up is unblocked. +- [ ] 6.4 Document the **spec linter/formatter** follow-up (design Decision 13): note that this PR makes synced/archived spec output canonical and byte-stable, and scope a separable authoring-side formatter + organization lint adjacent to `openspec-conventions`/`validate`. Out of scope to build here. ## 7. Docs & supersession From d2525dff82789470ce7d7c4f81104a4a4cd854bf Mon Sep 17 00:00:00 2001 From: Clay Good Date: Mon, 29 Jun 2026 17:06:44 -0500 Subject: [PATCH 5/8] docs(openspec): fold the deterministic spec formatter into scope (openspec format) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per request, promote the spec linter/formatter from a deferred follow-up to a first-class capability in this PR. New capability cli-format: `openspec format [target]` — a deterministic, behavior-preserving spec/delta formatter with a `--check` gate, sharing ONE canonicalizer with the merge engine (sync/archive) so their output cannot diverge: synced/archived specs pass `format --check` by construction. - Behavior-preserving: normalizes only presentation (whitespace, blank lines, list markers, heading spacing, canonical delta-section headers); never reorders requirements/scenarios or rewrites prose (parse-before==parse-after). - `--check` is a second model-free gate (CI + pre-commit) alongside sync --check; `--fix`/default writes; incremental via digests; --json. - Pure text → no agent, no /opsx:format skill. Not validate (semantic) and not code-vs-spec verify (inference) — those compose around it. Extract a shared src/core/spec-canonical.ts so merge engine + formatter use one canonicalizer. New design Decision 14; Decision 13's deferral removed. New tasks phase 2A + canonicalizer extraction (1.1a). Updated proposal, goals/non-goals. Validates with `openspec validate --strict`. No human names in PR content. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../design.md | 22 ++- .../proposal.md | 18 ++- .../specs/cli-format/spec.md | 144 ++++++++++++++++++ .../tasks.md | 17 ++- 4 files changed, 188 insertions(+), 13 deletions(-) create mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-format/spec.md diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/design.md b/openspec/changes/add-deterministic-sync-and-unarchive/design.md index 0c90643a64..3a906d6402 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/design.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/design.md @@ -1,4 +1,4 @@ -# Design: deterministic spec-merge engine — `sync` (forward), `unarchive` (reverse) +# Design: deterministic spec tooling — `sync` (forward), `unarchive` (reverse), `format` (canonical) ## Context @@ -16,6 +16,7 @@ This design covers that primitive and the commands built on it. It supersedes th - Early, deterministic conflict detection (delta-vs-base and cross-change) at commit/PR time, not at archive (Decision 11). - Per-edit provenance and a delta↔spec correspondence check, within the delta model (Decision 12). - Hash-optimized incremental checking: `--check` re-checks only specs whose digest changed, without changing any verdict (Decision 13). +- A deterministic, behavior-preserving spec formatter (`openspec format` + `--check`) sharing one canonicalizer with the merge engine (Decision 14). - Skills delegate the merge to the CLI; the agent never performs it. - Backward compatible with changes archived (and specs synced) before this feature. @@ -23,7 +24,8 @@ This design covers that primitive and the commands built on it. It supersedes th - Removing or replacing the `archive` lifecycle boundary, or editing `specs/` in place instead of via deltas (Decisions 5, 11). - Folding the spec merge into `apply` (Decision 5). - Building the spec-aware git diff driver (Decision 12 — provenance data is in scope; the diff driver is a deferred follow-up). -- Building a spec linter/formatter or reducing the committed artifact set (Decision 13 — output specs are made canonical here; an authoring formatter and the artifact-model question are separable). +- Inference-based "align spec to implementation" / code-vs-spec checking — that needs a model and is the `verify` direction (#880), not the deterministic `format` (Decision 14). +- Reducing the committed artifact set (commit-only-the-spec) — a separate product question served by schema flexibility (Decision 13). - Wiring a specific hook runner or CI job into this repo (primitives in scope; config staged — Decision 4). - A scenario-granular "smart" merge (Decision 8 — the conventions mandate whole-requirement deltas; whole-block apply is the correct, deterministic semantics). - Bulk `sync`/`unarchive`, archive-folder prefix scheme, lifecycle timestamps — out of scope, accommodated. @@ -131,9 +133,21 @@ Building three mechanisms would invite three drift bugs. Building one — the pr **Reaffirm (with nuance): deltas + planning artifacts stay.** The wish to commit "only the spec" and treat design/tasks as disposable runs against the reason those artifacts exist: in OpenSpec they *are* the reviewable, resumable product — the agreement about behavior *before* code, and the context a fresh session (or a reader six months later) needs. The delta layer is, again, what keeps *proposed* separate from *shipped* and lets parallel changes stay isolated and individually reversible (Decisions 5, 11). The valid kernel — that not every change needs the full artifact set — is real but is already served *outside this PR* by schema flexibility (a change can run a minimal, spec-only schema); this PR neither needs nor changes that. Recorded here so the "just commit the spec" question has a documented answer. -**Defer (delineated follow-up): the spec linter/formatter.** "How the spec looks, reads, and is organized" — a formatter (prettier-for-specs) and structural/organization lint — is a genuine, separable direction adjacent to `openspec-conventions` and `openspec validate`. This PR already contributes the half that belongs to the merge engine: its output specs are **canonical and byte-stable** (deterministic recomposition, newline normalization), so synced/archived specs have one well-defined form. A formatter for *authoring* input (deltas and hand-edited specs), and richer organization lint, are future work — not built here, to avoid overscoping an engine PR into an authoring-experience PR. +**In scope (Decision 14): the spec formatter.** "How the spec looks, reads, and is organized" — a deterministic formatter (prettier-for-specs) with a `--check` gate — is now folded into this PR rather than deferred, because it is the same canonicalization the merge engine already performs, exposed for authoring. See Decision 14. -**Why this split.** Incremental checking is a pure consequence of the digest this PR already records — high value, near-zero added surface. The formatter and the artifact-model questions are larger, separable products; folding them in would blur a focused engine PR. Adopt the kernel, document the boundary. +**Still out of scope: the artifact-model change.** Reducing the committed artifact set (commit-only-the-spec) is a separate product question, served by schema flexibility, not this PR. + +## Decision 14 — The deterministic spec formatter (`openspec format`), in scope + +**Context.** The discussion wanted the framework's job to include "how the spec looks and reads and is organized/formatted" — a linter/formatter, hash-optimized. Earlier this was deferred (an authoring concern, seemingly separable). Reconsidered: the merge engine *already* computes a canonical form for every spec it writes (deterministic recomposition, newline normalization, canonical spacing). A standalone formatter is that same canonicalizer pointed at authoring input. The marginal surface is small and the conceptual fit is exact, so it belongs in this PR. + +**Decision.** Add `openspec format [target]` — a deterministic, pure-code formatter for spec and delta files — with `--check` (read-only gate) and `--fix`/default (write). It shares one canonicalizer with `sync`/`archive`, so **the merge engine's output is, by construction, exactly what the formatter produces**: synced/archived specs always pass `format --check`. It is incremental (Decision 13's digest skip) and `--json`-capable, and it joins `sync --check` as a model-free gate the CI job and pre-commit hook run. + +**Behavior-preserving — the hard line.** The formatter changes only *presentation*: whitespace, blank-line policy, list markers/indentation, heading spacing, canonical delta-section headers. It MUST NOT reorder requirements or scenarios (order can carry meaning), rewrite prose, or add/remove/merge/split requirements. The invariant is testable: parse-before == parse-after (same requirements, scenarios, and delta operations); only surrounding whitespace may differ. This is what separates a *formatter* (safe, deterministic, automatic) from a *rewrite* (semantic, requires review). + +**What it is NOT.** Not `validate`: `validate` judges *semantic* validity (a requirement has a scenario, headers resolve), `format` judges *canonical form* (is it laid out canonically). They compose — `format` then `validate`. Not the inference-based "align spec to implementation" the discussion also mentioned: checking that *code* matches the spec needs a model and is the `verify` direction ([#880](https://github.com/Fission-AI/OpenSpec/issues/880)), explicitly out of scope here. `format` is pure text canonicalization, so it needs no agent and no `/opsx:format` skill — you run the binary (or the hook runs it). + +**Why now, not later.** Folding it in means one canonicalizer is specified and tested once and reused by `sync`, `archive`, and `format`, guaranteeing they cannot diverge. Deferring it would risk a future formatter that disagrees with the merge engine's output — the exact drift this PR exists to kill. ## Risks / trade-offs diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md b/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md index bcb35d447c..faac213467 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md @@ -38,11 +38,13 @@ Design principle the whole thread converges on, and the one this project is alre 4. **NO MODEL IN THE MERGE — skills delegate to the CLI (`specs-sync-skill` MODIFIED; `opsx-unarchive-skill` NEW).** `/opsx:sync` stops doing agent-driven edits and **invokes `openspec sync`**; the new `/opsx:unarchive` invokes `openspec unarchive`. The deterministic work lives in TypeScript; skills only select, confirm, and render. This is the direction [#863](https://github.com/Fission-AI/OpenSpec/issues/863)/[#799](https://github.com/Fission-AI/OpenSpec/issues/799)/[#656](https://github.com/Fission-AI/OpenSpec/issues/656) ask for, and it removes the [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) "intelligent merge drops scenarios" failure mode by construction. +5. **THE FORMATTER — `openspec format [target]` (`cli-format` NEW).** The discussion's "the framework's goal is how the spec looks and reads and is organized" — delivered as a deterministic, pure-code **formatter** with a `--check` gate, sharing **one canonicalizer** with the merge engine. The engine already computes a canonical form for everything it writes; `format` points that same canonicalizer at authoring input (spec and delta files), so synced/archived specs are canonical *by construction* — they always pass `format --check`. It is **behavior-preserving** (normalizes whitespace, blank-line policy, list markers, heading spacing, canonical delta-section headers; never reorders requirements/scenarios, never rewrites prose — `parse-before == parse-after`), **incremental** (digest skip, Decision 13), and needs no agent (`format` is pure text, so there is no `/opsx:format` skill — you run the binary or the hook does). It joins `sync --check` as the second model-free gate. It is *not* `validate` (semantic validity) and *not* code-vs-spec `verify` (inference); those compose around it (design Decision 14). + ### What this deliberately does *not* change The discussion proposed removing `archive` entirely — keep every change in its dated archive location for its whole life and fold the spec merge into `apply`. **Rejected, with the maintainer's reasoning, and it shapes the design:** `specs/` only ever describes *shipped* reality. Folding the merge into `apply` would let proposed-but-unmerged (or abandoned) changes pollute the source of truth and let parallel changes step on each other's specs before any land. So the archive boundary stays; what changes is that the merge across it becomes deterministic and reversible. The determinism is what makes the boundary cheap to cross in both directions, which is the real fix for the "awkward final step." (Full treatment in [design.md](./design.md) Decision 5.) -The discussion also raised the deeper question — *are deltas just simulating, above git, what git already does?* — and the matching proposals to **edit `specs/` in place** (the git diff *is* the delta) with a sidecar reasoning log, and to **commit only the spec**, treating proposal/design/tasks as disposable personal workflow. **Also rejected, and likewise constructive:** the delta layer is precisely what a raw git diff cannot give — clean isolation of *proposed* from *shipped*, so `specs/` stays canonical while several in-flight changes remain independently reviewable until each lands (it is also what makes reversing a single change, i.e. `unarchive`, possible); and the planning artifacts *are* the reviewable, resumable product, not crumbs. But the sharpest kernels are valid, so we **take them**: editing-the-spec-first "forces conflict resolution now" → `sync --check` moves conflict detection early (Decision 11); "reasoning travels with the change, every edit accounted for" → provenance + correspondence, within the delta model (Decision 12); "use hashes to optimize when checks are needed" → incremental `--check` (Decision 13). The framework-as-spec-formatter wish is acknowledged and bounded (Decision 13): this PR makes *output* specs canonical and byte-stable, while a full authoring formatter stays a separable follow-up. +The discussion also raised the deeper question — *are deltas just simulating, above git, what git already does?* — and the matching proposals to **edit `specs/` in place** (the git diff *is* the delta) with a sidecar reasoning log, and to **commit only the spec**, treating proposal/design/tasks as disposable personal workflow. **Also rejected, and likewise constructive:** the delta layer is precisely what a raw git diff cannot give — clean isolation of *proposed* from *shipped*, so `specs/` stays canonical while several in-flight changes remain independently reviewable until each lands (it is also what makes reversing a single change, i.e. `unarchive`, possible); and the planning artifacts *are* the reviewable, resumable product, not crumbs. But the sharpest kernels are valid, so we **take them**: editing-the-spec-first "forces conflict resolution now" → `sync --check` moves conflict detection early (Decision 11); "reasoning travels with the change, every edit accounted for" → provenance + correspondence, within the delta model (Decision 12); "use hashes to optimize when checks are needed" → incremental `--check` (Decision 13); and the framework-as-spec-formatter wish → a deterministic `openspec format` sharing the engine's canonicalizer (Decision 14). Only the artifact-model change (commit-only-the-spec) stays out — it is a schema-flexibility question, not a merge-engine one. ## Capabilities @@ -51,6 +53,7 @@ The discussion also raised the deeper question — *are deltas just simulating, - `cli-sync`: the deterministic, idempotent spec-merge engine exposed as `openspec sync [change]` — `--fix`/default writes `specs/` from the change's deltas (byte-identical for the same delta + base; re-running is a no-op; a revised delta regenerates with no crumbs), `--check` is a read-only, model-free gate for CI and pre-commit hooks that also surfaces conflicts early (delta-vs-base, and two active changes targeting one requirement) and verifies delta↔spec correspondence. Records a per-change applied-delta baseline (pre-image + digest + provenance) that powers reverse, drift detection, and tracing each spec edit to its delta. The same engine `archive` uses internally. - `cli-unarchive`: `openspec unarchive [change-name]` — the deterministic inverse of archive. Resolves an archived change (prefix-tolerant, never auto-picking ambiguous matches), reverses the spec merge from the baseline under a drift guard, moves the folder back, atomically. `--keep-specs` restores the folder without touching `specs/`; pre-baseline archives degrade gracefully. - `opsx-unarchive-skill`: a `/opsx:unarchive` workflow skill that delegates to `openspec unarchive` for all deterministic work (selection, confirmation, rendering only). Expanded profile; no cross-skill dependency. +- `cli-format`: `openspec format [target]` — a deterministic, behavior-preserving spec/delta formatter sharing one canonicalizer with the merge engine, so synced/archived specs are canonical by construction. `--check` is a read-only, model-free gate (CI + pre-commit) that names unformatted files; `--fix`/default writes canonical form; incremental via digests; `--json`-capable. Pure text — no agent, no `/opsx:format` skill. ### Modified Capabilities @@ -59,19 +62,21 @@ The discussion also raised the deeper question — *are deltas just simulating, ### Integration surface (primitives in scope; wiring staged) -- **CI drift gate** and **pre-commit `sync --fix` hook** are the payoff of `cli-sync --check`/`--fix`. This PR delivers the CLI primitives and documents both patterns; wiring a specific hook runner (husky/lefthook — none exists in the repo today) or a CI job is a small, separable follow-up the owner can stage. +- **CI drift gate** and **pre-commit hook** are the payoff of `cli-sync --check`/`--fix` and `cli-format --check`/`--fix` (the hook can `format --fix` then `sync --fix`, leaving specs canonical *and* in sync). This PR delivers the CLI primitives and documents both patterns; wiring a specific hook runner (husky/lefthook — none exists in the repo today) or a CI job is a small, separable follow-up the owner can stage. ## Impact - `src/core/specs-apply.ts` — make `buildUpdatedSpec`/`applySpecs` byte-deterministic and idempotent; add the inverse (delta-inversion for ADDED/RENAMED; pre-image restore for all ops); add baseline read/write. Forward output unchanged for existing callers. +- `src/core/spec-canonical.ts` (**new**) — extract the deterministic spec/delta canonicalizer (the recomposition `buildUpdatedSpec` already performs at [specs-apply.ts:311-348](../../../src/core/specs-apply.ts)) into one shared module used by the merge engine *and* the formatter, so their output cannot diverge. Behavior-preserving: `parse(canonicalize(x)) == parse(x)`. - `src/core/sync.ts` (**new**) — `SyncCommand` (default/`--fix`/`--check`), mirroring `ArchiveCommand`'s human + `--json` shape; writes/refreshes the applied-delta baseline. +- `src/core/format.ts` (**new**) — `FormatCommand` (default/`--fix`/`--check`, `--json`, incremental): runs the shared canonicalizer over main specs and active-change delta files; `--check` lists non-canonical files and exits non-zero without writing. - `src/core/unarchive.ts` (**new**) — `UnarchiveCommand`: resolve archived dir, drift-check, restore pre-images (or delta-invert / refuse for pre-baseline), move folder back, atomic abort. Reuse `moveDirectory()`/`copyDirRecursive()` ([src/core/archive.ts:96-128](../../../src/core/archive.ts)). - `src/core/archive.ts` — route the merge through the shared engine (already deterministic) and persist the baseline before `moveDirectory` (~414-506). No behavior/output change. - `src/core/change-metadata/` or a sibling baseline store — persist the applied-delta baseline per change (pre-image + digest, newline-normalized, scheme-tagged). Coordinate with [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278)'s digest ledger so the two drift layers share a digest convention. - `src/core/list.ts` / `src/core/view.ts` — reuse `getArchivedChangesData()` ([#399](https://github.com/Fission-AI/OpenSpec/pull/399)) to resolve/disambiguate archived candidates. -- `src/cli/index.ts` — register `sync [change]` (`--check`, `--fix`, `--json`, `--store`) and `unarchive [change-name]` (`--keep-specs`/`--skip-specs` alias, `-y/--yes`, `--no-validate`, `--json`, `--store`), mirroring archive (326-343). +- `src/cli/index.ts` — register `sync [change]` (`--check`, `--fix`, `--json`, `--store`), `unarchive [change-name]` (`--keep-specs`/`--skip-specs` alias, `-y/--yes`, `--no-validate`, `--json`, `--store`), and `format [target]` (`--check`, `--fix`, `--json`), mirroring archive (326-343). - `src/core/templates/workflows/sync-specs.ts` — rewrite `/opsx:sync` to invoke `openspec sync` (drop agent-driven edits). `src/core/templates/workflows/unarchive-change.ts` (**new**) — `/opsx:unarchive` delegating to the CLI. Registration: export from `skill-templates.ts`; add `unarchive` to `ALL_WORKFLOWS` ([src/core/profiles.ts:19](../../../src/core/profiles.ts)) and `WORKFLOW_TO_SKILL_DIR` ([src/core/init.ts:65](../../../src/core/init.ts)); **not** added to `CORE_WORKFLOWS`. `docs/opsx.md` gains a `/opsx:unarchive` row and a determinism note for `/opsx:sync`. -- Tests — byte-identical sync determinism (same delta+base → same bytes; CRLF/LF; Windows), idempotency (re-run no-op; revised delta no crumbs), `--check` exit codes, archive→unarchive byte-exact round-trip, drift refusal, `--keep-specs`, destination-collision abort, pre-baseline degradation, atomic "no files changed", skill-template delegation snapshots. Per [openspec/config.yaml](../../config.yaml), run on Windows CI. +- Tests — byte-identical sync determinism (same delta+base → same bytes; CRLF/LF; Windows), idempotency (re-run no-op; revised delta no crumbs), `--check` exit codes, archive→unarchive byte-exact round-trip, drift refusal, `--keep-specs`, destination-collision abort, pre-baseline degradation, atomic "no files changed", skill-template delegation snapshots; formatter determinism + idempotency + `parse-before==parse-after` (behavior-preserving) + `sync`/`archive` output passes `format --check` (shared-canonicalizer invariant). Per [openspec/config.yaml](../../config.yaml), run on Windows CI. ## Issues addressed @@ -83,6 +88,7 @@ Delivers (the Discord conclusions): - **`openspec unarchive` / `/opsx:unarchive`** that "moves the folder back and reverses the spec merge," including the hard case where the archive is buried in a multi-file commit. - **Idempotent, crumb-free regeneration** and the **`sync --fix` pre-commit hook** *(the discussion's "sync is a lint step… eslint --fix" / "sync acts as the auto-fixer")*. - **Hash-tracked drift** *(the discussion's "track the hash of the changes/ state applied to the spec")* → the baseline digest. +- **Deterministic spec formatter/linter** *(the discussion's "the framework's goal is how the spec looks and reads and is organized", as a linter/formatter with hash-optimized checks)* → `openspec format` + `--check`, sharing the engine's canonicalizer. Directly fixes / strengthens: @@ -101,8 +107,8 @@ Delineated from adjacent work (coordinate, don't collide): Deferred follow-ups (enabled by this PR, not built here): - **Spec-aware git diff driver** — a diff driver for spec files that follows the recorded provenance to splice each change's rationale inline, so reviewers see *what changed and why together* (the discussion's idea). This PR ships the data (provenance) and the correspondence check; the diff driver is a separable follow-up (design Decision 12). -- **CI drift gate + pre-commit `sync --fix` hook** — the CLI primitives (`--check`/`--fix`) are in scope; wiring a hook runner (none exists in the repo) or a CI job is staged (design Decision 4). -- **Spec linter/formatter** — a "prettier-for-specs" formatter and richer organization lint (the discussion's "the framework's goal is how the spec looks and reads and is organized"). This PR makes *output* specs canonical and byte-stable; an authoring-side formatter is adjacent to `openspec-conventions`/`validate` and stays a separable follow-up (design Decision 13). +- **CI drift gate + pre-commit hook** — the CLI primitives (`sync --check`/`--fix`, `format --check`/`--fix`) are in scope; wiring a hook runner (none exists in the repo) or a CI job is staged (design Decision 4). +- **Spec-aware authoring lint beyond canonical form** — `format` delivers deterministic canonicalization; richer *semantic* organization advice (which capability a requirement belongs in, etc.) would need judgment and stays separate from the pure-code formatter (design Decision 14). Considered and rejected (documented in design): diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-format/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-format/spec.md new file mode 100644 index 0000000000..436b4997cf --- /dev/null +++ b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-format/spec.md @@ -0,0 +1,144 @@ +## ADDED Requirements + +### Requirement: Spec Format Command + +The system SHALL provide an `openspec format [target]` command that rewrites OpenSpec spec files and delta spec files to a single deterministic canonical form, in pure code without AI inference. By default it writes; `--check` runs read-only. + +#### Scenario: Format main and delta specs + +- **WHEN** the user runs `openspec format` with no target +- **THEN** the command formats the main specs under `openspec/specs/` and the delta specs under active changes' `specs/` directories +- **AND** it reports which files were reformatted + +#### Scenario: Format a specific target + +- **WHEN** the user runs `openspec format ` for a spec or delta file or directory +- **THEN** the command formats only that target + +#### Scenario: No inference + +- **WHEN** the command formats a file +- **THEN** it computes the canonical form in code +- **AND** it does not call a language model or otherwise depend on non-deterministic input + +### Requirement: Deterministic Canonical Form + +The formatter SHALL produce byte-identical output for the same input on every platform, and SHALL be idempotent: formatting already-canonical content changes nothing. + +#### Scenario: Same input yields identical output + +- **WHEN** the formatter runs more than once on the same input +- **THEN** the output bytes are identical every time + +#### Scenario: Idempotent + +- **WHEN** the formatter is applied to content it has already formatted +- **THEN** the content is unchanged + +#### Scenario: Line endings normalized + +- **WHEN** the input contains CRLF or mixed line endings +- **THEN** the canonical output uses normalized line endings regardless of the platform + +### Requirement: Behavior-Preserving Normalization + +The formatter SHALL change only presentation — whitespace, blank-line policy, list markers and indentation, and heading spacing — and SHALL NOT change the meaning of a spec. It SHALL NOT reorder requirements or scenarios, rewrite prose, or add, remove, merge, or split requirements or scenarios. + +#### Scenario: Requirement and scenario order preserved + +- **WHEN** the formatter runs on a spec +- **THEN** the order of requirements and of scenarios within each requirement is unchanged + +#### Scenario: Prose is not rewritten + +- **WHEN** the formatter normalizes a requirement +- **THEN** the requirement's wording and scenario text are byte-for-byte unchanged except for surrounding whitespace normalization + +#### Scenario: Parsed content is identical before and after + +- **WHEN** a spec is parsed before formatting and after formatting +- **THEN** the parsed requirements, scenarios, and delta operations are identical + +### Requirement: Canonical Section Organization + +The formatter SHALL normalize the structural presentation of a spec deterministically — heading levels and nesting, the spacing between sections, and the canonical headers for delta sections — without changing which sections are present or their order. + +#### Scenario: Canonical headings and spacing + +- **WHEN** a spec uses inconsistent heading spacing or blank-line separation between requirements and scenarios +- **THEN** the formatter rewrites them to the canonical spacing defined by the conventions + +#### Scenario: Canonical delta section headers + +- **WHEN** a delta file contains `## ADDED/MODIFIED/REMOVED/RENAMED Requirements` sections +- **THEN** the formatter normalizes those headers to their canonical form +- **AND** it does not move requirements between sections + +### Requirement: Shared Canonicalization With The Merge Engine + +The canonical form produced by `openspec format` SHALL be the same canonical form emitted by the deterministic merge engine used by `openspec sync` and `openspec archive`, so that synced or archived specs are already canonical. + +#### Scenario: Merge output is already formatted + +- **WHEN** `openspec sync` or `openspec archive` writes a spec +- **THEN** running `openspec format --check` on that spec passes without changes + +#### Scenario: One canonicalizer + +- **WHEN** the same spec content is produced by the formatter and by the merge engine +- **THEN** the two results are byte-identical + +### Requirement: Check Mode + +The format command SHALL support a read-only `--check` mode that exits non-zero when any target is not in canonical form, naming the offending files and modifying nothing, so it can gate commits and CI as a plain binary. + +#### Scenario: Unformatted file detected + +- **WHEN** `openspec format --check` finds a file that is not in canonical form +- **THEN** the command reports the file +- **AND** it exits with a non-zero status code and modifies no files + +#### Scenario: All formatted + +- **WHEN** every target is already in canonical form +- **THEN** the command exits zero and modifies no files + +### Requirement: Fix Mode + +The format command SHALL, by default (or with `--fix`), rewrite targets to canonical form, suitable for use as an auto-fixer in a pre-commit hook. + +#### Scenario: Fix writes canonical form + +- **WHEN** the user runs `openspec format` (or `openspec format --fix`) +- **THEN** the command writes each target's canonical form +- **AND** a subsequent `openspec format --check` passes + +### Requirement: Incremental Checking + +The format check MAY use recorded content digests to skip files whose content is unchanged since they were last checked, re-checking only what changed. A skip SHALL be permitted only when it cannot change the result versus a full check. + +#### Scenario: Unchanged file skipped + +- **WHEN** `--check` runs and a file's current content digest matches the recorded digest +- **THEN** the command may skip re-checking that file +- **AND** the overall result is identical to checking it fully + +#### Scenario: Changed or unknown file fully checked + +- **WHEN** a file's digest does not match, no digest is recorded, or the recorded digest uses an unrecognized scheme +- **THEN** the command performs the full check for that file + +### Requirement: JSON Output + +The format command SHALL support `--json` for non-interactive use, emitting machine-readable results and diagnostics. + +#### Scenario: JSON reports unformatted files + +- **WHEN** `openspec format --check --json` finds files not in canonical form +- **THEN** it emits the list of offending files as JSON +- **AND** exits with a non-zero status code + +#### Scenario: JSON reports written files + +- **WHEN** `openspec format --json` rewrites files +- **THEN** it emits the list of changed files as JSON diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md b/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md index 60707d912e..54cc1553d2 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md @@ -1,10 +1,11 @@ -# Tasks: deterministic spec-merge engine — `sync` (forward) + `unarchive` (reverse) +# Tasks: deterministic spec tooling — `sync` (forward) + `unarchive` (reverse) + `format` (canonical) -> Phased so the keystone ships first and each phase is independently testable and shippable. Phase 1 (engine + baseline) and Phase 2 (`sync` + `--check`) deliver the deterministic, model-free drift path on their own; Phase 3 (`unarchive`) and Phase 4 (idempotency) build on the same baseline; Phase 5 (skills) removes the model from the merge; Phase 6 (integration) is the staged hook/CI follow-up. Cross-platform concerns (`path.join`, Windows `moveDirectory`, newline-normalized digests) are called out per [openspec/config.yaml](../../config.yaml). +> Phased so the keystone ships first and each phase is independently testable and shippable. Phase 1 (engine + baseline + shared canonicalizer) and Phase 2 (`sync` + `--check`) deliver the deterministic, model-free drift path; Phase 2A (`format`) reuses the canonicalizer from Phase 1; Phase 3 (`unarchive`) and Phase 4 (idempotency) build on the same baseline; Phase 5 (skills) removes the model from the merge; Phase 6 (integration) is the staged hook/CI follow-up. Cross-platform concerns (`path.join`, Windows `moveDirectory`, newline-normalized digests) are called out per [openspec/config.yaml](../../config.yaml). ## 1. The engine: deterministic, byte-stable merge + applied-delta baseline - [ ] 1.1 Make the merge byte-deterministic: audit `buildUpdatedSpec` ([src/core/specs-apply.ts:240-307](../../../src/core/specs-apply.ts)) for any nondeterministic ordering/whitespace; guarantee stable requirement ordering and newline normalization so the same (delta + base) yields byte-identical output on macOS/Linux/Windows. +- [ ] 1.1a Extract the canonicalizer: factor the recomposition/normalization `buildUpdatedSpec` performs ([specs-apply.ts:311-348](../../../src/core/specs-apply.ts)) into a shared `src/core/spec-canonical.ts` used by the merge engine *and* the formatter, so their output cannot diverge. Cover spec files and delta files (ADDED/MODIFIED/REMOVED/RENAMED sections). Behavior-preserving: assert `parse(canonicalize(x)) == parse(x)`. - [ ] 1.2 Define the **applied-delta baseline** format (per design Decision 1): per affected spec, the pre-merge content (or `absent` marker) plus a scheme-tagged, newline-normalized digest of the applied result. Store it with the change (e.g. `.openspec/merge-baseline/`); document the location. - [ ] 1.3 Add baseline read/write helpers (safe read-modify-write, preserving unrelated fields), coordinating the digest convention with [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278)'s ledger. - [ ] 1.4 Add the inverse merge in `specs-apply.ts`: delta-inversion for ADDED (remove by name) and RENAMED (rename TO→FROM, rewriting the header line); pre-image restore for all ops when a baseline exists. No change to forward behavior for existing callers. @@ -23,6 +24,15 @@ - [ ] 2.9 Incremental checking (design Decision 13): `--check` skips a spec whose current digest matches the recorded baseline; re-checks on mismatch; forces a full check when the baseline is missing or its scheme is unrecognized. Add an escape hatch to disable skips (e.g. `--no-incremental`) for the equivalence test. - [ ] 2.10 Tests: unchanged spec skipped; changed spec re-checked; missing/unknown-scheme baseline → full check; **incremental verdict == full verdict** on the same input (the correctness invariant); a touched spec in a large fixture is the only one re-checked. +## 2A. `openspec format` — the formatter (reuses the Phase 1 canonicalizer) + +- [ ] 2A.1 Create `src/core/format.ts` `FormatCommand` (human + `--json`); default/`--fix` rewrites targets via `spec-canonical.ts`; `--check` is read-only. +- [ ] 2A.2 Target resolution: no target → all main specs + active-change delta files; explicit path → just that file/dir. Handle both spec and delta formats. +- [ ] 2A.3 `--check`: list non-canonical files, exit non-zero, write nothing; default/`--fix`: write canonical form. +- [ ] 2A.4 Register `format [target]` in [src/cli/index.ts](../../../src/cli/index.ts) (`--check`, `--fix`, `--json`), mirroring archive's shape. No skill (pure code). +- [ ] 2A.5 Incremental: reuse the digest skip (Decision 13) so `--check` only re-reads changed files; skip never changes the verdict. +- [ ] 2A.6 Tests: determinism (same input → same bytes; CRLF/LF; Windows); idempotency (`format(format(x))==format(x)`); **behavior-preserving** (`parse-before==parse-after`; requirement/scenario order and prose unchanged); **shared-canonicalizer invariant** — `sync`/`archive` output passes `format --check`; `--check` exit codes; `--json` shape. + ## 3. `openspec unarchive` (reverse, built on the baseline) - [ ] 3.1 Create `src/core/unarchive.ts` `UnarchiveCommand` (human + `--json`, blocked-error → diagnostic). @@ -57,7 +67,7 @@ ## 7. Docs & supersession -- [ ] 7.1 `docs/opsx.md`: add `/opsx:unarchive`; update `/opsx:sync` to note CLI delegation + determinism; document `openspec sync` and `--check`/`--fix`. +- [ ] 7.1 `docs/opsx.md` + CLI docs: add `/opsx:unarchive`; update `/opsx:sync` to note CLI delegation + determinism; document `openspec sync` and `openspec format` with `--check`/`--fix` (and that `format` needs no agent/skill). - [ ] 7.2 Note the `specs/` = shipped-reality invariant and why `archive` is retained (design Decision 5), so the "why not just remove archive" question has a documented answer. ## 8. End-to-end verification @@ -66,5 +76,6 @@ - [ ] 8.2 E2E round-trip: `archive` then `unarchive` → `specs/` byte-identical to pre-archive, folder back under `changes//`. - [ ] 8.3 E2E stacked drift: change A MODIFIES req X and is archived; a second change re-MODIFIES X and is archived; `unarchive A` refuses spec reversal; `unarchive A --keep-specs` succeeds untouched. - [ ] 8.4 E2E no-crumbs: sync, revise the delta, re-sync → `specs/` reflects only the current delta. +- [ ] 8.4a E2E formatter: mangle a spec's whitespace → `format --check` fails → `format` fixes it → `--check` passes; and `sync`/`archive` output passes `format --check` unchanged (shared-canonicalizer invariant). - [ ] 8.5 Validation: `openspec validate add-deterministic-sync-and-unarchive --strict` passes; `openspec status` shows artifacts complete. - [ ] 8.6 Run the suite on macOS, Linux, and Windows CI. From 9dcc2265f58b0e32aa77072e3f635e392c104ecf Mon Sep 17 00:00:00 2001 From: Clay Good Date: Mon, 29 Jun 2026 17:14:28 -0500 Subject: [PATCH 6/8] docs(openspec): fold in the spec-aware diff driver (openspec diff); scope-complete MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Promote the spec-aware diff driver from a deferred follow-up to a first-class capability, completing the discussion's "review = read spec deltas + reasoning inline" vision deterministically. New capability cli-diff: `openspec diff [target] [--base ]` — a deterministic, pure-code renderer that splices each changed requirement's provenance and rationale inline (what changed and why, together). Usable standalone or, opt-in, as a git diff driver via .gitattributes (never alters git config without consent). Key insight: OpenSpec ALREADY has the "log" — the why lives in proposal.md, the what/where is the applied-delta provenance this PR records — so diff joins existing artifacts rather than building the sidecar reasoning database the discussion reached for. No inference; honest about unattributable changes; not semantic review (that's the verify direction). New design Decision 15 (+ Decision 12 deferral removed) and a "Scope completeness" note: the proposal now covers every deterministic, in-scope idea from the discussion, so later sessions should shift from expansion to sequencing/refinement for owner review. New tasks phase 2B. PR now 7 capabilities. Validates with `openspec validate --strict`. No human names in PR content. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../design.md | 23 ++++- .../proposal.md | 17 ++-- .../specs/cli-diff/spec.md | 86 +++++++++++++++++++ .../tasks.md | 17 +++- 4 files changed, 132 insertions(+), 11 deletions(-) create mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-diff/spec.md diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/design.md b/openspec/changes/add-deterministic-sync-and-unarchive/design.md index 3a906d6402..898cc63598 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/design.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/design.md @@ -1,4 +1,4 @@ -# Design: deterministic spec tooling — `sync` (forward), `unarchive` (reverse), `format` (canonical) +# Design: deterministic spec tooling — `sync` (forward), `unarchive` (reverse), `format` (canonical), `diff` (review) ## Context @@ -17,6 +17,7 @@ This design covers that primitive and the commands built on it. It supersedes th - Per-edit provenance and a delta↔spec correspondence check, within the delta model (Decision 12). - Hash-optimized incremental checking: `--check` re-checks only specs whose digest changed, without changing any verdict (Decision 13). - A deterministic, behavior-preserving spec formatter (`openspec format` + `--check`) sharing one canonicalizer with the merge engine (Decision 14). +- A deterministic spec-aware diff (`openspec diff`, opt-in git diff driver) that splices provenance + rationale inline, reusing existing artifacts (Decision 15). - Skills delegate the merge to the CLI; the agent never performs it. - Backward compatible with changes archived (and specs synced) before this feature. @@ -26,6 +27,8 @@ This design covers that primitive and the commands built on it. It supersedes th - Building the spec-aware git diff driver (Decision 12 — provenance data is in scope; the diff driver is a deferred follow-up). - Inference-based "align spec to implementation" / code-vs-spec checking — that needs a model and is the `verify` direction (#880), not the deterministic `format` (Decision 14). - Reducing the committed artifact set (commit-only-the-spec) — a separate product question served by schema flexibility (Decision 13). +- A separate "semantic delta" / reasoning-log database — unnecessary; `openspec diff` renders from the existing proposal + provenance (Decision 15). +- Semantic/AI summarization in the diff — `diff` mechanically splices recorded reasoning; model-based review is the `verify` direction (Decision 15). - Wiring a specific hook runner or CI job into this repo (primitives in scope; config staged — Decision 4). - A scenario-granular "smart" merge (Decision 8 — the conventions mandate whole-requirement deltas; whole-block apply is the correct, deterministic semantics). - Bulk `sync`/`unarchive`, archive-folder prefix scheme, lifecycle timestamps — out of scope, accommodated. @@ -119,7 +122,7 @@ Building three mechanisms would invite three drift bugs. Building one — the pr **Decision.** Record **provenance** as part of the applied-delta baseline: when the engine writes a spec change, it records which change and which delta operation produced it. Expose it (e.g. `openspec sync --explain` / a provenance entry). Use it for a deterministic **delta↔spec correspondence** check in `sync --check`: every committed `specs/` edit for a change must trace to one of its delta operations (no orphan edits), and every delta operation must have landed (no unapplied deltas). The prose "why" is not re-authored — provenance links each spec edit to its change, whose `proposal.md` already holds the rationale. -**Deferred (delineated follow-up): the spec-aware git diff driver.** A `git diff` driver for spec files that follows the provenance link and splices the change's rationale inline is a genuinely useful, separable feature. This PR delivers the **data** (provenance in the baseline) and the **check** (correspondence); the **presentation** (diff driver, custom textconv/diff attributes) is a follow-up — the same primitives-in-scope / wiring-staged split as the CI hook (Decision 4). Building a diff driver into a sync/unarchive PR would overscope it. +**In scope (Decision 15): the spec-aware diff driver.** The presentation side — a `git diff` driver for spec files that follows the provenance link and splices the change's rationale inline — is now folded in rather than deferred, because it consumes exactly the provenance this PR already records and needs no new data store. See Decision 15. **Why this is the right slice.** Provenance falls out of the merge for free (the engine already knows exactly what it applied), and correspondence reuses the baseline. Together they answer the discussion's consistency lint *within the delta model* (deltas are the source; specs are generated) rather than inverting it (specs as source, deltas as sidecar log) — which would reintroduce the edit-in-place problems of Decision 11. @@ -149,6 +152,22 @@ Building three mechanisms would invite three drift bugs. Building one — the pr **Why now, not later.** Folding it in means one canonicalizer is specified and tested once and reused by `sync`, `archive`, and `format`, guaranteeing they cannot diverge. Deferring it would risk a future formatter that disagrees with the merge engine's output — the exact drift this PR exists to kill. +## Decision 15 — The spec-aware diff driver (`openspec diff`), in scope + +**Context.** The discussion's review workflow: "review primarily becomes reading the spec deltas (in git)" with "a spec-aware diff driver that splices the reasoning log inline so reviewers see what changed and why together," and the related idea of a sidecar database of "semantic deltas as log entries." + +**Decision.** Add `openspec diff [target] [--base ]` — a deterministic, pure-code renderer that shows requirement-level spec changes with each change's provenance and rationale spliced inline. It is usable standalone and, opt-in, as a **git diff driver** (registered in `.gitattributes`) so `git diff` over spec/delta paths renders the annotated view. + +**Key insight — OpenSpec already has the "log"; no new store.** The discussion reached for a separate database of semantic deltas plus a reasoning log. But OpenSpec already keeps both halves: the *why* lives in the change's `proposal.md`, and the *what/where* lives in the applied-delta provenance this PR records. The diff driver simply joins them. So we explicitly **reject building a sidecar reasoning database** (Decision 11's edit-in-place family) and instead render from artifacts that already exist — less to maintain, nothing to keep in sync. + +**Deterministic and honest.** Rendering is a pure function of the git diff + recorded provenance + proposal text — byte-identical across runs and platforms, no inference. When a change cannot be attributed (a pre-baseline edit with no provenance), the diff says so rather than inventing a rationale. It does **not** judge review quality or summarize semantically (that would need a model); it mechanically splices recorded reasoning. Git config is never modified without explicit opt-in. + +**Why in scope now.** It is the presentation layer of the provenance this PR already produces, and it completes the discussion's review vision deterministically. Like the formatter (Decision 14), it reuses primitives this PR establishes rather than introducing new ones. + +## Scope completeness + +With `diff` folded in, this proposal now covers **every deterministic, in-scope idea** the discussion raised: the merge engine and applied-delta baseline (forward `sync`, reverse `unarchive`, the `archive` baseline), idempotent crumb-free re-merge, model-free `--check` drift gating, early conflict detection, provenance + delta↔spec correspondence, incremental checking, the canonical formatter, and the spec-aware diff driver. The ideas it deliberately leaves out are documented with reasons: editing specs in place / dropping the delta layer / committing only the spec (Decisions 5, 11, 13), inference-based code-vs-spec verification (Decision 14, the `verify` direction), and the *wiring* of a specific hook runner or CI job (Decision 4, a repo-policy choice). Subsequent sessions should therefore shift from **expansion** to **sequencing and refinement** for owner review — confirming the open decisions (e.g. `--keep-specs` vs `--skip-specs`, the `/opsx:sync` behavior change) and the phase order — rather than adding surface. + ## Risks / trade-offs - **Behavior shift for `/opsx:sync` users.** Replacing agent merge with deterministic merge changes output for anyone who relied on the agent's scenario-level merges. This is intended (it fixes [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)), but it is a real change; call it out in the changeset and docs, and keep the conventions' "complete requirement, not a diff" rule prominent so deltas are authored to merge cleanly. diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md b/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md index faac213467..e00dd24c45 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md @@ -40,11 +40,13 @@ Design principle the whole thread converges on, and the one this project is alre 5. **THE FORMATTER — `openspec format [target]` (`cli-format` NEW).** The discussion's "the framework's goal is how the spec looks and reads and is organized" — delivered as a deterministic, pure-code **formatter** with a `--check` gate, sharing **one canonicalizer** with the merge engine. The engine already computes a canonical form for everything it writes; `format` points that same canonicalizer at authoring input (spec and delta files), so synced/archived specs are canonical *by construction* — they always pass `format --check`. It is **behavior-preserving** (normalizes whitespace, blank-line policy, list markers, heading spacing, canonical delta-section headers; never reorders requirements/scenarios, never rewrites prose — `parse-before == parse-after`), **incremental** (digest skip, Decision 13), and needs no agent (`format` is pure text, so there is no `/opsx:format` skill — you run the binary or the hook does). It joins `sync --check` as the second model-free gate. It is *not* `validate` (semantic validity) and *not* code-vs-spec `verify` (inference); those compose around it (design Decision 14). +6. **THE REVIEW VIEW — `openspec diff [target]` (`cli-diff` NEW).** The discussion's "review primarily becomes reading the spec deltas, with a spec-aware diff driver that splices the reasoning log inline so reviewers see what changed and why together." Delivered as a deterministic, pure-code renderer that shows requirement-level changes with each change's **provenance and rationale spliced inline**, usable standalone or (opt-in) as a **git diff driver** via `.gitattributes`. The key move: OpenSpec **already has the "log"** — the *why* is in `proposal.md`, the *what/where* is the applied-delta provenance — so `diff` joins existing artifacts rather than building the sidecar reasoning database the discussion reached for. It is deterministic and honest (no inference; says so when a change is unattributable), never modifies git config without opt-in, and does not summarize semantically (that is the `verify` direction). (design Decision 15.) + ### What this deliberately does *not* change The discussion proposed removing `archive` entirely — keep every change in its dated archive location for its whole life and fold the spec merge into `apply`. **Rejected, with the maintainer's reasoning, and it shapes the design:** `specs/` only ever describes *shipped* reality. Folding the merge into `apply` would let proposed-but-unmerged (or abandoned) changes pollute the source of truth and let parallel changes step on each other's specs before any land. So the archive boundary stays; what changes is that the merge across it becomes deterministic and reversible. The determinism is what makes the boundary cheap to cross in both directions, which is the real fix for the "awkward final step." (Full treatment in [design.md](./design.md) Decision 5.) -The discussion also raised the deeper question — *are deltas just simulating, above git, what git already does?* — and the matching proposals to **edit `specs/` in place** (the git diff *is* the delta) with a sidecar reasoning log, and to **commit only the spec**, treating proposal/design/tasks as disposable personal workflow. **Also rejected, and likewise constructive:** the delta layer is precisely what a raw git diff cannot give — clean isolation of *proposed* from *shipped*, so `specs/` stays canonical while several in-flight changes remain independently reviewable until each lands (it is also what makes reversing a single change, i.e. `unarchive`, possible); and the planning artifacts *are* the reviewable, resumable product, not crumbs. But the sharpest kernels are valid, so we **take them**: editing-the-spec-first "forces conflict resolution now" → `sync --check` moves conflict detection early (Decision 11); "reasoning travels with the change, every edit accounted for" → provenance + correspondence, within the delta model (Decision 12); "use hashes to optimize when checks are needed" → incremental `--check` (Decision 13); and the framework-as-spec-formatter wish → a deterministic `openspec format` sharing the engine's canonicalizer (Decision 14). Only the artifact-model change (commit-only-the-spec) stays out — it is a schema-flexibility question, not a merge-engine one. +The discussion also raised the deeper question — *are deltas just simulating, above git, what git already does?* — and the matching proposals to **edit `specs/` in place** (the git diff *is* the delta) with a sidecar reasoning log, and to **commit only the spec**, treating proposal/design/tasks as disposable personal workflow. **Also rejected, and likewise constructive:** the delta layer is precisely what a raw git diff cannot give — clean isolation of *proposed* from *shipped*, so `specs/` stays canonical while several in-flight changes remain independently reviewable until each lands (it is also what makes reversing a single change, i.e. `unarchive`, possible); and the planning artifacts *are* the reviewable, resumable product, not crumbs. But the sharpest kernels are valid, so we **take them**: editing-the-spec-first "forces conflict resolution now" → `sync --check` moves conflict detection early (Decision 11); "reasoning travels with the change, every edit accounted for" → provenance + correspondence, within the delta model (Decision 12); "use hashes to optimize when checks are needed" → incremental `--check` (Decision 13); the framework-as-spec-formatter wish → a deterministic `openspec format` sharing the engine's canonicalizer (Decision 14); and "review = read spec deltas + reasoning inline" → `openspec diff`, which renders from the proposal + provenance OpenSpec already keeps, so no sidecar reasoning database is needed (Decision 15). Only the artifact-model change (commit-only-the-spec) stays out — it is a schema-flexibility question, not a merge-engine one. ## Capabilities @@ -54,6 +56,7 @@ The discussion also raised the deeper question — *are deltas just simulating, - `cli-unarchive`: `openspec unarchive [change-name]` — the deterministic inverse of archive. Resolves an archived change (prefix-tolerant, never auto-picking ambiguous matches), reverses the spec merge from the baseline under a drift guard, moves the folder back, atomically. `--keep-specs` restores the folder without touching `specs/`; pre-baseline archives degrade gracefully. - `opsx-unarchive-skill`: a `/opsx:unarchive` workflow skill that delegates to `openspec unarchive` for all deterministic work (selection, confirmation, rendering only). Expanded profile; no cross-skill dependency. - `cli-format`: `openspec format [target]` — a deterministic, behavior-preserving spec/delta formatter sharing one canonicalizer with the merge engine, so synced/archived specs are canonical by construction. `--check` is a read-only, model-free gate (CI + pre-commit) that names unformatted files; `--fix`/default writes canonical form; incremental via digests; `--json`-capable. Pure text — no agent, no `/opsx:format` skill. +- `cli-diff`: `openspec diff [target] [--base ]` — a deterministic, spec-aware diff that splices each changed requirement's provenance and rationale inline (what changed and why, together), usable standalone or as an opt-in git diff driver. Renders from the change's `proposal.md` + the recorded applied-delta provenance — no new reasoning store; no inference; `--json`-capable. ### Modified Capabilities @@ -70,13 +73,14 @@ The discussion also raised the deeper question — *are deltas just simulating, - `src/core/spec-canonical.ts` (**new**) — extract the deterministic spec/delta canonicalizer (the recomposition `buildUpdatedSpec` already performs at [specs-apply.ts:311-348](../../../src/core/specs-apply.ts)) into one shared module used by the merge engine *and* the formatter, so their output cannot diverge. Behavior-preserving: `parse(canonicalize(x)) == parse(x)`. - `src/core/sync.ts` (**new**) — `SyncCommand` (default/`--fix`/`--check`), mirroring `ArchiveCommand`'s human + `--json` shape; writes/refreshes the applied-delta baseline. - `src/core/format.ts` (**new**) — `FormatCommand` (default/`--fix`/`--check`, `--json`, incremental): runs the shared canonicalizer over main specs and active-change delta files; `--check` lists non-canonical files and exits non-zero without writing. +- `src/core/diff.ts` (**new**) — `DiffCommand` (`--base`, `--json`): renders requirement-level spec/delta changes annotated with provenance (from the baseline) and rationale (from the originating change's `proposal.md`); pure code; also invocable as a git diff driver. Plus a documented `.gitattributes` snippet for opt-in registration. - `src/core/unarchive.ts` (**new**) — `UnarchiveCommand`: resolve archived dir, drift-check, restore pre-images (or delta-invert / refuse for pre-baseline), move folder back, atomic abort. Reuse `moveDirectory()`/`copyDirRecursive()` ([src/core/archive.ts:96-128](../../../src/core/archive.ts)). - `src/core/archive.ts` — route the merge through the shared engine (already deterministic) and persist the baseline before `moveDirectory` (~414-506). No behavior/output change. - `src/core/change-metadata/` or a sibling baseline store — persist the applied-delta baseline per change (pre-image + digest, newline-normalized, scheme-tagged). Coordinate with [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278)'s digest ledger so the two drift layers share a digest convention. - `src/core/list.ts` / `src/core/view.ts` — reuse `getArchivedChangesData()` ([#399](https://github.com/Fission-AI/OpenSpec/pull/399)) to resolve/disambiguate archived candidates. -- `src/cli/index.ts` — register `sync [change]` (`--check`, `--fix`, `--json`, `--store`), `unarchive [change-name]` (`--keep-specs`/`--skip-specs` alias, `-y/--yes`, `--no-validate`, `--json`, `--store`), and `format [target]` (`--check`, `--fix`, `--json`), mirroring archive (326-343). +- `src/cli/index.ts` — register `sync [change]` (`--check`, `--fix`, `--json`, `--store`), `unarchive [change-name]` (`--keep-specs`/`--skip-specs` alias, `-y/--yes`, `--no-validate`, `--json`, `--store`), `format [target]` (`--check`, `--fix`, `--json`), and `diff [target]` (`--base`, `--json`), mirroring archive (326-343). - `src/core/templates/workflows/sync-specs.ts` — rewrite `/opsx:sync` to invoke `openspec sync` (drop agent-driven edits). `src/core/templates/workflows/unarchive-change.ts` (**new**) — `/opsx:unarchive` delegating to the CLI. Registration: export from `skill-templates.ts`; add `unarchive` to `ALL_WORKFLOWS` ([src/core/profiles.ts:19](../../../src/core/profiles.ts)) and `WORKFLOW_TO_SKILL_DIR` ([src/core/init.ts:65](../../../src/core/init.ts)); **not** added to `CORE_WORKFLOWS`. `docs/opsx.md` gains a `/opsx:unarchive` row and a determinism note for `/opsx:sync`. -- Tests — byte-identical sync determinism (same delta+base → same bytes; CRLF/LF; Windows), idempotency (re-run no-op; revised delta no crumbs), `--check` exit codes, archive→unarchive byte-exact round-trip, drift refusal, `--keep-specs`, destination-collision abort, pre-baseline degradation, atomic "no files changed", skill-template delegation snapshots; formatter determinism + idempotency + `parse-before==parse-after` (behavior-preserving) + `sync`/`archive` output passes `format --check` (shared-canonicalizer invariant). Per [openspec/config.yaml](../../config.yaml), run on Windows CI. +- Tests — byte-identical sync determinism (same delta+base → same bytes; CRLF/LF; Windows), idempotency (re-run no-op; revised delta no crumbs), `--check` exit codes, archive→unarchive byte-exact round-trip, drift refusal, `--keep-specs`, destination-collision abort, pre-baseline degradation, atomic "no files changed", skill-template delegation snapshots; formatter determinism + idempotency + `parse-before==parse-after` (behavior-preserving) + `sync`/`archive` output passes `format --check` (shared-canonicalizer invariant); diff determinism + provenance/rationale annotation + honest "unattributable" handling + opt-in-only git config. Per [openspec/config.yaml](../../config.yaml), run on Windows CI. ## Issues addressed @@ -89,6 +93,7 @@ Delivers (the Discord conclusions): - **Idempotent, crumb-free regeneration** and the **`sync --fix` pre-commit hook** *(the discussion's "sync is a lint step… eslint --fix" / "sync acts as the auto-fixer")*. - **Hash-tracked drift** *(the discussion's "track the hash of the changes/ state applied to the spec")* → the baseline digest. - **Deterministic spec formatter/linter** *(the discussion's "the framework's goal is how the spec looks and reads and is organized", as a linter/formatter with hash-optimized checks)* → `openspec format` + `--check`, sharing the engine's canonicalizer. +- **Spec-aware diff with reasoning inline** *(the discussion's "review primarily becomes reading the spec deltas" + "a spec-aware diff driver that splices the reasoning log inline")* → `openspec diff` (opt-in git diff driver), rendering from existing proposal + provenance, so no sidecar reasoning database is needed. Directly fixes / strengthens: @@ -106,8 +111,10 @@ Delineated from adjacent work (coordinate, don't collide): Deferred follow-ups (enabled by this PR, not built here): -- **Spec-aware git diff driver** — a diff driver for spec files that follows the recorded provenance to splice each change's rationale inline, so reviewers see *what changed and why together* (the discussion's idea). This PR ships the data (provenance) and the correspondence check; the diff driver is a separable follow-up (design Decision 12). -- **CI drift gate + pre-commit hook** — the CLI primitives (`sync --check`/`--fix`, `format --check`/`--fix`) are in scope; wiring a hook runner (none exists in the repo) or a CI job is staged (design Decision 4). +- **CI drift gate + pre-commit hook wiring** — the CLI primitives (`sync --check`/`--fix`, `format --check`/`--fix`) are in scope; wiring a hook runner (none exists in the repo) or a CI workflow is a repo-policy choice left staged (design Decision 4). +- **Inference-based review/verification** — semantic review or code-vs-spec checking needs a model; that is the `verify` direction ([#880](https://github.com/Fission-AI/OpenSpec/issues/880)), distinct from the deterministic `diff`/`format` shipped here (design Decisions 14–15). + +> **Scope note:** with `diff` and `format` folded in, this proposal now covers every deterministic, in-scope idea from the discussion (see design "Scope completeness"). Subsequent sessions should shift from expansion to sequencing and confirming the open decisions for owner review. - **Spec-aware authoring lint beyond canonical form** — `format` delivers deterministic canonicalization; richer *semantic* organization advice (which capability a requirement belongs in, etc.) would need judgment and stays separate from the pure-code formatter (design Decision 14). Considered and rejected (documented in design): diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-diff/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-diff/spec.md new file mode 100644 index 0000000000..67e1586a4c --- /dev/null +++ b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-diff/spec.md @@ -0,0 +1,86 @@ +## ADDED Requirements + +### Requirement: Spec Diff Command + +The system SHALL provide an `openspec diff [target] [--base ]` command that renders a deterministic, spec-aware diff of spec and delta files, splicing each changed requirement's provenance and rationale inline so a reviewer sees what changed and why together. It SHALL compute the rendering in pure code, without AI inference. + +#### Scenario: Diff a change's deltas + +- **WHEN** the user runs `openspec diff ` +- **THEN** the command shows the change's delta operations grouped by capability and requirement +- **AND** annotates each with the rationale drawn from the change's `proposal.md` + +#### Scenario: Diff main specs against a base revision + +- **WHEN** the user runs `openspec diff --base ` over `openspec/specs/` +- **THEN** the command shows the requirement-level differences since `` +- **AND** annotates each changed requirement with the change and delta operation that produced it, drawn from the recorded applied-delta provenance + +#### Scenario: No inference + +- **WHEN** the command renders a diff +- **THEN** it composes the result from the git diff and the recorded provenance/rationale in code +- **AND** it does not call a language model + +### Requirement: Inline Reasoning Annotation + +The diff SHALL annotate each changed requirement with the originating change and its rationale, sourced from existing OpenSpec artifacts, and SHALL NOT invent rationale that is not recorded. + +#### Scenario: Annotated with originating change and rationale + +- **WHEN** a changed requirement can be attributed to a change via provenance +- **THEN** the diff shows the originating change and a reference to or excerpt of its recorded rationale + +#### Scenario: Unattributable change shown honestly + +- **WHEN** a changed requirement cannot be attributed (no provenance recorded, e.g. a pre-baseline edit) +- **THEN** the diff shows the change without inventing a rationale +- **AND** it indicates that provenance is unavailable + +### Requirement: Reuses Existing Artifacts, No New Sidecar Store + +The rationale and provenance the diff splices SHALL come from artifacts OpenSpec already maintains — the change's `proposal.md` (the why) and the recorded applied-delta provenance (the what/where) — rather than a separate reasoning database. + +#### Scenario: Reasoning resolved from existing artifacts + +- **WHEN** the diff needs the reasoning for a changed requirement +- **THEN** it resolves the rationale from the originating change's `proposal.md` and the recorded provenance +- **AND** it requires no separate reasoning-log store + +### Requirement: Deterministic Rendering + +The diff rendering SHALL be a pure function of its inputs, producing byte-identical output for the same inputs on every platform. + +#### Scenario: Repeated runs are identical + +- **WHEN** `openspec diff` runs more than once on the same inputs +- **THEN** the output bytes are identical every time + +#### Scenario: Platform independence + +- **WHEN** the diff runs on different operating systems with the same inputs +- **THEN** the output is identical regardless of line-ending or path-separator differences + +### Requirement: Git Diff Driver Integration + +The command SHALL be usable as a git diff driver for spec files, documented as an opt-in `.gitattributes` registration, so that `git diff` over spec and delta files renders the spec-aware view. OpenSpec SHALL NOT modify the user's git configuration without explicit consent. + +#### Scenario: Registered as a diff driver + +- **WHEN** the user opts in by registering the driver for spec paths in `.gitattributes` +- **THEN** `git diff` over those paths renders the spec-aware, annotated diff + +#### Scenario: Opt-in only + +- **WHEN** the user has not registered the driver +- **THEN** OpenSpec does not alter git behavior +- **AND** `openspec diff` remains available as a standalone command + +### Requirement: JSON Output + +The diff command SHALL support `--json`, emitting a machine-readable, per-requirement structure (change operation, provenance, rationale reference) for review tooling. + +#### Scenario: JSON annotated diff + +- **WHEN** the user runs `openspec diff --json` +- **THEN** it emits, per changed requirement, the operation, the originating change, and a reference to the rationale diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md b/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md index 54cc1553d2..fe2964a189 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md @@ -1,6 +1,6 @@ # Tasks: deterministic spec tooling — `sync` (forward) + `unarchive` (reverse) + `format` (canonical) -> Phased so the keystone ships first and each phase is independently testable and shippable. Phase 1 (engine + baseline + shared canonicalizer) and Phase 2 (`sync` + `--check`) deliver the deterministic, model-free drift path; Phase 2A (`format`) reuses the canonicalizer from Phase 1; Phase 3 (`unarchive`) and Phase 4 (idempotency) build on the same baseline; Phase 5 (skills) removes the model from the merge; Phase 6 (integration) is the staged hook/CI follow-up. Cross-platform concerns (`path.join`, Windows `moveDirectory`, newline-normalized digests) are called out per [openspec/config.yaml](../../config.yaml). +> Phased so the keystone ships first and each phase is independently testable and shippable. Phase 1 (engine + baseline + shared canonicalizer) and Phase 2 (`sync` + `--check`) deliver the deterministic, model-free drift path; Phase 2A (`format`) reuses the canonicalizer from Phase 1, and Phase 2B (`diff`) reuses the provenance from Phase 1–2; Phase 3 (`unarchive`) and Phase 4 (idempotency) build on the same baseline; Phase 5 (skills) removes the model from the merge; Phase 6 (integration) is the staged hook/CI follow-up. Cross-platform concerns (`path.join`, Windows `moveDirectory`, newline-normalized digests) are called out per [openspec/config.yaml](../../config.yaml). ## 1. The engine: deterministic, byte-stable merge + applied-delta baseline @@ -33,6 +33,15 @@ - [ ] 2A.5 Incremental: reuse the digest skip (Decision 13) so `--check` only re-reads changed files; skip never changes the verdict. - [ ] 2A.6 Tests: determinism (same input → same bytes; CRLF/LF; Windows); idempotency (`format(format(x))==format(x)`); **behavior-preserving** (`parse-before==parse-after`; requirement/scenario order and prose unchanged); **shared-canonicalizer invariant** — `sync`/`archive` output passes `format --check`; `--check` exit codes; `--json` shape. +## 2B. `openspec diff` — spec-aware diff driver (consumes provenance) + +- [ ] 2B.1 Create `src/core/diff.ts` `DiffCommand` (human + `--json`, `--base `): render requirement-level spec/delta changes; resolve provenance from the applied-delta baseline and rationale from the originating change's `proposal.md`. +- [ ] 2B.2 Annotate each changed requirement with originating change + rationale reference; when a change is unattributable (no provenance, e.g. pre-baseline), show it honestly without inventing rationale. +- [ ] 2B.3 Deterministic rendering: pure function of git diff + provenance + proposal text; byte-identical across runs/platforms; no inference. +- [ ] 2B.4 Git diff driver integration: provide the renderer in a form usable as a git `diff`/`textconv` driver, plus a documented opt-in `.gitattributes` snippet for spec paths. Never modify the user's git config automatically. +- [ ] 2B.5 Register `diff [target]` in [src/cli/index.ts](../../../src/cli/index.ts) (`--base`, `--json`). No skill (pure code). +- [ ] 2B.6 Tests: deterministic output (repeat/CRLF/Windows); annotation resolves from proposal + provenance; unattributable change shown without invented rationale; no git-config mutation without opt-in; `--json` shape. + ## 3. `openspec unarchive` (reverse, built on the baseline) - [ ] 3.1 Create `src/core/unarchive.ts` `UnarchiveCommand` (human + `--json`, blocked-error → diagnostic). @@ -62,12 +71,11 @@ - [ ] 6.1 Document the CI drift-gate pattern (`openspec sync --check` as a plain binary, no model/API keys) in `docs/`; provide a copy-paste job snippet. (Wiring the actual workflow file is the owner's call.) - [ ] 6.2 Document the pre-commit hook pattern (`sync --check` to detect, `sync --fix` to auto-remediate before `git commit --amend`), eslint-`--fix` style. (Choosing husky/lefthook/native is a separable follow-up; none exists in the repo today.) -- [ ] 6.3 Document the **spec-aware git diff driver** follow-up (design Decision 12): a diff driver for spec files that follows the recorded provenance to splice each change's rationale inline. Out of scope to build here; capture the provenance shape it would consume so the follow-up is unblocked. -- [ ] 6.4 Document the **spec linter/formatter** follow-up (design Decision 13): note that this PR makes synced/archived spec output canonical and byte-stable, and scope a separable authoring-side formatter + organization lint adjacent to `openspec-conventions`/`validate`. Out of scope to build here. +- [ ] 6.3 Document the **opt-in git diff driver** registration for `openspec diff` (the `.gitattributes` snippet for spec/delta paths), making clear OpenSpec never alters git config without consent (design Decision 15). ## 7. Docs & supersession -- [ ] 7.1 `docs/opsx.md` + CLI docs: add `/opsx:unarchive`; update `/opsx:sync` to note CLI delegation + determinism; document `openspec sync` and `openspec format` with `--check`/`--fix` (and that `format` needs no agent/skill). +- [ ] 7.1 `docs/opsx.md` + CLI docs: add `/opsx:unarchive`; update `/opsx:sync` to note CLI delegation + determinism; document `openspec sync`, `openspec format`, and `openspec diff` with their flags (and that `format`/`diff` need no agent/skill). - [ ] 7.2 Note the `specs/` = shipped-reality invariant and why `archive` is retained (design Decision 5), so the "why not just remove archive" question has a documented answer. ## 8. End-to-end verification @@ -77,5 +85,6 @@ - [ ] 8.3 E2E stacked drift: change A MODIFIES req X and is archived; a second change re-MODIFIES X and is archived; `unarchive A` refuses spec reversal; `unarchive A --keep-specs` succeeds untouched. - [ ] 8.4 E2E no-crumbs: sync, revise the delta, re-sync → `specs/` reflects only the current delta. - [ ] 8.4a E2E formatter: mangle a spec's whitespace → `format --check` fails → `format` fixes it → `--check` passes; and `sync`/`archive` output passes `format --check` unchanged (shared-canonicalizer invariant). +- [ ] 8.4b E2E diff: edit a change's delta and proposal → `openspec diff ` shows the requirement change annotated with the proposal's rationale; a synced spec change is annotated with the originating change via provenance; a pre-baseline edit is shown without invented rationale. - [ ] 8.5 Validation: `openspec validate add-deterministic-sync-and-unarchive --strict` passes; `openspec status` shows artifacts complete. - [ ] 8.6 Run the suite on macOS, Linux, and Windows CI. From a06f702f32bd4e5d6af3ddd82d61556ab8782342 Mon Sep 17 00:00:00 2001 From: Clay Good Date: Mon, 29 Jun 2026 17:25:40 -0500 Subject: [PATCH 7/8] docs(openspec): address CodeRabbit findings; build the openspec check gate (pre-commit + CI) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address all four CodeRabbit review findings on PR #1279: - cli-archive: pin the applied-delta baseline storage location + versioned schema (.openspec/merge-baseline.json + .openspec/pre-merge//spec.md; unknown schema version => treated as no baseline). - cli-unarchive: specify the atomic validate->stage->commit sequence and the per-step failure/rollback semantics. - cli-unarchive: make pre-baseline reversal all-or-nothing (refuse entirely on any MODIFIED/REMOVED rather than partially reverse) for atomicity. - cli-sync: add the --all flag for cross-change conflict detection. Build the deferred CI/hook items (they serve the deterministic linter): new capability cli-check — `openspec check [--fix] [--all]`, the unified deterministic linter that runs format --check + sync --check + validate in one gate. Answers "pre-commit OR CI?": BOTH — same binary, identical verdict; a drift gate is a pure function of the committed files. Ships a runner-agnostic opt-in hook installer and a copy-paste CI step (no model, no API keys). --fix only does mechanical remediation; conflicts still fail. design Decision 16 (+ Decision 4 updated from "staged" to "built"). Now 8 capabilities. Validates with `openspec validate --strict`. No human names. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../design.md | 33 ++++-- .../proposal.md | 20 ++-- .../specs/cli-archive/spec.md | 7 ++ .../specs/cli-check/spec.md | 109 ++++++++++++++++++ .../specs/cli-sync/spec.md | 16 ++- .../specs/cli-unarchive/spec.md | 43 ++++--- .../tasks.md | 17 ++- 7 files changed, 206 insertions(+), 39 deletions(-) create mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-check/spec.md diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/design.md b/openspec/changes/add-deterministic-sync-and-unarchive/design.md index 898cc63598..b157771dcf 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/design.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/design.md @@ -1,4 +1,4 @@ -# Design: deterministic spec tooling — `sync` (forward), `unarchive` (reverse), `format` (canonical), `diff` (review) +# Design: deterministic spec tooling — `sync`, `unarchive`, `format`, `diff`, and the `check` gate ## Context @@ -18,6 +18,7 @@ This design covers that primitive and the commands built on it. It supersedes th - Hash-optimized incremental checking: `--check` re-checks only specs whose digest changed, without changing any verdict (Decision 13). - A deterministic, behavior-preserving spec formatter (`openspec format` + `--check`) sharing one canonicalizer with the merge engine (Decision 14). - A deterministic spec-aware diff (`openspec diff`, opt-in git diff driver) that splices provenance + rationale inline, reusing existing artifacts (Decision 15). +- A unified deterministic linter `openspec check` (+ `--fix`, `--all`, incremental) wired to run identically as a pre-commit hook and a CI gate, with an opt-in hook installer and a CI template (Decision 16). - Skills delegate the merge to the CLI; the agent never performs it. - Backward compatible with changes archived (and specs synced) before this feature. @@ -64,12 +65,12 @@ Building three mechanisms would invite three drift bugs. Building one — the pr ## Decision 4 — `--check` and `--fix`: the gate and the auto-fixer (wiring staged) -**Decision.** `sync --check` is read-only and exits non-zero when the change's deltas are not cleanly appliable to the base, or — when the team commits merged `specs/` during review — when committed `specs/` ≠ the regenerated output. `sync --fix` (and bare `sync`) regenerates `specs/`. This PR ships both CLI modes and documents two integration patterns; it does **not** add a hook runner or CI job to this repo (none exists today). +**Decision.** `sync --check` is read-only and exits non-zero when the change's deltas are not cleanly appliable to the base, or — when the team commits merged `specs/` during review — when committed `specs/` ≠ the regenerated output. `sync --fix` (and bare `sync`) regenerates `specs/`. The per-tool `--check`/`--fix` modes are the primitives; the unified gate over them is `openspec check` (Decision 16), which is what pre-commit and CI actually invoke. -- **CI drift gate**: a job runs `openspec sync --check` (or `--all`) and fails the PR on drift — "the same pattern as a codegen or IaC drift gate," as a plain binary. -- **pre-commit hook**: `openspec sync --check` detects drift before `git commit --amend`; `openspec sync --fix` is the eslint-`--fix`-style auto-remediation, after which the amend proceeds with `specs/` canonical. +- **CI drift gate**: a job runs `openspec check` and fails the PR on drift — "the same pattern as a codegen or IaC drift gate," as a plain binary, no model. +- **pre-commit hook**: `openspec check` detects drift before commit; `openspec check --fix` is the eslint-`--fix`-style auto-remediation, after which the commit proceeds with `specs/` canonical and in sync. -**Why staged.** The primitives are the reusable, testable part and belong here. Choosing husky vs. lefthook vs. native hooks, and the CI matrix, are repo-policy calls better made as a focused follow-up than bundled into a foundational engine PR. Keeping them separable also keeps this PR reviewable. +**What ships here vs. what stays policy.** Earlier revisions deferred all wiring. This revision **builds the integration** (Decision 16): the unified `openspec check` command, a runner-agnostic, opt-in hook installer, and a copy-paste CI step. What remains the owner's choice is only *which* runner to standardize on and whether to enable the gate by default — not whether the capability exists. ## Decision 5 — Keep the `archive` boundary; reject folding the merge into `apply` @@ -87,7 +88,7 @@ Building three mechanisms would invite three drift bugs. Building one — the pr ## Decision 7 — Reverse under a drift guard, atomically -**Decision.** Before reversing any spec, compare its current content to the baseline's applied-result digest. On drift (a later change touched the same requirement — [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246), [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md)), **refuse** and point to `--keep-specs`. Stage and validate the whole reversal first; commit order = restore/delete specs → move folder (`moveDirectory`, Windows EPERM/EXDEV fallback). On any failure: *"Abort. No files were changed."*, rolling restored specs back from the still-present baseline if the move fails. +**Decision.** Before reversing any spec, compare its current content to the baseline's applied-result digest. On drift (a later change touched the same requirement — [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246), [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md)), **refuse** and point to `--keep-specs`. The reversal runs as a defined **validate → stage → commit** sequence: (1) validate destination-absent + all affected specs present and drift-free; (2) compute reversed specs into a temporary staging area under `.openspec/` (no change to `specs/` yet); (3) swap staged specs into `specs/`; (4) move the folder. A failure in steps 1–2 leaves everything untouched; a failure of step 4 after the swap rolls `specs/` back from the still-present baseline. On any failure: *"Abort. No files were changed."*; if rollback itself cannot complete, report the partial state and exact recovery steps rather than leave a silent inconsistency. **Why.** Restoring a pre-image over a requirement a later change modified would silently delete that change's contribution — the loss OpenSpec guards against. Refusing is strictly safer, and `--keep-specs` always lets the user proceed. Atomicity gives archive the rollback [#682](https://github.com/Fission-AI/OpenSpec/issues/682) noted it lacks, and matches the abort contract [#1112](https://github.com/Fission-AI/OpenSpec/issues/1112) describes. @@ -97,7 +98,7 @@ Building three mechanisms would invite three drift bugs. Building one — the pr ## Decision 9 — Backward compatibility: pre-baseline archives and synced specs -**Decision.** Changes archived/synced before baselines existed have none. `unarchive` then reverses **ADDED**/**RENAMED** by delta inversion (the self-invertible half) and, for any **REMOVED**/**MODIFIED**, **refuses to guess** — naming each requirement it cannot safely restore and directing to `--keep-specs` (optionally offering opt-in git recovery of the pre-image). `sync` on a pre-baseline change establishes a baseline on its next run. Never silently emit a wrong spec. +**Decision.** Changes archived/synced before baselines existed have none. Pre-baseline reversal is **all-or-nothing**: if every delta operation across all affected specs is ADDED/RENAMED (self-invertible), `unarchive` reverses the whole change by delta inversion; if any MODIFIED/REMOVED is present anywhere (pre-image unrecoverable), it **refuses the entire spec reversal** — naming the requirements it cannot restore and directing to `--keep-specs` (optionally offering opt-in git recovery of the pre-image). It does **not** partially reverse the self-invertible specs and leave the rest, because a partial `specs/` is exactly the corruption the atomic guarantee (Decision 7) exists to prevent. `sync` on a pre-baseline change establishes a baseline on its next run. Never silently emit a wrong spec. **Why.** The feature must be useful on day one, including for already-archived changes, without ever degrading into corruption. Reverse what is provably reversible; stop honestly on the rest. @@ -164,9 +165,23 @@ Building three mechanisms would invite three drift bugs. Building one — the pr **Why in scope now.** It is the presentation layer of the provenance this PR already produces, and it completes the discussion's review vision deterministically. Like the formatter (Decision 14), it reuses primitives this PR establishes rather than introducing new ones. +## Decision 16 — `openspec check`: the unified deterministic linter for pre-commit *and* CI + +**Context.** The discussion's recurring ask: the sync/format gate should run "like a linter," both as a pre-commit hook (eslint-`--fix` style) and as a CI drift gate, with no model in CI. The question is whether the deterministic linter works for pre-commit *or* post-commit (CI). **Answer: both — it is the same binary invoked in two places.** A drift gate is a pure function of the committed files; *where* it runs (a local hook before the commit, or a CI job after) changes nothing about the verdict. + +**Decision.** Add `openspec check` — one command that runs the deterministic gates (`format --check`, `sync --check`, `validate`) and exits non-zero on any failure, with `--fix` for auto-remediation, `--all` for cross-change checks, incremental digest skipping, and `--json`. Ship the two integration points as well: a **runner-agnostic, opt-in hook installer** and a **copy-paste CI step**. This is the "deterministic linter" the discussion converged on, and it is the single entrypoint pre-commit and CI both call (so neither has to know the individual sub-checks). + +**Why one command, not three.** A hook or CI job shouldn't hard-code `format --check && sync --check && validate` and drift out of step with the toolchain. `openspec check` is the stable contract; the sub-gates can evolve behind it. It also gives one exit-code/`--json` shape for both contexts. + +**Pre-commit vs CI — same gate, complementary placement.** Pre-commit catches drift at authoring time and can `--fix` in place before the commit (fast feedback, fewer red CI runs). CI is the backstop that enforces the gate regardless of local setup (someone without the hook, or `--no-verify`). Because the verdict is identical and deterministic, the two never disagree — CI cannot fail something a correctly-installed hook would have passed. Recommended posture: hook runs `--fix` (convenience), CI runs `--check` (enforcement). + +**Honesty under `--fix`.** `--fix` only applies *mechanical* remediation (format + sync regeneration). Non-mechanical failures — a delta that doesn't cleanly apply, a cross-change conflict — are reported and still fail; `--fix` never invents a resolution or masks a real conflict. + +**Scope note.** This builds the deferred CI/hook items because they directly serve the deterministic-linter goal (the user's explicit ask). The one thing still left to the owner is policy: which runner to standardize and whether the gate is on by default. The inference-based "align spec to implementation" remains out (it needs a model — the `verify` direction, #880). + ## Scope completeness -With `diff` folded in, this proposal now covers **every deterministic, in-scope idea** the discussion raised: the merge engine and applied-delta baseline (forward `sync`, reverse `unarchive`, the `archive` baseline), idempotent crumb-free re-merge, model-free `--check` drift gating, early conflict detection, provenance + delta↔spec correspondence, incremental checking, the canonical formatter, and the spec-aware diff driver. The ideas it deliberately leaves out are documented with reasons: editing specs in place / dropping the delta layer / committing only the spec (Decisions 5, 11, 13), inference-based code-vs-spec verification (Decision 14, the `verify` direction), and the *wiring* of a specific hook runner or CI job (Decision 4, a repo-policy choice). Subsequent sessions should therefore shift from **expansion** to **sequencing and refinement** for owner review — confirming the open decisions (e.g. `--keep-specs` vs `--skip-specs`, the `/opsx:sync` behavior change) and the phase order — rather than adding surface. +This proposal now covers **every deterministic, in-scope idea** the discussion raised: the merge engine and applied-delta baseline (forward `sync`, reverse `unarchive`, the `archive` baseline), idempotent crumb-free re-merge, model-free `--check` drift gating, early conflict detection, provenance + delta↔spec correspondence, incremental checking, the canonical formatter, the spec-aware diff driver, and the unified `openspec check` linter wired for both pre-commit and CI. The ideas it deliberately leaves out are documented with reasons: editing specs in place / dropping the delta layer / committing only the spec (Decisions 5, 11, 13), and inference-based code-vs-spec verification (Decisions 14, 16 — the `verify` direction, #880). The only thing left to the owner is **policy**, not capability: which hook runner to standardize and whether the gate is on by default. Subsequent sessions should therefore shift from **expansion** to **sequencing and refinement** for owner review — confirming the open decisions (e.g. `--keep-specs` vs `--skip-specs`, the `/opsx:sync` behavior change) and the phase order — rather than adding surface. ## Risks / trade-offs @@ -177,4 +192,4 @@ With `diff` folded in, this proposal now covers **every deterministic, in-scope ## Migration / rollout -Additive and phased (see tasks). The engine + baseline land first (no user-visible change to `archive` output). `sync` and `unarchive` are new commands in the expanded profile. `/opsx:sync` delegation ships with a changeset noting the determinism shift. Pre-baseline changes degrade per Decision 9. Hook/CI wiring is a follow-up. +Additive and phased (see tasks). The engine + baseline land first (no user-visible change to `archive` output). `sync`, `unarchive`, `format`, `diff`, and `check` are new commands in the expanded profile. `/opsx:sync` delegation ships with a changeset noting the determinism shift. Pre-baseline changes degrade per Decision 9. The `openspec check` gate, opt-in hook installer, and CI template ship in this PR (Decision 16); enabling the gate by default is the owner's policy call. diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md b/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md index e00dd24c45..11a654a804 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md @@ -42,6 +42,8 @@ Design principle the whole thread converges on, and the one this project is alre 6. **THE REVIEW VIEW — `openspec diff [target]` (`cli-diff` NEW).** The discussion's "review primarily becomes reading the spec deltas, with a spec-aware diff driver that splices the reasoning log inline so reviewers see what changed and why together." Delivered as a deterministic, pure-code renderer that shows requirement-level changes with each change's **provenance and rationale spliced inline**, usable standalone or (opt-in) as a **git diff driver** via `.gitattributes`. The key move: OpenSpec **already has the "log"** — the *why* is in `proposal.md`, the *what/where* is the applied-delta provenance — so `diff` joins existing artifacts rather than building the sidecar reasoning database the discussion reached for. It is deterministic and honest (no inference; says so when a change is unattributable), never modifies git config without opt-in, and does not summarize semantically (that is the `verify` direction). (design Decision 15.) +7. **THE LINTER GATE — `openspec check` (`cli-check` NEW).** The discussion's "sync is a lint step… run within CI/tooling much like linters." One command runs all the deterministic gates (`format --check`, `sync --check`, `validate`), exits non-zero on any failure, and offers `--fix` (mechanical auto-remediation), `--all` (cross-change), incremental skipping, and `--json`. **It is the same binary for pre-commit and CI** — a drift gate is a pure function of the committed files, so *where* it runs (a local hook before the commit or a CI job after) never changes the verdict. This PR also **builds the wiring**: a runner-agnostic, opt-in pre-commit hook installer and a copy-paste CI step — no model, no API keys in CI. (design Decisions 4, 16.) + ### What this deliberately does *not* change The discussion proposed removing `archive` entirely — keep every change in its dated archive location for its whole life and fold the spec merge into `apply`. **Rejected, with the maintainer's reasoning, and it shapes the design:** `specs/` only ever describes *shipped* reality. Folding the merge into `apply` would let proposed-but-unmerged (or abandoned) changes pollute the source of truth and let parallel changes step on each other's specs before any land. So the archive boundary stays; what changes is that the merge across it becomes deterministic and reversible. The determinism is what makes the boundary cheap to cross in both directions, which is the real fix for the "awkward final step." (Full treatment in [design.md](./design.md) Decision 5.) @@ -57,15 +59,16 @@ The discussion also raised the deeper question — *are deltas just simulating, - `opsx-unarchive-skill`: a `/opsx:unarchive` workflow skill that delegates to `openspec unarchive` for all deterministic work (selection, confirmation, rendering only). Expanded profile; no cross-skill dependency. - `cli-format`: `openspec format [target]` — a deterministic, behavior-preserving spec/delta formatter sharing one canonicalizer with the merge engine, so synced/archived specs are canonical by construction. `--check` is a read-only, model-free gate (CI + pre-commit) that names unformatted files; `--fix`/default writes canonical form; incremental via digests; `--json`-capable. Pure text — no agent, no `/opsx:format` skill. - `cli-diff`: `openspec diff [target] [--base ]` — a deterministic, spec-aware diff that splices each changed requirement's provenance and rationale inline (what changed and why, together), usable standalone or as an opt-in git diff driver. Renders from the change's `proposal.md` + the recorded applied-delta provenance — no new reasoning store; no inference; `--json`-capable. +- `cli-check`: `openspec check [--fix] [--all]` — the unified deterministic linter that runs `format --check`, `sync --check`, and `validate` in one invocation, exiting non-zero on any failure. The **same binary for pre-commit and CI** (identical verdict in both); `--fix` applies mechanical auto-remediation (never invents resolutions); incremental via digests; `--json`. Ships with a runner-agnostic opt-in hook installer and a copy-paste CI step — no model, no API keys. ### Modified Capabilities - `cli-archive`: routes its spec merge through the shared deterministic engine and records the applied-delta baseline inside the change folder before moving it, so archiving becomes deterministically reversible. Forward-only and backward-compatible — no change to how archive merges, moves, validates, or what it prints. - `specs-sync-skill`: `/opsx:sync` delegates the merge to the deterministic `openspec sync` CLI instead of performing agent-driven edits to `specs/`, making the result byte-deterministic and removing the scenario-dropping "intelligent merge" failure mode. The skill handles selection, confirmation, and output only. -### Integration surface (primitives in scope; wiring staged) +### Integration surface (built, not just documented) -- **CI drift gate** and **pre-commit hook** are the payoff of `cli-sync --check`/`--fix` and `cli-format --check`/`--fix` (the hook can `format --fix` then `sync --fix`, leaving specs canonical *and* in sync). This PR delivers the CLI primitives and documents both patterns; wiring a specific hook runner (husky/lefthook — none exists in the repo today) or a CI job is a small, separable follow-up the owner can stage. +- **`openspec check` is the unified gate** over `format --check` + `sync --check` + `validate`, and it is the **same binary for pre-commit and CI** — the verdict is a pure function of the committed files, so it cannot differ by where it runs. This PR builds the gate, a runner-agnostic **opt-in hook installer**, and a **copy-paste CI step** (no model, no API keys). The only thing left to the owner is policy: which hook runner to standardize and whether to enable the gate by default (`cli-check`, design Decision 16). ## Impact @@ -74,11 +77,12 @@ The discussion also raised the deeper question — *are deltas just simulating, - `src/core/sync.ts` (**new**) — `SyncCommand` (default/`--fix`/`--check`), mirroring `ArchiveCommand`'s human + `--json` shape; writes/refreshes the applied-delta baseline. - `src/core/format.ts` (**new**) — `FormatCommand` (default/`--fix`/`--check`, `--json`, incremental): runs the shared canonicalizer over main specs and active-change delta files; `--check` lists non-canonical files and exits non-zero without writing. - `src/core/diff.ts` (**new**) — `DiffCommand` (`--base`, `--json`): renders requirement-level spec/delta changes annotated with provenance (from the baseline) and rationale (from the originating change's `proposal.md`); pure code; also invocable as a git diff driver. Plus a documented `.gitattributes` snippet for opt-in registration. +- `src/core/check.ts` (**new**) — `CheckCommand` (`--fix`, `--all`, `--json`, incremental): composes `format --check`, `sync --check`, and `validate` into one gate with a single exit-code/JSON contract; `--fix` runs `format --fix` + `sync --fix`. Plus an opt-in hook installer (composes with existing hooks, never auto-installs) and a committed CI workflow template that runs `openspec check`. - `src/core/unarchive.ts` (**new**) — `UnarchiveCommand`: resolve archived dir, drift-check, restore pre-images (or delta-invert / refuse for pre-baseline), move folder back, atomic abort. Reuse `moveDirectory()`/`copyDirRecursive()` ([src/core/archive.ts:96-128](../../../src/core/archive.ts)). - `src/core/archive.ts` — route the merge through the shared engine (already deterministic) and persist the baseline before `moveDirectory` (~414-506). No behavior/output change. - `src/core/change-metadata/` or a sibling baseline store — persist the applied-delta baseline per change (pre-image + digest, newline-normalized, scheme-tagged). Coordinate with [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278)'s digest ledger so the two drift layers share a digest convention. - `src/core/list.ts` / `src/core/view.ts` — reuse `getArchivedChangesData()` ([#399](https://github.com/Fission-AI/OpenSpec/pull/399)) to resolve/disambiguate archived candidates. -- `src/cli/index.ts` — register `sync [change]` (`--check`, `--fix`, `--json`, `--store`), `unarchive [change-name]` (`--keep-specs`/`--skip-specs` alias, `-y/--yes`, `--no-validate`, `--json`, `--store`), `format [target]` (`--check`, `--fix`, `--json`), and `diff [target]` (`--base`, `--json`), mirroring archive (326-343). +- `src/cli/index.ts` — register `sync [change]` (`--check`, `--fix`, `--all`, `--json`, `--store`), `unarchive [change-name]` (`--keep-specs`/`--skip-specs` alias, `-y/--yes`, `--no-validate`, `--json`, `--store`), `format [target]` (`--check`, `--fix`, `--json`), `diff [target]` (`--base`, `--json`), and `check` (`--fix`, `--all`, `--install-hook`, `--json`), mirroring archive (326-343). - `src/core/templates/workflows/sync-specs.ts` — rewrite `/opsx:sync` to invoke `openspec sync` (drop agent-driven edits). `src/core/templates/workflows/unarchive-change.ts` (**new**) — `/opsx:unarchive` delegating to the CLI. Registration: export from `skill-templates.ts`; add `unarchive` to `ALL_WORKFLOWS` ([src/core/profiles.ts:19](../../../src/core/profiles.ts)) and `WORKFLOW_TO_SKILL_DIR` ([src/core/init.ts:65](../../../src/core/init.ts)); **not** added to `CORE_WORKFLOWS`. `docs/opsx.md` gains a `/opsx:unarchive` row and a determinism note for `/opsx:sync`. - Tests — byte-identical sync determinism (same delta+base → same bytes; CRLF/LF; Windows), idempotency (re-run no-op; revised delta no crumbs), `--check` exit codes, archive→unarchive byte-exact round-trip, drift refusal, `--keep-specs`, destination-collision abort, pre-baseline degradation, atomic "no files changed", skill-template delegation snapshots; formatter determinism + idempotency + `parse-before==parse-after` (behavior-preserving) + `sync`/`archive` output passes `format --check` (shared-canonicalizer invariant); diff determinism + provenance/rationale annotation + honest "unattributable" handling + opt-in-only git config. Per [openspec/config.yaml](../../config.yaml), run on Windows CI. @@ -90,7 +94,7 @@ Delivers (the Discord conclusions): - **Deterministic, code-only sync path** *("the right fix is a deterministic, code-only openspec sync/archive path that CI can run as a plain binary… I'll fold this into the unarchive PR")* → `openspec sync` + `--check`. - **`openspec unarchive` / `/opsx:unarchive`** that "moves the folder back and reverses the spec merge," including the hard case where the archive is buried in a multi-file commit. -- **Idempotent, crumb-free regeneration** and the **`sync --fix` pre-commit hook** *(the discussion's "sync is a lint step… eslint --fix" / "sync acts as the auto-fixer")*. +- **Idempotent, crumb-free regeneration** and the **pre-commit / CI gate** *(the discussion's "sync is a lint step… eslint --fix" / "sync acts as the auto-fixer", run "within CI/tooling much like linters")* → `openspec check` (+ `--fix`), the same binary for pre-commit and CI, with an opt-in hook installer and a CI template. - **Hash-tracked drift** *(the discussion's "track the hash of the changes/ state applied to the spec")* → the baseline digest. - **Deterministic spec formatter/linter** *(the discussion's "the framework's goal is how the spec looks and reads and is organized", as a linter/formatter with hash-optimized checks)* → `openspec format` + `--check`, sharing the engine's canonicalizer. - **Spec-aware diff with reasoning inline** *(the discussion's "review primarily becomes reading the spec deltas" + "a spec-aware diff driver that splices the reasoning log inline")* → `openspec diff` (opt-in git diff driver), rendering from existing proposal + provenance, so no sidecar reasoning database is needed. @@ -109,12 +113,12 @@ Delineated from adjacent work (coordinate, don't collide): - [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md) — planning-time archive ordering. Unarchive's drift guard and sync's idempotency are the runtime spec-layer counterpart; the two compose. - [#704](https://github.com/Fission-AI/OpenSpec/issues/704)/[#682](https://github.com/Fission-AI/OpenSpec/issues/682) (archive hooks), [#709](https://github.com/Fission-AI/OpenSpec/issues/709) (`git mv`) — symmetric `unarchive` hook points and a shared `moveDirectory()` make a future switch cover both directions. -Deferred follow-ups (enabled by this PR, not built here): +Out of scope (with reasons): -- **CI drift gate + pre-commit hook wiring** — the CLI primitives (`sync --check`/`--fix`, `format --check`/`--fix`) are in scope; wiring a hook runner (none exists in the repo) or a CI workflow is a repo-policy choice left staged (design Decision 4). -- **Inference-based review/verification** — semantic review or code-vs-spec checking needs a model; that is the `verify` direction ([#880](https://github.com/Fission-AI/OpenSpec/issues/880)), distinct from the deterministic `diff`/`format` shipped here (design Decisions 14–15). +- **Inference-based review/verification** — semantic review or code-vs-spec checking needs a model; that is the `verify` direction ([#880](https://github.com/Fission-AI/OpenSpec/issues/880)), distinct from the deterministic `diff`/`format`/`check` shipped here (design Decisions 14–16). +- **Owner policy, not capability** — *which* hook runner to standardize and whether the `openspec check` gate is enabled by default are left to the owner; the capability itself (command + opt-in installer + CI template) is built here. -> **Scope note:** with `diff` and `format` folded in, this proposal now covers every deterministic, in-scope idea from the discussion (see design "Scope completeness"). Subsequent sessions should shift from expansion to sequencing and confirming the open decisions for owner review. +> **Scope note:** with `check` (and the pre-commit/CI wiring), `diff`, and `format` folded in, this proposal now covers every deterministic, in-scope idea from the discussion (see design "Scope completeness"). Subsequent sessions should shift from expansion to sequencing and confirming the open decisions for owner review. - **Spec-aware authoring lint beyond canonical form** — `format` delivers deterministic canonicalization; richer *semantic* organization advice (which capability a requirement belongs in, etc.) would need judgment and stays separate from the pure-code formatter (design Decision 14). Considered and rejected (documented in design): diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-archive/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-archive/spec.md index 5496cd72ac..e21d71d118 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-archive/spec.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-archive/spec.md @@ -10,6 +10,13 @@ When the archive operation rewrites main specs, it SHALL record a self-contained - **THEN** the command records, for each affected spec, its pre-merge content and its applied-result digest in an applied-delta baseline stored inside the change folder - **AND** the baseline moves into `openspec/changes/archive/-/` together with the rest of the change +#### Scenario: Defined storage location and versioned schema + +- **WHEN** the baseline is written +- **THEN** it is stored at a defined path inside the change folder — a versioned manifest `/.openspec/merge-baseline.json` plus, for each affected spec, its captured pre-merge content under `/.openspec/pre-merge//spec.md` +- **AND** the manifest carries a schema version and, per affected spec, an entry of the form `{ "capability": "...", "preImage": "pre-merge//spec.md" | null, "appliedDigest": ":" }` (`preImage: null` marks a spec the archive created) +- **AND** readers that encounter an unrecognized schema version treat the baseline as unavailable rather than misreading it + #### Scenario: Created specs marked absent - **WHEN** archiving creates a new spec that did not previously exist diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-check/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-check/spec.md new file mode 100644 index 0000000000..bb25077bde --- /dev/null +++ b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-check/spec.md @@ -0,0 +1,109 @@ +## ADDED Requirements + +### Requirement: Unified Check Command + +The system SHALL provide an `openspec check` command — the deterministic spec linter — that runs the repository's deterministic gates (format canonical-form, sync delta↔spec consistency and conflicts, and spec validation) in one invocation and exits non-zero if any gate fails. It SHALL run in pure code without AI inference. + +#### Scenario: Run all deterministic gates + +- **WHEN** the user runs `openspec check` +- **THEN** the command runs `format --check`, `sync --check`, and `validate` across the repository's specs and active changes +- **AND** it exits zero only if every gate passes, and non-zero (naming the failing gate and files) otherwise + +#### Scenario: Check all active changes + +- **WHEN** the user runs `openspec check --all` +- **THEN** it includes cross-change checks (e.g. two active changes targeting the same requirement) in addition to per-change checks + +#### Scenario: No inference + +- **WHEN** `openspec check` runs +- **THEN** it computes every result in code +- **AND** it does not call a language model — it is safe to run with no API keys + +### Requirement: Same Gate For Pre-Commit And CI + +The check command SHALL be invocable identically as a local pre-commit hook and as a post-commit CI job, with the only difference being where it is invoked — the same binary, flags, and exit-code contract in both. + +#### Scenario: Pre-commit usage + +- **WHEN** `openspec check` runs from a pre-commit hook +- **THEN** a passing check allows the commit to proceed +- **AND** a failing check blocks the commit with a non-zero exit and a report of what to fix + +#### Scenario: CI usage + +- **WHEN** `openspec check` runs as a CI step on a pull request +- **THEN** a passing check allows the job to succeed +- **AND** a failing check fails the job with the same non-zero exit and report as the pre-commit run + +#### Scenario: Identical verdict in both contexts + +- **WHEN** the same repository state is checked locally and in CI +- **THEN** both reach the same pass or fail verdict (the gate is deterministic and environment-independent) + +### Requirement: Auto-Fix Mode + +The check command SHALL support a `--fix` mode that applies the deterministic auto-fixes (format and sync regeneration) so a pre-commit hook can remediate drift in place before the commit proceeds, leaving specs canonical and in sync. + +#### Scenario: Fix remediates drift + +- **WHEN** the user runs `openspec check --fix` +- **THEN** the command runs `format --fix` and `sync --fix` to regenerate canonical, in-sync specs +- **AND** a subsequent `openspec check` passes + +#### Scenario: Fix does not mask non-mechanical failures + +- **WHEN** `--fix` runs and a failure cannot be resolved deterministically (for example, a delta that does not cleanly apply, or a cross-change conflict) +- **THEN** the command applies what it safely can, leaves the unfixable issue reported, and still exits non-zero +- **AND** it does not invent a resolution + +### Requirement: Incremental Checking + +The check command MAY use recorded content digests to skip specs and files whose content is unchanged since they were last checked, re-checking only what changed. A skip SHALL be permitted only when it cannot change the verdict versus a full check. + +#### Scenario: Unchanged inputs skipped + +- **WHEN** `openspec check` runs and an input's digest matches its recorded digest +- **THEN** the command may skip re-checking it +- **AND** the overall verdict is identical to a full check + +#### Scenario: Changed or unknown inputs fully checked + +- **WHEN** an input's digest does not match, is missing, or uses an unrecognized scheme +- **THEN** the command performs the full check for it + +### Requirement: Hook Installation + +The system SHALL provide a runner-agnostic way to install `openspec check` as a git pre-commit hook, and SHALL NOT modify the user's git configuration or hooks without explicit action. + +#### Scenario: Install on request + +- **WHEN** the user opts in to installing the hook (for example, `openspec check --install-hook`) +- **THEN** the command installs a pre-commit hook that runs `openspec check` +- **AND** it composes with an existing hook rather than silently overwriting it + +#### Scenario: Never automatic + +- **WHEN** the user has not opted in +- **THEN** OpenSpec installs no hook and changes no git configuration + +### Requirement: CI Template + +The system SHALL document and provide a copy-paste CI configuration that runs `openspec check` as a drift gate, requiring no model and no API keys. + +#### Scenario: Documented CI gate + +- **WHEN** a maintainer wants a CI drift gate +- **THEN** the project provides a ready CI step that runs `openspec check` +- **AND** the step fails the build when committed specs are not canonical, not in sync with deltas, or otherwise invalid + +### Requirement: JSON Output + +The check command SHALL support `--json`, emitting a machine-readable summary of which gates ran, which passed or failed, and the offending files. + +#### Scenario: JSON summary + +- **WHEN** the user runs `openspec check --json` +- **THEN** it emits a structured result per gate (format, sync, validate) with pass/fail and the files involved +- **AND** the process exit code reflects the overall verdict diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md index 6b13f9932b..d472c4a8d7 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md @@ -72,6 +72,12 @@ When sync applies deltas to `openspec/specs/`, it SHALL record a per-change appl - **WHEN** sync re-applies a revised delta - **THEN** it refreshes the baseline to reflect the new pre-merge state and applied-result digest +#### Scenario: Defined storage location and versioned schema + +- **WHEN** sync writes the baseline +- **THEN** it uses the same defined, versioned storage as archive — a manifest at `/.openspec/merge-baseline.json` plus captured pre-merge content under `/.openspec/pre-merge//spec.md` +- **AND** an unrecognized schema version is treated as no baseline rather than misread + ### Requirement: Drift Check Mode The sync command SHALL support a read-only `--check` mode that verifies spec consistency and exits non-zero on a problem, without modifying any files, so it can gate commits and CI as a plain binary. @@ -167,7 +173,7 @@ In `--check` mode, sync SHALL verify a two-way correspondence between a change's ### Requirement: Cross-Change Conflict Detection -Sync SHALL surface conflicts deterministically and early — at commit or PR time rather than only at archive — including when a change's deltas no longer apply to the current base specs and when two active changes target the same requirement. +Sync SHALL surface conflicts deterministically and early — at commit or PR time rather than only at archive — including when a change's deltas no longer apply to the current base specs and when two active changes target the same requirement. Cross-change detection requires visibility across active changes, so the command SHALL accept an `--all` flag that operates over every active change rather than a single named one. #### Scenario: Delta no longer applies to the current base @@ -175,9 +181,15 @@ Sync SHALL surface conflicts deterministically and early — at commit or PR tim - **THEN** the command reports the conflict with the specific requirement - **AND** it exits with a non-zero status code and modifies no files +#### Scenario: Check all active changes + +- **WHEN** the user runs `openspec sync --check --all` +- **THEN** the command checks every active change for cross-change conflicts (and per-change appliability) +- **AND** it reports any requirement targeted by more than one active change + #### Scenario: Two active changes target the same requirement -- **WHEN** `--check` runs across active changes and more than one active change modifies, removes, or renames the same requirement +- **WHEN** `--check --all` runs and more than one active change modifies, removes, or renames the same requirement - **THEN** the command reports the overlapping changes and requirement as a potential conflict to resolve before archiving - **AND** it exits with a non-zero status code diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-unarchive/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-unarchive/spec.md index 264bd52b01..60a892daae 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-unarchive/spec.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-unarchive/spec.md @@ -120,13 +120,29 @@ The command SHALL support a `--keep-specs` flag that restores the change folder ### Requirement: Atomic Operation -The command SHALL apply the reversal atomically: it stages and validates all changes first, and on any failure it leaves the filesystem unchanged. +The command SHALL apply the reversal atomically using a defined sequence: validate, then stage, then commit, so that any failure before the final commit step leaves the filesystem unchanged. -#### Scenario: Failure leaves no partial state +#### Scenario: Defined sequence -- **WHEN** any step of the reversal fails (for example, the folder move fails after specs were restored) +- **WHEN** the command performs a reversal that touches specs +- **THEN** it executes in this order: + 1. validate that the destination `openspec/changes//` does not exist and that every affected spec is present and drift-free against the baseline; + 2. compute the reversed spec content and write it to a temporary staging area inside `.openspec/` (no change yet to `openspec/specs/`); + 3. swap the staged specs into `openspec/specs/`; + 4. move the change folder from archive back to active. + +#### Scenario: Failure before the folder move leaves specs untouched + +- **WHEN** validation or staging (steps 1–2) fails - **THEN** the command reports `Abort. No files were changed.` -- **AND** both `openspec/specs/` and the folder location are returned to their pre-command state +- **AND** `openspec/specs/` and the archive folder are exactly as they were + +#### Scenario: Failure of the final move rolls specs back + +- **WHEN** the spec swap (step 3) has occurred but the folder move (step 4) fails +- **THEN** the command restores `openspec/specs/` from the still-present baseline +- **AND** reports `Abort. No files were changed.` +- **AND** if rollback itself cannot complete, it reports the partial state and the exact recovery steps rather than leaving a silent inconsistency #### Scenario: Destination already exists @@ -136,22 +152,21 @@ The command SHALL apply the reversal atomically: it stages and validates all cha ### Requirement: Backward Compatibility For Pre-Baseline Archives -For changes archived before applied-delta baselines existed, the command SHALL reverse the operations it can invert from the archived delta alone and SHALL refuse to guess the operations it cannot, never producing an incorrect spec. +For changes archived before applied-delta baselines existed, the command SHALL reverse the spec merge only when the whole change is invertible from the archived delta alone, and SHALL otherwise refuse the spec reversal entirely (all-or-nothing) rather than leaving a partially reversed `openspec/specs/`. This preserves the same atomic, never-corrupt guarantee in the degraded path. -#### Scenario: Reverse the self-invertible operations +#### Scenario: Fully self-invertible change is reversed - **WHEN** an archived change has no applied-delta baseline -- **AND** its delta contains only ADDED and/or RENAMED requirements -- **THEN** the command reverses them by delta inversion (removing added requirements, renaming renamed requirements back) -- **AND** restores `openspec/specs/` accordingly +- **AND** every delta operation across all of its affected specs is ADDED and/or RENAMED +- **THEN** the command reverses them by delta inversion (removing added requirements, renaming renamed requirements back) and restores `openspec/specs/` accordingly -#### Scenario: Refuse to guess irreversible operations +#### Scenario: Mixed invertibility refuses all spec reversal - **WHEN** an archived change has no applied-delta baseline -- **AND** its delta contains MODIFIED or REMOVED requirements (whose pre-image is not recoverable from the delta) -- **THEN** the command does not modify those specs -- **AND** it reports which requirements cannot be safely reversed -- **AND** it directs the user to `--keep-specs` (and optionally to git-based recovery) +- **AND** its delta contains at least one MODIFIED or REMOVED requirement (whose pre-image is not recoverable from the delta), possibly alongside ADDED/RENAMED in the same or other specs +- **THEN** the command modifies no specs at all (it does not partially reverse the self-invertible specs) +- **AND** it reports which requirements cannot be safely reversed and why +- **AND** it directs the user to `--keep-specs` (and optionally to git-based recovery of the pre-image) #### Scenario: Keep-specs always available diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md b/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md index fe2964a189..d0ed9b7a48 100644 --- a/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md +++ b/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md @@ -1,6 +1,6 @@ # Tasks: deterministic spec tooling — `sync` (forward) + `unarchive` (reverse) + `format` (canonical) -> Phased so the keystone ships first and each phase is independently testable and shippable. Phase 1 (engine + baseline + shared canonicalizer) and Phase 2 (`sync` + `--check`) deliver the deterministic, model-free drift path; Phase 2A (`format`) reuses the canonicalizer from Phase 1, and Phase 2B (`diff`) reuses the provenance from Phase 1–2; Phase 3 (`unarchive`) and Phase 4 (idempotency) build on the same baseline; Phase 5 (skills) removes the model from the merge; Phase 6 (integration) is the staged hook/CI follow-up. Cross-platform concerns (`path.join`, Windows `moveDirectory`, newline-normalized digests) are called out per [openspec/config.yaml](../../config.yaml). +> Phased so the keystone ships first and each phase is independently testable and shippable. Phase 1 (engine + baseline + shared canonicalizer) and Phase 2 (`sync` + `--check`) deliver the deterministic, model-free drift path; Phase 2A (`format`) reuses the canonicalizer from Phase 1, and Phase 2B (`diff`) reuses the provenance from Phase 1–2; Phase 3 (`unarchive`) and Phase 4 (idempotency) build on the same baseline; Phase 5 (skills) removes the model from the merge; Phase 6 builds the unified `openspec check` gate plus its opt-in pre-commit hook installer and CI template (same binary both places). Cross-platform concerns (`path.join`, Windows `moveDirectory`, newline-normalized digests) are called out per [openspec/config.yaml](../../config.yaml). ## 1. The engine: deterministic, byte-stable merge + applied-delta baseline @@ -67,15 +67,19 @@ - [ ] 5.4 Tests: template snapshots assert `/opsx:sync` and `/opsx:unarchive` call the CLI and contain no manual merge/move instructions (anti-#863/#1246 guard). - [ ] 5.5 Changeset + docs note: `/opsx:sync` is now deterministic; the agent no longer performs scenario-level merges (fixes #1246); author deltas as complete requirements per the conventions. -## 6. Integration surface (staged): CI gate + pre-commit hook +## 6. `openspec check` — the unified gate (pre-commit + CI), built -- [ ] 6.1 Document the CI drift-gate pattern (`openspec sync --check` as a plain binary, no model/API keys) in `docs/`; provide a copy-paste job snippet. (Wiring the actual workflow file is the owner's call.) -- [ ] 6.2 Document the pre-commit hook pattern (`sync --check` to detect, `sync --fix` to auto-remediate before `git commit --amend`), eslint-`--fix` style. (Choosing husky/lefthook/native is a separable follow-up; none exists in the repo today.) -- [ ] 6.3 Document the **opt-in git diff driver** registration for `openspec diff` (the `.gitattributes` snippet for spec/delta paths), making clear OpenSpec never alters git config without consent (design Decision 15). +- [ ] 6.1 Create `src/core/check.ts` `CheckCommand` (`--fix`, `--all`, `--json`, incremental): compose `format --check`, `sync --check`, and `validate` into one run with a single exit-code/JSON contract; `--fix` runs `format --fix` + `sync --fix` and never invents non-mechanical resolutions (cross-change conflicts / un-appliable deltas still fail). +- [ ] 6.2 Register `check` in [src/cli/index.ts](../../../src/cli/index.ts) (`--fix`, `--all`, `--install-hook`, `--json`). +- [ ] 6.3 Hook installer: `openspec check --install-hook` installs a pre-commit hook that runs `openspec check`; runner-agnostic; composes with an existing hook; never installs automatically or alters git config without the explicit flag. +- [ ] 6.4 CI template: add a committed, copy-paste CI step (e.g. a GitHub Actions job) that runs `openspec check` as a drift gate — no model, no API keys. +- [ ] 6.5 Assert the pre-commit/CI symmetry: the same repo state yields the same `openspec check` verdict whether invoked from the hook or the CI step (one binary, one contract). +- [ ] 6.6 Document the **opt-in git diff driver** registration for `openspec diff` (the `.gitattributes` snippet for spec/delta paths), making clear OpenSpec never alters git config without consent (design Decision 15). +- [ ] 6.7 Tests: `check` aggregates the sub-gates and exit codes; `--fix` remediates mechanical drift but still fails on conflicts; `--all` runs cross-change checks; incremental skip never changes the verdict; hook installer composes with an existing hook and is never automatic; `--json` shape. ## 7. Docs & supersession -- [ ] 7.1 `docs/opsx.md` + CLI docs: add `/opsx:unarchive`; update `/opsx:sync` to note CLI delegation + determinism; document `openspec sync`, `openspec format`, and `openspec diff` with their flags (and that `format`/`diff` need no agent/skill). +- [ ] 7.1 `docs/opsx.md` + CLI docs: add `/opsx:unarchive`; update `/opsx:sync` to note CLI delegation + determinism; document `openspec sync`, `openspec format`, `openspec diff`, and `openspec check` with their flags (and that `format`/`diff`/`check` need no agent/skill); document running `check` as a pre-commit hook and in CI. - [ ] 7.2 Note the `specs/` = shipped-reality invariant and why `archive` is retained (design Decision 5), so the "why not just remove archive" question has a documented answer. ## 8. End-to-end verification @@ -86,5 +90,6 @@ - [ ] 8.4 E2E no-crumbs: sync, revise the delta, re-sync → `specs/` reflects only the current delta. - [ ] 8.4a E2E formatter: mangle a spec's whitespace → `format --check` fails → `format` fixes it → `--check` passes; and `sync`/`archive` output passes `format --check` unchanged (shared-canonicalizer invariant). - [ ] 8.4b E2E diff: edit a change's delta and proposal → `openspec diff ` shows the requirement change annotated with the proposal's rationale; a synced spec change is annotated with the originating change via provenance; a pre-baseline edit is shown without invented rationale. +- [ ] 8.4c E2E check gate: introduce drift (un-synced delta + non-canonical whitespace) → `openspec check` fails naming both gates → `openspec check --fix` remediates → `check` passes; then introduce a cross-change conflict → `check --all` still fails (not auto-fixable); confirm identical verdict whether run via the installed hook or the CI step. - [ ] 8.5 Validation: `openspec validate add-deterministic-sync-and-unarchive --strict` passes; `openspec status` shows artifacts complete. - [ ] 8.6 Run the suite on macOS, Linux, and Windows CI. From c17dcabe796687e059f3290af949d6e936fa664a Mon Sep 17 00:00:00 2001 From: Clay Good Date: Wed, 1 Jul 2026 10:10:45 -0500 Subject: [PATCH 8/8] docs(openspec): narrow to the deterministic spec-merge engine (first slice) Per @alfred-openspec's review on #1279 ("too broad for one implementation pass"), narrow this proposal to the load-bearing first slice and split the rest into a follow-up: Keep (this PR): the applied-delta baseline + deterministic merge engine, `openspec sync` (+ `--check`/`--fix`), `archive` routed through the engine, and `/opsx:sync` delegating to the CLI. Capabilities: cli-sync (new), cli-archive (modified), specs-sync-skill (modified). Design Decisions 1-5, 11-13. Move to follow-up (add-spec-tooling-suite): unarchive, format, diff, and the unified `check` gate. Capabilities cli-unarchive, opsx-unarchive-skill, cli-format, cli-diff, cli-check; design Decisions 6-10, 14-16. Rename the change add-deterministic-sync-and-unarchive -> add-deterministic- spec-sync to match the narrowed scope. Validates clean under `openspec validate add-deterministic-spec-sync --strict`. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../.openspec.yaml | 0 .../add-deterministic-spec-sync/design.md | 113 ++++++++++ .../add-deterministic-spec-sync/proposal.md | 90 ++++++++ .../specs/cli-archive/spec.md | 0 .../specs/cli-sync/spec.md | 0 .../specs/specs-sync-skill/spec.md | 0 .../add-deterministic-spec-sync/tasks.md | 52 +++++ .../design.md | 195 ------------------ .../proposal.md | 132 ------------ .../specs/cli-check/spec.md | 109 ---------- .../specs/cli-diff/spec.md | 86 -------- .../specs/cli-format/spec.md | 144 ------------- .../specs/cli-unarchive/spec.md | 190 ----------------- .../specs/opsx-unarchive-skill/spec.md | 56 ----- .../tasks.md | 95 --------- 15 files changed, 255 insertions(+), 1007 deletions(-) rename openspec/changes/{add-deterministic-sync-and-unarchive => add-deterministic-spec-sync}/.openspec.yaml (100%) create mode 100644 openspec/changes/add-deterministic-spec-sync/design.md create mode 100644 openspec/changes/add-deterministic-spec-sync/proposal.md rename openspec/changes/{add-deterministic-sync-and-unarchive => add-deterministic-spec-sync}/specs/cli-archive/spec.md (100%) rename openspec/changes/{add-deterministic-sync-and-unarchive => add-deterministic-spec-sync}/specs/cli-sync/spec.md (100%) rename openspec/changes/{add-deterministic-sync-and-unarchive => add-deterministic-spec-sync}/specs/specs-sync-skill/spec.md (100%) create mode 100644 openspec/changes/add-deterministic-spec-sync/tasks.md delete mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/design.md delete mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/proposal.md delete mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-check/spec.md delete mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-diff/spec.md delete mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-format/spec.md delete mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-unarchive/spec.md delete mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/specs/opsx-unarchive-skill/spec.md delete mode 100644 openspec/changes/add-deterministic-sync-and-unarchive/tasks.md diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/.openspec.yaml b/openspec/changes/add-deterministic-spec-sync/.openspec.yaml similarity index 100% rename from openspec/changes/add-deterministic-sync-and-unarchive/.openspec.yaml rename to openspec/changes/add-deterministic-spec-sync/.openspec.yaml diff --git a/openspec/changes/add-deterministic-spec-sync/design.md b/openspec/changes/add-deterministic-spec-sync/design.md new file mode 100644 index 0000000000..a1b44ab48e --- /dev/null +++ b/openspec/changes/add-deterministic-spec-sync/design.md @@ -0,0 +1,113 @@ +# Design: the deterministic spec-merge engine — `sync` + the applied-delta baseline + +## Context + +OpenSpec's spec merge (delta → `specs/`) is reached two ways today: deterministically but only *inside* `openspec archive` (fused to the folder move), or by an AI agent in `/opsx:sync` that "directly edits main specs." The standalone deterministic apply (`applySpecs()`, [src/core/specs-apply.ts:391](../../../src/core/specs-apply.ts)) has zero callers. The design discussion (2026-06) worked from "add unarchive" to a single root cause: **the merge must be deterministic, idempotent, and reversible pure code, exposed as commands** — then a code-only CI drift gate, a `sync --fix` pre-commit hook, crumb-free re-merge, and (later) reverse all fall out of one primitive. + +**Scope of this PR (the first slice).** Per review on [#1279](https://github.com/Fission-AI/OpenSpec/pull/1279), the proposal was narrowed to the load-bearing spine so it ships in one reviewable pass: the applied-delta **baseline**, the deterministic **merge engine**, `openspec sync` (+ `--check`/`--fix`), `archive` routed through the engine, and `/opsx:sync` delegating to the CLI. The commands that build on the baseline — `unarchive`, `format`, `diff`, and the unified `check` gate — are specified in the **follow-up** change `add-spec-tooling-suite` (design Decisions 6–10, 14–16 live there). This document keeps the decisions that define and constrain the primitive. + +## Goals / Non-Goals + +**Goals** +- A deterministic merge: same (delta + base) → byte-identical `specs/`, in pure code, on every platform. +- Idempotent: re-running `sync` is a no-op; revising a delta regenerates `specs/` with no crumbs. +- Code-only drift detection (`sync --check`) usable by CI and a pre-commit hook — no model, no API keys. +- Early, deterministic conflict detection (delta-vs-base and cross-change) at commit/PR time, not at archive (Decision 11). +- Per-edit provenance and a delta↔spec correspondence check, within the delta model (Decision 12). +- Hash-optimized incremental checking: `--check` re-checks only specs whose digest changed, without changing any verdict (Decision 13). +- `archive` records the baseline so a later change can reverse it deterministically (the follow-up `unarchive`). +- Skills delegate the merge to the CLI; the agent never performs it. +- Backward compatible with changes archived (and specs synced) before this feature. + +**Non-Goals (here)** +- Removing or replacing the `archive` lifecycle boundary, or editing `specs/` in place instead of via deltas (Decisions 5, 11). +- Folding the spec merge into `apply` (Decision 5). +- The reverse command `unarchive` and its resolution / drift-guard / atomicity / pre-baseline handling — **follow-up** (`add-spec-tooling-suite`). +- The `format` canonicalizer-for-authoring, the spec-aware `diff` driver, and the unified `check` gate + hook/CI wiring — **follow-up**. +- Inference-based "align spec to implementation" / code-vs-spec checking — that needs a model and is the `verify` direction (#880). +- Reducing the committed artifact set (commit-only-the-spec) — a separate product question served by schema flexibility (Decision 13). +- A scenario-granular "smart" merge — the conventions mandate whole-requirement deltas; whole-block apply is the correct, deterministic semantics (Decision 2). + +## Decision 1 — One primitive: the applied-delta baseline + +**Decision.** When a change's deltas are merged into `specs/` (by `sync` or `archive`), record a per-change **applied-delta baseline**: for each affected spec, the **pre-merge file content** (the pre-image, or an explicit `absent` marker when the spec is created), a **digest of the applied result** (newline-normalized CRLF→LF, scheme-tagged), and the **provenance** of each edit (originating change + delta operation). The baseline lives with the change; when archived, it travels into the archive folder. + +**Why one primitive.** The asks in the thread are the same computation viewed several ways: +- **Idempotent re-merge** (`sync` after a delta revision) = `specs_new = apply(delta_new, base)` where `base = current specs with delta_old reversed` (via the pre-image). This is the "no leftover crumbs" guarantee — the prior revision's contribution is removed, not layered over. +- **Drift** (`sync --check`, the hook, CI) = current `specs/` digest ≠ baseline digest. This is the discussion's "track the hash of the changes/ state applied to the spec," made precise. +- **Reverse** (the follow-up `unarchive`) = restore the pre-image. Deterministic for *every* op, including the REMOVED/MODIFIED ones the delta alone cannot invert. This PR **records** the baseline that makes reverse possible; the reverse command itself is the follow-up. + +Building separate mechanisms would invite separate drift bugs. Building one — the pre-image + digest + provenance — and deriving the rest is why this is the spine. + +**Form.** Store the whole pre-image (not a reverse-diff), so a future restore is a byte copy with no re-parsing. Cost is a copy of each affected spec's prior bytes per merge — negligible, and only for changes that touch specs. Coordinate the digest convention with [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278)'s artifact-graph digest ledger so the two drift layers (artifact staleness vs. spec-merge drift) speak the same digest dialect. + +## Decision 2 — Deterministic sync is pure parser code; the agent never merges + +**Decision.** Promote `applySpecs()`/`buildUpdatedSpec` into the canonical merge and expose it as `openspec sync`. The merge is a pure function of (delta files + base spec bytes) producing byte-identical output across runs and platforms (stable requirement ordering, newline normalization, deterministic recomposition). `archive` calls the same function. `/opsx:sync` is rewritten to **invoke the CLI**, not edit specs itself. + +**Why.** This is the thread's keystone verbatim: *"Moving it out of the agent prompt and into plain parser code that always produces byte-identical output for the same delta + base."* The agent path isn't just non-deterministic; its stated reason for existing — "intelligent merging (e.g., adding a scenario without copying the entire requirement)" — is precisely how [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) silently drops sibling scenarios. The conventions spec already requires deltas to carry "the complete modified requirement, not a diff," so whole-block apply is the *correct* semantics and the agent's cleverness is the bug. Deterministic code is therefore more faithful, not less. + +**Alternatives.** Keep agent sync and only diff its output in CI — rejected: needs a model in CI, the exact thing the thread set out to avoid. + +## Decision 3 — Idempotency and "no crumbs" + +**Decision.** `sync` is idempotent: running it on an already-synced, unchanged change writes nothing (`--check` is clean). When a delta is revised, `sync` reverses the prior revision's contribution via the baseline pre-image, then applies the new delta, then refreshes the baseline — so `specs/` equals `apply(current delta, original base)` with zero residue from the old revision. + +**Why.** Directly answers *"a revised delta should let sync regenerate the spec output from scratch, idempotently, no leftover crumbs."* Without the baseline, re-applying a revised delta against already-merged `specs/` either conflicts (ADDED now exists) or layers (renamed-then-renamed) — crumbs. The baseline makes regeneration a clean reverse-then-apply. + +**Edge.** If `specs/` drifted from the baseline since the last sync (a later change touched the same requirement), `sync` does not silently reverse-then-apply over someone else's edit — it reports drift and requires `--check`-style acknowledgement. Consistency over cleverness. (The follow-up `unarchive` applies the same drift guard on reverse.) + +## Decision 4 — `--check` and `--fix`: the gate and the auto-fixer + +**Decision.** `sync --check` is read-only and exits non-zero when the change's deltas are not cleanly appliable to the base, or — when the team commits merged `specs/` during review — when committed `specs/` ≠ the regenerated output. `sync --fix` (and bare `sync`) regenerates `specs/`. + +- **CI drift gate**: a job runs `openspec sync --check` (fan out over active changes) and fails the PR on drift — "the same pattern as a codegen or IaC drift gate," as a plain binary, no model. +- **pre-commit hook**: `openspec sync --check` detects drift before commit; `openspec sync --fix` is the eslint-`--fix`-style auto-remediation. + +**Relationship to the follow-up `check` gate.** The per-tool `--check`/`--fix` modes shipped here are the primitives. The **follow-up** adds `openspec check`, a single command that composes `sync --check` with `format --check` and `validate` behind one exit-code/JSON contract, plus an opt-in hook installer and a CI template. Until that lands, CI and hooks invoke `openspec sync --check` directly — the verdict is identical, just not yet bundled behind one entrypoint. + +## Decision 5 — Keep the `archive` boundary; reject folding the merge into `apply` + +**Decision.** The lifecycle boundary stays: `specs/` is written at `sync`/`archive`, and `archive` remains the "fold finished change into shipped specs, then move the folder" step. Reject the proposal to remove `archive`, keep every change in a dated archive dir for its whole life, and make `apply` do both code and spec application. + +**Why.** The invariant that pays for OpenSpec's value is **`specs/` describes only shipped reality**. Folding the merge into `apply` would (a) pollute the source of truth with proposed-but-unmerged or abandoned changes, and (b) let parallel changes overwrite each other's specs before any lands. The thread's own resolution: the awkwardness of the "final archive step" is not the boundary's fault, it is the *non-determinism* of crossing it. Make the crossing deterministic and reversible (Decisions 1–3, and the follow-up `unarchive`) and the boundary becomes cheap in both directions — which is the actual fix. This is why determinism, not deletion, is the spine. + +**Consequence for `sync` on active changes.** Because of the invariant, the *default* workflow still merges at archive. Standalone `sync` serves: the engine archive uses; `--check` (read-only, never pollutes `specs/`); and the opt-in "generated-artifact" workflow where a team chooses to commit merged `specs/` mid-review and gate it. The tool enables both policies; it does not force early merge. + +## Decision 11 — Keep deltas; don't edit specs in place — and get early conflict resolution anyway + +**Context.** The discussion pushed a deeper question than "remove archive": *are we simulating deltas above git when git already stores deltas?* The proposed alternative: edit `specs/` in place (the git diff **is** the delta), drop the delta folder, and keep the "why" in a sidecar reasoning log. Its sharpest argument is about **timing**: if in-flight changes are deltas applied at archive, conflicts surface late; editing the spec first "forces conflict resolution immediately." + +**Decision.** Keep the delta layer; do **not** make in-flight edits directly to `specs/`. But **adopt the timing argument's goal** by moving conflict detection earlier with `sync --check` (Decision 12). + +**Why keep deltas.** The delta layer is exactly what a raw git diff cannot give you: a clean separation between **proposed** and **shipped**. `specs/` always describes reality; multiple in-flight changes stay isolated and independently reviewable until each lands. Edit-in-place collapses that — proposed-but-unmerged or abandoned edits sit in the source of truth, and N parallel changes mutate the same files, so the only place conflicts can be resolved is one big merge at the end. The delta model makes each change a self-contained, reviewable unit and is what makes reversing one change (the follow-up `unarchive`) and isolation-preserving parallelism *possible at all*. Git stores byte deltas; OpenSpec's deltas are behavioral agreements one level up — they answer *why* and *what-should-be* before code exists, which a diff cannot. + +**The synthesis — adopt the valid kernel.** Rather than discovering at archive that a delta no longer applies (or that two changes touched the same requirement), `sync --check` surfaces those conflicts at commit/PR time, deterministically, as a plain binary (Decision 12, "Cross-Change Conflict Detection"). So we get "resolve conflicts immediately" **without** sacrificing proposed-vs-shipped isolation. Same shape as Decision 5: adopt the goal, reject the mechanism that would break the invariant. + +## Decision 12 — Provenance and delta↔spec correspondence + +**Context.** The discussion wanted the "why" to travel with the "what": a consistency lint where a spec change with no delta — or a delta with no spec change — fails. + +**Decision.** Record **provenance** as part of the applied-delta baseline: when the engine writes a spec change, it records which change and which delta operation produced it. Expose it (e.g. `openspec sync --explain` / a provenance entry). Use it for a deterministic **delta↔spec correspondence** check in `sync --check`: every committed `specs/` edit for a change must trace to one of its delta operations (no orphan edits), and every delta operation must have landed (no unapplied deltas). The prose "why" is not re-authored — provenance links each spec edit to its change, whose `proposal.md` already holds the rationale. + +**Why this is the right slice.** Provenance falls out of the merge for free (the engine already knows exactly what it applied), and correspondence reuses the baseline. Together they answer the discussion's consistency lint *within the delta model* (deltas are the source; specs are generated) rather than inverting it (specs as source, deltas as sidecar log) — which would reintroduce the edit-in-place problems of Decision 11. + +**Consumed by the follow-up.** The presentation side — a `git diff` driver that follows the provenance link and splices the change's rationale inline — is the follow-up's `openspec diff`. It needs no new data store because it consumes exactly the provenance recorded here. + +## Decision 13 — Incremental checking (hash-optimized) + +**Context.** The discussion asked for "hashes to optimize when checks are needed." + +**Decision.** The applied-delta baseline already carries a per-spec digest. So `sync --check` (and any gate built on it) can skip any spec whose content digest is unchanged since it was last reconciled, and re-check only what changed. The correctness invariant: a skip is allowed *only* on an exact digest match against a recorded baseline; a mismatch, a missing baseline, or an unknown digest scheme forces a full check. So incremental mode is an optimization that can **never change a verdict** versus a full check — important for a gate. This makes the gate cheap enough to run on every commit even in a repo with hundreds of specs. + +**Out of scope (recorded for the "just commit the spec" question).** The wish to commit "only the spec" and treat design/tasks as disposable runs against the reason those artifacts exist: in OpenSpec they *are* the reviewable, resumable product. The valid kernel — that not every change needs the full artifact set — is already served *outside this PR* by schema flexibility (a change can run a minimal schema). This PR neither needs nor changes that. The `format` canonicalizer that the same discussion asked for ("how the spec looks and reads") reuses this PR's canonicalizer and ships in the **follow-up** (`add-spec-tooling-suite`). + +## Risks / trade-offs + +- **Behavior shift for `/opsx:sync` users.** Replacing agent merge with deterministic merge changes output for anyone who relied on the agent's scenario-level merges. This is intended (it fixes [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)), but it is a real change; call it out in the changeset and docs, and keep the conventions' "complete requirement, not a diff" rule prominent so deltas are authored to merge cleanly. +- **Baseline storage in the change folder.** Adds a small artifact; inert for spec-less changes. Format is scheme-tagged so canonicalization can evolve without silent mis-compares. +- **Baseline is forward-only.** Changes synced/archived before this feature have no baseline; `sync` establishes one on its next run, and the follow-up `unarchive` degrades gracefully for pre-baseline archives (specified there). + +## Migration / rollout + +Additive and phased (see tasks). The engine + baseline land first (no user-visible change to `archive` output). `openspec sync` is a new command in the expanded profile. `/opsx:sync` delegation ships with a changeset noting the determinism shift. The follow-up PR (`add-spec-tooling-suite`) adds `unarchive`, `format`, `diff`, and the unified `check` gate on top of the baseline this PR establishes. diff --git a/openspec/changes/add-deterministic-spec-sync/proposal.md b/openspec/changes/add-deterministic-spec-sync/proposal.md new file mode 100644 index 0000000000..30e7bae61e --- /dev/null +++ b/openspec/changes/add-deterministic-spec-sync/proposal.md @@ -0,0 +1,90 @@ +## Why + +The spec merge (delta → `specs/`) is **the** load-bearing operation in OpenSpec, yet today it is reachable only two ways — buried *inside* `openspec archive` (fused to the folder move), or performed by an **AI agent** in `/opsx:sync` ("This is an agent-driven operation… you will read delta specs and directly edit main specs"). The agent path is non-deterministic: its "intelligent" scenario-merge is the very mechanism that silently drops scenarios in [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246). The deterministic engine `applySpecs()` ([src/core/specs-apply.ts:391](../../../src/core/specs-apply.ts)) has **zero callers** — it cannot be run on its own. + +So you cannot gate spec drift in CI without a model, you cannot cleanly re-merge a revised delta, and you cannot catch merge conflicts before archive. One missing primitive sits under all of it. This change builds it: a deterministic, idempotent spec-merge **engine** plus a per-change **applied-delta baseline**, exposed as a first-class `openspec sync` command and shared by `archive`. + +## Scope: the first slice (this PR) + +This is deliberately the **spine**, not the whole toolchain. Per review ([@alfred-openspec on #1279](https://github.com/Fission-AI/OpenSpec/pull/1279): *"narrow the first slice to deterministic merge/baseline, sync check/fix, archive using that path, and CLI-delegated /opsx:sync, then split unarchive, format, and diff into follow-ups once the primitive is proven"*), this PR ships exactly that first slice: + +1. the deterministic, idempotent merge **engine** + the **applied-delta baseline**; +2. `openspec sync` with its model-free `--check` gate and `--fix`; +3. `openspec archive` routed through the same engine (records the baseline); +4. `/opsx:sync` delegating to the CLI instead of merging in the agent. + +The commands that **build on** this baseline — `unarchive` (reverse), `format` (canonical), `diff` (spec-aware review), and the unified `check` gate — ship as a **follow-up PR** once the primitive is proven. See "Follow-up" below. + +## The primitive: one applied-delta baseline + +When a change's deltas are merged into `specs/` (by `sync` or `archive`), record a per-change **applied-delta baseline**: for each affected spec, the **pre-merge content** (the pre-image, or an explicit `absent` marker when the spec is created) plus a **digest of the applied result** (newline-normalized, scheme-tagged), and the **provenance** of each edit (which change and delta op produced it). The baseline lives with the change; when archived, it travels into the archive folder. + +One primitive, three payoffs (design Decision 1): idempotent crumb-free re-merge = reverse the old delta via the pre-image, then apply the new; deterministic drift detection = current `specs/` digest ≠ baseline digest; and (in the follow-up) byte-exact reverse for `unarchive`. Building one mechanism and deriving the rest is why this is the spine. + +## What Changes + +Design principle the whole thread converges on, and the one this project is already adopting elsewhere ([#1277](https://github.com/Fission-AI/OpenSpec/pull/1277), prevent-silent-spec-drop): **the merge is pure parser code that produces byte-identical output for the same delta + base; the agent never performs it.** Ordered by leverage: + +1. **THE ENGINE — deterministic, idempotent merge core (`cli-sync` NEW, shared by archive).** Promote the existing `applySpecs()`/`buildUpdatedSpec` into a first-class, **byte-deterministic** apply that records the applied-delta baseline, and make it idempotent (re-running is a no-op; a revised delta regenerates `specs/` with no crumbs). `openspec archive` keeps merging-then-moving but routes its merge through this shared core and writes the baseline before moving, so it travels into the archive. + +2. **FORWARD COMMAND + DRIFT GATE — `openspec sync [change]` (`cli-sync` NEW).** Apply a change's deltas to `specs/` without archiving, deterministically and idempotently: + - default / `--fix`: write `specs/` to the regenerated result (idempotent; a revised delta regenerates with no crumbs); + - `--check`: read-only; exit non-zero if the deltas are not cleanly appliable, or (when `specs/` has been synced) if committed `specs/` ≠ the regenerated output. This is the **codegen/IaC-style drift gate CI runs as a plain binary — no model, no API keys.** + + `--check` also surfaces conflicts **early** — a delta that no longer applies to the current base, or two active changes targeting the same requirement — at commit/PR time instead of at archive (design Decision 11). The engine records **provenance**, enabling a deterministic delta↔spec correspondence check (orphan edit / unapplied delta → fail; design Decision 12). Because the baseline carries a per-spec digest, `--check` is **incremental** — it re-checks only what moved, so the gate stays cheap even in a large repo, and the incremental verdict always equals the full verdict (design Decision 13). + +3. **NO MODEL IN THE MERGE — `/opsx:sync` delegates to the CLI (`specs-sync-skill` MODIFIED).** `/opsx:sync` stops doing agent-driven edits and **invokes `openspec sync`**. The deterministic work lives in TypeScript; the skill only selects, confirms, and renders. This removes the [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) "intelligent merge drops scenarios" failure mode by construction, and is the direction [#863](https://github.com/Fission-AI/OpenSpec/issues/863)/[#799](https://github.com/Fission-AI/OpenSpec/issues/799)/[#656](https://github.com/Fission-AI/OpenSpec/issues/656) ask for. + +### What this deliberately does *not* change + +The discussion proposed removing `archive` and folding the spec merge into `apply`. **Rejected, and it shapes the design:** `specs/` only ever describes *shipped* reality; folding the merge into `apply` would let proposed-but-unmerged (or abandoned) changes pollute the source of truth and let parallel changes step on each other's specs before any land. So the archive boundary stays; what changes is that crossing it becomes deterministic. The determinism is what makes the boundary cheap — the real fix for the "awkward final step" (design Decision 5). + +The discussion also proposed editing `specs/` in place (the git diff *is* the delta) with a sidecar reasoning log. **Also rejected:** the delta layer is precisely what a raw git diff cannot give — clean isolation of *proposed* from *shipped*, so `specs/` stays canonical while several in-flight changes remain independently reviewable. Its sharpest kernel is valid and adopted: "editing the spec first forces conflict resolution now" → `sync --check` moves conflict detection early (design Decisions 11–12). + +## Capabilities + +### New Capabilities + +- `cli-sync`: the deterministic, idempotent spec-merge engine exposed as `openspec sync [change]` — `--fix`/default writes `specs/` from the change's deltas (byte-identical for the same delta + base; re-running is a no-op; a revised delta regenerates with no crumbs); `--check` is a read-only, model-free gate for CI and pre-commit that also surfaces conflicts early (delta-vs-base, and two active changes on one requirement) and verifies delta↔spec correspondence. Records a per-change applied-delta baseline (pre-image + digest + provenance) that powers idempotent re-merge, drift detection, and tracing each spec edit to its delta. The same engine `archive` uses internally. + +### Modified Capabilities + +- `cli-archive`: routes its spec merge through the shared deterministic engine and records the applied-delta baseline inside the change folder before moving it, so archiving becomes deterministically reversible (unlocking the follow-up `unarchive`). Forward-only and backward-compatible — no change to how archive merges, moves, validates, or what it prints. +- `specs-sync-skill`: `/opsx:sync` delegates the merge to the deterministic `openspec sync` CLI instead of performing agent-driven edits to `specs/`, making the result byte-deterministic and removing the scenario-dropping "intelligent merge" failure mode. The skill handles selection, confirmation, and output only. + +## Follow-up (separate PR) + +Once this primitive is proven, a follow-up PR (`add-spec-tooling-suite`) adds the commands that build on the same baseline and canonicalizer: + +- `openspec unarchive` (+ `/opsx:unarchive`) — the deterministic inverse of archive: byte-exact reverse from the baseline pre-image under a drift guard, atomically. +- `openspec format` — a deterministic, behavior-preserving spec formatter sharing **one canonicalizer** with this engine, so synced/archived specs pass `format --check` by construction. +- `openspec diff` — a spec-aware diff that splices the provenance this PR records + the change's rationale inline; opt-in git diff driver. +- `openspec check` — the unified deterministic linter (`format --check` + `sync --check` + `validate`) plus an opt-in pre-commit hook installer and a CI step — the same binary for both. + +## Impact + +- `src/core/specs-apply.ts` — make `buildUpdatedSpec`/`applySpecs` byte-deterministic and idempotent; add baseline read/write. Forward output unchanged for existing callers. +- `src/core/spec-canonical.ts` (**new**) — extract the deterministic spec/delta canonicalizer (the recomposition `buildUpdatedSpec` already performs at [specs-apply.ts:311-348](../../../src/core/specs-apply.ts)) into one shared module, so the merge engine's output is canonical by construction. Behavior-preserving: `parse(canonicalize(x)) == parse(x)`. (Reused by the follow-up `format`.) +- `src/core/sync.ts` (**new**) — `SyncCommand` (default/`--fix`/`--check`, `--json`), mirroring `ArchiveCommand`'s human + `--json` shape; writes/refreshes the applied-delta baseline; records provenance; incremental `--check`. +- `src/core/archive.ts` — route the merge through the shared engine (already deterministic) and persist the baseline before `moveDirectory` (~414-506). No behavior/output change. +- `src/core/change-metadata/` or a sibling baseline store — persist the applied-delta baseline per change (pre-image + digest, newline-normalized, scheme-tagged). Coordinate the digest convention with [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278)'s digest ledger. +- `src/cli/index.ts` — register `sync [change]` (`--check`, `--fix`, `--all`/`--changes` for CI fan-out, `--json`, `--store`), mirroring archive (326-343). +- `src/core/templates/workflows/sync-specs.ts` — rewrite `/opsx:sync` to invoke `openspec sync` (drop agent-driven edits). `docs/opsx.md` gains a determinism note for `/opsx:sync`. +- Tests — byte-identical sync determinism (same delta+base → same bytes; CRLF/LF; Windows), idempotency (re-run no-op; revised delta no crumbs), `--check` exit codes, provenance + correspondence, incremental verdict == full verdict, skill-template delegation snapshots. Per [openspec/config.yaml](../../config.yaml), run on Windows CI. + +## Issues addressed + +All references verified against `Fission-AI/OpenSpec` at `main` (`546224e`, #1248). Delivers the Discord thread's deterministic-sync conclusion — *"the right fix is a deterministic, code-only openspec sync/archive path that CI can run as a plain binary… no model in CI"* — as `openspec sync` + `--check`. + +Directly fixes / strengthens: + +- [#863](https://github.com/Fission-AI/OpenSpec/issues/863), [#799](https://github.com/Fission-AI/OpenSpec/issues/799), [#656](https://github.com/Fission-AI/OpenSpec/issues/656) — "the sync skill re-implements the merge instead of calling the CLI." `/opsx:sync` becomes CLI-first; the merge is one deterministic code path. +- [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) — the agent "intelligent merge" silently drops scenarios. Deterministic whole-block apply (per the conventions spec's "complete modified requirement, not a diff") removes the failure mode, and the baseline retains the pre-image #1246 wants for drift detection. +- [#1112](https://github.com/Fission-AI/OpenSpec/issues/1112) — MODIFIED/REMOVED/RENAMED-from headers absent from base pass `validate` but abort at archive. `sync --check` surfaces the same appliability check earlier, as a gate. + +Delineated from adjacent work (coordinate, don't collide): + +- [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278) (sibling) — artifact-graph drift (proposal→design→tasks staleness) via a content-digest ledger. This PR is the *spec-merge* drift layer (delta→`specs/`). Same digest philosophy, different layer; share the newline-normalization convention. +- [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md) — planning-time archive ordering. `sync`'s idempotency and cross-change `--check` are the runtime spec-layer counterpart; the two compose. + +Out of scope here (moved to the follow-up PR, with reasons in design.md): `unarchive` and its reverse/drift-guard machinery (Decisions 6–10); the `format` canonicalizer-for-authoring (Decision 14); the spec-aware `diff` driver (Decision 15); the unified `check` gate + hook/CI wiring (Decision 16). Inference-based code-vs-spec verification (the `verify` direction, [#880](https://github.com/Fission-AI/OpenSpec/issues/880)) is out of scope for both. diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-archive/spec.md b/openspec/changes/add-deterministic-spec-sync/specs/cli-archive/spec.md similarity index 100% rename from openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-archive/spec.md rename to openspec/changes/add-deterministic-spec-sync/specs/cli-archive/spec.md diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md b/openspec/changes/add-deterministic-spec-sync/specs/cli-sync/spec.md similarity index 100% rename from openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-sync/spec.md rename to openspec/changes/add-deterministic-spec-sync/specs/cli-sync/spec.md diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/specs-sync-skill/spec.md b/openspec/changes/add-deterministic-spec-sync/specs/specs-sync-skill/spec.md similarity index 100% rename from openspec/changes/add-deterministic-sync-and-unarchive/specs/specs-sync-skill/spec.md rename to openspec/changes/add-deterministic-spec-sync/specs/specs-sync-skill/spec.md diff --git a/openspec/changes/add-deterministic-spec-sync/tasks.md b/openspec/changes/add-deterministic-spec-sync/tasks.md new file mode 100644 index 0000000000..e360a81e91 --- /dev/null +++ b/openspec/changes/add-deterministic-spec-sync/tasks.md @@ -0,0 +1,52 @@ +# Tasks: the deterministic spec-merge engine — `sync` + the applied-delta baseline + +> Phased so the keystone ships first and each phase is independently testable. Phase 1 (engine + baseline + shared canonicalizer) and Phase 2 (`sync` + `--check`) deliver the deterministic, model-free drift path; Phase 3 (idempotency) and Phase 4 (skill delegation) build on the same baseline. Cross-platform concerns (`path.join`, newline-normalized digests) are called out per [openspec/config.yaml](../../config.yaml). The follow-up change `add-spec-tooling-suite` builds `unarchive`, `format`, `diff`, and the unified `check` gate on the baseline established here. + +## 1. The engine: deterministic, byte-stable merge + applied-delta baseline + +- [ ] 1.1 Make the merge byte-deterministic: audit `buildUpdatedSpec` ([src/core/specs-apply.ts:240-307](../../../src/core/specs-apply.ts)) for any nondeterministic ordering/whitespace; guarantee stable requirement ordering and newline normalization so the same (delta + base) yields byte-identical output on macOS/Linux/Windows. +- [ ] 1.1a Extract the canonicalizer: factor the recomposition/normalization `buildUpdatedSpec` performs ([specs-apply.ts:311-348](../../../src/core/specs-apply.ts)) into a shared `src/core/spec-canonical.ts` used by the merge engine (and, in the follow-up, the formatter), so their output cannot diverge. Cover spec files and delta files (ADDED/MODIFIED/REMOVED/RENAMED sections). Behavior-preserving: assert `parse(canonicalize(x)) == parse(x)`. +- [ ] 1.2 Define the **applied-delta baseline** format (per design Decision 1): per affected spec, the pre-merge content (or `absent` marker), a scheme-tagged, newline-normalized digest of the applied result, and provenance (originating change + delta op). Store it with the change (e.g. `.openspec/merge-baseline/`); document the location. +- [ ] 1.3 Add baseline read/write helpers (safe read-modify-write, preserving unrelated fields), coordinating the digest convention with [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278)'s ledger. +- [ ] 1.4 Tests: same delta+base → byte-identical output across repeated runs and simulated CRLF/LF + Windows paths; baseline round-trips through read/write; canonicalizer is behavior-preserving (`parse(canonicalize(x)) == parse(x)`) for spec and delta files. + +## 2. `openspec sync` + the drift gate (the keystone command) + +- [ ] 2.1 Create `src/core/sync.ts` `SyncCommand` mirroring `ArchiveCommand`'s human + `--json` shape (`resolveOpenSpecRoot`, blocked-error → diagnostic, exit codes); default/`--fix` writes `specs/` from the deltas and refreshes the baseline. +- [ ] 2.2 `--check`: read-only; exit non-zero when deltas are not cleanly appliable to the base, or (when `specs/` was synced) when committed `specs/` ≠ regenerated output. Never writes `specs/`. Support `--all`/`--changes` style fan-out for CI (or document the loop). +- [ ] 2.3 Register `sync [change]` in [src/cli/index.ts](../../../src/cli/index.ts) (`--check`, `--fix`, `--all`, `--json`, `--store`, hidden store-path), mirroring archive (326-343). +- [ ] 2.4 Tests: write produces deterministic `specs/`; `--check` clean vs drifted exit codes; `--check` never mutates; unknown/ambiguous change diagnostics; JSON shape on success and each blocked path. +- [ ] 2.5 Provenance: record, with the baseline, the originating change + delta operation for each applied spec change; add an `--explain` (and JSON) output that maps each affected requirement to its source delta. Do not re-author rationale (link to the change's `proposal.md`). (design Decision 12) +- [ ] 2.6 Delta↔spec correspondence in `--check`: fail on an orphan spec edit (attributable to the change but matching no delta op) and on an unapplied delta (delta op not reflected in `specs/`); pass when both directions hold. (design Decision 12) +- [ ] 2.7 Cross-change conflict detection in `--check` (design Decision 11): (a) delta-vs-base — a MODIFIED/REMOVED/RENAMED-from header absent from the current base (surfaces #1112 early); (b) cross-change — two active changes targeting the same requirement; report specifics, non-zero exit, no writes. Coordinate the cross-change check with [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md). +- [ ] 2.8 Tests: provenance recorded + `--explain` mapping; orphan-edit and unapplied-delta both fail `--check`; delta-vs-base conflict fails with the right requirement; two active changes on one requirement flagged; clean case passes. +- [ ] 2.9 Incremental checking (design Decision 13): `--check` skips a spec whose current digest matches the recorded baseline; re-checks on mismatch; forces a full check when the baseline is missing or its scheme is unrecognized. Add an escape hatch to disable skips (e.g. `--no-incremental`) for the equivalence test. +- [ ] 2.10 Tests: unchanged spec skipped; changed spec re-checked; missing/unknown-scheme baseline → full check; **incremental verdict == full verdict** on the same input (the correctness invariant); a touched spec in a large fixture is the only one re-checked. + +## 3. Idempotency & "no crumbs" + +- [ ] 3.1 `sync` on an already-synced, unchanged change writes nothing and `--check` is clean (idempotent no-op). +- [ ] 3.2 Revised-delta re-merge: reverse the prior revision via the baseline pre-image, apply the new delta, refresh the baseline — assert `specs/` equals `apply(current delta, original base)` with no residue from the prior revision. +- [ ] 3.3 Drift-on-resync: if `specs/` drifted from the baseline since last sync, do not silently reverse-then-apply over the edit — report drift and require acknowledgement. +- [ ] 3.4 Tests: double-sync no-op; add→sync→revise(add+remove a requirement)→sync yields exactly the new delta's result, no crumbs; drift-on-resync refusal. + +## 4. `archive` integration + skill delegation (no model in the merge) + +- [ ] 4.1 Route `archive`'s spec merge through the shared engine and persist the applied-delta baseline before `moveDirectory` ([src/core/archive.ts](../../../src/core/archive.ts) ~414-506), so the baseline travels into the archive. No behavior/output change; existing archive tests stay green. +- [ ] 4.2 Rewrite `src/core/templates/workflows/sync-specs.ts` so `/opsx:sync` invokes `openspec sync` (drop the "agent-driven… directly edit main specs" instructions); skill does selection/confirmation/output only. +- [ ] 4.3 Tests: archive still produces identical merged `specs/` and output, and now writes a baseline; `/opsx:sync` template snapshot asserts it calls the CLI and contains no manual merge instructions (anti-#863/#1246 guard). +- [ ] 4.4 Changeset + docs note: `/opsx:sync` is now deterministic; the agent no longer performs scenario-level merges (fixes #1246); author deltas as complete requirements per the conventions. + +## 5. Docs + +- [ ] 5.1 `docs/opsx.md` + CLI docs: document `openspec sync` and its flags; update `/opsx:sync` to note CLI delegation + determinism. +- [ ] 5.2 Note the `specs/` = shipped-reality invariant and why `archive` is retained (design Decision 5), so the "why not just remove archive" question has a documented answer. Point to the follow-up `add-spec-tooling-suite` for `unarchive`/`format`/`diff`/`check`. + +## 6. End-to-end verification + +- [ ] 6.1 E2E determinism: scaffold a change with ADDED/MODIFIED/REMOVED/RENAMED; `openspec sync` twice → byte-identical `specs/`; `sync --check` clean. +- [ ] 6.2 E2E no-crumbs: sync, revise the delta, re-sync → `specs/` reflects only the current delta. +- [ ] 6.3 E2E drift gate: introduce an un-synced delta → `openspec sync --check` fails naming the drift → `sync --fix` remediates → `--check` passes; then a delta-vs-base conflict still fails (not auto-fixable). +- [ ] 6.4 E2E archive baseline: `archive` a change → merged `specs/` unchanged from today's output, and a baseline is recorded in the archived folder (the artifact the follow-up `unarchive` reverses from). +- [ ] 6.5 Validation: `openspec validate add-deterministic-spec-sync --strict` passes; `openspec status` shows artifacts complete. +- [ ] 6.6 Run the suite on macOS, Linux, and Windows CI. diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/design.md b/openspec/changes/add-deterministic-sync-and-unarchive/design.md deleted file mode 100644 index b157771dcf..0000000000 --- a/openspec/changes/add-deterministic-sync-and-unarchive/design.md +++ /dev/null @@ -1,195 +0,0 @@ -# Design: deterministic spec tooling — `sync`, `unarchive`, `format`, `diff`, and the `check` gate - -## Context - -OpenSpec's spec merge (delta → `specs/`) is reached two ways today: deterministically but only *inside* `openspec archive` (fused to the folder move), or by an AI agent in `/opsx:sync` that "directly edits main specs." The standalone deterministic apply (`applySpecs()`, [src/core/specs-apply.ts:391](../../../src/core/specs-apply.ts)) has zero callers. The design discussion (2026-06) worked from "add unarchive" to a single root cause: **the merge must be deterministic, idempotent, and reversible pure code, exposed as commands** — then unarchive, a code-only CI drift gate, a `sync --fix` pre-commit hook, and crumb-free re-merge all fall out of one primitive. - -This design covers that primitive and the commands built on it. It supersedes the earlier "unarchive + separable CI companion" framing of this PR: the deterministic engine is no longer a companion, it is the spine. - -## Goals / Non-Goals - -**Goals** -- A deterministic merge: same (delta + base) → byte-identical `specs/`, in pure code, on every platform. -- Idempotent: re-running `sync` is a no-op; revising a delta regenerates `specs/` with no crumbs. -- Reversible: `unarchive` restores `specs/` exactly and moves the folder back, atomically. -- Code-only drift detection (`sync --check`) usable by CI and a pre-commit hook — no model, no API keys. -- Early, deterministic conflict detection (delta-vs-base and cross-change) at commit/PR time, not at archive (Decision 11). -- Per-edit provenance and a delta↔spec correspondence check, within the delta model (Decision 12). -- Hash-optimized incremental checking: `--check` re-checks only specs whose digest changed, without changing any verdict (Decision 13). -- A deterministic, behavior-preserving spec formatter (`openspec format` + `--check`) sharing one canonicalizer with the merge engine (Decision 14). -- A deterministic spec-aware diff (`openspec diff`, opt-in git diff driver) that splices provenance + rationale inline, reusing existing artifacts (Decision 15). -- A unified deterministic linter `openspec check` (+ `--fix`, `--all`, incremental) wired to run identically as a pre-commit hook and a CI gate, with an opt-in hook installer and a CI template (Decision 16). -- Skills delegate the merge to the CLI; the agent never performs it. -- Backward compatible with changes archived (and specs synced) before this feature. - -**Non-Goals** -- Removing or replacing the `archive` lifecycle boundary, or editing `specs/` in place instead of via deltas (Decisions 5, 11). -- Folding the spec merge into `apply` (Decision 5). -- Building the spec-aware git diff driver (Decision 12 — provenance data is in scope; the diff driver is a deferred follow-up). -- Inference-based "align spec to implementation" / code-vs-spec checking — that needs a model and is the `verify` direction (#880), not the deterministic `format` (Decision 14). -- Reducing the committed artifact set (commit-only-the-spec) — a separate product question served by schema flexibility (Decision 13). -- A separate "semantic delta" / reasoning-log database — unnecessary; `openspec diff` renders from the existing proposal + provenance (Decision 15). -- Semantic/AI summarization in the diff — `diff` mechanically splices recorded reasoning; model-based review is the `verify` direction (Decision 15). -- Wiring a specific hook runner or CI job into this repo (primitives in scope; config staged — Decision 4). -- A scenario-granular "smart" merge (Decision 8 — the conventions mandate whole-requirement deltas; whole-block apply is the correct, deterministic semantics). -- Bulk `sync`/`unarchive`, archive-folder prefix scheme, lifecycle timestamps — out of scope, accommodated. - -## Decision 1 — One primitive: the applied-delta baseline - -**Decision.** When a change's deltas are merged into `specs/` (by `sync` or `archive`), record a per-change **applied-delta baseline**: for each affected spec, the **pre-merge file content** (the pre-image, or an explicit `absent` marker when the spec is created) and a **digest of the applied result** (newline-normalized CRLF→LF, scheme-tagged). The baseline lives with the change; when archived, it travels into the archive folder. - -**Why one primitive.** The three asks in the thread are the same computation viewed three ways: -- **Reverse** (`unarchive`) = restore the pre-image. Deterministic for *every* op, including the REMOVED/MODIFIED ones the delta alone cannot invert. -- **Idempotent re-merge** (`sync` after a delta revision) = `specs_new = apply(delta_new, base)` where `base = current specs with delta_old reversed` (via the pre-image). This is the "no leftover crumbs" guarantee — the prior revision's contribution is removed, not layered over. -- **Drift** (`sync --check`, the hook, CI) = current `specs/` digest ≠ baseline digest. This is the discussion's "track the hash of the changes/ state applied to the spec," made precise. - -Building three mechanisms would invite three drift bugs. Building one — the pre-image + digest — and deriving the rest is why this is the spine. - -**Form.** Store the whole pre-image (not a reverse-diff), so restore is a byte copy with no re-parsing. Cost is a copy of each affected spec's prior bytes per merge — negligible, and only for changes that touch specs. Coordinate the digest convention with [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278)'s artifact-graph digest ledger so the two drift layers (artifact staleness vs. spec-merge drift) speak the same digest dialect. - -## Decision 2 — Deterministic sync is pure parser code; the agent never merges - -**Decision.** Promote `applySpecs()`/`buildUpdatedSpec` into the canonical merge and expose it as `openspec sync`. The merge is a pure function of (delta files + base spec bytes) producing byte-identical output across runs and platforms (stable requirement ordering, newline normalization, deterministic recomposition). `archive` calls the same function. `/opsx:sync` is rewritten to **invoke the CLI**, not edit specs itself. - -**Why.** This is the thread's keystone verbatim: *"Moving it out of the agent prompt and into plain parser code that always produces byte-identical output for the same delta + base."* The agent path isn't just non-deterministic; its stated reason for existing — "intelligent merging (e.g., adding a scenario without copying the entire requirement)" — is precisely how [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) silently drops sibling scenarios. The conventions spec already requires deltas to carry "the complete modified requirement, not a diff," so whole-block apply is the *correct* semantics and the agent's cleverness is the bug. Deterministic code is therefore more faithful, not less (Decision 8). - -**Alternatives.** Keep agent sync and only diff its output in CI — rejected: needs a model in CI, the exact thing the thread set out to avoid. - -## Decision 3 — Idempotency and "no crumbs" - -**Decision.** `sync` is idempotent: running it on an already-synced, unchanged change writes nothing (`--check` is clean). When a delta is revised, `sync` reverses the prior revision's contribution via the baseline pre-image, then applies the new delta, then refreshes the baseline — so `specs/` equals `apply(current delta, original base)` with zero residue from the old revision. - -**Why.** Directly answers *"a revised delta should let sync regenerate the spec output from scratch, idempotently, no leftover crumbs."* Without the baseline, re-applying a revised delta against already-merged `specs/` either conflicts (ADDED now exists) or layers (renamed-then-renamed) — crumbs. The baseline makes regeneration a clean reverse-then-apply. - -**Edge.** If `specs/` drifted from the baseline since the last sync (a later change touched the same requirement), `sync` does not silently reverse-then-apply over someone else's edit — it reports drift and requires `--check`-style acknowledgement, mirroring `unarchive`'s drift guard (Decision 7). Consistency over cleverness. - -## Decision 4 — `--check` and `--fix`: the gate and the auto-fixer (wiring staged) - -**Decision.** `sync --check` is read-only and exits non-zero when the change's deltas are not cleanly appliable to the base, or — when the team commits merged `specs/` during review — when committed `specs/` ≠ the regenerated output. `sync --fix` (and bare `sync`) regenerates `specs/`. The per-tool `--check`/`--fix` modes are the primitives; the unified gate over them is `openspec check` (Decision 16), which is what pre-commit and CI actually invoke. - -- **CI drift gate**: a job runs `openspec check` and fails the PR on drift — "the same pattern as a codegen or IaC drift gate," as a plain binary, no model. -- **pre-commit hook**: `openspec check` detects drift before commit; `openspec check --fix` is the eslint-`--fix`-style auto-remediation, after which the commit proceeds with `specs/` canonical and in sync. - -**What ships here vs. what stays policy.** Earlier revisions deferred all wiring. This revision **builds the integration** (Decision 16): the unified `openspec check` command, a runner-agnostic, opt-in hook installer, and a copy-paste CI step. What remains the owner's choice is only *which* runner to standardize on and whether to enable the gate by default — not whether the capability exists. - -## Decision 5 — Keep the `archive` boundary; reject folding the merge into `apply` - -**Decision.** The lifecycle boundary stays: `specs/` is written at `sync`/`archive`, and `archive` remains the "fold finished change into shipped specs, then move the folder" step. Reject the proposal to remove `archive`, keep every change in a dated archive dir for its whole life, and make `apply` do both code and spec application. - -**Why.** The invariant that pays for OpenSpec's value is **`specs/` describes only shipped reality**. Folding the merge into `apply` would (a) pollute the source of truth with proposed-but-unmerged or abandoned changes, and (b) let parallel changes overwrite each other's specs before any lands. The thread's own resolution: the awkwardness of the "final archive step" is not the boundary's fault, it is the *non-determinism* of crossing it. Make the crossing deterministic and reversible (Decisions 1–3) and the boundary becomes cheap in both directions — which is the actual fix. This is why determinism, not deletion, is the spine of this PR. - -**Consequence for `sync` on active changes.** Because of the invariant, the *default* workflow still merges at archive. Standalone `sync` serves: the engine archive uses; `--check` (read-only, never pollutes `specs/`); and the opt-in "generated-artifact" workflow where a team chooses to commit merged `specs/` mid-review and gate it. The tool enables both policies; it does not force early merge. - -## Decision 6 — `unarchive`: resolution, prefix-tolerance, never auto-pick - -**Decision.** `unarchive [change-name]` accepts a bare `` or a full archived directory id, treating the leading prefix as **opaque up to ``** — strips `YYYY-MM-DD-` today, tolerates `NNN-`/ISO/configurable ([#409](https://github.com/Fission-AI/OpenSpec/issues/409)/[#787](https://github.com/Fission-AI/OpenSpec/pull/787)/[#1192](https://github.com/Fission-AI/OpenSpec/issues/1192)). Multiple matches for a bare name → interactive prompt (most-recent first, never auto-select); `--json`/non-interactive → error listing candidates, require the full id. Reuse `getArchivedChangesData()` ([#399](https://github.com/Fission-AI/OpenSpec/pull/399)). - -**Why.** Multiple archives of one name already happen (different dates), and the proposed sequence scheme makes it routine. Silent selection is a guess on the rare, costly path. - -## Decision 7 — Reverse under a drift guard, atomically - -**Decision.** Before reversing any spec, compare its current content to the baseline's applied-result digest. On drift (a later change touched the same requirement — [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246), [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md)), **refuse** and point to `--keep-specs`. The reversal runs as a defined **validate → stage → commit** sequence: (1) validate destination-absent + all affected specs present and drift-free; (2) compute reversed specs into a temporary staging area under `.openspec/` (no change to `specs/` yet); (3) swap staged specs into `specs/`; (4) move the folder. A failure in steps 1–2 leaves everything untouched; a failure of step 4 after the swap rolls `specs/` back from the still-present baseline. On any failure: *"Abort. No files were changed."*; if rollback itself cannot complete, report the partial state and exact recovery steps rather than leave a silent inconsistency. - -**Why.** Restoring a pre-image over a requirement a later change modified would silently delete that change's contribution — the loss OpenSpec guards against. Refusing is strictly safer, and `--keep-specs` always lets the user proceed. Atomicity gives archive the rollback [#682](https://github.com/Fission-AI/OpenSpec/issues/682) noted it lacks, and matches the abort contract [#1112](https://github.com/Fission-AI/OpenSpec/issues/1112) describes. - -## Decision 8 — `--keep-specs` and naming vs. `--skip-specs` - -**Decision.** `--keep-specs` moves the folder back and leaves `specs/` untouched — always deterministic, always safe. Primary spelling `--keep-specs` (the request's; reads naturally for the inverse). Accept `--skip-specs` as a hidden alias for muscle-memory symmetry with archive ([#28](https://github.com/Fission-AI/OpenSpec/pull/28)). When `unarchive` cannot safely reverse specs (drift, or pre-baseline REMOVED/MODIFIED) and `--keep-specs` was not given, it refuses and *recommends* `--keep-specs` — it does not silently fall back. **Open for owner confirmation:** if strict symmetry is preferred, swap which spelling is primary. - -## Decision 9 — Backward compatibility: pre-baseline archives and synced specs - -**Decision.** Changes archived/synced before baselines existed have none. Pre-baseline reversal is **all-or-nothing**: if every delta operation across all affected specs is ADDED/RENAMED (self-invertible), `unarchive` reverses the whole change by delta inversion; if any MODIFIED/REMOVED is present anywhere (pre-image unrecoverable), it **refuses the entire spec reversal** — naming the requirements it cannot restore and directing to `--keep-specs` (optionally offering opt-in git recovery of the pre-image). It does **not** partially reverse the self-invertible specs and leave the rest, because a partial `specs/` is exactly the corruption the atomic guarantee (Decision 7) exists to prevent. `sync` on a pre-baseline change establishes a baseline on its next run. Never silently emit a wrong spec. - -**Why.** The feature must be useful on day one, including for already-archived changes, without ever degrading into corruption. Reverse what is provably reversible; stop honestly on the rest. - -## Decision 10 — Move strategy and lifecycle metadata - -- **Move**: reuse `moveDirectory()`/`copyDirRecursive()` ([src/core/archive.ts:96-128](../../../src/core/archive.ts)) and their Windows fallbacks ([#605](https://github.com/Fission-AI/OpenSpec/pull/605)). A future `git mv` ([#709](https://github.com/Fission-AI/OpenSpec/issues/709)) covers both directions through this one helper. -- **Metadata**: archive persists no `archived` timestamp today (only the folder-name prefix). If [#1245](https://github.com/Fission-AI/OpenSpec/issues/1245) lands one, `unarchive` should null it on restore — noted, not built. - -## Decision 11 — Keep deltas; don't edit specs in place — and get early conflict resolution anyway - -**Context.** The discussion pushed a deeper question than "remove archive": *are we simulating deltas above git when git already stores deltas?* The proposed alternative: edit `specs/` in place (the git diff **is** the delta), drop the delta folder, and keep the "why" in a sidecar reasoning log; `sync` becomes a lint that checks spec-edits against log entries. Its sharpest argument is about **timing**: if in-flight changes are deltas applied at archive, conflicts surface late, at merge; editing the spec first "forces conflict resolution immediately." - -**Decision.** Keep the delta layer; do **not** make in-flight edits directly to `specs/`. But **adopt the timing argument's goal** by moving conflict detection earlier with `sync --check` (Decision 12). - -**Why keep deltas.** The delta layer is exactly what a raw git diff cannot give you: a clean separation between **proposed** and **shipped**. `specs/` always describes reality; multiple in-flight changes stay isolated and independently reviewable until each lands. Edit-in-place collapses that — proposed-but-unmerged or abandoned edits sit in the source of truth, and N parallel changes mutate the same files, so the only place conflicts can be resolved is one big merge at the end. The delta model makes each change a self-contained, reviewable unit and is what makes `unarchive` (reverse one change) and isolation-preserving parallelism *possible at all*. Git stores byte deltas; OpenSpec's deltas are behavioral agreements one level up — they answer *why* and *what-should-be* before code exists, which a diff cannot. - -**The synthesis — adopt the valid kernel.** The timing concern is real and we take it: rather than discovering at archive that a delta no longer applies (or that two changes touched the same requirement), `sync --check` surfaces those conflicts at commit/PR time, deterministically, as a plain binary (Decision 12, "Cross-Change Conflict Detection"). So we get "resolve conflicts immediately" **without** sacrificing proposed-vs-shipped isolation. This is the same shape as Decision 5: adopt the goal, reject the mechanism that would break the invariant. - -## Decision 12 — Provenance, delta↔spec correspondence, and the spec-aware diff driver (deferred) - -**Context.** The discussion also wanted the "why" to travel with the "what": a reasoning log spliced inline so a reviewer sees *what changed and why together* ("git as the delta store plus a spec-aware diff driver that splices the reasoning log inline"), and a consistency lint where a spec change with no log entry — or a log entry with no spec change — fails. - -**Decision.** Record **provenance** as part of the applied-delta baseline: when the engine writes a spec change, it records which change and which delta operation produced it. Expose it (e.g. `openspec sync --explain` / a provenance entry). Use it for a deterministic **delta↔spec correspondence** check in `sync --check`: every committed `specs/` edit for a change must trace to one of its delta operations (no orphan edits), and every delta operation must have landed (no unapplied deltas). The prose "why" is not re-authored — provenance links each spec edit to its change, whose `proposal.md` already holds the rationale. - -**In scope (Decision 15): the spec-aware diff driver.** The presentation side — a `git diff` driver for spec files that follows the provenance link and splices the change's rationale inline — is now folded in rather than deferred, because it consumes exactly the provenance this PR already records and needs no new data store. See Decision 15. - -**Why this is the right slice.** Provenance falls out of the merge for free (the engine already knows exactly what it applied), and correspondence reuses the baseline. Together they answer the discussion's consistency lint *within the delta model* (deltas are the source; specs are generated) rather than inverting it (specs as source, deltas as sidecar log) — which would reintroduce the edit-in-place problems of Decision 11. - -## Decision 13 — Spec-as-source-of-truth, incremental checking, and the spec linter - -**Context.** The discussion's end-state wish: treat *the spec* as the single source of truth ("new source code"), iterate on it directly, let tools "align the spec to implementation like a linter/formatter would," using "hashes to optimize when checks are needed," and reduce what gets committed to just the spec — treating proposal/design/tasks as "artifacts of an individual's workflow, not the product to commit." The framework's job becomes "how the spec looks and reads and is organized/formatted." - -**Decision.** Adopt the one part that is squarely in this PR's scope — **hash-optimized incremental checking** — and explicitly position the rest (spec linter/formatter; reduced artifact set) as adjacent, not folded in. - -**Adopt: incremental checking.** The applied-delta baseline already carries a per-spec digest. So `sync --check` (and the CI/pre-commit gate) can skip any spec whose content digest is unchanged since it was last reconciled, and re-check only what changed — exactly "use hashes to optimize when checks are needed." The correctness invariant: a skip is allowed *only* on an exact digest match against a recorded baseline; a mismatch, a missing baseline, or an unknown digest scheme forces a full check. So incremental mode is an optimization that can never change a verdict versus a full check — important for a gate. This makes the gate cheap enough to run on every commit even in a repo with hundreds of specs. - -**Reaffirm (with nuance): deltas + planning artifacts stay.** The wish to commit "only the spec" and treat design/tasks as disposable runs against the reason those artifacts exist: in OpenSpec they *are* the reviewable, resumable product — the agreement about behavior *before* code, and the context a fresh session (or a reader six months later) needs. The delta layer is, again, what keeps *proposed* separate from *shipped* and lets parallel changes stay isolated and individually reversible (Decisions 5, 11). The valid kernel — that not every change needs the full artifact set — is real but is already served *outside this PR* by schema flexibility (a change can run a minimal, spec-only schema); this PR neither needs nor changes that. Recorded here so the "just commit the spec" question has a documented answer. - -**In scope (Decision 14): the spec formatter.** "How the spec looks, reads, and is organized" — a deterministic formatter (prettier-for-specs) with a `--check` gate — is now folded into this PR rather than deferred, because it is the same canonicalization the merge engine already performs, exposed for authoring. See Decision 14. - -**Still out of scope: the artifact-model change.** Reducing the committed artifact set (commit-only-the-spec) is a separate product question, served by schema flexibility, not this PR. - -## Decision 14 — The deterministic spec formatter (`openspec format`), in scope - -**Context.** The discussion wanted the framework's job to include "how the spec looks and reads and is organized/formatted" — a linter/formatter, hash-optimized. Earlier this was deferred (an authoring concern, seemingly separable). Reconsidered: the merge engine *already* computes a canonical form for every spec it writes (deterministic recomposition, newline normalization, canonical spacing). A standalone formatter is that same canonicalizer pointed at authoring input. The marginal surface is small and the conceptual fit is exact, so it belongs in this PR. - -**Decision.** Add `openspec format [target]` — a deterministic, pure-code formatter for spec and delta files — with `--check` (read-only gate) and `--fix`/default (write). It shares one canonicalizer with `sync`/`archive`, so **the merge engine's output is, by construction, exactly what the formatter produces**: synced/archived specs always pass `format --check`. It is incremental (Decision 13's digest skip) and `--json`-capable, and it joins `sync --check` as a model-free gate the CI job and pre-commit hook run. - -**Behavior-preserving — the hard line.** The formatter changes only *presentation*: whitespace, blank-line policy, list markers/indentation, heading spacing, canonical delta-section headers. It MUST NOT reorder requirements or scenarios (order can carry meaning), rewrite prose, or add/remove/merge/split requirements. The invariant is testable: parse-before == parse-after (same requirements, scenarios, and delta operations); only surrounding whitespace may differ. This is what separates a *formatter* (safe, deterministic, automatic) from a *rewrite* (semantic, requires review). - -**What it is NOT.** Not `validate`: `validate` judges *semantic* validity (a requirement has a scenario, headers resolve), `format` judges *canonical form* (is it laid out canonically). They compose — `format` then `validate`. Not the inference-based "align spec to implementation" the discussion also mentioned: checking that *code* matches the spec needs a model and is the `verify` direction ([#880](https://github.com/Fission-AI/OpenSpec/issues/880)), explicitly out of scope here. `format` is pure text canonicalization, so it needs no agent and no `/opsx:format` skill — you run the binary (or the hook runs it). - -**Why now, not later.** Folding it in means one canonicalizer is specified and tested once and reused by `sync`, `archive`, and `format`, guaranteeing they cannot diverge. Deferring it would risk a future formatter that disagrees with the merge engine's output — the exact drift this PR exists to kill. - -## Decision 15 — The spec-aware diff driver (`openspec diff`), in scope - -**Context.** The discussion's review workflow: "review primarily becomes reading the spec deltas (in git)" with "a spec-aware diff driver that splices the reasoning log inline so reviewers see what changed and why together," and the related idea of a sidecar database of "semantic deltas as log entries." - -**Decision.** Add `openspec diff [target] [--base ]` — a deterministic, pure-code renderer that shows requirement-level spec changes with each change's provenance and rationale spliced inline. It is usable standalone and, opt-in, as a **git diff driver** (registered in `.gitattributes`) so `git diff` over spec/delta paths renders the annotated view. - -**Key insight — OpenSpec already has the "log"; no new store.** The discussion reached for a separate database of semantic deltas plus a reasoning log. But OpenSpec already keeps both halves: the *why* lives in the change's `proposal.md`, and the *what/where* lives in the applied-delta provenance this PR records. The diff driver simply joins them. So we explicitly **reject building a sidecar reasoning database** (Decision 11's edit-in-place family) and instead render from artifacts that already exist — less to maintain, nothing to keep in sync. - -**Deterministic and honest.** Rendering is a pure function of the git diff + recorded provenance + proposal text — byte-identical across runs and platforms, no inference. When a change cannot be attributed (a pre-baseline edit with no provenance), the diff says so rather than inventing a rationale. It does **not** judge review quality or summarize semantically (that would need a model); it mechanically splices recorded reasoning. Git config is never modified without explicit opt-in. - -**Why in scope now.** It is the presentation layer of the provenance this PR already produces, and it completes the discussion's review vision deterministically. Like the formatter (Decision 14), it reuses primitives this PR establishes rather than introducing new ones. - -## Decision 16 — `openspec check`: the unified deterministic linter for pre-commit *and* CI - -**Context.** The discussion's recurring ask: the sync/format gate should run "like a linter," both as a pre-commit hook (eslint-`--fix` style) and as a CI drift gate, with no model in CI. The question is whether the deterministic linter works for pre-commit *or* post-commit (CI). **Answer: both — it is the same binary invoked in two places.** A drift gate is a pure function of the committed files; *where* it runs (a local hook before the commit, or a CI job after) changes nothing about the verdict. - -**Decision.** Add `openspec check` — one command that runs the deterministic gates (`format --check`, `sync --check`, `validate`) and exits non-zero on any failure, with `--fix` for auto-remediation, `--all` for cross-change checks, incremental digest skipping, and `--json`. Ship the two integration points as well: a **runner-agnostic, opt-in hook installer** and a **copy-paste CI step**. This is the "deterministic linter" the discussion converged on, and it is the single entrypoint pre-commit and CI both call (so neither has to know the individual sub-checks). - -**Why one command, not three.** A hook or CI job shouldn't hard-code `format --check && sync --check && validate` and drift out of step with the toolchain. `openspec check` is the stable contract; the sub-gates can evolve behind it. It also gives one exit-code/`--json` shape for both contexts. - -**Pre-commit vs CI — same gate, complementary placement.** Pre-commit catches drift at authoring time and can `--fix` in place before the commit (fast feedback, fewer red CI runs). CI is the backstop that enforces the gate regardless of local setup (someone without the hook, or `--no-verify`). Because the verdict is identical and deterministic, the two never disagree — CI cannot fail something a correctly-installed hook would have passed. Recommended posture: hook runs `--fix` (convenience), CI runs `--check` (enforcement). - -**Honesty under `--fix`.** `--fix` only applies *mechanical* remediation (format + sync regeneration). Non-mechanical failures — a delta that doesn't cleanly apply, a cross-change conflict — are reported and still fail; `--fix` never invents a resolution or masks a real conflict. - -**Scope note.** This builds the deferred CI/hook items because they directly serve the deterministic-linter goal (the user's explicit ask). The one thing still left to the owner is policy: which runner to standardize and whether the gate is on by default. The inference-based "align spec to implementation" remains out (it needs a model — the `verify` direction, #880). - -## Scope completeness - -This proposal now covers **every deterministic, in-scope idea** the discussion raised: the merge engine and applied-delta baseline (forward `sync`, reverse `unarchive`, the `archive` baseline), idempotent crumb-free re-merge, model-free `--check` drift gating, early conflict detection, provenance + delta↔spec correspondence, incremental checking, the canonical formatter, the spec-aware diff driver, and the unified `openspec check` linter wired for both pre-commit and CI. The ideas it deliberately leaves out are documented with reasons: editing specs in place / dropping the delta layer / committing only the spec (Decisions 5, 11, 13), and inference-based code-vs-spec verification (Decisions 14, 16 — the `verify` direction, #880). The only thing left to the owner is **policy**, not capability: which hook runner to standardize and whether the gate is on by default. Subsequent sessions should therefore shift from **expansion** to **sequencing and refinement** for owner review — confirming the open decisions (e.g. `--keep-specs` vs `--skip-specs`, the `/opsx:sync` behavior change) and the phase order — rather than adding surface. - -## Risks / trade-offs - -- **Behavior shift for `/opsx:sync` users.** Replacing agent merge with deterministic merge changes output for anyone who relied on the agent's scenario-level merges. This is intended (it fixes [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)), but it is a real change; call it out in the changeset and docs, and keep the conventions' "complete requirement, not a diff" rule prominent so deltas are authored to merge cleanly. -- **Baseline storage in the change folder.** Adds a small artifact; inert for spec-less changes. Format is scheme-tagged so canonicalization can evolve without silent mis-compares. -- **Forward round-trip is not guaranteed identity.** `unarchive` restores the exact pre-archive `specs/`; a *subsequent* re-archive re-runs the forward merge, which inherits archive's existing cross-change caveats ([#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)). Documented, not solved here. -- **Scope.** This is larger than a lone `unarchive` command. It is intentionally the foundational engine the thread converged on; tasks phase it so the keystone (engine + `sync --check`) is independently shippable before reverse, idempotency, and skill delegation. - -## Migration / rollout - -Additive and phased (see tasks). The engine + baseline land first (no user-visible change to `archive` output). `sync`, `unarchive`, `format`, `diff`, and `check` are new commands in the expanded profile. `/opsx:sync` delegation ships with a changeset noting the determinism shift. Pre-baseline changes degrade per Decision 9. The `openspec check` gate, opt-in hook installer, and CI template ship in this PR (Decision 16); enabling the gate by default is the owner's policy call. diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md b/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md deleted file mode 100644 index 11a654a804..0000000000 --- a/openspec/changes/add-deterministic-sync-and-unarchive/proposal.md +++ /dev/null @@ -1,132 +0,0 @@ -## Why - -The Discord thread that asked for `openspec unarchive` ended somewhere bigger: the spec merge (delta → `specs/`) is **the** load-bearing operation in OpenSpec, and today it is reachable only two ways — buried *inside* `openspec archive`, or performed by an **AI agent** in `/opsx:sync` ("This is an agent-driven operation… you will read delta specs and directly edit main specs"). Both are problems. The agent path is non-deterministic (wording drift, and the "intelligent" scenario-merge is the very mechanism that silently drops scenarios in [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246)). The CLI path is deterministic but cannot be run on its own — `applySpecs()` has **zero callers**. - -So you cannot reverse an archive, you cannot gate drift in CI without a model, and you cannot re-merge a revised delta cleanly. One missing primitive sits under all three. This PR builds it: a deterministic, idempotent, **reversible** merge engine, exposed as first-class commands. - -## Background: one engine, three missing directions - -Verified against `Fission-AI/OpenSpec` at `main` (`546224e`, #1248) on 2026-06-29. The merge `buildUpdatedSpec` ([src/core/specs-apply.ts:240-307](../../../src/core/specs-apply.ts)) applies a change's deltas to a base spec in fixed order (RENAMED → REMOVED → MODIFIED → ADDED). That engine is missing three things the workflow needs: - -1. **A standalone forward command.** `openspec sync` does not exist as a binary. The deterministic merge runs only inside `archive` (fused to the folder move) or is re-implemented by the agent in `/opsx:sync`. So the Discord conclusion — *"a deterministic, code-only sync path that CI can run as a plain binary… no model in CI"* — is impossible today. CI can only gate drift by re-running archive or invoking the LLM. - -2. **A reverse direction.** Archiving merges deltas into `specs/` and moves the folder to `changes/archive/`. There is no undo. The maintainer's manual fix (`git revert`, or a hand-scoped `git checkout -- …`) fails in the motivating case — *"the archive is part of a commit with other changes, so I cannot simply revert."* And the reverse is not free: the archived delta is **not self-inverting**. ADDED and RENAMED can be undone from the delta; **REMOVED and MODIFIED cannot** — the forward merge discards the pre-image (a MODIFIED delta carries "the complete modified requirement, not a diff"; [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) documents the same whole-block replace as forward data loss). - -3. **Idempotent regeneration ("no crumbs").** When review feedback revises a delta, re-merging should regenerate `specs/` from scratch with no leftovers from the prior revision — *"a revised delta should let sync regenerate the spec output from scratch, idempotently, no leftover crumbs."* Today re-applying a revised delta against already-merged `specs/` conflicts or double-applies; there is no record of what the prior revision contributed. - -**The unification.** All three need the *same* missing primitive: a per-change **applied-delta baseline** — the pre-image of each affected spec plus a digest of the applied result, recorded whenever a change's deltas are merged. With it: - -- **reverse** is a byte-exact restore of the pre-image (deterministic for every op, including REMOVED/MODIFIED) → `unarchive`; -- **idempotent re-merge** is "reverse the old delta via the baseline, apply the new" → crumb-free `sync`; -- **drift detection** is "current `specs/` digest ≠ baseline digest" (the discussion's "track the hash of the changes/ state applied to the spec") → a code-only gate for CI and a `sync --fix` pre-commit hook (the eslint-`--fix` analogy the discussion arrived at). - -Frozen into the archived folder, that baseline *is* the unarchive reversal snapshot. One primitive, three payoffs. - -## What Changes - -Design principle the whole thread converges on, and the one this project is already adopting elsewhere ([#1278](https://github.com/Fission-AI/OpenSpec/pull/1278), prevent-silent-spec-drop): **the merge is pure parser code that produces byte-identical output for the same delta + base; the agent never performs it.** Ordered by leverage: - -1. **THE ENGINE — deterministic, idempotent, reversible merge core (`cli-sync` NEW, shared by archive).** Promote the existing `applySpecs()` into a first-class, **byte-deterministic** apply that records an applied-delta baseline, and add its inverse. `openspec archive` keeps merging-then-moving but routes its merge through this shared core and writes the baseline before moving (so it travels into the archive). This is the keystone the other three items stand on; it is the "rework it… plain parser code that always produces byte-identical output for the same delta + base" the thread calls for. - -2. **FORWARD COMMAND + DRIFT GATE — `openspec sync [change]` (`cli-sync` NEW).** Apply a change's deltas to `specs/` without archiving, deterministically and idempotently. Three modes: - - default / `--fix`: write `specs/` to the regenerated result (idempotent — re-running is a no-op; revising the delta regenerates with no crumbs); - - `--check`: read-only; exit non-zero if the change's deltas are not cleanly appliable, or (when `specs/` has been synced) if committed `specs/` ≠ the regenerated output. This is the **codegen/IaC-style drift gate CI runs as a plain binary, no model, no API keys** — and the auto-fixer a `pre-commit` hook can call. - - `--check` also surfaces conflicts **early** — at commit/PR time instead of at archive: a delta that no longer applies to the current base, or two active changes targeting the same requirement. This is the discussion's "force conflict resolution immediately rather than at a later merge step," delivered without abandoning the delta model (design Decision 11). And the engine records **provenance** — which change and delta op produced each spec edit — so every line in `specs/` traces to a delta, enabling a deterministic delta↔spec correspondence check (orphan edit / unapplied delta → fail) and a future spec-aware diff driver (design Decision 12). Because the baseline carries a per-spec digest, `--check` is **incremental** — it skips specs whose content is unchanged and re-checks only what moved (the discussion's "use hashes to optimize when checks are needed"), so the gate stays cheap on every commit even in a large repo (design Decision 13). - -3. **REVERSE COMMAND — `openspec unarchive [change-name]` (`cli-unarchive` NEW).** The deterministic inverse of archive: resolve the archived folder (prefix-tolerant, never auto-picking among ambiguous matches), reverse the spec merge from the baseline under a **drift guard** (refuse rather than clobber a requirement a later change has since touched), move the folder back to `changes//`, **atomically** ("Abort. No files were changed."). `--keep-specs` restores the folder without touching `specs/` (the always-safe escape hatch, mirror of archive's `--skip-specs`). Changes archived before this feature have no baseline and degrade gracefully — reverse the self-invertible half (ADDED/RENAMED), refuse to guess the rest. - -4. **NO MODEL IN THE MERGE — skills delegate to the CLI (`specs-sync-skill` MODIFIED; `opsx-unarchive-skill` NEW).** `/opsx:sync` stops doing agent-driven edits and **invokes `openspec sync`**; the new `/opsx:unarchive` invokes `openspec unarchive`. The deterministic work lives in TypeScript; skills only select, confirm, and render. This is the direction [#863](https://github.com/Fission-AI/OpenSpec/issues/863)/[#799](https://github.com/Fission-AI/OpenSpec/issues/799)/[#656](https://github.com/Fission-AI/OpenSpec/issues/656) ask for, and it removes the [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) "intelligent merge drops scenarios" failure mode by construction. - -5. **THE FORMATTER — `openspec format [target]` (`cli-format` NEW).** The discussion's "the framework's goal is how the spec looks and reads and is organized" — delivered as a deterministic, pure-code **formatter** with a `--check` gate, sharing **one canonicalizer** with the merge engine. The engine already computes a canonical form for everything it writes; `format` points that same canonicalizer at authoring input (spec and delta files), so synced/archived specs are canonical *by construction* — they always pass `format --check`. It is **behavior-preserving** (normalizes whitespace, blank-line policy, list markers, heading spacing, canonical delta-section headers; never reorders requirements/scenarios, never rewrites prose — `parse-before == parse-after`), **incremental** (digest skip, Decision 13), and needs no agent (`format` is pure text, so there is no `/opsx:format` skill — you run the binary or the hook does). It joins `sync --check` as the second model-free gate. It is *not* `validate` (semantic validity) and *not* code-vs-spec `verify` (inference); those compose around it (design Decision 14). - -6. **THE REVIEW VIEW — `openspec diff [target]` (`cli-diff` NEW).** The discussion's "review primarily becomes reading the spec deltas, with a spec-aware diff driver that splices the reasoning log inline so reviewers see what changed and why together." Delivered as a deterministic, pure-code renderer that shows requirement-level changes with each change's **provenance and rationale spliced inline**, usable standalone or (opt-in) as a **git diff driver** via `.gitattributes`. The key move: OpenSpec **already has the "log"** — the *why* is in `proposal.md`, the *what/where* is the applied-delta provenance — so `diff` joins existing artifacts rather than building the sidecar reasoning database the discussion reached for. It is deterministic and honest (no inference; says so when a change is unattributable), never modifies git config without opt-in, and does not summarize semantically (that is the `verify` direction). (design Decision 15.) - -7. **THE LINTER GATE — `openspec check` (`cli-check` NEW).** The discussion's "sync is a lint step… run within CI/tooling much like linters." One command runs all the deterministic gates (`format --check`, `sync --check`, `validate`), exits non-zero on any failure, and offers `--fix` (mechanical auto-remediation), `--all` (cross-change), incremental skipping, and `--json`. **It is the same binary for pre-commit and CI** — a drift gate is a pure function of the committed files, so *where* it runs (a local hook before the commit or a CI job after) never changes the verdict. This PR also **builds the wiring**: a runner-agnostic, opt-in pre-commit hook installer and a copy-paste CI step — no model, no API keys in CI. (design Decisions 4, 16.) - -### What this deliberately does *not* change - -The discussion proposed removing `archive` entirely — keep every change in its dated archive location for its whole life and fold the spec merge into `apply`. **Rejected, with the maintainer's reasoning, and it shapes the design:** `specs/` only ever describes *shipped* reality. Folding the merge into `apply` would let proposed-but-unmerged (or abandoned) changes pollute the source of truth and let parallel changes step on each other's specs before any land. So the archive boundary stays; what changes is that the merge across it becomes deterministic and reversible. The determinism is what makes the boundary cheap to cross in both directions, which is the real fix for the "awkward final step." (Full treatment in [design.md](./design.md) Decision 5.) - -The discussion also raised the deeper question — *are deltas just simulating, above git, what git already does?* — and the matching proposals to **edit `specs/` in place** (the git diff *is* the delta) with a sidecar reasoning log, and to **commit only the spec**, treating proposal/design/tasks as disposable personal workflow. **Also rejected, and likewise constructive:** the delta layer is precisely what a raw git diff cannot give — clean isolation of *proposed* from *shipped*, so `specs/` stays canonical while several in-flight changes remain independently reviewable until each lands (it is also what makes reversing a single change, i.e. `unarchive`, possible); and the planning artifacts *are* the reviewable, resumable product, not crumbs. But the sharpest kernels are valid, so we **take them**: editing-the-spec-first "forces conflict resolution now" → `sync --check` moves conflict detection early (Decision 11); "reasoning travels with the change, every edit accounted for" → provenance + correspondence, within the delta model (Decision 12); "use hashes to optimize when checks are needed" → incremental `--check` (Decision 13); the framework-as-spec-formatter wish → a deterministic `openspec format` sharing the engine's canonicalizer (Decision 14); and "review = read spec deltas + reasoning inline" → `openspec diff`, which renders from the proposal + provenance OpenSpec already keeps, so no sidecar reasoning database is needed (Decision 15). Only the artifact-model change (commit-only-the-spec) stays out — it is a schema-flexibility question, not a merge-engine one. - -## Capabilities - -### New Capabilities - -- `cli-sync`: the deterministic, idempotent spec-merge engine exposed as `openspec sync [change]` — `--fix`/default writes `specs/` from the change's deltas (byte-identical for the same delta + base; re-running is a no-op; a revised delta regenerates with no crumbs), `--check` is a read-only, model-free gate for CI and pre-commit hooks that also surfaces conflicts early (delta-vs-base, and two active changes targeting one requirement) and verifies delta↔spec correspondence. Records a per-change applied-delta baseline (pre-image + digest + provenance) that powers reverse, drift detection, and tracing each spec edit to its delta. The same engine `archive` uses internally. -- `cli-unarchive`: `openspec unarchive [change-name]` — the deterministic inverse of archive. Resolves an archived change (prefix-tolerant, never auto-picking ambiguous matches), reverses the spec merge from the baseline under a drift guard, moves the folder back, atomically. `--keep-specs` restores the folder without touching `specs/`; pre-baseline archives degrade gracefully. -- `opsx-unarchive-skill`: a `/opsx:unarchive` workflow skill that delegates to `openspec unarchive` for all deterministic work (selection, confirmation, rendering only). Expanded profile; no cross-skill dependency. -- `cli-format`: `openspec format [target]` — a deterministic, behavior-preserving spec/delta formatter sharing one canonicalizer with the merge engine, so synced/archived specs are canonical by construction. `--check` is a read-only, model-free gate (CI + pre-commit) that names unformatted files; `--fix`/default writes canonical form; incremental via digests; `--json`-capable. Pure text — no agent, no `/opsx:format` skill. -- `cli-diff`: `openspec diff [target] [--base ]` — a deterministic, spec-aware diff that splices each changed requirement's provenance and rationale inline (what changed and why, together), usable standalone or as an opt-in git diff driver. Renders from the change's `proposal.md` + the recorded applied-delta provenance — no new reasoning store; no inference; `--json`-capable. -- `cli-check`: `openspec check [--fix] [--all]` — the unified deterministic linter that runs `format --check`, `sync --check`, and `validate` in one invocation, exiting non-zero on any failure. The **same binary for pre-commit and CI** (identical verdict in both); `--fix` applies mechanical auto-remediation (never invents resolutions); incremental via digests; `--json`. Ships with a runner-agnostic opt-in hook installer and a copy-paste CI step — no model, no API keys. - -### Modified Capabilities - -- `cli-archive`: routes its spec merge through the shared deterministic engine and records the applied-delta baseline inside the change folder before moving it, so archiving becomes deterministically reversible. Forward-only and backward-compatible — no change to how archive merges, moves, validates, or what it prints. -- `specs-sync-skill`: `/opsx:sync` delegates the merge to the deterministic `openspec sync` CLI instead of performing agent-driven edits to `specs/`, making the result byte-deterministic and removing the scenario-dropping "intelligent merge" failure mode. The skill handles selection, confirmation, and output only. - -### Integration surface (built, not just documented) - -- **`openspec check` is the unified gate** over `format --check` + `sync --check` + `validate`, and it is the **same binary for pre-commit and CI** — the verdict is a pure function of the committed files, so it cannot differ by where it runs. This PR builds the gate, a runner-agnostic **opt-in hook installer**, and a **copy-paste CI step** (no model, no API keys). The only thing left to the owner is policy: which hook runner to standardize and whether to enable the gate by default (`cli-check`, design Decision 16). - -## Impact - -- `src/core/specs-apply.ts` — make `buildUpdatedSpec`/`applySpecs` byte-deterministic and idempotent; add the inverse (delta-inversion for ADDED/RENAMED; pre-image restore for all ops); add baseline read/write. Forward output unchanged for existing callers. -- `src/core/spec-canonical.ts` (**new**) — extract the deterministic spec/delta canonicalizer (the recomposition `buildUpdatedSpec` already performs at [specs-apply.ts:311-348](../../../src/core/specs-apply.ts)) into one shared module used by the merge engine *and* the formatter, so their output cannot diverge. Behavior-preserving: `parse(canonicalize(x)) == parse(x)`. -- `src/core/sync.ts` (**new**) — `SyncCommand` (default/`--fix`/`--check`), mirroring `ArchiveCommand`'s human + `--json` shape; writes/refreshes the applied-delta baseline. -- `src/core/format.ts` (**new**) — `FormatCommand` (default/`--fix`/`--check`, `--json`, incremental): runs the shared canonicalizer over main specs and active-change delta files; `--check` lists non-canonical files and exits non-zero without writing. -- `src/core/diff.ts` (**new**) — `DiffCommand` (`--base`, `--json`): renders requirement-level spec/delta changes annotated with provenance (from the baseline) and rationale (from the originating change's `proposal.md`); pure code; also invocable as a git diff driver. Plus a documented `.gitattributes` snippet for opt-in registration. -- `src/core/check.ts` (**new**) — `CheckCommand` (`--fix`, `--all`, `--json`, incremental): composes `format --check`, `sync --check`, and `validate` into one gate with a single exit-code/JSON contract; `--fix` runs `format --fix` + `sync --fix`. Plus an opt-in hook installer (composes with existing hooks, never auto-installs) and a committed CI workflow template that runs `openspec check`. -- `src/core/unarchive.ts` (**new**) — `UnarchiveCommand`: resolve archived dir, drift-check, restore pre-images (or delta-invert / refuse for pre-baseline), move folder back, atomic abort. Reuse `moveDirectory()`/`copyDirRecursive()` ([src/core/archive.ts:96-128](../../../src/core/archive.ts)). -- `src/core/archive.ts` — route the merge through the shared engine (already deterministic) and persist the baseline before `moveDirectory` (~414-506). No behavior/output change. -- `src/core/change-metadata/` or a sibling baseline store — persist the applied-delta baseline per change (pre-image + digest, newline-normalized, scheme-tagged). Coordinate with [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278)'s digest ledger so the two drift layers share a digest convention. -- `src/core/list.ts` / `src/core/view.ts` — reuse `getArchivedChangesData()` ([#399](https://github.com/Fission-AI/OpenSpec/pull/399)) to resolve/disambiguate archived candidates. -- `src/cli/index.ts` — register `sync [change]` (`--check`, `--fix`, `--all`, `--json`, `--store`), `unarchive [change-name]` (`--keep-specs`/`--skip-specs` alias, `-y/--yes`, `--no-validate`, `--json`, `--store`), `format [target]` (`--check`, `--fix`, `--json`), `diff [target]` (`--base`, `--json`), and `check` (`--fix`, `--all`, `--install-hook`, `--json`), mirroring archive (326-343). -- `src/core/templates/workflows/sync-specs.ts` — rewrite `/opsx:sync` to invoke `openspec sync` (drop agent-driven edits). `src/core/templates/workflows/unarchive-change.ts` (**new**) — `/opsx:unarchive` delegating to the CLI. Registration: export from `skill-templates.ts`; add `unarchive` to `ALL_WORKFLOWS` ([src/core/profiles.ts:19](../../../src/core/profiles.ts)) and `WORKFLOW_TO_SKILL_DIR` ([src/core/init.ts:65](../../../src/core/init.ts)); **not** added to `CORE_WORKFLOWS`. `docs/opsx.md` gains a `/opsx:unarchive` row and a determinism note for `/opsx:sync`. -- Tests — byte-identical sync determinism (same delta+base → same bytes; CRLF/LF; Windows), idempotency (re-run no-op; revised delta no crumbs), `--check` exit codes, archive→unarchive byte-exact round-trip, drift refusal, `--keep-specs`, destination-collision abort, pre-baseline degradation, atomic "no files changed", skill-template delegation snapshots; formatter determinism + idempotency + `parse-before==parse-after` (behavior-preserving) + `sync`/`archive` output passes `format --check` (shared-canonicalizer invariant); diff determinism + provenance/rationale annotation + honest "unattributable" handling + opt-in-only git config. Per [openspec/config.yaml](../../config.yaml), run on Windows CI. - -## Issues addressed - -All references verified against `Fission-AI/OpenSpec` at `main` (`546224e`) on 2026-06-29. No prior unarchive/sync-command/rollback issue or PR exists — this is greenfield. - -Delivers (the Discord conclusions): - -- **Deterministic, code-only sync path** *("the right fix is a deterministic, code-only openspec sync/archive path that CI can run as a plain binary… I'll fold this into the unarchive PR")* → `openspec sync` + `--check`. -- **`openspec unarchive` / `/opsx:unarchive`** that "moves the folder back and reverses the spec merge," including the hard case where the archive is buried in a multi-file commit. -- **Idempotent, crumb-free regeneration** and the **pre-commit / CI gate** *(the discussion's "sync is a lint step… eslint --fix" / "sync acts as the auto-fixer", run "within CI/tooling much like linters")* → `openspec check` (+ `--fix`), the same binary for pre-commit and CI, with an opt-in hook installer and a CI template. -- **Hash-tracked drift** *(the discussion's "track the hash of the changes/ state applied to the spec")* → the baseline digest. -- **Deterministic spec formatter/linter** *(the discussion's "the framework's goal is how the spec looks and reads and is organized", as a linter/formatter with hash-optimized checks)* → `openspec format` + `--check`, sharing the engine's canonicalizer. -- **Spec-aware diff with reasoning inline** *(the discussion's "review primarily becomes reading the spec deltas" + "a spec-aware diff driver that splices the reasoning log inline")* → `openspec diff` (opt-in git diff driver), rendering from existing proposal + provenance, so no sidecar reasoning database is needed. - -Directly fixes / strengthens: - -- [#863](https://github.com/Fission-AI/OpenSpec/issues/863), [#799](https://github.com/Fission-AI/OpenSpec/issues/799), [#656](https://github.com/Fission-AI/OpenSpec/issues/656) — "the archive/sync skill re-implements the merge instead of calling the CLI." Both skills become CLI-first; the merge is one deterministic code path. -- [#1246](https://github.com/Fission-AI/OpenSpec/issues/1246) — the agent "intelligent merge" silently drops scenarios. Deterministic whole-block apply (per the conventions spec's "complete modified requirement, not a diff") removes the failure mode, and the baseline retains the pre-image #1246 wants for drift detection. -- [#682](https://github.com/Fission-AI/OpenSpec/issues/682) — archive "is not transactional… there's no rollback." `unarchive` is that rollback, itself atomic. -- [#1112](https://github.com/Fission-AI/OpenSpec/issues/1112) — MODIFIED/REMOVED/RENAMED-from headers absent from base pass `validate` but abort at archive. `sync --check` surfaces the same appliability check earlier, as a gate. - -Delineated from adjacent work (coordinate, don't collide): - -- [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278) (this author's sibling) — artifact-graph drift (proposal→design→tasks staleness) via a content-digest ledger. This PR is the *spec-merge* drift layer (delta→`specs/`). Same digest philosophy, different layer; share the digest/newline-normalization convention. -- [#409](https://github.com/Fission-AI/OpenSpec/issues/409) / [#787](https://github.com/Fission-AI/OpenSpec/pull/787) / [#1192](https://github.com/Fission-AI/OpenSpec/issues/1192) (archive-folder prefix scheme) — unarchive's resolver is prefix-tolerant; it does not pick a scheme. -- [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md) — planning-time archive ordering. Unarchive's drift guard and sync's idempotency are the runtime spec-layer counterpart; the two compose. -- [#704](https://github.com/Fission-AI/OpenSpec/issues/704)/[#682](https://github.com/Fission-AI/OpenSpec/issues/682) (archive hooks), [#709](https://github.com/Fission-AI/OpenSpec/issues/709) (`git mv`) — symmetric `unarchive` hook points and a shared `moveDirectory()` make a future switch cover both directions. - -Out of scope (with reasons): - -- **Inference-based review/verification** — semantic review or code-vs-spec checking needs a model; that is the `verify` direction ([#880](https://github.com/Fission-AI/OpenSpec/issues/880)), distinct from the deterministic `diff`/`format`/`check` shipped here (design Decisions 14–16). -- **Owner policy, not capability** — *which* hook runner to standardize and whether the `openspec check` gate is enabled by default are left to the owner; the capability itself (command + opt-in installer + CI template) is built here. - -> **Scope note:** with `check` (and the pre-commit/CI wiring), `diff`, and `format` folded in, this proposal now covers every deterministic, in-scope idea from the discussion (see design "Scope completeness"). Subsequent sessions should shift from expansion to sequencing and confirming the open decisions for owner review. -- **Spec-aware authoring lint beyond canonical form** — `format` delivers deterministic canonicalization; richer *semantic* organization advice (which capability a requirement belongs in, etc.) would need judgment and stays separate from the pure-code formatter (design Decision 14). - -Considered and rejected (documented in design): - -- **Remove `archive`; fold the merge into `apply`** (raised in the discussion). Rejected to preserve the invariant that `specs/` describes only shipped reality (design Decision 5). -- **Edit `specs/` in place; treat the git diff as the delta; keep semantic deltas in a sidecar log** (raised in the discussion). Rejected because the delta layer is what isolates *proposed* from *shipped* and makes parallel changes and single-change reverse possible; its valid kernel (early conflict resolution, reasoning-with-the-change) is adopted via `sync --check` conflict detection and provenance/correspondence instead (design Decisions 11–12). -- **Commit only the spec; treat proposal/design/tasks as disposable personal workflow** (raised in the discussion). Rejected because those artifacts are the reviewable, resumable product — the behavioral agreement before code. The valid kernel (lightweight changes needn't carry every artifact) is already served by schema flexibility, outside this PR's scope (design Decision 13). - -Related, out of scope (referenced, not closed): - -- [#1245](https://github.com/Fission-AI/OpenSpec/issues/1245) — first-class lifecycle timestamps. If/when `archived` is persisted, `unarchive` should clear it; noted, not built. diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-check/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-check/spec.md deleted file mode 100644 index bb25077bde..0000000000 --- a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-check/spec.md +++ /dev/null @@ -1,109 +0,0 @@ -## ADDED Requirements - -### Requirement: Unified Check Command - -The system SHALL provide an `openspec check` command — the deterministic spec linter — that runs the repository's deterministic gates (format canonical-form, sync delta↔spec consistency and conflicts, and spec validation) in one invocation and exits non-zero if any gate fails. It SHALL run in pure code without AI inference. - -#### Scenario: Run all deterministic gates - -- **WHEN** the user runs `openspec check` -- **THEN** the command runs `format --check`, `sync --check`, and `validate` across the repository's specs and active changes -- **AND** it exits zero only if every gate passes, and non-zero (naming the failing gate and files) otherwise - -#### Scenario: Check all active changes - -- **WHEN** the user runs `openspec check --all` -- **THEN** it includes cross-change checks (e.g. two active changes targeting the same requirement) in addition to per-change checks - -#### Scenario: No inference - -- **WHEN** `openspec check` runs -- **THEN** it computes every result in code -- **AND** it does not call a language model — it is safe to run with no API keys - -### Requirement: Same Gate For Pre-Commit And CI - -The check command SHALL be invocable identically as a local pre-commit hook and as a post-commit CI job, with the only difference being where it is invoked — the same binary, flags, and exit-code contract in both. - -#### Scenario: Pre-commit usage - -- **WHEN** `openspec check` runs from a pre-commit hook -- **THEN** a passing check allows the commit to proceed -- **AND** a failing check blocks the commit with a non-zero exit and a report of what to fix - -#### Scenario: CI usage - -- **WHEN** `openspec check` runs as a CI step on a pull request -- **THEN** a passing check allows the job to succeed -- **AND** a failing check fails the job with the same non-zero exit and report as the pre-commit run - -#### Scenario: Identical verdict in both contexts - -- **WHEN** the same repository state is checked locally and in CI -- **THEN** both reach the same pass or fail verdict (the gate is deterministic and environment-independent) - -### Requirement: Auto-Fix Mode - -The check command SHALL support a `--fix` mode that applies the deterministic auto-fixes (format and sync regeneration) so a pre-commit hook can remediate drift in place before the commit proceeds, leaving specs canonical and in sync. - -#### Scenario: Fix remediates drift - -- **WHEN** the user runs `openspec check --fix` -- **THEN** the command runs `format --fix` and `sync --fix` to regenerate canonical, in-sync specs -- **AND** a subsequent `openspec check` passes - -#### Scenario: Fix does not mask non-mechanical failures - -- **WHEN** `--fix` runs and a failure cannot be resolved deterministically (for example, a delta that does not cleanly apply, or a cross-change conflict) -- **THEN** the command applies what it safely can, leaves the unfixable issue reported, and still exits non-zero -- **AND** it does not invent a resolution - -### Requirement: Incremental Checking - -The check command MAY use recorded content digests to skip specs and files whose content is unchanged since they were last checked, re-checking only what changed. A skip SHALL be permitted only when it cannot change the verdict versus a full check. - -#### Scenario: Unchanged inputs skipped - -- **WHEN** `openspec check` runs and an input's digest matches its recorded digest -- **THEN** the command may skip re-checking it -- **AND** the overall verdict is identical to a full check - -#### Scenario: Changed or unknown inputs fully checked - -- **WHEN** an input's digest does not match, is missing, or uses an unrecognized scheme -- **THEN** the command performs the full check for it - -### Requirement: Hook Installation - -The system SHALL provide a runner-agnostic way to install `openspec check` as a git pre-commit hook, and SHALL NOT modify the user's git configuration or hooks without explicit action. - -#### Scenario: Install on request - -- **WHEN** the user opts in to installing the hook (for example, `openspec check --install-hook`) -- **THEN** the command installs a pre-commit hook that runs `openspec check` -- **AND** it composes with an existing hook rather than silently overwriting it - -#### Scenario: Never automatic - -- **WHEN** the user has not opted in -- **THEN** OpenSpec installs no hook and changes no git configuration - -### Requirement: CI Template - -The system SHALL document and provide a copy-paste CI configuration that runs `openspec check` as a drift gate, requiring no model and no API keys. - -#### Scenario: Documented CI gate - -- **WHEN** a maintainer wants a CI drift gate -- **THEN** the project provides a ready CI step that runs `openspec check` -- **AND** the step fails the build when committed specs are not canonical, not in sync with deltas, or otherwise invalid - -### Requirement: JSON Output - -The check command SHALL support `--json`, emitting a machine-readable summary of which gates ran, which passed or failed, and the offending files. - -#### Scenario: JSON summary - -- **WHEN** the user runs `openspec check --json` -- **THEN** it emits a structured result per gate (format, sync, validate) with pass/fail and the files involved -- **AND** the process exit code reflects the overall verdict diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-diff/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-diff/spec.md deleted file mode 100644 index 67e1586a4c..0000000000 --- a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-diff/spec.md +++ /dev/null @@ -1,86 +0,0 @@ -## ADDED Requirements - -### Requirement: Spec Diff Command - -The system SHALL provide an `openspec diff [target] [--base ]` command that renders a deterministic, spec-aware diff of spec and delta files, splicing each changed requirement's provenance and rationale inline so a reviewer sees what changed and why together. It SHALL compute the rendering in pure code, without AI inference. - -#### Scenario: Diff a change's deltas - -- **WHEN** the user runs `openspec diff ` -- **THEN** the command shows the change's delta operations grouped by capability and requirement -- **AND** annotates each with the rationale drawn from the change's `proposal.md` - -#### Scenario: Diff main specs against a base revision - -- **WHEN** the user runs `openspec diff --base ` over `openspec/specs/` -- **THEN** the command shows the requirement-level differences since `` -- **AND** annotates each changed requirement with the change and delta operation that produced it, drawn from the recorded applied-delta provenance - -#### Scenario: No inference - -- **WHEN** the command renders a diff -- **THEN** it composes the result from the git diff and the recorded provenance/rationale in code -- **AND** it does not call a language model - -### Requirement: Inline Reasoning Annotation - -The diff SHALL annotate each changed requirement with the originating change and its rationale, sourced from existing OpenSpec artifacts, and SHALL NOT invent rationale that is not recorded. - -#### Scenario: Annotated with originating change and rationale - -- **WHEN** a changed requirement can be attributed to a change via provenance -- **THEN** the diff shows the originating change and a reference to or excerpt of its recorded rationale - -#### Scenario: Unattributable change shown honestly - -- **WHEN** a changed requirement cannot be attributed (no provenance recorded, e.g. a pre-baseline edit) -- **THEN** the diff shows the change without inventing a rationale -- **AND** it indicates that provenance is unavailable - -### Requirement: Reuses Existing Artifacts, No New Sidecar Store - -The rationale and provenance the diff splices SHALL come from artifacts OpenSpec already maintains — the change's `proposal.md` (the why) and the recorded applied-delta provenance (the what/where) — rather than a separate reasoning database. - -#### Scenario: Reasoning resolved from existing artifacts - -- **WHEN** the diff needs the reasoning for a changed requirement -- **THEN** it resolves the rationale from the originating change's `proposal.md` and the recorded provenance -- **AND** it requires no separate reasoning-log store - -### Requirement: Deterministic Rendering - -The diff rendering SHALL be a pure function of its inputs, producing byte-identical output for the same inputs on every platform. - -#### Scenario: Repeated runs are identical - -- **WHEN** `openspec diff` runs more than once on the same inputs -- **THEN** the output bytes are identical every time - -#### Scenario: Platform independence - -- **WHEN** the diff runs on different operating systems with the same inputs -- **THEN** the output is identical regardless of line-ending or path-separator differences - -### Requirement: Git Diff Driver Integration - -The command SHALL be usable as a git diff driver for spec files, documented as an opt-in `.gitattributes` registration, so that `git diff` over spec and delta files renders the spec-aware view. OpenSpec SHALL NOT modify the user's git configuration without explicit consent. - -#### Scenario: Registered as a diff driver - -- **WHEN** the user opts in by registering the driver for spec paths in `.gitattributes` -- **THEN** `git diff` over those paths renders the spec-aware, annotated diff - -#### Scenario: Opt-in only - -- **WHEN** the user has not registered the driver -- **THEN** OpenSpec does not alter git behavior -- **AND** `openspec diff` remains available as a standalone command - -### Requirement: JSON Output - -The diff command SHALL support `--json`, emitting a machine-readable, per-requirement structure (change operation, provenance, rationale reference) for review tooling. - -#### Scenario: JSON annotated diff - -- **WHEN** the user runs `openspec diff --json` -- **THEN** it emits, per changed requirement, the operation, the originating change, and a reference to the rationale diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-format/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-format/spec.md deleted file mode 100644 index 436b4997cf..0000000000 --- a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-format/spec.md +++ /dev/null @@ -1,144 +0,0 @@ -## ADDED Requirements - -### Requirement: Spec Format Command - -The system SHALL provide an `openspec format [target]` command that rewrites OpenSpec spec files and delta spec files to a single deterministic canonical form, in pure code without AI inference. By default it writes; `--check` runs read-only. - -#### Scenario: Format main and delta specs - -- **WHEN** the user runs `openspec format` with no target -- **THEN** the command formats the main specs under `openspec/specs/` and the delta specs under active changes' `specs/` directories -- **AND** it reports which files were reformatted - -#### Scenario: Format a specific target - -- **WHEN** the user runs `openspec format ` for a spec or delta file or directory -- **THEN** the command formats only that target - -#### Scenario: No inference - -- **WHEN** the command formats a file -- **THEN** it computes the canonical form in code -- **AND** it does not call a language model or otherwise depend on non-deterministic input - -### Requirement: Deterministic Canonical Form - -The formatter SHALL produce byte-identical output for the same input on every platform, and SHALL be idempotent: formatting already-canonical content changes nothing. - -#### Scenario: Same input yields identical output - -- **WHEN** the formatter runs more than once on the same input -- **THEN** the output bytes are identical every time - -#### Scenario: Idempotent - -- **WHEN** the formatter is applied to content it has already formatted -- **THEN** the content is unchanged - -#### Scenario: Line endings normalized - -- **WHEN** the input contains CRLF or mixed line endings -- **THEN** the canonical output uses normalized line endings regardless of the platform - -### Requirement: Behavior-Preserving Normalization - -The formatter SHALL change only presentation — whitespace, blank-line policy, list markers and indentation, and heading spacing — and SHALL NOT change the meaning of a spec. It SHALL NOT reorder requirements or scenarios, rewrite prose, or add, remove, merge, or split requirements or scenarios. - -#### Scenario: Requirement and scenario order preserved - -- **WHEN** the formatter runs on a spec -- **THEN** the order of requirements and of scenarios within each requirement is unchanged - -#### Scenario: Prose is not rewritten - -- **WHEN** the formatter normalizes a requirement -- **THEN** the requirement's wording and scenario text are byte-for-byte unchanged except for surrounding whitespace normalization - -#### Scenario: Parsed content is identical before and after - -- **WHEN** a spec is parsed before formatting and after formatting -- **THEN** the parsed requirements, scenarios, and delta operations are identical - -### Requirement: Canonical Section Organization - -The formatter SHALL normalize the structural presentation of a spec deterministically — heading levels and nesting, the spacing between sections, and the canonical headers for delta sections — without changing which sections are present or their order. - -#### Scenario: Canonical headings and spacing - -- **WHEN** a spec uses inconsistent heading spacing or blank-line separation between requirements and scenarios -- **THEN** the formatter rewrites them to the canonical spacing defined by the conventions - -#### Scenario: Canonical delta section headers - -- **WHEN** a delta file contains `## ADDED/MODIFIED/REMOVED/RENAMED Requirements` sections -- **THEN** the formatter normalizes those headers to their canonical form -- **AND** it does not move requirements between sections - -### Requirement: Shared Canonicalization With The Merge Engine - -The canonical form produced by `openspec format` SHALL be the same canonical form emitted by the deterministic merge engine used by `openspec sync` and `openspec archive`, so that synced or archived specs are already canonical. - -#### Scenario: Merge output is already formatted - -- **WHEN** `openspec sync` or `openspec archive` writes a spec -- **THEN** running `openspec format --check` on that spec passes without changes - -#### Scenario: One canonicalizer - -- **WHEN** the same spec content is produced by the formatter and by the merge engine -- **THEN** the two results are byte-identical - -### Requirement: Check Mode - -The format command SHALL support a read-only `--check` mode that exits non-zero when any target is not in canonical form, naming the offending files and modifying nothing, so it can gate commits and CI as a plain binary. - -#### Scenario: Unformatted file detected - -- **WHEN** `openspec format --check` finds a file that is not in canonical form -- **THEN** the command reports the file -- **AND** it exits with a non-zero status code and modifies no files - -#### Scenario: All formatted - -- **WHEN** every target is already in canonical form -- **THEN** the command exits zero and modifies no files - -### Requirement: Fix Mode - -The format command SHALL, by default (or with `--fix`), rewrite targets to canonical form, suitable for use as an auto-fixer in a pre-commit hook. - -#### Scenario: Fix writes canonical form - -- **WHEN** the user runs `openspec format` (or `openspec format --fix`) -- **THEN** the command writes each target's canonical form -- **AND** a subsequent `openspec format --check` passes - -### Requirement: Incremental Checking - -The format check MAY use recorded content digests to skip files whose content is unchanged since they were last checked, re-checking only what changed. A skip SHALL be permitted only when it cannot change the result versus a full check. - -#### Scenario: Unchanged file skipped - -- **WHEN** `--check` runs and a file's current content digest matches the recorded digest -- **THEN** the command may skip re-checking that file -- **AND** the overall result is identical to checking it fully - -#### Scenario: Changed or unknown file fully checked - -- **WHEN** a file's digest does not match, no digest is recorded, or the recorded digest uses an unrecognized scheme -- **THEN** the command performs the full check for that file - -### Requirement: JSON Output - -The format command SHALL support `--json` for non-interactive use, emitting machine-readable results and diagnostics. - -#### Scenario: JSON reports unformatted files - -- **WHEN** `openspec format --check --json` finds files not in canonical form -- **THEN** it emits the list of offending files as JSON -- **AND** exits with a non-zero status code - -#### Scenario: JSON reports written files - -- **WHEN** `openspec format --json` rewrites files -- **THEN** it emits the list of changed files as JSON diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-unarchive/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-unarchive/spec.md deleted file mode 100644 index 60a892daae..0000000000 --- a/openspec/changes/add-deterministic-sync-and-unarchive/specs/cli-unarchive/spec.md +++ /dev/null @@ -1,190 +0,0 @@ -## ADDED Requirements - -### Requirement: Unarchive Command - -The system SHALL provide an `openspec unarchive [change-name]` command that is the inverse of `openspec archive`: it moves a change folder out of `openspec/changes/archive/-/` back to `openspec/changes//` and reverses the spec merge that archiving applied to `openspec/specs/`. - -#### Scenario: Restore an archived change - -- **WHEN** the user runs `openspec unarchive ` for a change that was archived by this version or later -- **THEN** the command restores `openspec/specs/` to its pre-archive state -- **AND** moves the change folder back to `openspec/changes//` -- **AND** reports the restored change name and location - -#### Scenario: Interactive selection - -- **WHEN** no change-name is provided in an interactive session -- **THEN** the command lists archived changes (most-recently-archived first) and prompts the user to select one -- **AND** it does not auto-select - -#### Scenario: Non-interactive requires a name - -- **WHEN** the command is run with `--json` (or otherwise non-interactively) and no change-name is provided -- **THEN** it fails with a machine-readable diagnostic stating a change name is required -- **AND** exits with a non-zero status code - -### Requirement: Archived Change Resolution - -The command SHALL resolve the target archived directory from either a bare change `` or a full archived directory id, treating the leading prefix as opaque up to the name, and SHALL never silently choose among multiple matches. - -#### Scenario: Resolve a bare name with a single match - -- **WHEN** exactly one archived directory matches `` after stripping its leading prefix -- **THEN** the command resolves to that directory - -#### Scenario: Tolerate differing prefix schemes - -- **WHEN** an archived directory uses a date prefix (`YYYY-MM-DD-`) or a sequence/other prefix (e.g. `NNN-`) -- **THEN** the command resolves `` regardless of which prefix scheme produced the directory - -#### Scenario: Ambiguous bare name in interactive mode - -- **WHEN** more than one archived directory matches a bare `` -- **AND** the session is interactive -- **THEN** the command lists the matching directories (most-recent first) and prompts the user to choose -- **AND** it does not auto-select - -#### Scenario: Ambiguous bare name in non-interactive mode - -- **WHEN** more than one archived directory matches a bare `` -- **AND** the command is run with `--json` or otherwise non-interactively -- **THEN** it fails with a diagnostic listing the candidate directory ids -- **AND** directs the user to re-run with the full archived directory id - -#### Scenario: Full directory id resolves unambiguously - -- **WHEN** the user passes the full archived directory id (including its prefix) -- **THEN** the command resolves to exactly that directory without prompting - -#### Scenario: No matching archive - -- **WHEN** no archived directory matches the provided name or id -- **THEN** the command fails with a clear not-found error and makes no changes - -### Requirement: Deterministic Spec Reversal - -The command SHALL reverse the spec merge deterministically by restoring, for each affected spec, the pre-merge content recorded in the change's applied-delta baseline — recreating specs that archiving deleted and deleting specs that archiving created — without re-parsing or inferring requirement content. - -#### Scenario: Restore modified and removed requirements exactly - -- **WHEN** the archived change's baseline records pre-merge content for an affected spec -- **THEN** the command restores that spec to the recorded pre-merge bytes -- **AND** requirements that were MODIFIED or REMOVED during archiving are restored to their exact prior content - -#### Scenario: Reverse added requirements - -- **WHEN** archiving added requirements to a spec -- **THEN** restoring the recorded pre-merge content removes exactly those added requirements - -#### Scenario: Recreate a spec that archiving created - -- **WHEN** archiving created a new spec (the baseline marks the pre-image as absent) -- **THEN** the command deletes that spec on reversal, returning `openspec/specs/` to its pre-archive shape - -#### Scenario: Round-trip is byte-exact - -- **WHEN** a change is archived and then unarchived with no intervening edits to the affected specs -- **THEN** `openspec/specs/` is byte-for-byte identical to its state before the archive - -### Requirement: Drift Guard - -The command SHALL verify, before reversing any spec, that each affected spec still matches the applied-result state recorded in the applied-delta baseline, and SHALL refuse to reverse a spec that has drifted rather than overwrite later changes. - -#### Scenario: Refuse on drift - -- **WHEN** an affected spec's current content no longer matches the applied-result digest recorded in the baseline (for example, a later change modified the same requirement) -- **THEN** the command refuses to reverse the spec merge -- **AND** it reports which specs drifted -- **AND** it directs the user to re-run with `--keep-specs` to restore the folder without touching specs - -#### Scenario: Proceed when no drift - -- **WHEN** every affected spec still matches its recorded applied-result state -- **THEN** the command proceeds with the deterministic spec reversal - -### Requirement: Keep Specs Option - -The command SHALL support a `--keep-specs` flag that restores the change folder to active without modifying `openspec/specs/`, and SHALL accept `--skip-specs` as an equivalent alias. - -#### Scenario: Restore folder without touching specs - -- **WHEN** the user runs `openspec unarchive --keep-specs` -- **THEN** the command moves the change folder back to `openspec/changes//` -- **AND** it makes no changes to `openspec/specs/` -- **AND** it does not perform drift or reversal checks on specs - -#### Scenario: Skip-specs alias - -- **WHEN** the user runs `openspec unarchive --skip-specs` -- **THEN** the command behaves identically to `--keep-specs` - -### Requirement: Atomic Operation - -The command SHALL apply the reversal atomically using a defined sequence: validate, then stage, then commit, so that any failure before the final commit step leaves the filesystem unchanged. - -#### Scenario: Defined sequence - -- **WHEN** the command performs a reversal that touches specs -- **THEN** it executes in this order: - 1. validate that the destination `openspec/changes//` does not exist and that every affected spec is present and drift-free against the baseline; - 2. compute the reversed spec content and write it to a temporary staging area inside `.openspec/` (no change yet to `openspec/specs/`); - 3. swap the staged specs into `openspec/specs/`; - 4. move the change folder from archive back to active. - -#### Scenario: Failure before the folder move leaves specs untouched - -- **WHEN** validation or staging (steps 1–2) fails -- **THEN** the command reports `Abort. No files were changed.` -- **AND** `openspec/specs/` and the archive folder are exactly as they were - -#### Scenario: Failure of the final move rolls specs back - -- **WHEN** the spec swap (step 3) has occurred but the folder move (step 4) fails -- **THEN** the command restores `openspec/specs/` from the still-present baseline -- **AND** reports `Abort. No files were changed.` -- **AND** if rollback itself cannot complete, it reports the partial state and the exact recovery steps rather than leaving a silent inconsistency - -#### Scenario: Destination already exists - -- **WHEN** an active change directory `openspec/changes//` already exists -- **THEN** the command fails without overwriting it -- **AND** it makes no changes to specs - -### Requirement: Backward Compatibility For Pre-Baseline Archives - -For changes archived before applied-delta baselines existed, the command SHALL reverse the spec merge only when the whole change is invertible from the archived delta alone, and SHALL otherwise refuse the spec reversal entirely (all-or-nothing) rather than leaving a partially reversed `openspec/specs/`. This preserves the same atomic, never-corrupt guarantee in the degraded path. - -#### Scenario: Fully self-invertible change is reversed - -- **WHEN** an archived change has no applied-delta baseline -- **AND** every delta operation across all of its affected specs is ADDED and/or RENAMED -- **THEN** the command reverses them by delta inversion (removing added requirements, renaming renamed requirements back) and restores `openspec/specs/` accordingly - -#### Scenario: Mixed invertibility refuses all spec reversal - -- **WHEN** an archived change has no applied-delta baseline -- **AND** its delta contains at least one MODIFIED or REMOVED requirement (whose pre-image is not recoverable from the delta), possibly alongside ADDED/RENAMED in the same or other specs -- **THEN** the command modifies no specs at all (it does not partially reverse the self-invertible specs) -- **AND** it reports which requirements cannot be safely reversed and why -- **AND** it directs the user to `--keep-specs` (and optionally to git-based recovery of the pre-image) - -#### Scenario: Keep-specs always available - -- **WHEN** an archived change has no applied-delta baseline -- **AND** the user runs `openspec unarchive --keep-specs` -- **THEN** the command restores the folder without touching specs, regardless of which delta operations the change contains - -### Requirement: Error Conditions - -The command SHALL handle error conditions gracefully and consistently with `openspec archive`. - -#### Scenario: Missing archive directory - -- **WHEN** no `openspec/changes/archive/` directory exists -- **THEN** the command fails with a clear message and makes no changes - -#### Scenario: JSON diagnostics - -- **WHEN** the command is run with `--json` and a blocked condition occurs (not found, ambiguous, drift, destination exists, or pre-baseline irreversibility) -- **THEN** it emits a machine-readable diagnostic with a stable code -- **AND** exits with a non-zero status code diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/specs/opsx-unarchive-skill/spec.md b/openspec/changes/add-deterministic-sync-and-unarchive/specs/opsx-unarchive-skill/spec.md deleted file mode 100644 index 742592f506..0000000000 --- a/openspec/changes/add-deterministic-sync-and-unarchive/specs/opsx-unarchive-skill/spec.md +++ /dev/null @@ -1,56 +0,0 @@ -## ADDED Requirements - -### Requirement: OPSX Unarchive Skill - -The system SHALL provide an `/opsx:unarchive` skill that restores an archived change to active, delegating the deterministic work to the `openspec unarchive` CLI rather than moving folders or editing specs itself. - -#### Scenario: Unarchive a selected change - -- **WHEN** the agent executes `/opsx:unarchive` with a change name -- **THEN** the skill invokes `openspec unarchive ` to perform the restore -- **AND** it displays the CLI's result (restored location and spec-reversal outcome) - -#### Scenario: Change selection prompt - -- **WHEN** the agent executes `/opsx:unarchive` without specifying a change -- **AND** the change cannot be inferred from conversation context -- **THEN** the skill lists archived changes (via `openspec list --archived --json`) and asks the user to choose -- **AND** it never auto-selects, including when several archives share a name - -### Requirement: CLI Delegation - -The skill SHALL perform all deterministic operations — resolution, spec reversal, drift checking, folder move, and atomicity — by calling the `openspec unarchive` CLI, and SHALL NOT re-implement that logic in prose. - -#### Scenario: No manual file operations - -- **WHEN** the skill restores a change -- **THEN** it does not manually move the change directory -- **AND** it does not manually edit files under `openspec/specs/` -- **AND** it relies on the CLI for the reversal - -#### Scenario: Surface CLI guardrails verbatim - -- **WHEN** the CLI refuses to reverse specs because they have drifted, or because a pre-snapshot change contains irreversible operations -- **THEN** the skill reports that refusal to the user -- **AND** it offers the `--keep-specs` option the CLI recommends rather than attempting its own workaround - -### Requirement: Confirmation And Output - -The skill SHALL confirm the action before running it and present a clear summary of the outcome. - -#### Scenario: Confirm before restoring - -- **WHEN** the skill has resolved which archived change to restore -- **THEN** it shows the user what it will do (target change, whether specs will be reversed or kept) before invoking the CLI -- **AND** it proceeds only after the user confirms - -#### Scenario: Summarize the result - -- **WHEN** the CLI completes -- **THEN** the skill displays the restored change name, its new active location, and whether specs were reversed or kept - -#### Scenario: Report a clean failure - -- **WHEN** the CLI aborts (for example, the destination already exists or the change name is ambiguous) -- **THEN** the skill reports the CLI's diagnostic and the suggested next step -- **AND** it does not attempt a manual fallback diff --git a/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md b/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md deleted file mode 100644 index d0ed9b7a48..0000000000 --- a/openspec/changes/add-deterministic-sync-and-unarchive/tasks.md +++ /dev/null @@ -1,95 +0,0 @@ -# Tasks: deterministic spec tooling — `sync` (forward) + `unarchive` (reverse) + `format` (canonical) - -> Phased so the keystone ships first and each phase is independently testable and shippable. Phase 1 (engine + baseline + shared canonicalizer) and Phase 2 (`sync` + `--check`) deliver the deterministic, model-free drift path; Phase 2A (`format`) reuses the canonicalizer from Phase 1, and Phase 2B (`diff`) reuses the provenance from Phase 1–2; Phase 3 (`unarchive`) and Phase 4 (idempotency) build on the same baseline; Phase 5 (skills) removes the model from the merge; Phase 6 builds the unified `openspec check` gate plus its opt-in pre-commit hook installer and CI template (same binary both places). Cross-platform concerns (`path.join`, Windows `moveDirectory`, newline-normalized digests) are called out per [openspec/config.yaml](../../config.yaml). - -## 1. The engine: deterministic, byte-stable merge + applied-delta baseline - -- [ ] 1.1 Make the merge byte-deterministic: audit `buildUpdatedSpec` ([src/core/specs-apply.ts:240-307](../../../src/core/specs-apply.ts)) for any nondeterministic ordering/whitespace; guarantee stable requirement ordering and newline normalization so the same (delta + base) yields byte-identical output on macOS/Linux/Windows. -- [ ] 1.1a Extract the canonicalizer: factor the recomposition/normalization `buildUpdatedSpec` performs ([specs-apply.ts:311-348](../../../src/core/specs-apply.ts)) into a shared `src/core/spec-canonical.ts` used by the merge engine *and* the formatter, so their output cannot diverge. Cover spec files and delta files (ADDED/MODIFIED/REMOVED/RENAMED sections). Behavior-preserving: assert `parse(canonicalize(x)) == parse(x)`. -- [ ] 1.2 Define the **applied-delta baseline** format (per design Decision 1): per affected spec, the pre-merge content (or `absent` marker) plus a scheme-tagged, newline-normalized digest of the applied result. Store it with the change (e.g. `.openspec/merge-baseline/`); document the location. -- [ ] 1.3 Add baseline read/write helpers (safe read-modify-write, preserving unrelated fields), coordinating the digest convention with [#1278](https://github.com/Fission-AI/OpenSpec/pull/1278)'s ledger. -- [ ] 1.4 Add the inverse merge in `specs-apply.ts`: delta-inversion for ADDED (remove by name) and RENAMED (rename TO→FROM, rewriting the header line); pre-image restore for all ops when a baseline exists. No change to forward behavior for existing callers. -- [ ] 1.5 Tests: same delta+base → byte-identical output across repeated runs and simulated CRLF/LF + Windows paths; baseline round-trips through read/write; inverse of a forward apply restores the base exactly (ADD/MODIFY/REMOVE/RENAME/create). - -## 2. `openspec sync` + the drift gate (the keystone command) - -- [ ] 2.1 Create `src/core/sync.ts` `SyncCommand` mirroring `ArchiveCommand`'s human + `--json` shape (`resolveOpenSpecRoot`, blocked-error → diagnostic, exit codes); default/`--fix` writes `specs/` from the deltas and refreshes the baseline. -- [ ] 2.2 `--check`: read-only; exit non-zero when deltas are not cleanly appliable to the base, or (when `specs/` was synced) when committed `specs/` ≠ regenerated output. Never writes `specs/`. Support `--all`/`--changes` style fan-out for CI (or document the loop). -- [ ] 2.3 Register `sync [change]` in [src/cli/index.ts](../../../src/cli/index.ts) (`--check`, `--fix`, `--json`, `--store`, hidden store-path), mirroring archive (326-343). -- [ ] 2.4 Tests: write produces deterministic `specs/`; `--check` clean vs drifted exit codes; `--check` never mutates; unknown/ambiguous change diagnostics; JSON shape on success and each blocked path. -- [ ] 2.5 Provenance: record, with the baseline, the originating change + delta operation for each applied spec change; add an `--explain` (and JSON) output that maps each affected requirement to its source delta. Do not re-author rationale (link to the change's `proposal.md`). (design Decision 12) -- [ ] 2.6 Delta↔spec correspondence in `--check`: fail on an orphan spec edit (attributable to the change but matching no delta op) and on an unapplied delta (delta op not reflected in `specs/`); pass when both directions hold. (design Decision 12) -- [ ] 2.7 Cross-change conflict detection in `--check` (design Decision 11): (a) delta-vs-base — a MODIFIED/REMOVED/RENAMED-from header absent from the current base (surfaces #1112 early); (b) cross-change — two active changes targeting the same requirement; report specifics, non-zero exit, no writes. Coordinate the cross-change check with [add-change-stacking-awareness](../add-change-stacking-awareness/proposal.md). -- [ ] 2.8 Tests: provenance recorded + `--explain` mapping; orphan-edit and unapplied-delta both fail `--check`; delta-vs-base conflict fails with the right requirement; two active changes on one requirement flagged; clean case passes. -- [ ] 2.9 Incremental checking (design Decision 13): `--check` skips a spec whose current digest matches the recorded baseline; re-checks on mismatch; forces a full check when the baseline is missing or its scheme is unrecognized. Add an escape hatch to disable skips (e.g. `--no-incremental`) for the equivalence test. -- [ ] 2.10 Tests: unchanged spec skipped; changed spec re-checked; missing/unknown-scheme baseline → full check; **incremental verdict == full verdict** on the same input (the correctness invariant); a touched spec in a large fixture is the only one re-checked. - -## 2A. `openspec format` — the formatter (reuses the Phase 1 canonicalizer) - -- [ ] 2A.1 Create `src/core/format.ts` `FormatCommand` (human + `--json`); default/`--fix` rewrites targets via `spec-canonical.ts`; `--check` is read-only. -- [ ] 2A.2 Target resolution: no target → all main specs + active-change delta files; explicit path → just that file/dir. Handle both spec and delta formats. -- [ ] 2A.3 `--check`: list non-canonical files, exit non-zero, write nothing; default/`--fix`: write canonical form. -- [ ] 2A.4 Register `format [target]` in [src/cli/index.ts](../../../src/cli/index.ts) (`--check`, `--fix`, `--json`), mirroring archive's shape. No skill (pure code). -- [ ] 2A.5 Incremental: reuse the digest skip (Decision 13) so `--check` only re-reads changed files; skip never changes the verdict. -- [ ] 2A.6 Tests: determinism (same input → same bytes; CRLF/LF; Windows); idempotency (`format(format(x))==format(x)`); **behavior-preserving** (`parse-before==parse-after`; requirement/scenario order and prose unchanged); **shared-canonicalizer invariant** — `sync`/`archive` output passes `format --check`; `--check` exit codes; `--json` shape. - -## 2B. `openspec diff` — spec-aware diff driver (consumes provenance) - -- [ ] 2B.1 Create `src/core/diff.ts` `DiffCommand` (human + `--json`, `--base `): render requirement-level spec/delta changes; resolve provenance from the applied-delta baseline and rationale from the originating change's `proposal.md`. -- [ ] 2B.2 Annotate each changed requirement with originating change + rationale reference; when a change is unattributable (no provenance, e.g. pre-baseline), show it honestly without inventing rationale. -- [ ] 2B.3 Deterministic rendering: pure function of git diff + provenance + proposal text; byte-identical across runs/platforms; no inference. -- [ ] 2B.4 Git diff driver integration: provide the renderer in a form usable as a git `diff`/`textconv` driver, plus a documented opt-in `.gitattributes` snippet for spec paths. Never modify the user's git config automatically. -- [ ] 2B.5 Register `diff [target]` in [src/cli/index.ts](../../../src/cli/index.ts) (`--base`, `--json`). No skill (pure code). -- [ ] 2B.6 Tests: deterministic output (repeat/CRLF/Windows); annotation resolves from proposal + provenance; unattributable change shown without invented rationale; no git-config mutation without opt-in; `--json` shape. - -## 3. `openspec unarchive` (reverse, built on the baseline) - -- [ ] 3.1 Create `src/core/unarchive.ts` `UnarchiveCommand` (human + `--json`, blocked-error → diagnostic). -- [ ] 3.2 Resolver: bare `` or full archived id; treat prefix as opaque up to `` (strip `YYYY-MM-DD-`; tolerate `NNN-`/ISO/configurable, #409/#787/#1192); reuse `getArchivedChangesData()` (#399). Ambiguity → interactive prompt (most-recent first, never auto-pick) / `--json` error with candidates. -- [ ] 3.3 Reverse from baseline: restore each affected spec's pre-image (rewrite modified, recreate deleted, delete archive-created), under the drift guard (current content vs baseline applied-result digest); on drift, refuse and direct to `--keep-specs`. -- [ ] 3.4 Destination-collision guard (`changes//` exists → abort, no overwrite). Atomicity: stage + validate first; commit order restore specs → move folder (`moveDirectory`, Windows fallback); on failure *"Abort. No files were changed."*, rolling specs back from the baseline if the move fails. -- [ ] 3.5 `--keep-specs` (with hidden `--skip-specs` alias): pure folder move, no spec work, no drift/reversal checks. -- [ ] 3.6 Register `unarchive [change-name]` in `src/cli/index.ts` (`--keep-specs`/`--skip-specs`, `-y/--yes`, `--no-validate`, `--json`, `--store`). -- [ ] 3.7 Tests: byte-exact archive→unarchive round-trip for ADD/MODIFY/REMOVE/RENAME/create; drift refusal (no writes); destination collision abort; `--keep-specs` leaves `specs/` untouched; atomic abort leaves zero partial state; Windows move path; `--json` blocked-path diagnostics. - -## 4. Idempotency & "no crumbs" - -- [ ] 4.1 `sync` on an already-synced, unchanged change writes nothing and `--check` is clean (idempotent no-op). -- [ ] 4.2 Revised-delta re-merge: reverse the prior revision via the baseline, apply the new delta, refresh the baseline — assert `specs/` equals `apply(current delta, original base)` with no residue from the prior revision. -- [ ] 4.3 Drift-on-resync: if `specs/` drifted from the baseline since last sync, do not silently reverse-then-apply over the edit — report drift and require acknowledgement (consistent with unarchive's guard). -- [ ] 4.4 Tests: double-sync no-op; add→sync→revise(add+remove a requirement)→sync yields exactly the new delta's result, no crumbs; drift-on-resync refusal. - -## 5. Skills delegate to the deterministic CLI (no model in the merge) - -- [ ] 5.1 Rewrite `src/core/templates/workflows/sync-specs.ts` so `/opsx:sync` invokes `openspec sync` (drop the "agent-driven… directly edit main specs" instructions); skill does selection/confirmation/output only. -- [ ] 5.2 Create `src/core/templates/workflows/unarchive-change.ts` (`getUnarchiveChangeSkillTemplate()` + `getOpsxUnarchiveCommandTemplate()`) delegating to `openspec unarchive`; surface the CLI's drift refusal and `--keep-specs` option verbatim; never hand-move folders or hand-edit specs. -- [ ] 5.3 Export from `skill-templates.ts`; add `unarchive` to `ALL_WORKFLOWS` ([src/core/profiles.ts:19](../../../src/core/profiles.ts)) and `WORKFLOW_TO_SKILL_DIR` ([src/core/init.ts:65](../../../src/core/init.ts)); **not** to `CORE_WORKFLOWS` (#913/#762). -- [ ] 5.4 Tests: template snapshots assert `/opsx:sync` and `/opsx:unarchive` call the CLI and contain no manual merge/move instructions (anti-#863/#1246 guard). -- [ ] 5.5 Changeset + docs note: `/opsx:sync` is now deterministic; the agent no longer performs scenario-level merges (fixes #1246); author deltas as complete requirements per the conventions. - -## 6. `openspec check` — the unified gate (pre-commit + CI), built - -- [ ] 6.1 Create `src/core/check.ts` `CheckCommand` (`--fix`, `--all`, `--json`, incremental): compose `format --check`, `sync --check`, and `validate` into one run with a single exit-code/JSON contract; `--fix` runs `format --fix` + `sync --fix` and never invents non-mechanical resolutions (cross-change conflicts / un-appliable deltas still fail). -- [ ] 6.2 Register `check` in [src/cli/index.ts](../../../src/cli/index.ts) (`--fix`, `--all`, `--install-hook`, `--json`). -- [ ] 6.3 Hook installer: `openspec check --install-hook` installs a pre-commit hook that runs `openspec check`; runner-agnostic; composes with an existing hook; never installs automatically or alters git config without the explicit flag. -- [ ] 6.4 CI template: add a committed, copy-paste CI step (e.g. a GitHub Actions job) that runs `openspec check` as a drift gate — no model, no API keys. -- [ ] 6.5 Assert the pre-commit/CI symmetry: the same repo state yields the same `openspec check` verdict whether invoked from the hook or the CI step (one binary, one contract). -- [ ] 6.6 Document the **opt-in git diff driver** registration for `openspec diff` (the `.gitattributes` snippet for spec/delta paths), making clear OpenSpec never alters git config without consent (design Decision 15). -- [ ] 6.7 Tests: `check` aggregates the sub-gates and exit codes; `--fix` remediates mechanical drift but still fails on conflicts; `--all` runs cross-change checks; incremental skip never changes the verdict; hook installer composes with an existing hook and is never automatic; `--json` shape. - -## 7. Docs & supersession - -- [ ] 7.1 `docs/opsx.md` + CLI docs: add `/opsx:unarchive`; update `/opsx:sync` to note CLI delegation + determinism; document `openspec sync`, `openspec format`, `openspec diff`, and `openspec check` with their flags (and that `format`/`diff`/`check` need no agent/skill); document running `check` as a pre-commit hook and in CI. -- [ ] 7.2 Note the `specs/` = shipped-reality invariant and why `archive` is retained (design Decision 5), so the "why not just remove archive" question has a documented answer. - -## 8. End-to-end verification - -- [ ] 8.1 E2E determinism: scaffold a change with ADDED/MODIFIED/REMOVED/RENAMED; `openspec sync` twice → byte-identical `specs/`; `sync --check` clean. -- [ ] 8.2 E2E round-trip: `archive` then `unarchive` → `specs/` byte-identical to pre-archive, folder back under `changes//`. -- [ ] 8.3 E2E stacked drift: change A MODIFIES req X and is archived; a second change re-MODIFIES X and is archived; `unarchive A` refuses spec reversal; `unarchive A --keep-specs` succeeds untouched. -- [ ] 8.4 E2E no-crumbs: sync, revise the delta, re-sync → `specs/` reflects only the current delta. -- [ ] 8.4a E2E formatter: mangle a spec's whitespace → `format --check` fails → `format` fixes it → `--check` passes; and `sync`/`archive` output passes `format --check` unchanged (shared-canonicalizer invariant). -- [ ] 8.4b E2E diff: edit a change's delta and proposal → `openspec diff ` shows the requirement change annotated with the proposal's rationale; a synced spec change is annotated with the originating change via provenance; a pre-baseline edit is shown without invented rationale. -- [ ] 8.4c E2E check gate: introduce drift (un-synced delta + non-canonical whitespace) → `openspec check` fails naming both gates → `openspec check --fix` remediates → `check` passes; then introduce a cross-change conflict → `check --all` still fails (not auto-fixable); confirm identical verdict whether run via the installed hook or the CI step. -- [ ] 8.5 Validation: `openspec validate add-deterministic-sync-and-unarchive --strict` passes; `openspec status` shows artifacts complete. -- [ ] 8.6 Run the suite on macOS, Linux, and Windows CI.