Skip to content

Add typed exit codes and confirmation gate to skill sync/upgrade#5897

Open
samuv wants to merge 2 commits into
skills-lock/05-upgradefrom
skills-lock/06-cli-exitcodes
Open

Add typed exit codes and confirmation gate to skill sync/upgrade#5897
samuv wants to merge 2 commits into
skills-lock/05-upgradefrom
skills-lock/06-cli-exitcodes

Conversation

@samuv

@samuv samuv commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Why: RFC THV-0080 gives CI a scriptable contract for sync/upgrade: exit 2 for check/freshness failures, 3 for partial failures, 4 for policy rejections — distinct from the generic 1 cobra already uses everywhere else. It also requires a pre-install confirmation gate: skill content is a set of AI-followed instructions, so a non-interactive run must refuse outright without --yes rather than silently proceeding. This is the last PR of Stack 1.
  • What:
    • cmd/thv/app/exitcode.go: exitCodeError lets a RunE carry a specific exit code back through cobra; main.go maps it via ExitCodeFromError instead of a hardcoded os.Exit(1).
    • cmd/thv/app/skill_confirm.go: requireConfirmation prompts on a TTY, refuses with ExitCodePolicyRejection off one (matching the RFC's stated rationale, not the workload-upgrade command's precedent of silently skipping the prompt when non-interactive — skill content is higher-stakes).
    • sync/upgrade gain --yes, skipped automatically by --check/--preview (which never install anything). Exit-code mapping prioritizes partial failure over drift/ref-change-blocked (a real operational failure is a stronger signal than a guard doing its job).
    • docs/arch/12-skills-system.md: new "Project Lock File" section covering the rollout gate, schema, install/uninstall hooks, sync, upgrade, and the exit-code contract — plus updates to the existing storage/API/CLI tables.
    • CLI E2E coverage for exit codes 0, 2, and 4 (--check clean/drifted, non-interactive refusal, --yes bypass).

Part of the production stack for #5715 (RFC THV-0080). Stack: 6/6 (final PR of Stack 1) — lockfile → lock-service → install-hooks → sync → upgrade → cli-exitcodes. Based on #5896 (PR5).

Type of change

  • New feature

Test plan

  • Unit tests (task test)
  • E2E tests (task test-e2e, scoped: ginkgo --label-filter=skills-lock)
  • Linting (task lint-fix)

Unit tests for ExitCodeFromError/withExitCode (including error wrapping), syncExitError/upgradeExitError precedence rules, requireConfirmation's --yes and non-interactive-refusal paths (the interactive TTY-prompt branch is untestable in a normal go test process and is left uncovered, matching the existing confirmUpgrade helper for workload upgrades — this codebase's established precedent for interactive-only code). New CLI E2E spec file drives the real thv binary against a real server process and asserts actual process exit codes via *exec.ExitError.

Also verified against a clean origin/main worktree that a batch of 24 unrelated skills E2E failures on this dev machine (extracting skill: target path validation: symlink found at "/var") are a pre-existing macOS-local environment quirk (/var/private/var), not something this stack introduced.

Does this introduce a user-facing change?

Not yet — same TOOLHIVE_SKILLS_LOCK_ENABLED gate as the rest of the stack. Once the full RFC v1 (this stack + the later Sigstore stack) lands: thv skill sync/upgrade will prompt for confirmation before installing and use the documented exit codes for scripting.

Special notes for reviewers

  • This PR intentionally does not flip the TOOLHIVE_SKILLS_LOCK_ENABLED default — per the plan agreed for this stack, the feature stays gated until the Sigstore stack (built on top of this one) also lands, since the RFC frames "v1" as lock-file and signing together.

🤖 Generated with Claude Code

@github-actions github-actions Bot added the size/L Large PR: 600-999 lines changed label Jul 21, 2026
@samuv
samuv force-pushed the skills-lock/05-upgrade branch from dd81407 to a7e9269 Compare July 21, 2026 15:31
@samuv
samuv force-pushed the skills-lock/06-cli-exitcodes branch from 726ce33 to 9efdc60 Compare July 21, 2026 15:31
@github-actions github-actions Bot added size/L Large PR: 600-999 lines changed and removed size/L Large PR: 600-999 lines changed labels Jul 21, 2026
@codecov

codecov Bot commented Jul 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (skills-lock/05-upgrade@7aa5c69). Learn more about missing BASE report.

Additional details and impacted files
@@                    Coverage Diff                    @@
##             skills-lock/05-upgrade    #5897   +/-   ##
=========================================================
  Coverage                          ?   71.64%           
=========================================================
  Files                             ?      707           
  Lines                             ?    72271           
  Branches                          ?        0           
=========================================================
  Hits                              ?    51777           
  Misses                            ?    16739           
  Partials                          ?     3755           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@samuv samuv self-assigned this Jul 21, 2026
@samuv
samuv force-pushed the skills-lock/05-upgrade branch from a7e9269 to 9deff64 Compare July 21, 2026 16:58
@samuv
samuv force-pushed the skills-lock/06-cli-exitcodes branch from 9efdc60 to 0e58495 Compare July 21, 2026 16:58
@github-actions github-actions Bot added size/L Large PR: 600-999 lines changed and removed size/L Large PR: 600-999 lines changed labels Jul 21, 2026
@samuv
samuv force-pushed the skills-lock/05-upgrade branch from 9deff64 to 3e750f4 Compare July 21, 2026 17:08
@samuv
samuv force-pushed the skills-lock/06-cli-exitcodes branch from 0e58495 to 58e1107 Compare July 21, 2026 17:08
@github-actions github-actions Bot added size/L Large PR: 600-999 lines changed and removed size/L Large PR: 600-999 lines changed labels Jul 21, 2026
@samuv
samuv force-pushed the skills-lock/05-upgrade branch from 3e750f4 to cf3c890 Compare July 21, 2026 17:14
@samuv
samuv force-pushed the skills-lock/06-cli-exitcodes branch from 58e1107 to 80ae30f Compare July 21, 2026 17:14
@github-actions github-actions Bot removed the size/L Large PR: 600-999 lines changed label Jul 21, 2026
@github-actions github-actions Bot added the size/L Large PR: 600-999 lines changed label Jul 21, 2026
@samuv
samuv force-pushed the skills-lock/05-upgrade branch from cf3c890 to aecb406 Compare July 21, 2026 17:17
@samuv
samuv force-pushed the skills-lock/06-cli-exitcodes branch from 80ae30f to 5f504ce Compare July 21, 2026 17:18
@github-actions github-actions Bot added size/L Large PR: 600-999 lines changed and removed size/L Large PR: 600-999 lines changed labels Jul 21, 2026
@samuv
samuv force-pushed the skills-lock/06-cli-exitcodes branch from 5f504ce to 02ef32f Compare July 21, 2026 17:25
@github-actions github-actions Bot added size/L Large PR: 600-999 lines changed and removed size/L Large PR: 600-999 lines changed labels Jul 21, 2026
samuv added 2 commits July 21, 2026 19:34
RFC THV-0080 gives CI a scriptable contract for sync/upgrade: exit
2 for check/freshness failures, 3 for partial failures, 4 for
policy rejections, distinct from the generic 1 cobra already uses.
It also requires a pre-install confirmation gate — skill content
is a set of AI-followed instructions, so a non-interactive run
must refuse outright without --yes rather than silently proceeding.

- cmd/thv/app/exitcode.go: exitCodeError carries a specific exit
  code through a RunE return; main.go maps it via
  ExitCodeFromError instead of a hardcoded os.Exit(1)
- cmd/thv/app/skill_confirm.go: requireConfirmation prompts on a
  TTY, refuses with ExitCodePolicyRejection off one
- sync/upgrade gain --yes, skipped automatically by --check/
  --preview (which never install anything); their exit-code
  mapping prioritizes partial failure over drift/ref-change-blocked
- docs/arch/12-skills-system.md: new Project Lock File section
  covering rollout gating, schema, install/uninstall hooks, sync,
  upgrade, and the exit-code contract
- CLI E2E coverage for all four exit codes (0/2/4, plus --yes)

This is the last PR of Stack 1 (lock file + sync + upgrade) for
RFC THV-0080. Stack 2 (Sigstore signing/verification) builds on
top once this stack is fully merged; TOOLHIVE_SKILLS_LOCK_ENABLED
stays off by default until that full v1 lands.

Part of the production stack for #5715 (RFC THV-0080). Stack: 6/6.
upgradeExitError mapped a ref-change-blocked outcome to exit 4
unconditionally, even during --preview — where nothing was actually
blocked, only reported. Thread preview through, mirroring how
syncExitError already gates Drifted behind --check. Also reconcile a
docs contradiction about --preview's side effects, drop a stale
signer-change reference in the exit-code doc comment (that guard
doesn't exist until Stack 2), and add upgrade exit-code E2E coverage.
@samuv
samuv force-pushed the skills-lock/05-upgrade branch from 664f512 to 7aa5c69 Compare July 21, 2026 17:34
@samuv
samuv force-pushed the skills-lock/06-cli-exitcodes branch from f0d9cc9 to 840b9d4 Compare July 21, 2026 17:34
@github-actions github-actions Bot added size/L Large PR: 600-999 lines changed and removed size/L Large PR: 600-999 lines changed labels Jul 21, 2026
@samuv

samuv commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

Tracked in #5899 (RFC THV-0080 stack).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Large PR: 600-999 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants