Skip to content

refactor(agent-task): persist CookRunState; write it only through replay-keyed transitions (#18498) - #18510

Merged
chubes4 merged 1 commit into
mainfrom
fix/issue-18498-homeboy
Nov 19, 2026
Merged

chubes4 merged 1 commit into
mainfrom
fix/issue-18498-homeboy

Conversation

@chubes4

@chubes4 chubes4 commented Nov 19, 2026

Copy link
Copy Markdown
Member

Slice 1b of #18436 (tracked in #18498). Completes slice 1, together with #18495.

Summary

CookRunState is now persisted on the authoritative lifecycle record and written only through the replay-keyed transition API in cook_run_state.rs. The same event with the same key is a no-op, and an illegal transition is a typed error. CookRunState::of(record) keeps its signature. It prefers the persisted state and falls back to the legacy derivation for records written before this change; that fallback is the migration path and carries a removal condition.

What changed

  • records.rs: a persisted Cook state field with serde defaults, so existing records still load.
  • cook_run_state.rs: the transition API, replay keys, and a write restriction that keeps writes inside the state module.
  • Lifecycle writers that change the Cook phase route through transitions.
  • Tests: idempotency and replay, illegal-transition refusal, legacy-record fallback, and store round-trip.
  • docs/internals/developer-guide/cook-recovery.md is updated.

Verification

  • Candidate 28ca1b0, run locally by the operator: cargo fmt --check passed. The full homeboy-agents lib suite passed: 1749 tests, plus 732 rerun after the hermetic harness timed out under load, plus 1 ignored. 3 timing flakes pass when run alone. Per-test timing matches the base branch.
  • After the rebase onto main, cargo check -p homeboy-core -p homeboy-agents -p homeboy-cli --all-targets passes.
  • A Cook rework run will repeat the full gates on this PR, with a 3600s gate limit; the full suite exceeds Lab's 1800s default under load.
  • Public JSON contracts are unchanged.

Remaining on #18436

AI assistance

Homeboy Cook (roadie/cook-heavy, served claude-opus-5-5) wrote the change. Claude (via Roadie/Pi) verified it locally, rebased the branch, and opened this PR. Chris Huber directed the work and owns it.

…lay-keyed transitions (#18498)

Slice 1b of #18436. Cook candidate 28ca1b0 from agent-task-9d181fe8.
@chubes4

chubes4 commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

Verified on head aeb1b55. cargo fmt --check passes. cargo test -p homeboy-agents --lib: all 2489 tests pass. Under Lab load the full run hit the local hermetic timeout. Four timing flakes failed under load but passed with --exact --test-threads=1 (dispatch_promotes_provider_config_secret_env, output_liveness_marker_keeps_a_delegated_phase_alive, provider_timeout_persists_bounded_redacted_last_activity, promotion_owner_liveness_decides_terminal_disposition). The rest passed in a follow-up run. The rework Cook agent-task-f7e29e82 only moved a doc comment and failed gate 4 on the same timing flakes, so it was cancelled. Merges cleanly with #18511. (Claude via Roadie.)

@chubes4
chubes4 marked this pull request as ready for review November 19, 2026 04:41
@chubes4
chubes4 merged commit c303bf1 into main Nov 19, 2026
17 checks passed
@chubes4
chubes4 deleted the fix/issue-18498-homeboy branch November 19, 2026 06:41
chubes4 added a commit that referenced this pull request Nov 19, 2026
chubes4 added a commit that referenced this pull request Nov 19, 2026
…8511 read projections (#18524)

Main is red after #18510 and #18511 merged one after the other. Each PR
was green against its own base. Together, these fail deterministically
on the Lab and in CI:
- `homeboy-agents
orchestration::tests::review_preserves_authoritative_aggregate_absence`:
expected `CandidateRecoverable`, got `Succeeded`.
- `homeboy-agents
agent_task_service::cook::tests::cook_observation_agrees_across_real_stores_without_mutation`
- `homeboy-cli
…status_run_fixtures::agent_task_status_published_receipt_wins_over_terminal_and_promotion_snapshots`

Bisect: 1ae3148 (before #18510) passes, and c303bf1 (#18510) fails.
#18510 makes the persisted CookRunState override derivation, while
#18511's read projections and these tests depend on the derived verdict.
One example is an unverified success that reads as
`CandidateRecoverable`.

This reverts #18510's CookRunState persistence. It keeps #18510's
unrelated `deps.rs` fix, where a deadline that wins the cancel race
stays a timeout. #18498 reopens so the persistence can reland rebased on
#18511.

Lab at this SHA: the 3 tests above pass, and `cook_run_state` passes
5/5.

---
Written by Claude (claude-opus via Claude Code): bisected the red main
and prepared the revert.
homeboy-ci Bot added a commit that referenced this pull request Nov 19, 2026
### Added
- report runner pressure and Homeboy-owned root sizes in status and runner doctor

### Changed
- persisted CookRunState (#18510) conflicts with #18511 read projections

### Fixed
- a superseded Cook step settles when its owner's mission is fenced
chubes4 added a commit that referenced this pull request Nov 19, 2026
…ence verdicts (#18546)

## Summary
Relands persistence for #18436 and #18498, following #18510 and its
revert in #18524. Only transition-owned execution facts are persisted;
the shared classifier layers current evidence over that phase so
unverified success remains recoverable. Additive: legacy records remain
readable and public JSON contracts are unchanged.

## What changed
- Restore revision-fenced, replay-keyed phase persistence with one write
owner.
- Keep evidence refinements, fencing and liveness out of durable
verdicts.
- Preserve read projections and replay, refusal, fallback and round-trip
coverage.
- Add regression coverage for persisted and legacy evidence agreement.
- Document the persistence and observation boundary.

## Verification
-
0-verify-194fabc0129ce4a90d69f396041f7dfad35653be552de104cfe6bc624f183b5f.txt
(`cargo fmt --check`): passed on `f785dc4`. Base `main`@`3cbb92a`
advanced to `5c0e5e7`. 27593 ms; [retained gate metadata](#cook-run).

-
1-verify-97abbbd2ffe64d2ede888d26337c6c105146db892f530a8a1328477a1db13564.txt
(`cargo check -p homeboy-core -p homeboy-agents -p homeboy-cli
--all-targets`): passed on `f785dc4`. Base `main`@`3cbb92a` advanced to
`5c0e5e7`. 191595 ms; [retained gate metadata](#cook-run).

-
2-verify-f6401ed85d7b0a6e2f3622f222798d4447309ec16e461166d6d4d0b9ca7bf9cc.txt
(`cargo test -p homeboy-lifecycle-contract -p
homeboy-control-plane-contract --lib`): passed (63/63) on `f785dc4`.
Base `main`@`3cbb92a` advanced to `5c0e5e7`. 11321 ms; [retained gate
metadata](#cook-run).

-
3-verify-adb385d9d27ece7111020558a1ad5b1dde1db8aa933324e0bdbfc61905b8b2e0.txt
(`cargo test -p homeboy-cli status_run_fixtures`): passed (2/2) on
`f785dc4`. Base `main`@`3cbb92a` advanced to `5c0e5e7`. 149615 ms;
[retained gate metadata](#cook-run).

-
4-verify-3c6e494f77a85c48b076eb2ae2b565b036e0c82a28585bdb46ed835f1c2494ad.txt
(`cargo test -p homeboy-agents --lib -- --test-threads=8`): passed
(2502/2504) on `f785dc4`. Base `main`@`3cbb92a` advanced to `5c0e5e7`.
1235223 ms; [retained gate metadata](#cook-run).

- HOMEBOY_LINT_BASE=3cbb92a8545e63e752255486b662e994c2bad209 homeboy
--placement l
(`HOMEBOY_LINT_BASE=3cbb92a8545e63e752255486b662e994c2bad209 homeboy
--placement local review lint homeboy --path . --cha…`): passed on
`f785dc4`. Base `main`@`3cbb92a` advanced to `5c0e5e7`. 267024 ms;
[retained gate metadata](#cook-run).
- repro: none (no repro gate was declared, so these gates do not prove
the issue).
- CI: pending

<a id="cook-run"></a>
<details><summary>Cook run</summary>

- Gate 1: tree `50922c56b339a5104939b6927020d8a825ad30f8`
- Gate 2: tree `50922c56b339a5104939b6927020d8a825ad30f8`
- Gate 3: tree `50922c56b339a5104939b6927020d8a825ad30f8`; 0 failed, 0
ignored
- Gate 4: tree `50922c56b339a5104939b6927020d8a825ad30f8`; 0 failed, 0
ignored
- Gate 5: tree `50922c56b339a5104939b6927020d8a825ad30f8`; 0 failed, 2
ignored
- Gate 6: tree `50922c56b339a5104939b6927020d8a825ad30f8`

</details>

## Evidence
- The three reported regressions pass, along with focused state, store
and transition tests; formatting and changed-code lint checks pass.

## AI disclosure
Tool: Homeboy Cook. Agent: Homeboy (roadie). Model: roadie · route
cook-heavy · served gpt-6.1-sol. Cook time: 2h58m.

AI assistance: AI reviewed the issues and revert, implemented the
reland, and ran focused regression and lint checks, under the direction
of the Cook requester. Gates were run by Homeboy. A human reviews before
merge.

Closes #18498
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant