agent: coop agents could not tell a finished peer from a silent one - #79
Merged
Conversation
A coop pair has exactly two channels for coordinating, and both reported success while conveying nothing. Messaging: when an agent submitted and exited, its peer kept sending into a mailbox nobody would read again. send() queued to Redis and returned success unconditionally, with no liveness check. Over 8 flash_10 pairs: 16 messages sent, 11 delivered (31% lost); in pallets_click_task/2800/f1_f7 all 3 were lost, sent 35s after the peer's final turn. The sender was told "Message sent to agent2" (rc 0) each time and recorded in its own summary that "coordination with agent2 is ongoing ... no expected conflicts" immediately before submitting a patch that merge-conflicted. --wait was documented in the prompt but never implemented: the call site guarded on hasattr(comm, "send_and_wait") and no such method existed, so a blocking question silently became fire-and-forget. Git: the prompt presents team/<peer> as the sanctioned way to read a colleague's code, but setup() pushes the base commit once and nothing updates it afterwards. Across 19 agent runs there were zero pushes -- one agent ran 24 fetch/diff commands against team/agent2, saw the untouched baseline every time, and asked "have you submitted your branch yet?". Fixes: - send() returns False for a departed peer; the agent gets a non-zero result naming the cause and the recovery, not a false success. - send_and_wait() implemented; ends on reply OR peer exit rather than burning the full timeout. - each agent publishes its SUBMITTED PATCH to team/<agent_id> on exit. patch.txt is what gets evaluated and may be a subset of the tree, so publishing the tree would show the peer something other than what is merged. Built in a detached worktree from the pristine base: the agent's branch, index and working tree are untouched, and a patch is never double-applied when the agent had already committed. Publication is best-effort, so the exit marker records whether it actually landed and peers are pointed at the branch only when it really holds the submission. - peers are told once when a colleague exits, with the branch to reconcile. - prompt states what team/<peer> holds and when, that --wait can return early, and that a colleague may exit first. Also adds the missing 0.0.20 / 0.0.21 changelog entries.
The connector tests cover Redis; these cover what the agent actually observes, which is where the bug lived -- _handle_send_message reported returncode 0 and 'Message sent to agent2' no matter what, and the sender believed it. Driven directly rather than through a live rollout: whether an agent calls send_message at all is up to the model. A real 1-pair run on the pair that lost all 3 messages pre-fix completed with neither agent messaging, so it could not confirm the fix either way. Pins in particular that _publish_final_work branches from team/main rather than HEAD (else a patch double-applies when the agent already committed) and that a failed publish reports False so peers are never sent to read a baseline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A coop pair has exactly two channels for coordinating. Both reported success while conveying nothing.
Found while analysing why Qwen3.5-9B scores 0/9 pairs on
flash_10— the agents were coordinating, and the harness was throwing it away.1. Messages to a finished peer vanish, and the sender is told they succeeded
When an agent submits and exits, its peer keeps sending into a mailbox nobody will read again.
send()queued to Redis and returned success unconditionally — no liveness check.Measured over 8
flash_10pairs: 16 sent, 11 delivered (31% lost).pallets_click/2800/f1_f7openai_tiktoken/0/f6_f8pillow/290/f4_f5go_chi/26/f1_f2In the click pair all three were sent 35s after the peer's final turn. The sender got
Message sent to agent2(rc 0) each time, then wrote in its own summary:…and submitted a patch that merge-conflicted. A merge conflict is an automatic double failure, so that pair scored 0 on both features.
2.
--waitwas documented but never implemented_handle_send_messageguarded onhasattr(self.comm, "send_and_wait");MessagingConnectorhad no such method. Sosend_message --wait— advertised in the prompt as "block until your colleague responds" — silently degraded to fire-and-forget.3.
team/<peer>never contained the peer's workGitConnector.setup()pushes the base commit once; nothing updates the branch afterwards. The prompt presents that remote as the way to see a colleague's code.Zero pushes across all 19 agent runs. One agent ran 24
git fetch/git diff team/agent2commands, saw the untouched baseline every time, and asked "have you submitted your branch yet?" — to a peer that had already finished.Fixes
send()returnsFalsefor a departed peer; the agent gets a non-zero result naming the cause and the recovery path.send_and_wait()implemented — ends on reply or peer exit, instead of burning the full 60s.team/<agent_id>on exit.patch.txtis the evaluated artifact and may be a subset of the working tree, so publishing the tree would show the peer something other than what gets merged.team/<peer>holds and when, that--waitcan return early, and that a colleague may exit first.Publication is careful about two things
It must not lie. Publication is best-effort, so the exit marker records whether the patch actually landed (
mark_exited(published=...)), and peers are pointed at the branch only when it really holds the submission. Otherwise this would reintroduce exactly the bug being fixed.It must not corrupt the agent's state. Built in a detached worktree from the pristine base (
team/main), not fromHEAD— if the agent had already committed its work, applyingpatch.txton top ofHEADwould double-apply it. The agent's branch, index and working tree are untouched, and the adapter still readspatch.txtafterwards.Testing
tests/: 392 passed, 63 skipped. The 2test_team_wiring.pyfailures are a missing optionalopenhandsmodule and reproduce on cleanmain.test_messaging.pycovering: delivery to a live vs departed peer, thepublishedflag distinguishing a failed publish,--waitreturning in <5s on a departed peer (vs 60s), abandoning the wait when the peer exits mid-wait, and stale exit markers being cleared on a fresh run.HEADand working tree unchanged, no worktrees leaked, and a missingpatch.txtexits non-zero rather than reporting a successful publication.Bumps to 0.0.22; adds the missing 0.0.20 / 0.0.21 changelog entries.