Skip to content

agent: coop agents could not tell a finished peer from a silent one - #79

Merged
akhatua2 merged 2 commits into
mainfrom
coop-visibility-fixes
Aug 4, 2026
Merged

agent: coop agents could not tell a finished peer from a silent one#79
akhatua2 merged 2 commits into
mainfrom
coop-visibility-fixes

Conversation

@akhatua2

@akhatua2 akhatua2 commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

A coop pair has exactly two channels for coordinating. Both reported success while conveying nothing.

Found while analysing why Qwen3.5-9B scores 0/9 pairs on flash_10 — the agents were coordinating, and the harness was throwing it away.

1. Messages to a finished peer vanish, and the sender is told they succeeded

When an agent submits and exits, its peer keeps sending into a mailbox nobody will read again. send() queued to Redis and returned success unconditionally — no liveness check.

Measured over 8 flash_10 pairs: 16 sent, 11 delivered (31% lost).

pair sent delivered
pallets_click/2800/f1_f7 3 0
openai_tiktoken/0/f6_f8 4 3
pillow/290/f4_f5 6 6
go_chi/26/f1_f2 2 1

In the click pair all three were sent 35s after the peer's final turn. The sender got Message sent to agent2 (rc 0) each time, then wrote in its own summary:

"Coordination with agent2 is ongoing via send_message — edits target different files, no expected conflicts."

…and submitted a patch that merge-conflicted. A merge conflict is an automatic double failure, so that pair scored 0 on both features.

2. --wait was documented but never implemented

_handle_send_message guarded on hasattr(self.comm, "send_and_wait"); MessagingConnector had no such method. So send_message --wait — advertised in the prompt as "block until your colleague responds" — silently degraded to fire-and-forget.

3. team/<peer> never contained the peer's work

GitConnector.setup() pushes the base commit once; nothing updates the branch afterwards. The prompt presents that remote as the way to see a colleague's code.

Zero pushes across all 19 agent runs. One agent ran 24 git fetch / git diff team/agent2 commands, saw the untouched baseline every time, and asked "have you submitted your branch yet?" — to a peer that had already finished.

Fixes

  • send() returns False for a departed peer; the agent gets a non-zero result naming the cause and the recovery path.
  • send_and_wait() implemented — ends on reply or peer exit, instead of burning the full 60s.
  • Each agent publishes its submitted patch to team/<agent_id> on exit. patch.txt is the evaluated artifact and may be a subset of the working tree, so publishing the tree would show the peer something other than what gets merged.
  • Peers are told once when a colleague exits, with the branch to reconcile against.
  • Prompt states what team/<peer> holds and when, that --wait can return early, and that a colleague may exit first.

Publication is careful about two things

It must not lie. Publication is best-effort, so the exit marker records whether the patch actually landed (mark_exited(published=...)), and peers are pointed at the branch only when it really holds the submission. Otherwise this would reintroduce exactly the bug being fixed.

It must not corrupt the agent's state. Built in a detached worktree from the pristine base (team/main), not from HEAD — if the agent had already committed its work, applying patch.txt on top of HEAD would double-apply it. The agent's branch, index and working tree are untouched, and the adapter still reads patch.txt afterwards.

Testing

  • tests/: 392 passed, 63 skipped. The 2 test_team_wiring.py failures are a missing optional openhands module and reproduce on clean main.
  • 9 new tests in test_messaging.py covering: delivery to a live vs departed peer, the published flag distinguishing a failed publish, --wait returning in <5s on a departed peer (vs 60s), abandoning the wait when the peer exits mid-wait, and stale exit markers being cleared on a fresh run.
  • The publish script was verified against a real git fixture including the "agent already committed" case: published branch is base+patch applied exactly once, agent's HEAD and working tree unchanged, no worktrees leaked, and a missing patch.txt exits non-zero rather than reporting a successful publication.

Bumps to 0.0.22; adds the missing 0.0.20 / 0.0.21 changelog entries.

A coop pair has exactly two channels for coordinating, and both reported success
while conveying nothing.

Messaging: when an agent submitted and exited, its peer kept sending into a
mailbox nobody would read again. send() queued to Redis and returned success
unconditionally, with no liveness check. Over 8 flash_10 pairs: 16 messages sent,
11 delivered (31% lost); in pallets_click_task/2800/f1_f7 all 3 were lost, sent
35s after the peer's final turn. The sender was told "Message sent to agent2"
(rc 0) each time and recorded in its own summary that "coordination with agent2
is ongoing ... no expected conflicts" immediately before submitting a patch that
merge-conflicted.

--wait was documented in the prompt but never implemented: the call site guarded
on hasattr(comm, "send_and_wait") and no such method existed, so a blocking
question silently became fire-and-forget.

Git: the prompt presents team/<peer> as the sanctioned way to read a colleague's
code, but setup() pushes the base commit once and nothing updates it afterwards.
Across 19 agent runs there were zero pushes -- one agent ran 24 fetch/diff
commands against team/agent2, saw the untouched baseline every time, and asked
"have you submitted your branch yet?".

Fixes:
- send() returns False for a departed peer; the agent gets a non-zero result
  naming the cause and the recovery, not a false success.
- send_and_wait() implemented; ends on reply OR peer exit rather than burning
  the full timeout.
- each agent publishes its SUBMITTED PATCH to team/<agent_id> on exit. patch.txt
  is what gets evaluated and may be a subset of the tree, so publishing the tree
  would show the peer something other than what is merged. Built in a detached
  worktree from the pristine base: the agent's branch, index and working tree are
  untouched, and a patch is never double-applied when the agent had already
  committed. Publication is best-effort, so the exit marker records whether it
  actually landed and peers are pointed at the branch only when it really holds
  the submission.
- peers are told once when a colleague exits, with the branch to reconcile.
- prompt states what team/<peer> holds and when, that --wait can return early,
  and that a colleague may exit first.

Also adds the missing 0.0.20 / 0.0.21 changelog entries.
The connector tests cover Redis; these cover what the agent actually observes,
which is where the bug lived -- _handle_send_message reported returncode 0 and
'Message sent to agent2' no matter what, and the sender believed it.

Driven directly rather than through a live rollout: whether an agent calls
send_message at all is up to the model. A real 1-pair run on the pair that lost
all 3 messages pre-fix completed with neither agent messaging, so it could not
confirm the fix either way.

Pins in particular that _publish_final_work branches from team/main rather than
HEAD (else a patch double-applies when the agent already committed) and that a
failed publish reports False so peers are never sent to read a baseline.
@akhatua2
akhatua2 merged commit 4db20a0 into main Aug 4, 2026
3 checks passed
@akhatua2
akhatua2 deleted the coop-visibility-fixes branch August 4, 2026 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant