Skip to content

fix: terminate timed-out worker process groups - #80

Open
BunsDev wants to merge 1 commit into
mainfrom
codex/fix-orphaned-processes-in-timeout-handler
Open

fix: terminate timed-out worker process groups#80
BunsDev wants to merge 1 commit into
mainfrom
codex/fix-orphaned-processes-in-timeout-handler

Conversation

@BunsDev

@BunsDev BunsDev commented Aug 14, 2026

Copy link
Copy Markdown
Member

Motivation

  • The host-backed worker timed out coven-code by killing only the immediate child, allowing background descendants to survive and retain repository tokens or resources.
  • Surviving descendants can be left behind across retries or after workspace cleanup, undermining the timeout_secs security guarantee and enabling cross-task leakage or resource exhaustion.

Description

  • Start host-launched coven-code in its own Unix process group and terminate the whole group on timeout by adding a ProcessGroup kill path and calling libc::kill(-pid, SIGKILL) on Unix (no-op fallback on non-Unix). (crates/worker/src/backend.rs)
  • Introduce a KillTarget enum and update await_child to handle process-group termination for host runs and retain the existing container kill flow for container backend runs. (crates/worker/src/backend.rs)
  • Add libc as a workspace dependency and enable libc for the worker crate so process-group signaling is available. (Cargo.toml, crates/worker/Cargo.toml)
  • Strengthen the timeout regression test to spawn a background descendant that attempts a delayed write and assert that the descendant cannot outlive the timeout cleanup. (crates/worker/src/lib.rs)
  • Commits include the required DCO Signed-off-by: trailer per project policy; modified files: Cargo.toml, Cargo.lock, crates/worker/Cargo.toml, crates/worker/src/backend.rs, and crates/worker/src/lib.rs.

Testing

  • Ran cargo check --all-targets and it completed successfully.
  • Ran cargo clippy --all-targets -- -D warnings and it completed successfully with no warnings.
  • Executed the timeout regression and unit suites: NO_PROXY=127.0.0.1,localhost no_proxy=127.0.0.1,localhost cargo test --all which passed; the focused test process_tests::coven_code_process_is_stopped_after_configured_timeout was also run and passed locally.
  • Note: an earlier cargo test --all invocation in this environment failed due to an HTTP proxy / Wiremock routing issue (HTTP 403) unrelated to the change; re-running tests with NO_PROXY scoped to localhost produced all-green results.

Codex Task

Signed-off-by: Codex <codex@openai.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the host-backed worker timeout behavior by ensuring coven-code is launched in its own Unix process group and that a timeout terminates the entire group, preventing background descendants from surviving and retaining tokens/resources.

Changes:

  • Add a KillTarget abstraction and host process-group setup/termination path in the worker backend.
  • Add libc as a workspace dependency to support Unix kill(-pgid, SIGKILL) signaling.
  • Strengthen the timeout regression test to detect surviving background descendants after timeout.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
Cargo.toml Adds libc to workspace dependencies for Unix process-group signaling.
Cargo.lock Locks libc dependency resolution in the workspace.
crates/worker/Cargo.toml Enables libc for the worker crate.
crates/worker/src/backend.rs Launches host sessions in a process group and adds timeout logic to kill process groups or containers based on KillTarget.
crates/worker/src/lib.rs Updates the timeout regression test to spawn a background descendant and assert it cannot outlive timeout cleanup.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +211 to 220
if matches!(kill, KillTarget::ProcessGroup) {
kill_process_group(&mut child).await;
} else {
let _ = child.kill().await;
}
let _ = child.wait().await;
// …then the container itself: killing the docker CLI does not
// Killing the docker CLI does not
// reliably stop the container it launched.
if let Some(kill) = kill {
if let KillTarget::Container(kill) = kill {
match Command::new(&kill.docker_bin)
Comment on lines +237 to +244
// SAFETY: `pid` belongs to the child we spawned as a new process-group
// leader. A negative pid asks kill(2) to signal that entire group.
unsafe {
libc::kill(-(pid as i32), libc::SIGKILL);
}
} else {
let _ = child.kill().await;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants