fix(runner): stop the timeout watchdog pinning the caller's output - #1273
Merged
Chemaclass merged 2 commits intoAug 16, 2026
Conversation
`run_with_timeout` forks a watchdog that sleeps out the timeout budget. It was detached on 0/1/2 only, so it still inherited both descriptors the runner dups from stdout -- FD 3, handed to the test body, and FD 5, kept for the EXIT trap -- plus the outer run's pair when the test spawns a nested bashunit. A watchdog that outlived its parent therefore held the pipe its caller was reading with no visible writer, and that caller waited out the whole budget for an EOF that could not arrive. Verified on a real Bash 3.0 build: a SIGKILLed run blocked its capture for 20 of 20 seconds. Only a `--test-timeout` run reaches this, which is why the suites acceptance test is the one that shows it: `[suite:acceptance]` carries `test-timeout = 60`, so the union case is the only test there whose nested run arms a watchdog at all. The conditions for an orphan -- the group kill that ends it "intermittently misses", per the comment already in that function -- are load-dependent, matching a flake seen only on loaded CI. The watchdog now closes both descriptors with `exec`, which closes for good; a per-command `3>&- 5>&-` would not, because bash implements that by dup'ing each descriptor to a fd >= 10 that the child inherits anyway. It also refuses to fire once the run it was policing is gone: after a full budget the pid it holds may name something else, and the two kills that follow signal a process GROUP. Related #1137
Closing the inherited descriptors keeps an orphaned watchdog from pinning its caller's output, but it still slept out the whole budget before checking anything -- so a killed run left a process behind for up to `--test-timeout` seconds, and CI reported it as an orphan. Wait in one-second steps instead, giving up as soon as either the run or the test being watched is gone. That also retires the "intermittently misses" group kill as a correctness dependency: a watchdog the signal misses now exits on its own within a second rather than waking up later to signal a process GROUP by a pid the kernel may have reused. The regression test now generates its fixture so the body can announce itself, and the kill lands at that moment rather than after a guessed delay. A fixed 2s sleep killed the run before it had started a test -- or printed anything -- on the slower macOS runner, which is what failed the build-verify job rather than the code under test. Related #1137
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤔 Background
Related #1137
[suite:acceptance]in that test's fixture carriestest-timeout = 60, so the union case is the only test in the file whose nested run arms a timeout watchdog — the real difference between it and its neighbours, not theparallel/no-parallelconflict the issue guessed at (suite options resolve in argv order, deterministically).That watchdog was detached on stdin/stdout/stderr only, so it still inherited both descriptors the runner dups from stdout, plus the outer run's pair when a test spawns a nested bashunit. One that outlived its parent held the pipe its caller was reading with no visible writer, and the caller waited out the whole timeout budget for an EOF that could not arrive. Verified on a real Bash 3.0 build: a killed run blocked its capture for 20 of 20 seconds, 5 after.
💡 Changes
exec; a per-command3>&- 5>&-would not, since bash implements that by dup'ing each to a fd >= 10 the child inherits anyway