[JENKINS-73447] Do not fail the build when docker top is unavailable - #752
Open
AbdulMateenzwl wants to merge 1 commit into
Open
[JENKINS-73447] Do not fail the build when docker top is unavailable#752AbdulMateenzwl wants to merge 1 commit into
docker top is unavailable#752AbdulMateenzwl wants to merge 1 commit into
Conversation
`docker top` requires cgroups, so it fails on rootless Docker and rootless dind even though the container is running normally. Its result is only a diagnostic — a mismatch merely logs an error and the build continues — but an `IOException` from `listProcess` propagated out of the step and aborted the build. Treat an unobtainable process list the same way: report it and skip the check. `InterruptedException` still propagates.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #716 / JENKINS-73447.
Problem
After starting the container,
WithContainerStepcallsdocker topto check that thecontainer is actually running the expected command (
cat/cmd.exe) rather thansomething from an
ENTRYPOINT.docker toprequires cgroups, so on rootless Docker androotless dind it fails outright:
Error response from daemon: runc did not terminate successfully: exit status 1:
unable to get all container pids: read /sys/fs/cgroup//cgroup.procs: operation not supported
The check is only ever advisory — when the process list does not contain the expected
command the step just logs an error and carries on — but an
IOExceptionfromlistProcesspropagated out ofExecution.start()and failed the build, even though thecontainer was healthy.
Fix
Catch the
IOException, log it atFINE, print a short note to the build log, and skipthe check.
InterruptedExceptionis deliberately not caught, so aborts still work.listProcessis left throwing rather than returning an empty list, since an empty listwould trip the
!ps.contains(command)branch and print the misleading "container startedbut didn't run the expected command" message.
Testing
Added
WithContainerStepTest.topFailureIsNotFatal, which installs aDockerToolwhosebin/dockeris a wrapper that fails ontopwith the daemon error above and delegatesevery other subcommand to the real
docker— a stand-in for a cgroup-less daemon thatneeds no refactoring of the step to inject a mock.
java.io.IOException: Failed to run top '<id>'andFinished: FAILURE, reproducing the reported bug.Could not verify the command running in the container: ....mvn verify -Dtest=WithContainerStepTestpasses locally against a real Linux daemon(15 tests, 0 failures, SpotBugs clean). The new test calls
assumeNotWindows(), since thewrapper is a
/bin/shscript, so it skips on the Windows CI leg.Note
The reporter also suggested a system property to disable the process-list check outright.
I left that out — making the check non-fatal resolves the reported failure without adding
configuration surface — but I'm happy to add one if you'd prefer it.