Skip to content

Per-test cleanup reads the whole shared temp directory, so old leftovers slow every later run #1269

Description

@Chemaclass

Problem

bashunit::cleanup_testcase_temp_files runs in every test's EXIT trap and expands:

matches=("$BASHUNIT_TEMP_DIR/${BASHUNIT_CURRENT_TEST_ID}"_*)

Expanding that glob makes bash read all of BASHUNIT_TEMP_DIR. That directory is shared (${TMPDIR:-/tmp}/bashunit/tmp) and persists between runs, and nothing ever reaps it — so every file an interrupted run left behind is re-examined by every test of every later run.

Worse, the scan can never match one of them: the id is built as <fn>_$$_<suffix>, so leftovers always carry a different PID. The work is pure overhead.

Measured

--no-parallel, one file of 100 trivial tests, varying only the number of unrelated files in BASHUNIT_TEMP_DIR:

leftovers run time
0 498 ms
500 513 ms
2000 635 ms
5000 978 ms

Linear and unbounded — the framework gets slower the longer the machine has been used. Profiling a 200-test run against a real 4,242-file directory, this one call was 3.16 ms of an 8.5 ms per-test total (37%), larger than the test body, both hooks, mock teardown and result encoding combined.

I hit this on my own machine: 4,242 files, 45 MB, all from runs killed mid-flight during a long session.

Proposal

Stat one known path instead of listing the directory. temp_file/temp_dir write a <id>_.mark marker (a redirect, not a fork); the trap returns early when it is absent, which is the common case since most tests create no temp file. The marker shares the <id>_ prefix, so the existing rm removes it along with everything else.

Cost becomes O(1) in the size of the directory:

leftovers before after
0 498 ms 520 ms
2000 635 ms 520 ms
5000 978 ms 542 ms

Behaviour change

A file a test writes into BASHUNIT_TEMP_DIR by hand, named with its own test id, is no longer removed — only files handed out by temp_file/temp_dir are. Finding a hand-rolled one is exactly the directory scan being removed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

  • Status
    Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions