Skip to content

fix(libc): run guest exit handlers before the host exit handlers - #1658

Merged
oneandonlydean merged 1 commit into
boykopovar:mainfrom
morten-holm:pr/exit-guest-handlers-first
Oct 10, 2026
Merged

oneandonlydean merged 1 commit into
boykopovar:mainfrom
morten-holm:pr/exit-guest-handlers-first

Conversation

@morten-holm

Copy link
Copy Markdown
Contributor

What

LibcExit_nid_no_patch called std::exit after the shutdown callbacks. std::exit runs the static destructors of the host libraries (singletons that hold a std::mutex) before the guest exit handlers, because the guest handlers are run by an atexit runner that is registered at the first guest __cxa_atexit. A guest handler that freed memory then locked a destroyed std::mutex. On macOS std::mutex::lock then throws std::system_error ("Invalid argument"), and the noexcept destructor of the guest ended in std::terminate: terminate called without an active exception, exit code 134, on a window close and on SIGTERM.

LibcExit_nid_no_patch now calls CxaFinalize_nid_no_patch(nullptr) before std::exit. The guest handlers run first, while the library objects are alive. CxaFinalize removes each entry as it runs it, so the later atexit runner finds the list empty and nothing runs twice. A guest handler that calls exit again behaves as before (LibcExit already guards that).

Tested

  • New test guest_exit_order (UNIX only): a forked child registers a guest handler with __cxa_atexit, then a host std::atexit handler, then calls LibcExit; the parent reads the order through a pipe. It fails without the change (host handler first) and passes with it, 3 of 3 runs.
  • macOS 27 (Apple M2), x86-64 under Rosetta: one retail title. Before, a window close and SIGTERM ended with exit code 134. After, both end with exit code 0. I ran the test and the title on a macOS port tree whose libc builds; main does not build libc on macOS yet.
  • Not tested: Linux and Windows (CI). The test is UNIX only, so Windows has no coverage. The change is in shared code and moves the guest handlers from inside std::exit to just before it; I did not check other titles or platforms.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source (none added)
  • Generic behaviour, not specific to one title
  • Depends on: none
  • AI-assisted: yes

🤖 Generated with Claude Code

LibcExit called std::exit, which ran the static destructors of the libraries
before the guest exit handlers. A guest handler that freed memory then locked
a destroyed std::mutex, which throws, and a noexcept guest destructor ended in
std::terminate. The guest handlers now run first, while the library objects
are alive.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@morten-holm morten-holm mentioned this pull request Oct 8, 2026
@morten-holm

Copy link
Copy Markdown
Contributor Author

Additional result on mugurc's macos-port branch (4dc4045, plus local changes needed to run the title: fixed-base load address, registry and maxprot for the no-access segment, conditional VK_KHR_portability_enumeration, and the one-line change from this PR).

Setup: macOS 27, Apple M2, x86-64 under Rosetta, MoltenVK 1.4.2, one retail title (not named). The title runs for 120 s without a crash.

Window close:

  • Without this change the process ended with libc++abi: terminating due to uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument.
  • With this change, a click on the window close button after 20 s ends the run with exit code 0. The log ends after the MoltenVK device and instance are destroyed, and no process is left.

This is one run on macOS. SIGTERM, Linux and Windows are not tested on this tree; the unit test (guest_exit_order) covers the order on UNIX.

@hikazey

hikazey commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Linux and Windows runs, since the description lists both as untested. This commit merged onto main 17350cf5, which merges cleanly.

Linux (Ubuntu 24.04 container, GCC 14.2, Release, Ryzen 9 5900X host):

build guest_exit_order
this PR 5/5 passed
CxaFinalize_nid_no_patch(nullptr) removed, test kept 5/5 failed (guest exit handlers must run before the host exit handlers)
full suite with this PR 465/465. import_audit hit my 120 s ctest timeout in the full run; alone it passes in 236 s, which is slow I/O through the container's bind mount, not this change

Windows (Windows 11 Pro 26200, MinGW-w64 GCC 15.2.0 / WinLibs r7, Release, configured like CI). guest_exit_order is UNIX only because of fork, so I checked the order with a small local test without fork. It registers a guest handler with __cxa_atexit, then a host std::atexit handler, calls LibcExit_nid_no_patch(0), and the handlers write G/H to the stdout handle:

build output
this PR GH, 5/5
CxaFinalize_nid_no_patch(nullptr) removed HG, 5/5
full suite with this PR all 458 project tests passed

So the reorder takes effect on Windows as well. If you want Windows coverage in the PR, this is the test I used (registered under if(WIN32) next to the other libc guest tests, with PASS_REGULAR_EXPRESSION "GH", FAIL_REGULAR_EXPRESSION "HG" and the usual PATH=path_list_prepend:$<TARGET_FILE_DIR:libc>):

#include "prx/libc/include/Shutdown.hpp"
#include "prx/libc/include/general/VabiMacros.hpp"
#include <cstdlib>
#include <windows.h>

extern "C" int APS5_VABI __cxa_atexit_nid_postfix(void (APS5_VABI *func)(void*), void* arg, void* dsoHandle);

namespace {
void Record(const char* value) {
    DWORD written = 0;
    WriteFile(GetStdHandle(STD_OUTPUT_HANDLE), value, 1, &written, nullptr);
}
void APS5_VABI GuestHandler(void*) { Record("G"); }
void HostHandler() { Record("H"); }
}

int main() {
    __cxa_atexit_nid_postfix(GuestHandler, nullptr, nullptr);
    std::atexit(HostHandler);
    LibcExit_nid_no_patch(0);
}

AI-assisted: yes (Claude Code).

@fisherjoey

Copy link
Copy Markdown
Contributor

Linux test with a retail title, since the PR hasn't been tested there.

Demon's Souls (PPSA01341), Nobara 44 (Linux 7.2), GCC 16.2.1, Release, RTX 5070 Ti (595.91.07). Our stack is main df16c4c plus #1071, #1201/#1202, #1184 and some local performance commits. This PR was cherry-picked onto it; only the test list in libc/CMakeLists.txt conflicted, and guest_exit_order_tests passes. The window was closed at the attract screen (t+150 s) with kdotool windowclose, 3 runs per arm, interleaved:

how each run ended last guest messages
stack SIGABRT, SIGABRT, SIGSEGV AJM: decoding codec 4026830656 is not implemented, AJM: decoding codec 27 is not implemented; the SIGSEGV is a guest job-worker fiber jumping to address 0
stack + this PR still running 15 s after the close (killed, no teardown started), SIGABRT, SIGABRT APR: cannot open <garbage path>, double free or corruption (out)

None of the six runs exited cleanly. Before this test, every window close on our stack since #1071 ended in -11 or -6, mostly from level-length runs, with NCA DecoderManager: sceAjmBatchWait() failed with error code 0x80930004 ... halting now and then one of a fault on a guest job-worker thread, double free or corruption, or AJM: decoding codec <garbage> is not implemented. With the PR the failures look the same: guest code still reads freed or garbage state during teardown (codec ids, file paths). So on Linux this PR doesn't fix that teardown fault, which must be a different one. I didn't see any new failure caused by the PR either; the one run that ignored the close is new, but with three runs per arm I can't tell it apart from noise.

@github-actions

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
#2545 core/libs/tests/GuestExitOrder.cpp

@oneandonlydean
oneandonlydean merged commit 6b9f6d5 into boykopovar:main Oct 10, 2026
7 checks passed
SleglJan pushed a commit to SleglJan/AnyPS5 that referenced this pull request Oct 10, 2026
…dlers-first

fix(libc): run guest exit handlers before the host exit handlers
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants