fix(libc): leave a range that a mutation waits for out of new leases

[xXPBSXx]

What

Brigandine Abyss (PPSA27752) aborts 5 to 6 minutes after start with

[gpu] leased allocation 0x11a4000000+0xea00000 still pinned after 8 recorder syncs
what():  guest allocation 0x11a4000000+0xea00000 is still pinned by GPU work or an import (waited 60000 ms for the lease (APS5_PIN_WAIT_MS))

(PPSA27752_rH/run_210801.err lines 132 and 135; run_211614.err lines 131 and 136 show it for 0x11a6000000+0x1a00000, after [gpu] storage image …: memory no longer registered; 1 pending ranges dropped. Storage images listed at lines 124-128 of the first run lie inside the pinned allocation.)

GuestAllocationsRequireUnpinned waits for the GPU work that leases a range through the driver's waiter. GuestAllocationsAcquireAll and GuestAllocationsAcquire copy every registered range into each new lease, including the range that is being released. A driver worker that starts an address-based build between the waiter's return and the re-lock therefore pins the range again; the comment in RequireUnpinned describes this as rare. A title that draws every frame repeats it each round, and after APS5_PIN_WAIT_MS the free fails.

• A mutation that finds a pinned range records it as retiring until the wait ends (also when the wait throws).
• Acquire and AcquireAll leave retiring ranges out and do not cache the lease while one is recorded; the caches are cleared when the record is removed.
• Debt entry: GPU work started while a release waits does not see that memory; whether the console keeps it visible is unknown.

guest_memory gains CheckPinnedRangeIsLeftOutOfNewLeases: a pin waiter replaces the lease with a new AcquireAll lease the way a worker does, and the range is unmapped. Without the change the unmap fails with the line above (still pinned after 8 recorder syncs, GuestMemory.cpp:1622); with it the unmap succeeds, the old lease no longer holds the range, and a later AcquireAll lease holds a new mapping.

This shows the mechanism, not that Brigandine hangs this way: a diagnostic run of the title (PPSA27752_rD1/run_215729.err) ended earlier in a different stop (the engine's 120 s watchdog on the render thread) and did not reach the pin, so the lease source in the title is not measured.

Tested

Windows 10 (19045), MinGW-w64 GCC 15.2.0, Release, on main 324e0e48e + this commit:

• ctest -R ^guest_memory$: passes with the change; fails without it (the source file reverted, test kept): Guest memory check failed at core/libs/tests/GuestMemory.cpp:1622, exit code 0xc0000409.
• python tools/check_conventions.py --base upstream/main passes.
• Not tested: Linux and macOS (the test runs there but was not built), a console, the title with this change.
• Overlap: perf(agc): keep a host import across a split of its guest range #6499 (open, keeps a host import across a split of its guest range) changes lines near GuestAllocations.cpp L40-L47 (the overlap bot lists it); the topics are different, whichever lands second needs a rebase. perf(memory): restrict registry work to affected ranges #1324 also changes core/libs/prx/libc/src/GuestAllocations.cpp and include/GuestAllocations.hpp (registry work per range); the two are unrelated topics, whichever lands second needs a rebase.

Checklist

• Based on current main; no other open PR implements the same change
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt (docs/dev/debt/libc.md)
• No notes, investigation .md files or images added to the repository
• New third-party code is a submodule built from source (none added)
• Generic behaviour, not specific to one title
• Depends on: none
• Unblocks: PPSA27752 (Brigandine Abyss) at this stop, not confirmed in the title
• GPUs tested: n/a (registry logic; the title runs used an NVIDIA GeForce RTX 4080 SUPER, Windows 10)
• AI-assisted: yes

AI-assisted (Claude Code).

🤖 Generated with Claude Code

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
PR 6975	docs/dev/debt/libc.md L117
PR 7448	core/libs/tests/GuestMemory.cpp L2894

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 1324	core/libs/prx/libc/src/GuestAllocations.cpp
PR 1835	core/libs/prx/libc/src/GuestAllocations.cpp
PR 5021	core/libs/prx/libc/src/GuestAllocations.cpp core/libs/tests/GuestMemory.cpp docs/dev/debt/libc.md
PR 6044	core/libs/tests/GuestMemory.cpp
PR 6226	docs/dev/debt/libc.md
PR 6321	core/libs/prx/libc/src/GuestAllocations.cpp core/libs/tests/GuestMemory.cpp
PR 6469	core/libs/tests/GuestMemory.cpp docs/dev/debt/libc.md
PR 6879	core/libs/prx/libc/src/GuestAllocations.cpp
PR 7081	docs/dev/debt/libc.md
PR 7105	core/libs/tests/GuestMemory.cpp
PR 7201	core/libs/prx/libc/src/GuestAllocations.cpp
PR 7332	core/libs/prx/libc/src/GuestAllocations.cpp core/libs/tests/GuestMemory.cpp docs/dev/debt/libc.md
PR 7417	docs/dev/debt/libc.md
PR 7473	core/libs/tests/GuestMemory.cpp docs/dev/debt/libc.md
PR 7715	docs/dev/debt/libc.md

[jotitadev]

Linux run, since the description lists Linux as not built.

Ubuntu 26.04, Intel i7-8700, GCC 15.2, Ninja, Release, BUILD_TESTING=ON, full build. main 31cbb55 with head 46d1178 merged on top.

• With the PR: guest_memory passes 3 runs of 3 (0.6 s each).
• Test kept, core/libs/prx/libc/src/GuestAllocations.cpp reverted to main: guest_memory aborts with the same failure as on Windows:

  [gpu] leased allocation 0x200d68000+0x8000 still pinned after 8 recorder syncs
  Guest memory check failed at core/libs/tests/GuestMemory.cpp:1497
  

  It fails in about 2 s, so the new setenv("APS5_PIN_WAIT_MS", "1000", 1) for non-Windows hosts is what keeps it short.
• Non-GPU suite (ctest -E "^agc|gpu|vulkan|video_out" -j8): 648/650, same as main on this machine. The 2 failures (file_writer_binary, file_writer_text) also fail on main here.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

[xXPBSXx]

@adriadam10 a short request for review of this PR. Head 46d11782e is MERGEABLE / CLEAN, the Conventions, Eligibility and Intake checks are green. The new guest_memory case CheckPinnedRangeIsLeftOutOfNewLeases passes with the change and fails without it (still pinned after 8 recorder syncs). The description names the title (Brigandine Abyss, PPSA27752) and the log lines, and says what is shown (the mechanism) and what is not (that the title hangs this way). The overlap bot lists #6499 near GuestAllocations.cpp L40-L47; the topics differ, and I will rebase whichever of the two lands second if asked.

AI-assisted (Claude Code).

[blawesom]

Rebase handoff — rebased onto current main (authorship unchanged). One docs-debt conflict, unioned.

Verified on Linux (CachyOS, kernel 7.2.9, GCC 16.2.1, Ninja, Release, BUILD_TESTING=ON, AMD RX 9070 XT / RADV):

• builds; tools/check_conventions.py --base origin/main passes
• full suite: 1338/1339 — the only failure is the known main regression (agc_driver_tessellation segfault, feat(shader): read hull inputs at a control-point index computed from data #4727), not this PR

To pick it up:

git fetch https://github.com/blawesom/AnyPS5 rebase/6565
git checkout <your branch>
git merge --ff-only FETCH_HEAD

Head is a850691ed.

[xXPBSXx]

Rebased onto main 324e0e48e; the only conflict was in docs/dev/debt/libc.md (both lines kept). The code is unchanged: the patch id of the changes to GuestAllocations.cpp and GuestMemory.cpp is 89ba80ecc119, the same as before the rebase and as in blawesom's rebase/6565 (thanks for the handoff; it is based on an older main, so I rebased the same commit myself instead of fast-forwarding).

Tested on main 324e0e48e + head 95fabdbef (Windows 10, MinGW-w64 GCC 15.2.0, Release, rebuilt after the submodules of main were fetched): ctest -R ^guest_memory$ passes with the change (4.0 s) and fails without it (GuestAllocations.cpp taken from main, test kept): [gpu] leased allocation 0x200d60000+0x8000 still pinned after 8 recorder syncs, Guest memory check failed at core/libs/tests/GuestMemory.cpp:1622, exit code 8. The earlier run on 4eb91eacd gave the same result. The head commit author address is the account's noreply address.

AI-assisted (Claude Code).

[xXPBSXx]

Rebased onto main 4ffb0cd12 (new head b2dbb48bb). One conflict, in core/libs/prx/libc/src/GuestAllocations.cpp: #6917 made the registry keep its lease snapshots (std::shared_ptr) until the registry changes, where this change had std::weak_ptr. I kept the shared snapshots and added the list of retiring ranges next to them. A retiring range is only added after the snapshots were reset in the scan loop and no snapshot is stored while that list is not empty; unmark resets them again, so a snapshot never holds a range that is being waited for.

Tested on main 4ffb0cd12 + head b2dbb48bb (Windows 10, NVIDIA GeForce RTX 4080 SUPER, driver 610.74, MinGW-w64 GCC 15.2.0, Release): ctest -R ^guest_memory$ passes with the change (4.9 s). Without it (GuestAllocations.cpp taken from main, test kept) the test aborts (exit code 0xc0000409) after [gpu] leased allocation 0x200d60000+0x8000 still pinned after 8 recorder syncs. python tools/check_conventions.py --base upstream/main passes. The head commit author address is the account's noreply address; the previous head 95fabdbef is kept as tag backup/6565-95fabdbef on the fork.

AI-assisted (Claude Code).

[xXPBSXx]

Rebased again onto main fb0dc118f (new head b10615d5e; merge-tree against the newer main d8d0222d0 is clean). The conflict was in core/libs/prx/libc/src/GuestAllocations.cpp, in three places: #6917-follow-up f9caa0974 added backingSerials / renewBacking next to the lease snapshots, and RequireUnpinned now records state->unpinned. I kept those and added the list of retiring ranges beside them; the range is removed again (unmark) before unpinned is recorded, so the snapshots are reset and never hold a range that is being waited for.

Tested on main fb0dc118f + head b10615d5e (Windows 10, NVIDIA GeForce RTX 4080 SUPER, driver 610.74, MinGW-w64 GCC 15.2.0, Release): ctest -R ^guest_memory$ passes with the change (3.1 s). Without it (GuestAllocations.cpp taken from main, test kept) the test aborts (exit code 0xc0000409) after [gpu] leased allocation 0x200d60000+0x8000 still pinned after 8 recorder syncs. python tools/check_conventions.py --base upstream/main passes. The head commit author is the account's noreply address; the previous head b2dbb48bb is kept as tag backup/6565-b2dbb48bb on the fork.

AI-assisted (Claude Code).

[xXPBSXx]

Pushed 2ca591f5d (docs only, no force): the technical-debt entry for the lease change now states the gap first, per the new CONTRIBUTING rule: whether the console keeps memory visible to GPU work submitted while a release or reprotect is pending is unknown and not measured on hardware; the implemented behaviour (a mutation leaves its leased range out of every lease taken until it ends) follows as far as the gap needs. python tools/check_conventions.py --base upstream/main passes and merge-tree against main 3fdd1f52e is clean; code unchanged since b10615d5e.

AI-assisted (Claude Code).