fix(agc): drain library prebuilds before device replacement

[MarcZX10]

What

Closes #6805. Device replacement now drains queued registration library prebuilds before clearing the old device's pipeline caches. Pipeline-library clearing also waits for in-flight library part builds, including the interval where vkCreateGraphicsPipelines runs without the library-store mutex, before erasing the per-device maps. A deterministic mock test blocks pipeline creation while cache clearing runs, then checks that clearing waits and releases all pipeline objects.

Open #5925 and #6641 touch nearby lifecycle code; neither drains registration library prebuilds during device replacement.

Tested

• Linux, Debian 12, GCC 12.2: agc_driver_graphics_tests built successfully. This host lacks the PS5 CPU feature set, so I temporarily bypassed the test binary's startup CPU guard to run the mock-only test; the guard change was restored and is not in this PR.
• ctest --test-dir /tmp/build-6805 -R '^agc_pipeline_library_clear_waits_for_build$' --output-on-failure: passed, 1/1. The run used the same temporary local CPU-guard bypass. No GPU was used; the regression test uses mocked Vulkan calls.
• git diff --check and python3 tools/check_conventions.py --base upstream/main: passed.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Unblocks: PPSA02100 AGC: since #5875, a registration library prebuild racing the window attach segfaults in LinkPipelineFromLibraries (Stray PPSA02100, warm shader cache) #6805
• GPUs tested: none (mock Vulkan test only)
• AI-assisted: yes

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
PR 6320	core/libs/prx/libSceAgcDriver/tests/Graphics.cpp L7823
PR 6641	core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp L55-56
PR 6973	core/libs/prx/libSceAgcDriver/tests/Graphics.cpp L5268, L7940
PR 6987	core/libs/prx/libSceAgcDriver/tests/Graphics.cpp L5268, L7940
PR 6989	core/libs/prx/libSceAgcDriver/tests/Graphics.cpp L5268, L7940
PR 6995	core/libs/prx/libSceAgcDriver/tests/Graphics.cpp L5268, L7940
PR 7002	core/libs/prx/libSceAgcDriver/tests/Graphics.cpp L5268, L7940

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 1341	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 1368	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 1395	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 4040	core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 4072	core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 4339	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 4472	core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 4650	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 4891	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 5021	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 5115	core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp
PR 5371	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5390	core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 5426	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5431	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 5441	core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 5499	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 5500	core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 5925	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6044	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6134	core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp
PR 6176	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6177	core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6188	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6215	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6222	core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6235	core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6259	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6297	core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6373	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6412	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6503	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6529	core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6535	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6548	core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6647	core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6718	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6747	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6763	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6808	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6937	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6946	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp
PR 6947	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp

[milescoviello]

Ghost of Tsushima (PPSA02225) hits the same crash on Linux/NVIDIA, and this PR's head fixes it there.

Host: Ubuntu 24.04, RTX 4070 Ti (NVIDIA 595.99.02), i7-13700K, release-like libs, pad input, warm shader cache.

• main 82ddae7: EXIT-139 0–1 s in, in both runs (a sweep run and a rerun). The rerun's core matches AGC: since #5875, a registration library prebuild racing the window attach segfaults in LinkPipelineFromLibraries (Stray PPSA02100, warm shader cache) #6805. The registration prebuild thread is the one that faults, in std::map<std::vector<std::byte>, VkPipeline>::find ← LinkPipelineFromLibraries ← Pipeline::Prebuild ← VulkanDevice::PrebuildGraphicsLibraries ← the PrebuildStageLibrary task. At the same moment the presenter thread is in Driver::AttachWindow → VulkanDevice::VulkanDevice, and the log ends with the window device's constructor lines.
• Before perf(agc): journal the pipelines of a run and build them ahead on the next one #5875: four sweep runs of the same title on main from 10-14 and 10-15 ran to their 180 s / 600 s limits.
• This PR's head 81e3db8 (on 82ddae7): 3 of 3 runs reach the 180 s limit with the same warm cache. Their logs match the pre-perf(agc): journal the pipelines of a run and build them ahead on the next one #5875 runs (156–158 lines, both devices created, no new messages).

Forspoken (PPSA03984) hits it too, with a warm cache on the same host:

• main 82ddae7: EXIT-139 1 s in. The core has the same two stacks (prebuild thread in LinkPipelineFromLibraries, presenter in AttachWindow → VulkanDevice::VulkanDevice).
• This PR's head: it gets past that point and plays to 21 s, where it stops on the DB_RENDER_CONTROL 0xc0 refusal it hit before perf(agc): journal the pipelines of a run and build them ahead on the next one #5875 (fix(agc): accept a copy's sample selection on a draw that copies nothing #5937).

Three ps5-opengl homebrew apps, PS5 OpenGL Showcase (PPSA99005), PS5MC (PPSA99420) and OpenNOW (PPSA99082), show the same pattern:

• On main 82ddae7 + perf(agc): refresh only the layers a use of an array color target covers #6202, with the warm cache a cold run had just written, all three end EXIT-139 0 s in. No cores were taken.
• With this PR added to the same tree, all three run to their 60 s limit.

Not checked: AMD, Intel, Windows.

AI-assisted: yes

[MarcZX10]

Thanks for the detailed title runs and core backtraces. The matching prebuild-thread fault on Ghost of Tsushima and Forspoken, and the head getting past that crash in the latter, are useful independent evidence for the lifecycle race. I will keep the stated validation scope as mock Vulkan plus your reported Linux/NVIDIA title runs; I have not run a title or GPU validation locally.

[yabanana]

What this change does: it keeps the library maps alive while Vulkan creates a part or links a pipeline, and drains registration prebuilds before replacing the device. I checked head 81e3db853c00453a04668e2c62e8b969f24d324b and reproduced the original lifetime failure independently.

1. Must fix: the new test is missing its Windows DLL search path.
agc_pipeline_library_clear_waits_for_build runs agc_driver_graphics_tests, but the adjacent if(WIN32) PATH setup names only agc_pipeline_journal. The test executable is in build/tests; the PRX DLLs are in the library output directory. Add the new test to that existing ENVIRONMENT_MODIFICATION list so it can load those DLLs in the standard Windows CTest setup. Otherwise the regression can fail in the loader before exercising this fix. This is a configuration finding; I have not run Windows.

Test patch and one-line CMake correction, based on this exact head. The regression now also fails a library creation and the final link while clearing waits, checks exception propagation and object release, and uses atomic accounting for concurrent mock destruction. It reuses the existing Pipeline constructor.

Verified:

• macOS 27.2 ARM64, Apple Clang 21, ASan+UBSan, production PipelineLibrary.cpp with mocked Vulkan: main 82ddae75b reproduces a heap-use-after-free; this PR passes 40 cases (part/link × success/error × 10). The probe disables background LTO and does not run journal replay.
• Ubuntu 24.04 amd64, GCC 13.3 Release, QEMU 10.2.3 -cpu max: the patched focused CTest passes 10/10 fresh processes. CPU guards and assertions are unchanged. agc_pipeline_journal, agc_driver_address_space_teardown, and agc_embedded_vertex_fetch also pass.
• The general graphics suite still hits the pre-existing zero-word vertex fixture failure tracked in #6792. I am not counting that run as a pass or attributing it to this PR.
• Conventions and diff whitespace checks pass.

Merge preview with #6641 has one conflict in LibraryStore: keep this PR's activeBuilds and #6641's worker/stopping; do not restore the replaced started field. This is the textual resolution, not yet a combined runtime result.

Independent sanitizer probes, source pins and reproduction commands.

Verdict: fix the Windows test registration before merging; the lifetime fix passes the exercised paths.
Not checked: Stray, physical GPUs, Windows execution, or the full platform suites. No title-unblock or performance claim.

AI-assisted: yes.

[MarcZX10]

Fixed in 83eb3ed04: the Windows ENVIRONMENT_MODIFICATION now applies the libc DLL search path to both agc_pipeline_journal and agc_pipeline_library_clear_waits_for_build. git diff --check passes. Windows runtime validation remains for CI; I have not run the test locally.

[MarcZX10]

Thanks for the additional title runs. The matching EXIT-139 on main and the three 60-second runs with this head add useful evidence across Ghost of Tsushima, Forspoken, and the three homebrew apps. I’ll keep the conclusion scoped to the reported Linux/NVIDIA setup; AMD, Intel, and Windows remain unvalidated.

[yabanana]

The combination with #6641 is now verified at dd8824b63629d68b7f205e66bcc578abb4a42e60. Keep activeBuilds plus worker/stopping in the one LibraryStore conflict; the combined lifetime, journal and teardown checks pass, as do 60 shutdown processes and the fast/optimized D32 readback on llvmpipe. Exact composition, test patch and limits.

This includes the existing #6789 fixture fix. The general graphics CTest exceeds 30 s under emulation; a separate bounded diagnostic completes, with write-watch cases unavailable. No full-suite, Windows or title result is claimed. AI-assisted: yes.

[MarcZX10]

Thanks for verifying the combined #6814/#6641 behavior and recording the llvmpipe, teardown, and D32 checks. I have noted the timeout and unavailable write-watch coverage, and will keep the claim scoped to that integration and host.

[blawesom]

Duplicate pair heads-up for the maintainers: this PR and #6827 carry byte-identical diffs (217 lines across the same six files — ShaderRegistry.hpp/.cpp, VulkanDevice.cpp, PipelineLibrary.cpp, the CMakeLists and Graphics.cpp — both adding AwaitLibraryPrebuilds()). This PR was opened first (13:03 UTC); #6827 followed 28 minutes later. Whichever survives, the other can be closed as a duplicate — flagging both rather than choosing, since the call is yours.

[MarcZX10]

Thanks for flagging the duplicate pair. I checked #6827: its body marks it as a follow-up to #6814 and adds the Windows DLL search-path correction for the new CTest. The current #6814 head has the same correction, so that change is already present here too. The two PR diffs now overlap byte-for-byte; I’ll leave the maintainer choice about which PR remains open to you. No separate patch is needed on #6814 for the DLL path.

[MarcZX10]

A duplicate check confirms #6827 is also open with the Windows DLL search-path correction and carries a byte-identical production diff. Since #6827 is specifically the test configuration follow-up, could a maintainer advise whether #6814 or #6827 should be the surviving PR? I will keep both untouched pending that direction.

[MarcZX10]

Correction to my earlier note: I described #6827 as a duplicate, but its author explicitly presents it as a follow-up depending on #6814. The branch includes #6814’s implementation and adds the separate Windows test PATH correction, so these are stacked changes rather than duplicate submissions. I withdraw the request for a maintainer to choose between them; #6827 correctly records Depends on: #6814.

[milescoviello]

Two homebrew apps added to the catalog today crash on the same race on Linux/NVIDIA. Host: RTX 4070 Ti (NVIDIA 595.99.02), main 1b1f848, 30–60 s runs.

• PS5LM (PPSA99575): on main, 6 of 9 launches crash 0–1 s in. Some are EXIT-139 in LinkPipelineFromLibraries ← Pipeline::Prebuild ← PrebuildGraphicsLibraries on a RegistrationPreparePool thread. Others are EXIT-134, a glibc corrupted double-linked list raised inside NVIDIA's compiler called from the same prebuild. With this PR's head 83eb3ed merged onto the same main, 4 of 4 runs reach the limit.
• Afterglow (PPSA47770): on main, most launches crash 1 s in with the same prebuild stack. With this PR, all runs get past that point, but the app then exits on its own 3–9 s in with [ps5-agc] submission retirement failed. That message is the 2 s retirement watchdog of the ps5-opengl 1.0.0 runtime the app bundles; 1.0.2 waits 30 s. Thread stacks at the exit show the draw pipeline thread still in LinkPipelineFromLibraries (inside NVIDIA's compiler) for the fifth submission's draw, with the queue worker waiting on the GPU mutex. With the app's limit patched to 30 s in a local copy, it runs 60 s, 2 of 2. So I don't think this PR causes that exit; it shows how long that first link takes here on cold caches. Once a patched run has cached that pipeline, the unpatched app also runs with this PR (4 of 4 20 s runs, where every library part built in at most 253 ms). The earlier warm-cache runs had all exited before the pipeline was cached. Surround Sound Studio (PPSA99051) bundles the same ps5-opengl 1.0.0 runtime and shows the same exit with this PR (2 of 2, 3 s); with its limit patched it runs 40 s, 2 of 2. On main, only 3 of 15 Afterglow launches got past the crash: two reached 30 s and one hit the same exit. That's too few to tell whether the link is faster there.

AI-assisted: yes

[MarcZX10]

Thanks for the new PS5LM and Afterglow results. The PS5LM comparison adds another Linux/NVIDIA reproduction of the registration-prebuild crash and shows 4/4 runs passing the affected point on this PR head. For Afterglow, I understand the later retirement failure is attributable to its bundled 1.0.0 watchdog: with the local 30-second adjustment, this PR reaches the 60-second limit 2/2, so I will not attribute that exit to this change. I have not reproduced either title locally; I will keep the evidence scoped to your reported setup and runs.

[MarcZX10]

Thanks for consolidating the Linux/NVIDIA title runs and the Windows results. I see the focused pipeline-clear test and agc_pipeline_journal pass on Windows, while the remaining agc_shader_disk_cache timeout is reported as the same runner hang seen on main and other PRs. The current #6814 head includes the Windows DLL path correction, and I’ll keep the claim scoped to the lifetime-race evidence and tests reported here. I also see #6827 is closed without merging, so I’m not treating it as a merged dependency.

[oneandonlydean]

Right, #6827 is closed without merging, so nothing here depends on it, and its Windows DLL path correction is already in this head. If #6641 lands first, keep activeBuilds plus worker/stopping in the LibraryStore conflict, as @yabanana verified.

[MarcZX10]

Thanks for confirming. I verified that #6827 is closed without merging and that the Windows DLL path correction is already in this head. #6641 is still open, so there is no integration change to make now; if it lands first, I will preserve the activeBuilds and worker/stopping handling in LibraryStore as verified.

[milescoviello]

Another title with #6805's crash: Ghost of Tsushima (PPSA02225), Linux 6.8, RTX 4070 Ti (NVIDIA 595.99.02), Release, pad input, warm shader cache.

Build	Runs	Result
main f6a2091	9	segfault 0–1 s after start, at the window attach (9 of 9)
main f6a2091 + this PR (83eb3ed04a, merged locally as bc8b0d8dee)	5	all reach their limit (60–90 s); shader disk cache 666–711 hits, 0 misses

main passed once more, in the run right after a run with this PR; I didn't count that one above.

Crashing thread from the core of a main run, the same frames as in #6805:

#0 std::_Rb_tree<std::vector<std::byte>, std::pair<const std::vector<std::byte>, VkPipeline_T*>, ...>::find
#1 AgcDriver::Graphics::LinkPipelineFromLibraries(...)
#2 AgcDriver::Graphics::Pipeline::Pipeline(...)
#3 AgcDriver::Graphics::Pipeline::Prebuild(...)
#4 AgcDriver::VulkanDevice::PrebuildGraphicsLibraries(...)
#5 AgcDriver::DriverDetail::(anonymous namespace)::PrebuildStageLibrary(...) lambda
#6 AgcDriver::DriverDetail::(anonymous namespace)::RegistrationPreparePool::Main(void*)

Ghost ran to its limit on main 940eba9 in a sweep on 10-15.

Update (homebrew): ProsperoEden 1.004.000 (PPSA99008, the catalog build from today), pad input, warm cache, 30–60 s runs. It has the same frames, from the core of a crashing run: _Rb_tree::find ← LinkPipelineFromLibraries ← Pipeline::Prebuild ← PrebuildGraphicsLibraries ← RegistrationPreparePool::Main.

Build	Runs	Result
main ea5c27b	7	stops 3 s after start in 5 of 7 (4 segfaults, 1 abort without a kept core), the other 2 reach the limit
main f6a2091 + this PR (83eb3ed04a)	5	all reach the limit

Earlier ProsperoEden versions (1.001.000, 1.003.000) reached their limits in all my earlier sweeps (40–180 s).

AI-assisted: yes (Claude Code)

[cadenasynyster]

Windows check of head 83eb3ed, since Windows execution was still listed as unvalidated:

Windows 11 10.0.26200.9711, WinLibs GCC 15.2.0 posix UCRT r7, Release, BUILD_TESTING=ON, PR merged onto main f3263df (clean merge, no new build errors), standard ctest setup.

• agc_pipeline_library_clear_waits_for_build: loads and passes 10/10 (ctest --repeat until-fail:10), so the DLL search-path change in 83eb3ed works in the standard Windows ctest setup.
• agc_pipeline_journal, agc_driver_address_space_teardown, agc_embedded_vertex_fetch: pass.

Not run: a title, the full graphics suite, or a negative control against main's PipelineLibrary.cpp.

AI-assisted: yes

[MarcZX10]

Thanks for checking the exact head on Windows and repeating agc_pipeline_library_clear_waits_for_build 10 times. That verifies the DLL search-path change in the standard CTest setup; the three related teardown/journal tests also pass in your report. I’ve noted the limits you called out: no title run, full graphics suite, or negative control against main’s PipelineLibrary.cpp.

[oneandonlydean]

Ghost of Tsushima on main f6a2091 segfaults 9 of 9 times at the window attach, with the same LinkPipelineFromLibraries frames as #6805; with this head merged, 5 of 5 runs reach their limit (666 to 711 shader cache hits, no misses), @milescoviello. @cadenasynyster's Windows check shows the DLL search path change works: agc_pipeline_library_clear_waits_for_build loads and passes 10/10 in the standard ctest setup on this head, next to agc_pipeline_journal, agc_driver_address_space_teardown and agc_embedded_vertex_fetch. Still not run by anyone: a Windows title, the full graphics suite, or a negative control against main's PipelineLibrary.cpp.

[snoozedDev]

Another title with the #6805 crash, Grand Theft Auto III – The Definitive Edition (PPSA03527 v01.007.000), and one cost of the fix: this PR (head 83eb3ed) stops the crash, but the window attach now waits about 67 s on a warm cache.

Tree: main af0fff7 + #6236 (merged since) + #6681. The title needs #6681 first: without it, it faults in its own code at the same moment (data on #6681). Runs use APS5_PLAYGO_UNLISTED_CHUNKS=absent. Linux (kernel 7.2.5), GCC 16.2.1, Release, RTX 3090, NVIDIA 610.57.04, inside gamescope --backend headless, no input; T+ is seconds since launch. The warm cache holds 3,617 shaders (3617 hits, 0 misses in the first 10 s tick).

Libraries	Cache	Window device created	Result
without this PR	warm	T+8 s	SIGSEGV (SI_KERNEL) at T+9 s in std::_Rb_tree<std::vector<std::byte>, …, VkPipeline_T*>::find, called from LinkPipelineFromLibraries (a tree node pointer reads 0xbeef0004beef0003)
without this PR, APS5_NO_LIBRARY_PREBUILD=1	warm	T+3 s	no stop up to the 900 s limit
with this PR	warm, 2 runs	T+75 s, T+75 s	no stop up to the 600 s and 240 s limits
with this PR	cold (3,676 shaders registered)	T+626 s	no stop up to the 1500 s limit

Every run that went on to the limit presented 900 frames in each 30 s summary interval (30 fps, p50 frame about 33.37 ms) after its first interval.

Until the window device is created, the title logs RHISubmitCommandsHintEEE once a second (68 lines in the second run). PrepareForReplacement now calls AwaitLibraryPrebuilds(), which waits for every prebuild already submitted, and on this cache that takes about 67 s. #6805 also suggested cancelling the queued prebuilds for the device instead, which would avoid the wait. I can rerun either variant on this title.

agc_pipeline_library_clear_waits_for_build and the other agc_pipeline_library* tests pass on Linux (RTX 3090) with the PR.

AI-assisted: yes (Claude Code, Claude Opus 5.5), operating under my supervision; I am accountable for this report and will respond to review.

[MarcZX10]

Thanks for the GTA III measurements. The 67 s attach delay is a material cost of the current fix: AwaitLibraryPrebuilds() calls AwaitSubmitted() on the shared prebuild pool, so it waits for all submitted tasks, including queued work for this device. The wait does preserve the required lifetime boundary before device destruction.

Since you offered to rerun either variant, would you be willing to compare a device-scoped cancellation variant on the same warm-cache setup? The useful distinction would be canceling queued prebuilds for the retiring device while still waiting for any prebuild already active on it, then recording attach time and whether the title progresses. I have not implemented or validated that alternative, so I am treating the 67 s result as an observed tradeoff rather than assuming cancellation is safe.

[snoozedDev]

I ran the device-scoped cancellation on GTA III DE. It brings the window attach back to 8–10 s, and the title runs on as it does with 83eb3ed.

Variant (local, measurement only, on top of 83eb3ed): each prebuild pool task records the VulkanDevice* it was submitted for. PrepareForReplacement calls a CancelLibraryPrebuilds(this) in place of AwaitLibraryPrebuilds(). Under the pool mutex it removes this device's queued tasks (and their sequence numbers from unfinished), then waits until no task for this device is running. The other devices' tasks are untouched. A temporary fprintf logs the counts.

Setup: same tree and settings as my earlier comment (main af0fff7 + #6236 + #6681 + this PR, APS5_PLAYGO_UNLISTED_CHUNKS=absent, RTX 3090, NVIDIA 610.57.04, headless gamescope, no input). The two library builds alternated on one warm shader cache, 240 s limit per run. The cache holds the 3,676 shaders of the earlier cold run. Every run hits all 3,676: by T+22 s with the variant, by T+92 s with 83eb3ed (whose first 10 s report shows 3,617 hits, as in my earlier comment). Run 2 wrote 4 new shaders at T+86 s; no later run loaded them.

Run	Libraries	Window device created	Cancel log	30 s intervals summarized	Result
1	83eb3ed	T+83 s	–	5	no stop up to the limit
2	variant	T+9 s	cancelled 290 queued, waited for 1 active, 712 ms	7	no stop up to the limit
3	83eb3ed	T+78 s	–	5	no stop up to the limit
4	variant	T+10 s	cancelled 288 queued, waited for 1 active, 480 ms	7	no stop up to the limit
5	variant	T+8 s	cancelled 288 queued, waited for 1 active, 160 ms	7	no stop up to the limit

In all five runs, the first interval had 779–794 presents and every later one 900, with a p50 frame of 33.37 ms. Before the window device was created, the title logged RHISubmitCommandsHintEEE 74 and 70 times with 83eb3ed and not at all with the variant. In each variant run one prebuild for the old device was running when the cancel came; the whole cancel call, including that wait, took 160–712 ms. None of the three variant runs crashed. Three runs is a small sample; for comparison, the one warm run without this PR (with #6681, prebuild enabled) crashed on #6805 at T+9 s.

What the variant does not cover: a registration preparation that is still running can submit a prebuild for the old device after the cancel has returned. The ticket in AwaitSubmitted() does not cover that case in 83eb3ed either. I did not see it in these runs, and nothing logged shader library prebuild skipped.

agc_pipeline_library*, agc_pipeline_journal and agc_driver_address_space_teardown pass with the variant on Linux, agc_pipeline_library_clear_waits_for_build 10/10 with --repeat until-fail:10. None of them exercises the cancellation path. I can post the diff (43 lines added, 11 removed, in ShaderRegistry.{hpp,cpp} and VulkanDevice.cpp) if it helps.

AI-assisted: yes (Claude Code, Claude Opus 5.5), operating under my supervision; I am accountable for this report and will respond to review.

[MarcZX10]

Thanks for the additional Ghost of Tsushima A/B and the standard Windows CTest run. The current head includes the DLL-path correction, and the reported 9/9 main crash versus 5/5 runs reaching the limit adds useful evidence for the lifetime fix.

[milescoviello]

On the attach delay: in the two titles from my earlier comment it doesn't show. I ran them with this head (83eb3ed04a, on main f6a2091) and APS5_PROFILE_DRAW=1, warm cache, 70 s, two runs each. The [present] lines, presents per 10 s from the first one:

Title	Run 1	Run 2
Ghost of Tsushima (PPSA02225)	591 600 600 600 600 600	591 600 600 600 600 600
ProsperoEden 1.004.000 (PPSA99008)	602 600 600 600 600 600	602 600 600 600 600 600

So the first 10 s window already runs at the full rate, and the wait in AwaitLibraryPrebuilds() at the attach is short in both. Their registration queues at that point are probably much shorter than GTA III's. On main ea5c27b in the same session, ProsperoEden crashed at 3 s in one run, and the other reached the limit without a single [present] line.

AI-assisted: yes (Claude Code)

[MarcZX10]

Thanks for measuring both titles on the current head. Starting the first 10-second window at 591/600 and 602/600 presents shows this drain did not delay attach in those two warm-cache runs. That is useful alongside the Stray reproductions; it does not establish cold-cache or broader title behavior.

[MarcZX10]

Thanks for identifying the late-enqueue window. I traced it through RegisterShader and the prebuild pool: AwaitLibraryPrebuilds() snapshots that pool's submitted ticket, so it cannot include a prebuild that an already-running registration-preparation task submits afterward.

I also checked the lifetime impact. The prebuild closure captures a shared_ptr<VulkanDevice>, and AttachWindow retains replaced devices in replacedDevices until driver shutdown, so this does not by itself show a device-use-after-free. A late task can still repopulate pipeline-library state after PrepareForReplacement has cleared that device's caches, leaving work and resources on the retired device until shutdown.

A complete fix needs to stop new RegisterShader submissions while replacement is in progress, then await registration preparations before draining/canceling prebuilds. The current regression does not exercise that producer-to-consumer race, so I’m not treating the existing drain as covering it or proposing an untested patch yet. If you have a deterministic test seam or a measured case where the late work causes a functional problem, please share it; that will help establish the right regression and whether to extend this PR.

[snoozedDev]

I have neither a test seam nor a measured case. In the three variant runs the title did not crash or stall: all reached the 240 s limit at 900 presents per 30 s interval after the first. But the variant does not log a prebuild submitted after the cancel returns, and the title presents only black frames in this setup (#6994), so these runs cannot show whether a late prebuild happened or had an effect. If a version that also gates RegisterShader during replacement comes up, I can run it on GTA III warm and cold the same way.

AI-assisted: yes (Claude Code, Claude Opus 5.5), operating under my supervision; I am accountable for this report and will respond to review.

[Tamalero]

Title check for #6805 on High on Life (PPSA15388), Linux, RTX 4090, GCC 16.2.1, Release. Tree: main f26eb67c8 + the PlayGo setting (now #6236) + this PR's two commits (cherry-picked unchanged). Same relinked executable and a warm shader cache in every run; the runs with this PR used the same cache folder as the last run without it.

Build	Runs	At the window attach (Presentation window context=… extent=1920x1080)
without this PR	4 (2 on f26eb67c8, 2 on 07e741481)	4 × SIGSEGV in std::_Rb_tree<…, VkPipeline_T*>::find (the #6805 frame)
with this PR	3	3 × passes

With the PR, two runs reached the main menu. The third passed the window attach and then hit a different crash about 80 s in: a jump to 0x468 from the title's own code. That one is an older, intermittent High on Life start-up crash, seen on builds from before #5875 too, so it's not related to this change.

Tested and written by Claude (AI assistant) on a contributor's machine.

[Tamalero]

The same on Guardians of the Galaxy (PPSA01748), same tree as above (f26eb67c8 + this PR), Guardians' own warm shader cache:

Build	Runs	At the window attach
without this PR	4	3 × SIGSEGV in std::_Rb_tree<…, VkPipeline_T*>::find (#6805), 1 × passes
with this PR	3	3 × passes, all three reach the main menu

Tested and written by Claude (AI assistant) on a contributor's machine.

[teepean]

Stray (PPSA02100), the title from #6805, on current main 6a32550 with and without this PR (83eb3ed, merged locally). Linux (CachyOS, kernel 7.2.10), Ryzen 9 9950X, RTX 3090 (615.78.08), GCC Release, stock game config, no input, 90 s per run. All runs use a warm shader cache (3393 hits, 0 misses in the first 10 s), APS5_NO_IDLE_STORE=1 (an unrelated Stray startup crash) and APS5_PROFILE_DRAW=1.

Libraries	Library prebuild	Runs	Result	Window device constructed
main 6a32550	on	2	2 × crash 1.6 s after start	1.5 s
main 6a32550	off (APS5_NO_LIBRARY_PREBUILD=1)	1	reaches the limit	1.5 s
main + this PR	on	4	all reach the limit	1.7–1.8 s
main + this PR	off	2	both reach the limit	1.5 s

The two crashes on main, from the cores:

• SIGSEGV on the prebuild thread with the AGC: since #5875, a registration library prebuild racing the window attach segfaults in LinkPipelineFromLibraries (Stray PPSA02100, warm shader cache) #6805 frames: _Rb_tree<…, VkPipeline_T*>::find ← LinkPipelineFromLibraries ← Pipeline::Prebuild ← PrebuildGraphicsLibraries ← PrebuildStageLibrary.
• SIGABRT, glibc corrupted double-linked list from posix_memalign inside libnvidia-gpucomp.so, the other form @milescoviello saw on PS5LM.

On the attach delay: AwaitLibraryPrebuilds() runs in AttachWindow before the window device is constructed, so the gap between the last log line before the constructor and the constructor's VulkanDevice constructor window=… line shows its cost. That gap is 0.27–0.32 s with the prebuild on, against 0.08 s with it off. So on Stray the drain costs about 0.2–0.25 s, far from the 67 s on GTA III, like Ghost of Tsushima and ProsperoEden in @milescoviello's runs.

AI-assisted: yes (analysis and text written with Claude Code; the runs and core dumps are from my machine).

[LeandroLP23]

Windows and AMD, on a title with no warm cache at all: ProsperoLight (PPSA99002, catalog 01.000.100), the stop I reported in the catalog update (libSceAgcDriver.prx+0x19b24d in LinkPipelineFromLibraries).

Windows 11 (26300), Radeon 860M (RDNA 3.5, driver 32.0.13058.8001), WinLibs GCC 15.2.0, Release. Tree: main 6a32550fd + #6220 (libkernel sysctl only, needed by another title on this build) on both sides; with this head 83eb3ed04 merged cleanly. Each cold run starts from an empty shader cache; audio muted.

Build	Runs	Result
without this PR, cold	5	5 crashes at 6-7 s: two 0xC0000374 (heap corruption), three 0xC0000005 (in ucrtbase or libstdc++)
without this PR, warm (runs 2-3 after a cold run)	2	both run 60 s at 7.2-7.6 FPS
this PR, cold, 25 s	5	no crash; still building shaders at 25 s (11-19 presents)
this PR, cold, 90 s	2	no crash; the title's own screen (an update notice) at 6.8 and 7.2 FPS, and it closes cleanly

The update notice is for the current release, 01.000.110 (from the project's GitHub releases, SHA-256 checked). That release on the same tree with this PR, cold, 90 s: no crash, 8.0 FPS, and the dumped frames show its home screen ("Your PCs", "No PCs yet", tabs PCs, Games, Settings, About).

AI-assisted: yes (Claude Code).