fix(shader): map retained SPIR-V from shader cache entries

[eljackjoestar-cmd]

What

Replace this PR's opt-in temporary-file backing with read-only mappings of the shader disk-cache entries themselves. APS5_FILE_BACKED_SPIRV and the extra temporary files are removed from the implementation.

This adopts ThatsLeo's perf/shader-spirv-on-disk implementation (0bc9120bb), preserving its authorship, and adds the Windows mapping path. Cache-loaded variants retain a shared mapping owner; newly compiled variants are persisted to the normal cache and mapped back before retention. Reading or copying a module shares that storage; mutation detaches into a private vector. Words() returns a const span, and explicit vector consumers use Copy().

Windows mappings share read, write and delete access, so atomic replacement/removal of a cache file does not invalidate its existing readers. Failed cache persistence/mapping and disabled caches retain the existing vector path. No dedicated backing-enable switch, temporary-volume dependency, memory-budget knob or cache-version bypass is introduced. This does not add cache eviction; normal cache files still use disk space.

The cache payload is aligned and its format advances from 30 to 31. Length, key and payload validation remain in force.

Tested

Windows 11 Pro 10.0.26300, GCC 15.2, Release, SPIRV-Tools OFF, current main f87799aa3 plus this PR alone.

The cache regression verifies actual Windows MEM_MAPPED storage, round-trip words/metadata, copy-on-write isolation, and retained word access after the original cache pathname is atomically replaced and removed. Existing cross-process, corruption, cache-invalidation, binding, vertex specialization and invocation-isolation cases also pass.

Full Windows build passed on the final rebased branch. Four focused cache/shader/rendering checks passed on NVIDIA RTX 5070 / driver 617.14 in 6.81 s; graphics and prepared-shader checks passed on AMD Radeon integrated gfx1036 / driver 32.0.21036.18 in 5.26 s. The cache regression also exercises loading while a published writer retains write/rename access. Conventions and whitespace checks pass. Linux/macOS execution and full-suite CI are not claimed for this revised head.

The previous opt-in implementation's memory measurements are superseded and are not evidence for this implementation. The independent Demon's Souls report in #4806 (comment) applies to ThatsLeo's original POSIX branch: it reached the tutorial where its comparison run exhausted the memory cap. That report used a warm cache and test-only format/source-version overrides, which are not included here. It is external Linux title evidence, not a Windows title validation of this revised head. No new Windows title-memory or speed improvement is claimed yet.

Checklist

• Based on current main; no equivalent open cache-mapping PR found
• One topic per PR; replaces the earlier implementation
• No new code comments
• Unimplemented paths throw; no silent stubs
• No investigation artifacts committed
• No third-party code added
• Generic shader-cache storage behaviour
• Depends on: none
• Unblocks: none (addresses reported memory/storage failures; revised Windows title outcome pending)
• GPUs tested: NVIDIA RTX 5070 / 617.14 and AMD Radeon integrated gfx1036 / 32.0.21036.18 on Windows 11 Pro 10.0.26300
• AI-assisted: yes, OpenAI Codex

Related: #5552 changes the fallback file reader. Its head �f2b86928 merges cleanly with this branch in a pairwise merge simulation; that simulation is not a combined build/test claim.

[kr4y4]

In a plain build the two new tests don't run. cmake --build build followed by --target libs leaves agc_shared_spirv_tests unbuilt, so ctest reports agc_shared_spirv_heap and agc_shared_spirv_file as Not Run.

Setup: Windows 11 (26200), WinLibs GCC 15.2.0 r7, Release, CI configure flags, an RTX 4070 Laptop GPU and an Intel Raptor Lake-S iGPU. The head 501f197 merged into main 495bf91. It now conflicts with main ea2bbef in docs/dev/debt/shader.md.

• add_test_executable(agc_shared_spirv_tests EXCLUDE_FROM_ALL ...) has no set_target_properties(agc_shared_spirv_tests PROPERTIES EXCLUDE_FROM_ALL OFF). The other registered tests in this file set that inside if(BUILD_TESTING), and the two add_test lines here are outside that block.
• After the CI build steps the executable doesn't exist. The full suite (ctest --timeout 120 --parallel 4) then passes 1023 of 1025 on both GPUs, and the two failures are these Not Run entries. tools/check_test_registration.py still passes, because it only checks executables that were built.
• With the target built by hand (--target agc_shared_spirv_tests), both tests pass on both GPUs with --repeat until-fail:3. With APS5_FILE_BACKED_SPIRV=1 it prints sharing, write isolation, lifetime and reads passed, with a private-bytes delta of 163840.

Also, the new block sits between add_test_executable(agc_shader_disk_cache_tests ...) and that target's own target_include_directories and target_link_libraries lines.

Written with AI assistance.

[eljackjoestar-cmd]

Rebased onto main fc1f063f5, preserving the newer shader-debt entries. Also corrected the reported registration defect: the shared-SPIR-V test target is included in the normal build when BUILD_TESTING=ON, its two add_test calls are guarded by that option, and its block is separate from the disk-cache target setup.

The default full Windows Release build now builds the executable without naming its target. Heap/file storage, shader memory and shader disk cache all pass without skips (four tests, 0.78 s). The 64 MiB private-byte deltas are 67,244,032 bytes for heap storage and 159,744 bytes for file backing. Separate cache-disabled heap/file replay processes emit identical 134,780-word SPIR-V for the captured title shader. Conventions and whitespace checks pass. No new game-runtime claim is made for this rebase.

[oneandonlydean]

APS5_FILE_BACKED_SPIRV is an opt-in environment switch; a change like this goes in as the one path, not behind a knob. With it on, the temporary volume filled after about three minutes in the run reported above. Mapping the disk-cache entry itself, as in ThatsLeo's branch, needs neither the switch nor extra disk space; could you take that route instead?

[fisherjoey]

A Demon's Souls (PPSA01341 v1.07) data point for the route @oneandonlydean suggested. @ThatsLeo, your perf/shader-spirv-on-disk (0bc9120) gets the title into its tutorial level on Linux. Without it, the title runs out of memory while the level loads.

Setup: Linux, RTX 5070 Ti, NVIDIA 595.91, 30 GiB RAM, ANYPS5_ENABLE_SPIRV_TOOLS=OFF. Base main 7a0c838 with local merges of #758, #5181 (ceda63b, b130903) and a local stencil-export workaround for #4276. The run is capped at 22 GiB RAM plus 12 GiB swap. Warm shader cache, which needed two test-only tweaks to stay valid across the builds (I kept the cache format at 30 and overrode the source version). Neither is part of your change.

	without the branch	with 0bc9120
outcome	OOM-killed at the cap at about t+337, still loading the level	no OOM in 1200 s, reaches the level (HUD, first tunnel)
memory	12.2 GiB anon + 12 GiB swap at the kill	peak ~14 GiB anon + swap (anon 3.5-6.3 GiB, swap levels off near 8.7 GiB), plus 3-6 GiB of reclaimable file-backed pages
shader variants	5.0k at the title, 8.3k at the kill	9614 disk-cache hits, 250 misses

A heap profile of the run without it puts the growth in SPIR-V decoded from the disk cache and kept in SourceEntry::variants: 3.1 GiB live at t+40 and 10.0 GiB at t+340, about 1.2 MiB per cache hit.

This PR's file-backed storage doesn't help on this machine, because /tmp is tmpfs and the spilled modules stay in RAM.

If you open the branch as a PR, I can rerun it there.

AI-assisted: yes

[eljackjoestar-cmd]

Replaced the opt-in temporary-file implementation with the suggested cache-entry mapping route. The branch now carries ThatsLeo's original change with its authorship preserved, plus Windows mapping support at 4a0489a7a. There is no APS5_FILE_BACKED_SPIRV switch or separate temporary backing file. Cache-loaded and freshly persisted variants retain their cache mapping; mutation detaches a private copy.

Rebased onto f87799aa3. Full Windows Release build passed. Four cache/shader/rendering checks passed on RTX 5070 (617.14), and the two rendering/prepared-shader checks passed on AMD integrated gfx1036 (32.0.21036.18). The regression verifies mapped storage, copy isolation, a retained writer's write/rename access, and reader survival after replacement/removal of the pathname. Conventions pass. The body records the Demon's Souls result above as external evidence for the original POSIX branch; its test-only format/source-version overrides are not included, and no revised Windows title-memory result is claimed.

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
#757	core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp L726-792, L812-862 core/shader/recompiler/ShaderDiskCache.cpp L80-83
#758	core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp L726-792, L817-862 core/shader/recompiler/ShaderDiskCache.cpp L80-83 core/shader/recompiler/ShaderDiskCache.hpp L16
#4732	core/shader/recompiler/ShaderDiskCache.cpp L16-17

[oneandonlydean]

The Linux build with SPIRV-Tools failed in SpirvNarrowConstantStore.cpp: SpirvTools::Validate has no overload for a span, so I pushed a one-line fix that passes the word pointer and count. It builds with ANYPS5_ENABLE_SPIRV_TOOLS=ON and the shader tests pass; running the full build again.

[fisherjoey]

Demon's Souls (PPSA01341 v1.07) on Linux with this head (4a0489a), as offered above. With it, the title loads the tutorial level and stays in it, where it was OOM-killed without it.

Setup: Linux, RTX 5070 Ti, NVIDIA 595.91, ANYPS5_ENABLE_SPIRV_TOOLS=OFF. Base main 6546a33 with local merges of #758, #5181 (three commits) and #5747, plus a local stencil-export workaround for #4276. The run is capped at 22 GiB RAM and 12 GiB swap. Warm shader cache: 9067 hits, 757 misses.

	without the cache mapping (warm)	with this PR (warm)
outcome	OOM-killed at the cap at about t+337, during the level load	ran the full 1500 s with no fatal error, in the level from about t+180
at about t+335	anon 12.2, file 2.7, shmem 1.0 GiB, swap 12 GiB (the cap)	anon 8.0, file-mapped 1.6, shmem 2.3, swap 4.4 GiB
later		levels off from about t+830: anon 3.9-4.5, file 1.0-1.1, shmem 0.5-0.7, swap 9.9 GiB. Peak anon + swap 14.4 GiB

One caveat: the run without the mapping was on an older base (main 7a0c838 with #5181). The newer base also includes #290's larger import budget, so the base differs a little besides this PR.

The Linux side works like ThatsLeo's original branch did here (numbers in my earlier comment).

AI-assisted: yes