perf(shader): avoid redundancy traversal map copies

[JJenkx]

What

The bundled redundancy-elimination traversal copies inherited value maps for sibling branches. Replace those copies with an iterative depth-first traversal, one scoped value map, and a log of values inserted by each subtree. Roll back each subtree's insertions on return so ancestor reuse is preserved and sibling values remain isolated. Optimization passes, type restrictions, value numbering and validation are unchanged.

CMake patches a build-directory copy of the pinned SPIRV-Tools source and rejects drift in the traversal and elimination helper. The dependency checkout is untouched. This extends SPIRV-Tools #6962, which releases processed maps but retains sibling map copies; it does not replace the separate AnyPS5 IR work in #4948.

Tested

Current published head b00931842321f466b8ce1e7a48ac952d4f06f1ec is three topic/documentation commits directly above main 56429064fc3074ed5192e32a06ba0cf7813af4fc, with zero merge commits. Additive shader-debt conflicts were resolved while retaining main's structurizer entry. Conventions and whitespace checks pass. The production patch and optimizer regression file are byte-for-byte identical to measured head 5aea65b4b1c363c8783beb804f7e7579a2e9001e. The current-head Linux Release runtime builds with bundled SPIRV-Tools; default/full/none optimizer checks pass (3 passed, 0 skipped). Fresh full Linux/Windows CI passes this head against base 56429064fc3074ed5192e32a06ba0cf7813af4fc: Linux 1,073 passed / 48 skipped / 1,121 total; Windows 739 passed / 406 skipped / 1,145 total. All three optimizer modes pass on Linux. Windows disables SPIRV-Tools and does not validate the traversal patch; skipped cases remain unvalidated. The measurements below retain their original revisions; no performance measurement against the new base is claimed.

• At measured head 5aea65b4b1c3, Linux Release builds with bundled SPIRV-Tools enabled pass. Default/full/none optimizer checks pass on both main and this PR, with zero skips, including ancestor reuse, sibling scopes, conditional control flow, volatile stores and storage-only byte output. The earlier omitted-rollback negative control fails validation because the sibling value does not dominate its use.
• Existing generated-workload measurements demonstrate reduced map-copy work, but are synthetic compiler evidence and establish no title speed gain.
• Historical full Linux/Windows CI passes at topic 5aea65b4b1c363c8783beb804f7e7579a2e9001e and base 1e7272196389c30eff9af9e91a51c72d781f46c6: Linux 1068 passed / 46 skipped / 1114 total; Windows 735 passed / 402 skipped / 1137 total. Linux exercises all three optimizer modes. The Windows configuration does not enable SPIRV-Tools and does not validate the dependency traversal patch. Skips remain unvalidated.

Cold-title measurements

Astro Bot PPSA21564, untouched startup with no input, main 1e7272196389 versus main plus this PR 5aea65b4b1c3. Same machine, executable, modules, assets and settings; separate empty AnyPS5 shader, Vulkan pipeline and Mesa caches. Both builds enable APS5_ENABLE_TIMING_LOG=ON to retain preparation before first present. Official APS5_TRACE_SECONDS=0:60 traces start at first present; no other trace/profile switch and console output not redirected. This deliberately measures cold loading, as requested, rather than the performance form's warm-cache gameplay protocol.

First-flip boundary metric	Main, pair 1	PR, pair 1	Main, reverse-order repeat	PR, reverse-order repeat
First submit to flip completion, seconds	349.480	303.908	344.934	289.638
Inclusive shader preparation elapsed, seconds	1578.487	1206.310	1537.397	1128.035
Preparation calls	2243	2245	2245	2245
Maximum preparation call, seconds	13.907	8.457	14.065	8.021

First-flip wait is 13.0% shorter in the first pair and 16.0% shorter in the reverse-order repeat. Inclusive preparation is 23.6% / 26.6% lower. Repeat call counts are identical (2,245); the first pair differs by two calls. Both repeat runs reached the ASTRO BOT title screen in PID-matched snapshots at approximately 612 seconds, without input, and closed normally (exit 0); temporary kernel settings were restored.

Shader preparation includes translation, optimization and cache work. It sums inclusive elapsed calls across concurrent threads, so it exceeds wall-clock time and is not optimizer-only CPU time. First flip completion has display_confirmed=0; it is not the time to a visible frame or completed title loading. Sparse title-screen snapshots establish the destination, not a precisely measured time-to-menu gain. The traces cover initial loading, with different present counts (the seconds-based window ends after its final present). No gameplay or steady-FPS improvement is claimed. Build timing adds overhead to both builds. Two pairs establish a repeated observed gain on this host, not broad hardware coverage.

The first pair closed earlier during resource loading and both processes exited 139 after the close request. Those unresolved shutdown observations remain recorded; both longer repeat runs closed normally. No game assets, screenshots, core dumps or raw console logs are published.

Download all four traces, summary lines, digests and extracted timing comparison. The fork prerelease is a timing attachment, not an emulator build release; each .json.gz opens in Perfetto.

Official summary lines (pair 1 main/PR, then reverse-order repeat main/PR):

{"commit":"1e7272196389","dirty":false,"env":{"APS5_DRAW_CACHE_ENTRIES":"4096","APS5_HOST_IMPORT_MIB":"16384","APS5_PAD_INPUT_FILE":"<path>","APS5_PIPELINED_DRAWS":"1","APS5_TRACE_SECONDS":"0:60"},"gpu":"AMD Radeon RX 6650 XT (RADV NAVI23)","driver":"radv Mesa 26.2.4-arch1.1","os":"Linux 7.2.9-arch1-1","resolution":"3840x2160","window":{"spec":"seconds 0:60","first_present":0,"ms":64056.804},"presents":105,"frame_ms":{"p50":37.269,"p90":2364.364,"p99":4295.703,"max":4548.802},"gpu_busy_ms_per_present":33.152,"worker_busy_pct":{"queue worker 0x0":7.853,"queue worker 0x48":5.405,"queue worker 0x40":7.332},"top_phases_ms_per_frame":{"Driver.Packet":125.119,"Driver.Packet.memory_wait":68.363,"Shader.PrepareArtifact":43.275,"Driver.Packet.dispatch":31.749,"Driver.Dispatch":31.69,"Vulkan.Dispatch":19.241,"Vulkan.Dispatch.shader_resources":18.996,"Vulkan.GraphicsPipelinePrebuild":16.309},"trace_dropped":false,"trace":"aps5-trace-1e7272196389-3451464.json.gz"}
{"commit":"5aea65b4b1c3","dirty":false,"env":{"APS5_DRAW_CACHE_ENTRIES":"4096","APS5_HOST_IMPORT_MIB":"16384","APS5_PAD_INPUT_FILE":"<path>","APS5_PIPELINED_DRAWS":"1","APS5_TRACE_SECONDS":"0:60"},"gpu":"AMD Radeon RX 6650 XT (RADV NAVI23)","driver":"radv Mesa 26.2.4-arch1.1","os":"Linux 7.2.9-arch1-1","resolution":"3840x2160","window":{"spec":"seconds 0:60","first_present":0,"ms":62409.625},"presents":105,"frame_ms":{"p50":37.947,"p90":2207.181,"p99":3281.585,"max":3371.253},"gpu_busy_ms_per_present":34.569,"worker_busy_pct":{"queue worker 0x0":7.324,"queue worker 0x40":6.717,"queue worker 0x48":4.487},"top_phases_ms_per_frame":{"Driver.Packet":109.736,"Driver.Packet.memory_wait":57.306,"Shader.PrepareArtifact":44.191,"Driver.Dispatch":28.25,"Driver.Packet.dispatch":28.208,"Vulkan.GraphicsPipelinePrebuild":17.346,"Vulkan.GraphicsPipeline":17.322,"Vulkan.GraphicsPipelineLibrary":17.162},"trace_dropped":false,"trace":"aps5-trace-5aea65b4b1c3-3620398.json.gz"}
{"commit":"1e7272196389","dirty":false,"env":{"APS5_DRAW_CACHE_ENTRIES":"4096","APS5_HOST_IMPORT_MIB":"16384","APS5_PAD_INPUT_FILE":"<path>","APS5_PIPELINED_DRAWS":"1","APS5_TRACE_SECONDS":"0:60"},"gpu":"AMD Radeon RX 6650 XT (RADV NAVI23)","driver":"radv Mesa 26.2.4-arch1.1","os":"Linux 7.2.9-arch1-1","resolution":"3840x2160","window":{"spec":"seconds 0:60","first_present":0,"ms":62789.392},"presents":91,"frame_ms":{"p50":39.617,"p90":2610.342,"p99":4203.167,"max":4203.167},"gpu_busy_ms_per_present":34.887,"worker_busy_pct":{"queue worker 0x0":7.23,"queue worker 0x40":6.691,"queue worker 0x48":4.982},"top_phases_ms_per_frame":{"Driver.Packet":129.92,"Driver.Packet.memory_wait":71.194,"Shader.PrepareArtifact":44.54,"Driver.Packet.dispatch":34.86,"Driver.Dispatch":34.808,"Vulkan.Dispatch":22.51,"Vulkan.Dispatch.shader_resources":22.246,"Vulkan.GraphicsPipelinePrebuild":19.135},"trace_dropped":false,"trace":"aps5-trace-1e7272196389-4086320.json.gz"}
{"commit":"5aea65b4b1c3","dirty":false,"env":{"APS5_DRAW_CACHE_ENTRIES":"4096","APS5_HOST_IMPORT_MIB":"16384","APS5_PAD_INPUT_FILE":"<path>","APS5_PIPELINED_DRAWS":"1","APS5_TRACE_SECONDS":"0:60"},"gpu":"AMD Radeon RX 6650 XT (RADV NAVI23)","driver":"radv Mesa 26.2.4-arch1.1","os":"Linux 7.2.9-arch1-1","resolution":"3840x2160","window":{"spec":"seconds 0:60","first_present":0,"ms":61633.391},"presents":125,"frame_ms":{"p50":31.313,"p90":1939.731,"p99":3029.308,"max":3053.024},"gpu_busy_ms_per_present":32.66,"worker_busy_pct":{"queue worker 0x0":8.393,"queue worker 0x48":5.481,"queue worker 0x40":7.758},"top_phases_ms_per_frame":{"Driver.Packet":106.213,"Driver.Packet.memory_wait":56.589,"Shader.PrepareArtifact":33.973,"Driver.Packet.dispatch":26.534,"Driver.Dispatch":26.468,"Vulkan.Dispatch":15.256,"Vulkan.GraphicsPipelinePrebuild":15.075,"Vulkan.GraphicsPipeline":15.051},"trace_dropped":false,"trace":"aps5-trace-5aea65b4b1c3-3880445.json.gz"}

Independent title validation

milescoviello’s Forspoken PPSA03984 report tests this PR’s code commit 8ed0bc395 on main e34c47921 plus #5937, using an RTX 4070 Ti on Linux. Under a 15 GiB cap, the cold baseline reached the cap after 130 seconds and 1,624 shaders; the candidate completed the 150-second window with 2,505 shaders and 8.3 GiB peak anonymous memory versus 11.7 GiB. Two warm candidate launches reached a separate guest-memory-read stop rather than the baseline’s memory cap. That remaining stop prevents a playability claim.

The same contributor reports byte-identical optimized output for 40 captured production modules; the module with 1,925,310 words dropped from 8,069 MiB peak RSS to 371 MiB. These measurements belong to that contributor’s setup and dependency stack; they complement the Astro measurements above and were not rerun locally.

Checklist

• Based on recorded main; only this topic's commits above it
• One topic per PR
• No code comments added except technical debt
• No unimplemented paths or silent stubs added
• No investigation files, game assets or images added
• Existing pinned third-party source only
• Generic behavior
• Depends on: none
• Unblocks: none
• GPUs tested: AMD Radeon RX 6650 XT, RADV Mesa 26.2.4, Arch Linux; matched cold-start comparison below
• AI-assisted: yes

[MarcZX10]

I checked the merge state against current main (bf8dc61). GitHub marks this PR DIRTY; a local merge-tree check finds the only conflict in docs/dev/debt/shader.md, where this PR appends a new Build entry. The contribution docs describe resolving additive debt-file conflicts during rebase with python3 tools/debt_split.py resolve, then splitting with python3 tools/debt_split.py split if needed. The source/test/workflow files otherwise merge cleanly. Please refresh the branch and rerun the checks.

[JJenkx]

Refreshed and rechecked against current main 30f961266ade83b41d5c9de7dec1393e7fcecfb7. Published head is d7828747a19fe04b69c5cc63b5228a5d5c5b4c0e, which contains current main. The additive docs/dev/debt/shader.md conflict is resolved, preserving the upstream SDWA Functional entry and this PR's Build entry. The source patch is unchanged.

GitHub now reports MERGEABLE / CLEAN, and a fresh local git merge-tree --write-tree origin/main HEAD exits successfully with no conflicts. Conventions and whitespace checks pass. Fresh full fork CI passes for this head/base: Linux 1,037 passed / 46 expected skips; Windows 713 passed / 389 expected skips; zero failures. Skipped cases remain unvalidated.

The earlier DIRTY result against bf8dc61 describes the old branch state.

[JJenkx]

Resolved the reported overlaps in published head 180d47af0daa91bef8dac60d24361411599619ba, which includes current main d1670009fbf. Fresh local merge-tree checks merge this branch without conflicts with #4926, #4964 and #5028. The PR body describes the edits; these are independent features, so Depends on: none remains correct.

Conventions and whitespace checks pass. The compiler patch is unchanged; the duplicate Build section was consolidated into the existing section. Fresh full Linux/Windows CI passes on Linux and Windows for this published head; expected skips remain unvalidated.

The overlap workflow runs on main pushes and hourly, rather than PR pushes, so its earlier comment may remain until the next sweep.

[MarcZX10]

Reviewed the updated SPIRV-Tools traversal and regression coverage. The published head 180d47af0daa91bef8dac60d24361411599619ba contains current upstream main d1670009fbfdfe57f8f26ee09ced77fe6b18c2ca; the exact-head full Linux and Windows run 38485823046 completed successfully. The current PR check set also has passing conventions, eligibility and report checks; the Linux/Windows jobs on that upstream run are skipped by path filters. No remaining blocker found in the diff I reviewed.\n\nAI-assisted: yes; reviewed by MarcZX100.

[oneandonlydean]

Please rebase without the merge commits; the branch should be your commits on top of main.
Perf PRs need the summary lines of main and of the change from the same machine and scene (CONTRIBUTING, performance report form). The synthetic compiler numbers don't show what a title gains; a cold-cache load of a title, main against main plus this PR, with the shader compile time, would.

[JJenkx]

Rebased to three topic/documentation commits directly on main 1e7272196389, with no merge commits; published head 5aea65b4b1c3. Full Linux/Windows CI passes (Linux enables the patched SPIRV-Tools; Windows does not).

The description now includes matched Astro Bot PPSA21564 cold-start results from two pairs, with reverse order on the repeat. First submit to first flip is 349.480→303.908 seconds and 344.934→289.638 seconds (13.0% and16.0% shorter). Repeat shader-preparation call counts match at2,245; inclusive preparation elapsed is26.6% lower. Both repeat runs reach the title screen and close normally. This is cold-loading evidence, not a gameplay-FPS claim or an optimizer-only CPU measurement.

All four official summary lines, the protocol, limitations and a downloadable trace/digest bundle are in the description. The initial pair’s shutdown failures remain disclosed. Returned the PR to ready for review.

AI-assisted: yes.

[milescoviello]

A title comparison, since a cold-cache load of a title was asked for above: Forspoken (PPSA03984), relinked with --to-intel, on Linux (Ubuntu 24.04, GCC 13.3, Release with CI's flags, RTX 4070 Ti, NVIDIA 595.99.02). Both builds are main e34c479 plus #5937, without which the title stops at 43 s; one also has this PR's code commit (8ed0bc3), and they differ only in libSceAgcDriver.prx. Each run had a 15 GiB memory cap without swap, and the runs alternated, starting from an empty shader cache:

run	main	this PR
cold, 150 s	reached the cap at 130 s, 1,624 shaders compiled, peak anonymous 11.7 GiB	ran the full 150 s, 2,505 shaders compiled, peak anonymous 8.3 GiB
warm start, cache from the cold run	reached the cap at 10 s, 13.0 GiB anonymous	ran 52 s with 2,502 cache hits, peak 6.4 GiB
warm start again	reached the cap at 9 s, 12.0 GiB	ran 51 s with 2,503 hits, peak 5.4 GiB

The warm runs with this PR end at an unrelated stop (AGC driver: guest memory is not readable at 0x1037024000), which I'm looking at separately. A jemalloc heap profile of a warm main start had about 3.5 GiB of its live heap in std::map<uint32_t, uint32_t> copies under RedundancyEliminationPass::EliminateRedundanciesFrom.

Offline, I passed the 40 largest modules the title sends to ValidateAndOptimizeSpirv (1.0 to 5.3 M words, dumped by a local change) through spirv-opt built from the pinned SPIRV-Tools, once with main's patched redundancy_elimination.cpp and once with this PR's, using the default pass list from SpirvOptimizer.cpp and one process at a time. All 40 outputs are byte-identical. The largest differences in peak RSS:

module	main	this PR
1,925,310 words	8,069 MiB, 16.8 s	371 MiB, 2.1 s
1,792,844 words	7,262 MiB, 15.0 s	341 MiB, 1.9 s
1,660,134 words	6,455 MiB, 12.9 s	310 MiB, 1.7 s
1,527,464 words	5,652 MiB, 11.3 s	283 MiB, 1.5 s
5,291,434 words	1,268 MiB, 16.1 s	1,043 MiB, 7.9 s

AI-assisted: yes