fix(agc): preserve empty images in stencil feedback checks

[fdfawef324]

What

Keep supported empty typed-image elements out of stencil replay's attachment-feedback scan. A null element currently reaches texture decoding and throws before an otherwise valid draw. The existing empty-image predicate skips that element while non-null depth aliases still receive the feedback refusal.

This is the separate follow-up requested in #5466, which has merged. The diff against main is now only this change: one production line in Draw.cpp and its own native fixture tests/execution/StencilNullFeedback.cpp.

Tested

• Rebased onto main 341613ab6 (137 commits past ab9a47684, several touching Draw.cpp) with no conflicts; head da0d5b2be, same three commits. Conventions check passes.
• The fixture binds a real image followed by an empty array element at a nonzero binding, checks the ordinary path, replay colors, both stencil classes, and rejection of a non-null image aliasing depth.
• Windows Server 2025 10.0.26100, NVIDIA GeForce RTX 3060 12 GB, driver 591.86, head da0d5b2be: ctest -R 'stencil|masked_select|replay' passed 3/3 runs (agc_masked_select, agc_driver_sample_mask_depth_stencil, agc_driver_stencil_null_feedback, agc_driver_stencil_ref_export_unused, agc_driver_stencil_replay, agc_driver_stencil_replay_no_gpl, agc_driver_pm4_replay, agc_driver_pm4_replay_tool). agc_driver_stencil_ref_export skips because this GPU lacks VK_EXT_shader_stencil_export. agc_driver_stencil_null_feedback also passed 3/3 runs with APS5_NO_GPL=1.
• On this head, with only the Draw.cpp line reverted, agc_driver_stencil_null_feedback fails with guest texture descriptor has a null base address; with it restored, the test passes.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR (follow-up to merged feat(agc): replay proven finite stencil reference exports #5466)
• No comments in code except technical debt
• Unimplemented paths throw; silent stubs are listed in their library's technical debt
• No notes, investigation .md files or images added to the repository
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none (feat(agc): replay proven finite stencil reference exports #5466 merged)
• Unblocks: synthetic empty-image stencil replay cases; no title advancement claimed
• GPUs tested: NVIDIA GeForce RTX 3060 12 GB, driver 591.86 (32.0.15.9186), Vulkan 1.4.325, Windows Server 2025 10.0.26100
• AI-assisted: yes

Co-Authored-By: Codex noreply@openai.com

[fdfawef324]

Rebased onto main ab9a47684 now that #5466 has merged. Its commits dropped out with no conflicts, so the diff is only the one-line Draw.cpp change and StencilNullFeedback.cpp (head 20e14ab0a -> 5cfd7bf6f).

On an RTX 3060 (driver 591.86, Windows Server 2025), ctest -R 'stencil|masked_select|replay' passed 3/3 runs, and stencil_null_feedback also passed 3/3 with APS5_NO_GPL=1. stencil_ref_export skips because the GPU lacks VK_EXT_shader_stencil_export. With only the product line reverted, the fixture fails with the null-base-address error. Marked ready for review.

AI-assisted: yes

[fdfawef324]

Rebased onto main 341613ab6 (137 commits, several touching Draw.cpp) with no conflicts: head 5cfd7bf6f -> da0d5b2be, same three commits.

On the RTX 3060 (driver 591.86, Windows Server 2025), ctest -R 'stencil|masked_select|replay' passed 3/3 runs at the new head, now also covering agc_driver_sample_mask_depth_stencil and agc_driver_stencil_ref_export_unused from main. stencil_null_feedback passed 3/3 with APS5_NO_GPL=1, and still fails with the null-base-address error when only the product line is reverted. Body updated.

AI-assisted: yes

[github-actions]

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 1191	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 1341	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 1368	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 1395	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 2434	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 4072	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 4650	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 4891	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5021	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5371	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5426	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5431	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 5441	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 5499	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 5609	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 5925	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 5958	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5998	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6044	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6054	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6082	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6176	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6182	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6188	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6192	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6222	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6279	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6292	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6300	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6373	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6414	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6457	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6479	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6499	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6503	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6535	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6539	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6615	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6636	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6678	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6683	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6727	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6730	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6741	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6747	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6763	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6788	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6808	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6941	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6946	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6947	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6973	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6987	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6989	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 6995	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 7002	core/libs/prx/libSceAgcDriver/CMakeLists.txt core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 7154	core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp
PR 7201	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7223	core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7332	core/libs/prx/libSceAgcDriver/CMakeLists.txt

[nicolasfalesy]

Tested on Linux (Arch), c++ (GCC) 16.2.1 20260810, Release, BUILD_TESTING=ON, main at 5d9dcc42d2ec5401aebd089028b8d4d7410a77b4. GPU tests ran on lavapipe (Mesa 26.2.2, llvmpipe with LLVM 22.1.8).

agc_driver_stencil_null_feedback (this PR's test) fails on main plus this PR (2 of 2):

AGC graphics: vkCreateGraphicsPipelines library: Vulkan result -13

With APS5_NO_GPL=1 it still fails, at the plain pipeline creation (AGC graphics: vkCreateGraphicsPipelines: Vulkan result -13), so it is not only the pipeline library path. CONTRIBUTING says CI runs the Vulkan tests on lavapipe, so this test would be red there. The description reports RTX 3060 only.

I did not find which part of the pipeline state lavapipe rejects. A skip on that device, or a different fixture state, is needed for the test to run on lavapipe.

AI-assisted: yes

[adriadam10]

The Draw.cpp:2096 change looks right, but the new fixture does not run outside your NVIDIA setup:

• agc_driver_stencil_null_feedback segfaults inside libvulkan_radeon on RADV (Mesa), in LinkPipelineFromLibraries on the first draw, and also with APS5_NO_GPL=1. @nicolasfalesy confirmed it independently on lavapipe (vkCreateGraphicsPipelines result -13, with and without GPL). CI runs the Vulkan tests on lavapipe, so it would be red.
• Likely cause: AttachNullImages (StencilNullFeedback.cpp:92-139) splices a descriptor-array variable into the SPIR-V by hand and appends a binding to result.bindings. NVIDIA seems to tolerate a module/layout mismatch; other drivers do not. Please make the module and the pipeline layout agree (e.g. take the image array from a real shader, or validate the module with spirv-val), and run the test on lavapipe or RADV before pushing.

Once the fixture passes on a non-NVIDIA driver (and still fails with the Draw.cpp line reverted), this is ready.

[fdfawef324]

Thanks both for running the fixture on RADV and lavapipe. We only have an NVIDIA GPU, so we could not reproduce the crash. What we could do without a GPU: compile the fixture's two pixel shaders with the repo's recompiler, run the old splice on them, and check the result with spirv-val built from our 3rdparty/SPIRV-Tools submodule. That showed two real problems in AttachNullImages.

1. The old module was not valid SPIR-V. spirv-val rejects all 8 modules we generated (SPIR-V 1.3 and 1.4, with and without graphics-pipeline-library style modules, both pixel shaders) with "Decorate is in an invalid layout section". The splice put its decorations before the first OpTypeVoid, but the recompiler declares its float, vector and pointer types and the Input/Output variables before that, so the decorations ended up between types. This is consistent with the lavapipe -13 and the RADV crash, but we cannot confirm that on those drivers.
2. The descriptor did not match the layout rules the driver uses. The old binding was set 1, binding 30. Fragment-stage bindings are numbered 65 plus the heap kind, and for a float 2D storage image heap that is binding 96. Binding 30 is the vertex-group number of the float 1D-array heap, not the 2D image the module declares. ShaderResources picks the set of a binding from its number (binding / 65), so the draw's set layout put binding 30 in set 0 while the module declared set 1. DescribeSetLayouts uses the metadata's set and would have said 1. We read this from the code. We did not run the old fixture with the layout check; the fixed one passes with APS5_VERIFY_SET_LAYOUTS=1 on our RTX 3060, which compares those two layouts.

What changed (commit d707066, test and CMake only, the product line in Draw.cpp is untouched):

• The decorations go after the existing annotations, before the first type, constant or variable.
• The variable has the type the recompiler emits for a float 2D storage heap (format Unknown, no NonWritable) and is listed in the SPIR-V 1.4 entry point interface like the recompiler does.
• The binding is numbered like the recompiler numbers it for the fragment stage, in set 1 when the target uses registration modules and set 0 otherwise.
• When built with ANYPS5_ENABLE_SPIRV_TOOLS=ON (the Linux CI configuration) the fixture validates its module with SPIRV-Tools before drawing; CMake links SPIRV-Tools-static into this one test in that case.

Check: with the same 8 combinations, spirv-val accepts the new modules, the aliasing variant used by the depth feedback case, and the unmodified recompiler output, in the vulkan1.1, vulkan1.1spv1.4, vulkan1.2 and vulkan1.3 environments (72 of 72 runs; the 8 old modules fail in all three of the environments they were checked in). We ran the validation function on the dumped modules in a separate small program; we have not built the whole tree with ANYPS5_ENABLE_SPIRV_TOOLS=ON here (the same SPIRV-Tools-static link is already used by other tests in that configuration).

The fixture still feeds the same descriptor words: element 0 is a valid image and element 1 is all zeros, and the product line in Draw.cpp is untouched. We ran it on our RTX 3060 (driver 591.86, Windows Server 2025): agc_driver_stencil_null_feedback passes with the default settings, with APS5_NO_GPL=1, with APS5_VERIFY_SET_LAYOUTS=1 and with both (4 of 4), and agc_driver_stencil_replay, agc_driver_stencil_replay_no_gpl and agc_driver_stencil_ref_export_unused pass (agc_driver_stencil_ref_export skips: the GPU lacks VK_EXT_shader_stencil_export). With the Draw.cpp line removed (a build with exactly that line deleted), the fixture fails with "guest texture descriptor has a null base address", so it still fails without the fix.

Could one of you rerun agc_driver_stencil_null_feedback on RADV and on lavapipe, with and without APS5_NO_GPL=1? Setting APS5_VERIFY_SET_LAYOUTS=1 as well makes the driver compare the set layouts described from the shaders with the ones the draw built, which is the check for problem 2.

AI-assisted: yes

[adriadam10]

Re-checked the new head: the Draw.cpp:2096 skip is unchanged and correct, and agc_driver_stencil_null_feedback now passes on RADV (Mesa) and on lavapipe, including with ANYPS5_ENABLE_SPIRV_TOOLS=ON validating the spliced module. With the Draw.cpp line reverted the test fails, so it covers the change. Merging once the full Build is green.

[adriadam10]

Re-checked against current main: the PR still merges cleanly, the Draw.cpp:2096 skip is unchanged and agc_driver_stencil_null_feedback passes; with that line reverted it fails. Merging once the full Build is green.

[oneandonlydean]

Picking this up while @adriadam10 is away: we will merge it once full CI and an independent review pass on this head.