fix(shader): bind an unnormalized S# normalized when it samples only null images

[gjonn]

What

A compute shader that does image_sample_lz through an S# forcing unnormalized coordinates was refused when that S# is paired with a T# the shader declares 3D, 2D-array, cube or multisampled:

• DescriptorBindingBuilder::Populate unnormalized guest sampler samples a 1D-array, 2D-array, 3D, cube or multisampled image, which is not implemented;
• and, past that, the driver's guest sampler descriptor uses unnormalized coordinates which are not implemented.

Vulkan allows unnormalized coordinates only on a single-level 1D or 2D view, so a real T# of those shapes still cannot take such a sampler. But a null T# reads zeros at any coordinate, and a T# whose four channel selects are all constants is emitted as those constants without a sample. When an unnormalized S# samples nothing else, its coordinates cannot change any result.

PPSA01317 compute shader 0x1258d12200 does exactly this during boot: S# 00008092 00fff000 02500000 00000000 (clamp on all axes, bilinear, explicit LOD) is paired only with a null T# and a 1x1x1 R8 3D T# whose channel selects are all 0.

Change:

• ProveUnnormalized skips a sampled pair whose shape mismatch comes from a null T#. If the S# then samples no real T#, its binding element gets the new samplerCoordinatesIgnored flag instead of samplerUnnormalized.
• ShaderResources::addImageBinding clears the force-unnormalized bit of such an S# before creating the sampler.
• An S# that also samples a real T# while a null T# of such a shape shares it is still refused, with the same message as before.
• The flag is part of the binding content key and is stored in the shader disk cache; FormatVersion goes 37 -> 38 (main is at 37 since feat(agc): implement 11_11_10 float color targets through 32-bit uint attachments #7207), as earlier binding-flag additions (e.g. samplerIntegerBorder) did.

Tested

Windows 11, MinGW-w64 GCC, NVIDIA GeForce RTX 5090 Laptop GPU (driver 616.92). Rebased on main at c0a7468.

• agc_shader_memory (new verifyNullVolumeUnnormalizedSampler, which runs DescriptorBindingBuilder::Populate with the S# above):
  • S# paired only with a null 3D T#: accepted, samplerCoordinatesIgnored = {true}, samplerUnnormalized = {false}.
  • S# paired with a null 3D T# and a 3D T# whose channels all select 0: accepted the same way.
  • S# paired with a real 3D T#: still refused.
  • S# shared by a null 3D T# and a real 2D T#: still refused.
  • With the non-test files at main (only the new DescriptorBinding::samplerCoordinatesIgnored field kept so the test compiles): fails with DescriptorBindingBuilder::Populate unnormalized guest sampler samples a 1D-array, 2D-array, 3D, cube or multisampled image, which is not implemented.
  • With this change: passes.
• agc_shader_disk_cache (binding round-trip now covers samplerCoordinatesIgnored), agc_shader_disk_cache_large_file, agc_driver_graphics: pass.
• python tools/check_conventions.py --base origin/main and python tools/check_test_registration.py pass.

Not tested: AMD, Intel, Linux, macOS. The title was not re-run on this exact branch.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Fixes: none
• Unblocks: PPSA01317 Gran Turismo 7 (PPSA01317) #4697
• GPUs tested: NVIDIA GeForce RTX 5090 Laptop GPU, driver 616.92, Windows 11
• AI-assisted: yes

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
PR 5656	core/shader/recompiler/ShaderDiskCache.cpp L88-90
PR 6454	core/shader/recompiler/Optimization/include/Optimization/ResourceMaterializer.hpp L58
PR 6470	core/shader/recompiler/ShaderDiskCache.hpp L16
PR 6486	core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp L87 core/shader/recompiler/ShaderDiskCache.cpp L90
PR 6683	core/shader/recompiler/ShaderDiskCache.hpp L16
PR 6858	core/shader/recompiler/ShaderDiskCache.hpp L16
PR 6924	core/shader/recompiler/ShaderDiskCache.hpp L16
PR 6941	core/shader/recompiler/ShaderDiskCache.cpp L89-90
PR 6946	core/shader/recompiler/ShaderDiskCache.cpp L88-90
PR 6947	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp L4116 core/shader/recompiler/ShaderDiskCache.cpp L88-90 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 6970	core/shader/recompiler/Recompiler.hpp L428-432 core/shader/recompiler/ShaderDiskCache.cpp L90

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 907	core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp
PR 1341	core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp
PR 3247	core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp core/shader/recompiler/Recompiler.hpp core/shader/recompiler/ShaderDiskCache.cpp
PR 4650	core/shader/recompiler/Recompiler.hpp
PR 5371	core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp core/shader/recompiler/Optimization/include/Optimization/ResourceMaterializer.hpp core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp
PR 5500	core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp core/shader/recompiler/Recompiler.hpp core/shader/recompiler/ShaderDiskCache.cpp
PR 5609	core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp core/shader/recompiler/Recompiler.hpp core/shader/recompiler/ShaderDiskCache.cpp
PR 5925	core/shader/recompiler/Recompiler.hpp core/shader/recompiler/ShaderDiskCache.cpp core/shader/recompiler/ShaderDiskCache.hpp
PR 6177	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp core/shader/recompiler/Recompiler.hpp
PR 6188	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp core/shader/recompiler/Recompiler.hpp
PR 6192	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6194	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6223	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6232	core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp core/shader/recompiler/ShaderDiskCache.cpp
PR 6235	core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp core/shader/recompiler/Recompiler.hpp
PR 6280	core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp core/shader/recompiler/Optimization/include/Optimization/ResourceMaterializer.hpp core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp core/shader/recompiler/ShaderDiskCache.cpp core/shader/recompiler/ShaderDiskCache.hpp
PR 6292	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6312	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp
PR 6412	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp core/shader/recompiler/Optimization/include/Optimization/ResourceMaterializer.hpp core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp core/shader/recompiler/Recompiler.hpp core/shader/recompiler/ShaderDiskCache.cpp core/shader/recompiler/ShaderDiskCache.hpp
PR 6414	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6481	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6488	core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp core/shader/recompiler/ShaderDiskCache.cpp core/shader/recompiler/ShaderDiskCache.hpp
PR 6503	core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp core/shader/recompiler/Recompiler.hpp
PR 6539	core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp core/shader/recompiler/Recompiler.hpp core/shader/recompiler/ShaderDiskCache.cpp
PR 6615	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6645	core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
PR 6879	core/shader/recompiler/Recompiler.hpp core/shader/recompiler/ShaderDiskCache.cpp
PR 6937	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6972	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6977	core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp core/shader/recompiler/ShaderDiskCache.cpp
PR 7002	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp core/shader/recompiler/Recompiler.hpp
PR 7085	core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp core/shader/recompiler/Optimization/include/Optimization/ResourceMaterializer.hpp core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp core/shader/recompiler/ShaderDiskCache.cpp core/shader/recompiler/ShaderDiskCache.hpp
PR 7201	core/shader/recompiler/Recompiler.hpp
PR 7223	core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp core/shader/recompiler/ShaderDiskCache.cpp core/shader/recompiler/ShaderDiskCache.hpp
PR 7332	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp core/shader/recompiler/Recompiler.hpp
PR 7390	core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp core/shader/recompiler/Recompiler.hpp core/shader/recompiler/ShaderDiskCache.cpp core/shader/recompiler/ShaderDiskCache.hpp
PR 7426	core/shader/recompiler/Recompiler.hpp

[oneandonlydean]

Full CI on 6bbbf0a (merged into main) fails to compile on Linux only:

core/shader/recompiler/ShaderDiskCache.cpp:90:41: error: static assertion failed: DescriptorBinding changed: update the binding encoder; the comparison reduces to '(632 == 592)'.

The new std::vector<bool> samplerCoordinatesIgnored field in DescriptorBinding (Recompiler.hpp:429) adds 40 bytes under libstdc++. The size check at ShaderDiskCache.cpp:90 is compiled only for __linux__ && __x86_64__ && __GLIBCXX__, so it still says 592 and Windows passes. The encoder and decoder already carry the new flag, so only the number is out of date. DescriptorBinding hasn't changed on main since this PR's base.

Fix: change the assert to static_assert(sizeof(DescriptorBinding) == 632, ...). This is the same one-line change 05b3f6e made for samplerIntegerBorder (488 -> 528).

Separately, #7085, #6959, #6941, #7223 and #6924 also raise ShaderDiskCache FormatVersion from 36 to 37. Whichever of them lands after this PR needs 38 instead (or this PR does, if one of them lands first).

[gjonn]

Thanks. Fixed in ec2453d: the assert now expects 632, the same one-line change as 05b3f6e. I couldn't build the Linux-only assert here, so the value comes from your CI output.

On FormatVersion: noted. If one of #7085, #6959, #6941, #7223 or #6924 lands first, I'll rebase and move this one to 38.

AI-assisted: yes

[gjonn]

Rebased onto main 385ed71 (force-push, head ca7f2f6). Git merged cleanly, but main is already at ShaderDiskCache FormatVersion 37 since #7207, so the identical 36 -> 37 change merged away silently. This PR now raises it to 38. DescriptorBinding hasn't changed on main since the old base, so the Linux size assert stays at 632.

agc_shader_memory, agc_shader_disk_cache and agc_shader_disk_cache_large_file pass (Windows, MinGW GCC, Release). check_conventions.py --base origin/main and check_test_registration.py are clean.

AI-assisted: yes

[LeandroLP23]

AMD on Windows: the tests this PR names pass on an RDNA 3.5 GPU.

Radeon 860M (RDNA 3.5, driver 32.0.13058.8001), Windows 11 26300, WinLibs GCC 15.2.0 r7, Release. main 6efb7261b with this PR (ca7f2f60b) merged, together with #7446, #7377, #7445, #6708, #6748, #6785, #6706, #6613, #7449, #7391, #7489, #7484, #7514. They were built once, so the result is for the combination; each merge was clean.

• agc_driver_graphics, agc_shader_disk_cache, agc_shader_disk_cache_large_file, agc_shader_memory: pass, 3 runs of 3.
• Every agc_* test on the combined build: 713 of 716 pass. The 3 that fail, agc_driver_host_import_lifetime, agc_driver_quad_pixel_mask and agc_driver_layered_volume_vertex_no_gpl, fail the same way on unchanged main here, so none comes from these PRs.

AI-assisted