fix(shader): propagate color view slice to pixel ancillary input

[ivica00]

What

When a draw does not export Layer, the pixel ancillary render-target index was 0 even if CB_COLOR_VIEW started at a later slice. PixelLayerBase uses the 3D depth slice, or the 2D array slice, in that case, and 0 when a layer is exported or no color target is bound. DrawDecode stores it on the pixel stage and sameDecode compares it. The recompiler adds that value to the exported layer in the packed ancillary word.

PipelineSpecialization::LayerBase is 8236 (8235 is StencilReferenceOffset on main). Recompile requests are version 28; versions through 27 leave the base at 0.

The color-view warning suppression and the 2D-array color.layers edit are not part of this PR. The slice addend is this driver's model of SLICE_START. It is not a console measurement; the ancillary bit positions follow the RDNA 3 / GCN guides already cited in docs/dev/debt/shader.md.

Tested

pixelLayerBaseTests in agc graphics tests checks the 2D slice, the 3D depth slice, an exported layer, and a draw with no color target. pixel_ancillary and agc_shader_memory were not re-run after the merge onto current main.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Unblocks: none
• GPUs tested: n/a for this push
• AI-assisted: yes

[MarcZX10]

The conventions check is failing because the required ### What section is missing or empty. The body currently starts with the implementation description but uses a BOM before the heading, so the template parser does not recognize it. Please remove the leading BOM and use a plain ### What heading so the check can parse this section.

[xXPBSXx]

Windows build of #5500: Windows 10 (19045), MinGW-w64 GCC 15.2, Release, ninja -j 4. Tree: main feddf7f4c + merge of PR head 1fa0de855fc10d8058265b548f22a39cfcbe7355 (the merge itself had no conflict). Targets: agc_driver_pixel_ancillary_word_tests, agc_driver_relative_ancillary_tests, agc_shader_memory_tests, pixel_ancillary_tests. Log: rtx5500_result.txt.

Result: the build stops (ninja exit code 1) at shader/recompiler/Recompiler.cpp:184:22: error: 'struct ShaderRecompiler::TranslateOptions' has no member named 'integerDot4x8'. The line translateOptions.integerDot4x8 = ...(spv::CapabilityDotProductInput4x8BitPacked) is added by this PR's commit (git diff 1fa0de855~1 1fa0de855 -- core/shader/recompiler/Recompiler.cpp, merge base cf5f5b53b); no member integerDot4x8 exists in TranslateOptions on that base or on main (git grep integerDot4x8 upstream/main -- core finds nothing). Open #5337 (perf: v_dot4 with packed integer dot products) mentions the same capability; I did not check whether it defines the member.
Because of the compile error none of the PR's tests could be built or run (0 tests run). The PR's Windows/Linux CI jobs show as skipped, so no CI build covers this.

Not tested: everything that needs a build of this PR, whether the line comes from a stacked branch, Linux, a title run.

AI-assisted (Claude Code).

[MarcZX10]

I checked the current TranslateOptions declaration at core/shader/recompiler/Translation/include/Translation/InstructionTranslator.hpp: it has no integerDot4x8 member. The same assignment appears in #5337, but that PR also does not add the member, so stacking on #5337 alone would not resolve this compile failure. Please coordinate the option/API change with #5337 (or adjust this patch to the current translator API) and ensure the supported-capability path is covered by a build before merging.

[ivica00]

Removed the translateOptions.integerDot4x8 assignment. TranslateOptions has no such member, and #5337 does not add one, so that line did not compile on this branch or on main.

Also merged current main. The ShaderDiskCache conflict keeps layerBase on the shader-info encode and decode, inside the split encodeInfo / decodeInfo helpers.

Pushed in 720127c.

[adriadam10]

Thanks for tracking this down. Points before this can go in:

• Source for the semantics. The description says the rasterizer feeds CB_COLOR_VIEW.SLICE_START into ancillary bits 16..28 when no layer is exported, but cites nothing. As far as I know the render target index there comes from the layer exported by the previous stage, and the CB applies the view slice itself when addressing the target. Please cite a source (ISA guide, Mesa/LLPC, a measurement) or a test that shows the slice offset reaches the pixel shader. The Hello Neighbor 2 screenshot only shows the symptom; it does not show where the value comes from.
• Unrelated/undescribed changes. State.cpp:873 now suppresses the "layer/viewport index vertex exports are ignored" message for non-zero slices, and State.cpp:1119 changes color.layers for 2D arrays. Neither is in the description and neither has a test. Please justify them or drop them.
• No test for the new wiring. DrawDecode.cpp:120-122 (layerBase from depthSlice/arraySlice) and sameDecode are not exercised; the tests only cover the IR and the serializer.
• Conflicts with main. main is now at request serialization version 27 and PipelineSpecialization id 8235u is taken by StencilReferenceOffset; StageIO.hpp, Recompiler.hpp, DrawDecode.cpp, DrawRegisterKey.cpp and ShaderMemory.cpp conflict too. Please rebase and renumber the version and the constant, and rerun pixel_ancillary and agc_shader_memory.

[ivica00]

Merged current main. LayerBase is specialization id 8236 and recompile requests are version 28, so versions through 27 still read a base of 0. The warning suppression and the 2D-array color.layers edit are removed. PixelLayerBase is what DrawDecode stores, and pixelLayerBaseTests covers a 2D slice, a 3D depth slice, an exported layer, and a draw with no color target. sameDecode compares layerBase together with layerExported and dithered coverage. The slice addend is this driver's model of SLICE_START; I do not have a console measurement that the rasterizer itself inserts that register into the ancillary word, and docs/dev/debt/shader.md says so. pixel_ancillary and agc_shader_memory were not re-run after this merge.

[ivica00]

The two open points are still open, and I did not invent a source for them.

There is still no ISA, Mesa or LLPC reference that feeds CB_COLOR_VIEW.SLICE_START into ancillary bits 16..28, and no measurement or capture of a pixel shader reading a non-zero layer with a non-zero slice and no exported layer. The debt line already says the addend is this driver's model, not a console measurement. Adding it in the shader would apply the slice twice if the color block already adds SLICE_START when it addresses the target. I will not rebase or renumber until that evidence is there, as the last review asked.

DrawDecode storing layerBase, and sameDecode comparing it, are still not covered by a test. pixelLayerBaseTests only calls PixelLayerBase.