AMD Windows: draws whose pixel shader reads the packed ancillary VGPR leave pixels unwritten (agc_driver_relative_ancillary fails on main) #4800

Activity

1. LeandroLP23 commented on Oct 12, 2026

   

   LeandroLP23

   on Oct 12, 2026

   ContributorAuthor

   More actions

   First finding: the read of gl_Layer is what loses the pixels.

   I dumped the SPIR-V that agc_driver_relative_ancillary's first draw (wave 32 packed read) passes to vkCreateShaderModule on main 584cc4c57. The fragment shader composes the ancillary word from gl_SampleID (bits 8+) and gl_Layer (bits 16+), both Flat inputs, with SampleRateShading and Geometry declared. It writes out_mrt_0 only in non-helper lanes and kills nothing else. The vertex shader writes Position, ClipDistance, and reads VertexIndex and InstanceIndex. It does not write Layer.

   Then, with a local hook that swaps in an edited fragment module (one OpLoad replaced by an OpBitcast of the constant 0, nothing else changed):

   Fragment module	First draw
   as emitted	pixel unwritten (incorrect pixel: 64, the fill)
   gl_SampleID load replaced by 0	pixel unwritten (64)
   gl_Layer load replaced by 0	written; the test moves on and fails later only because the edited module is reused for its other cases
   both replaced by 0	written

   So on this AMD driver, a fragment shader that reads Layer, in a pipeline whose vertex stage doesn't write it, leaves the color target untouched. lavapipe returns 0 there. I haven't confirmed whether the Vulkan spec defines the value when no pre-rasterization stage writes Layer. Either way, the draw shouldn't lose its output on AMD.

   Next I'll check the other direction: the same pipeline with a vertex shader that writes Layer = 0.

   AI-assisted: yes (Claude Code)
2. LeandroLP23 commented on Oct 12, 2026

   

   LeandroLP23

   on Oct 12, 2026

   ContributorAuthor

   More actions

   One more case narrows it down. With the gl_Layer load left in place but its result unused, the first draw writes its pixel. I changed only the OpBitcast that consumed the loaded value, so the value no longer reaches the ancillary word.

   In the emitted module, the layer lands only in bits 16+ of the word, and the output takes bits 8–11 (OpBitFieldUExtract … 8 4). So the layer value never reaches the colour, yet a live use of it loses the write. That looks like the AMD driver taking the fragment's render target layer from a vertex-stage export that is never written, when the fragment shader consumes Layer. The write then goes to a layer the one-layer target doesn't have. I haven't confirmed that mechanism on the hardware side. The SPIR-V results are what's established.

   Two ways the emulator could avoid it, for whoever picks this up:

   1. Fold Layer to 0 in the fragment shader when the pipeline's pre-rasterization stage doesn't write Layer, which is the common case. That is also the value the RT index has when the guest vertex shader doesn't export one. The pipeline knows both stages, so it could set it with a specialization constant, like the other per-pipeline values.
   2. Make the vertex stage write Layer = 0 whenever the fragment shader reads it and the guest vertex shader exports none. That needs shaderOutputLayer, and a vertex variant keyed on the fragment shader's needs.

   Option 1 keeps the vertex shaders as they are. I can test either one on this GPU.

   AI-assisted: yes (Claude Code)
3. LeandroLP23 commented on Oct 12, 2026

   

   LeandroLP23

   on Oct 12, 2026

   ContributorAuthor

   More actions

   Fix proposed in #4842: the packed ancillary lowering uses 0 for the render target index field instead of reading the unwritten Layer. On the Radeon 860M agc_driver_relative_ancillary passes with it (445/445 agc_), and with #4031 merged on top (plus the same 0 at its whole-word read) #4031's agc_driver_pixel_ancillary_word passes too.
4. oneandonlydean

   added 2 commits that reference this issue on Oct 13, 2026

   fix(shader): move the primitive class to request version 19 and compo…

   ...

   c52321f

   fix(shader): read the ancillary render target index as 0

   ...

   3334e41
5. oneandonlydean

   closed this as completedin #4842on Oct 13, 2026
6. RINNECODER

   added a commit that references this issue on Oct 13, 2026

   Upstream sync 6: shader recompiler to upstream/main (slice B) (#448)

   ...

   8c5770c