Skip to content

fix(shader): load a BDA span in place from its own function - #5022

Merged
oneandonlydean merged 1 commit into
boykopovar:mainfrom
oneandonlydean:up4/bda-read-selection-nvidia
Oct 13, 2026
Merged

oneandonlydean merged 1 commit into
boykopovar:mainfrom
oneandonlydean:up4/bda-read-selection-nvidia

Conversation

@oneandonlydean

Copy link
Copy Markdown
Collaborator

What

read_bda_span chooses between a span's in-place dword loads and its per-dword byte fallback, which calls read_bda_dword_bytes. Both shapes of that choice so far hang a group of NVIDIA drivers (a dispatch never finishes and the device is lost), and #4975 shows that reverting #4759 only moves the hang:

shape hangs reports
one selection, the loads emitted in its arm (before #4759) 591.86, 590.48.01 #3291
two selections, loads under the condition and fallback under its negation (#4759) 595.97, 596.36, 610.62, 610.74, 610.57.04 #4975, #4759 comment, here

The loads now live in load_bda_span, a DontInline function that calls nothing, and read_bda_span is one selection again: one arm calls load_bda_span, the other is the fallback as before. Neither earlier shape is emitted: the selection is single, and no function holds both the in-place loads and the nested calls. The probe, the loads, the lookups, the fault records and the invocation stops are unchanged for every lane. A read site in main is unchanged, so its size is too.

This is the shape described in a comment on #3291 as passing on 591.86 (RTX 4060 Laptop) with no pipeline creation cost over main (1.95 s and 7.2 s for 256 and 512 read sites). That was another implementation of it, not this commit.

docs/dev/debt/shader.md: the BDA span reads entry now lists both hanging shapes with their drivers and says where the current one is measured.

Tested

Linux (CachyOS, kernel 7.2, GCC 16.2), RTX 4090, NVIDIA 610.57.04, Release, BUILD_TESTING=ON, built with -DANYPS5_ENABLE_SPIRV_TOOLS=ON and OFF. main was measured at c2565f0 and the branch is on cac4e96; nothing under core/shader or libSceAgcDriver differs between the two.

The 23 tests of #4759 plus agc_driver_scalar_wide_loads, run one at a time with ANYPS5_NO_SHADER_CACHE=1:

SPIRV-Tools on SPIRV-Tools off
main (1 run) 9 of 24: agc_driver_scalar_wide_loads and agc_driver_flat_apertures_0 to _13 hang 22 of 24: agc_driver_bda_span_reads and agc_driver_pixel_interlock hang
main with SpirvBdaRead.cpp from e51428b^1 (4 of the tests) 4 of 4 4 of 4
this branch (3 runs) 24 of 24, slowest 5.9 s (agc_driver_bda_device) 24 of 24, slowest 2.1 s
  • Each hang prints [gpu] recorded batch still running on the GPU after 5 s, logs one NVIDIA Xid 109 in the kernel log and ends in guest allocation ... is still pinned by GPU work or an import. No Xid with the branch.
  • A 3D title also loses the device on this driver, 0.2 to 0.4 s after a level starts (vkQueueSubmit recorder: Vulkan result -4, Xid 109), in a build without SPIRV-Tools. Bisected to e51428b: its parent doesn't, and main with only fix(shader): split a BDA span read's in-place loads from its fallback #4759 reverted ran 5 times without it. This branch was not run in that title.
  • So on this driver the optimizer decides which tests the fix(shader): split a BDA span read's in-place loads from its fallback #4759 shape hangs, and the two that fail without SPIRV-Tools are the two reported on Windows in NVIDIA: agc_driver_bda_span_reads and agc_driver_pixel_interlock lose the device since #4759 (RTX 4070 Laptop, driver 596.36) #4975.
  • The emitted module of agc_driver_scalar_wide_loads passes spirv-val --target-env vulkan1.3 in all three shapes; nothing invalid was found in the hanging one.
  • agc_shader_memory, "BDA read functions": a span read function must hold no load and call its load function once, and the load function must hold the four loads and no call. On main it fails with the span load function does not hold the four in-place loads and nothing else. It needs no GPU, so it runs on lavapipe CI.
  • ctest -R "agc|shader" with SPIRV-Tools on: 476 of 476 pass, no Xid. On main the 15 hangs above are the failures.
  • python3 tools/check_conventions.py --base origin/main and git diff --check pass.

Not tested: drivers 591.86, 590.48.01, 595.97, 596.36, 610.62 and 610.74, any Windows build, AMD, Intel, and any title. A run on one driver of each group would settle it; the #4759 comment above offers one on 595.97 and a #4975 comment one on 610.74.

#4976 and #3655 change the same function, so whichever merges later needs a rebase.

Checklist

read_bda_span chooses between the span's in-place dword loads and its
per-dword byte fallback, which calls read_bda_dword_bytes. Each shape of
that choice so far hangs a group of NVIDIA drivers: a dispatch never
finishes and the device is lost.

As the two arms of one selection, with the loads emitted in the arm,
it hung drivers 591.86 and 590.48.01 when lanes took both arms (boykopovar#3291).
As two selections, the loads under the span's condition and the fallback
under its negation (boykopovar#4759), it hangs 595.97, 596.36, 610.62, 610.74 and
610.57.04 (boykopovar#4975).

The loads now live in load_bda_span, a DontInline function that calls
nothing, and read_bda_span is one selection again: one arm calls
load_bda_span, the other is the fallback as before. Probe, loads,
lookups, faults and stops are unchanged for every lane.

On an RTX 4090 (Linux, driver 610.57.04) the two-selection shape hangs
agc_driver_scalar_wide_loads and agc_driver_flat_apertures_0..13 in a
build with SPIRV-Tools, and agc_driver_bda_span_reads and
agc_driver_pixel_interlock in a build without. All pass with this
change in both builds. Drivers 591.86, 590.48.01, 595.97, 596.36,
610.62 and 610.74 are not tested; a shape with the loads in their own
DontInline function was reported passing on 591.86 in boykopovar#3291.

agc_shader_memory now requires that a span read function holds no load
and calls its load function once, and that the load function holds the
four loads and no call.

Refs boykopovar#4975
@kr4y4

kr4y4 commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Driver 595.97, which the commit message lists as not tested, passes with this change, in a default build and in one with SPIRV-Tools.

Setup: Windows 11 (26200), WinLibs GCC 15.2.0 r7, Release, CI configure flags, an RTX 4070 Laptop GPU (NVIDIA 595.97) and an Intel Raptor Lake-S iGPU (32.0.101.6790), main 0f2683c. This is the machine from my report on #4759, where agc_driver_bda_span_reads and agc_driver_pixel_interlock lost the device on main ea2bbef.

  • 25 tests, run with ANYPS5_NO_SHADER_CACHE=1, --repeat until-fail:3 and --parallel 4: agc_shader_memory, agc_driver_scalar_wide_loads, agc_driver_bda_device, agc_driver_bda_span_reads, agc_driver_pixel_interlock, agc_driver_flat_apertures_0 to 13, and the six flat_loads and global_loads tests.
    • Default build: 25 of 25 on both GPUs. The slowest test took 2.8 s on NVIDIA and 3.5 s on Intel.
    • With -DANYPS5_ENABLE_SPIRV_TOOLS=ON: 25 of 25 on both GPUs. The slowest took 7.3 s on NVIDIA and 27.0 s on Intel.
  • Full suite (ctest --timeout 120 --parallel 4), default build: 1069 of 1069 on both GPUs.
  • Windows logged no nvlddmkm events during these runs.

Written with AI assistance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants