feat(recompiler): FLAT addresses in the shared and private apertures reach LDS and scratch

[MichonGoddijn231849]

What

A compute FLAT access whose address has the console's shared aperture in its high dword (0x8000xxxx) reads or writes the workgroup's LDS at its low dword. One in the private aperture (0x7000xxxx) reaches the lane's scratch. Other addresses stay guest memory through BDA, as before. This holds for every FLAT access kind in compute:

• dword loads and stores; sub-dword stores and unaligned dword accesses go byte by byte;
• 32- and 64-bit FLAT atomics route to LDS (64-bit ones under the workgroup lock), to scratch (a read-modify-write, since SPIR-V has no atomics on Function storage) or to BDA; an unaligned atomic records a BDA fault.

GTA V's ray traversal keeps its stack in LDS and spills it to scratch through such addresses. Treated as guest memory, the pushes were lost and the traversal never ended (device lost).

tests/execution/FlatApertures.cpp runs a table of 121 rows: dword loads and stores, sub-dword stores (aligned and crossing a dword), 32- and 64-bit atomics on shared, private and guest addresses, a wave whose lanes mix LDS, scratch and guest addresses, and wave64.

Source of the aperture bases: the values GTA V's traversal addresses use (shared 0x8000, private 0x7000 in the high 16 bits). No public source confirms them for gfx10 or the console; they, the unaligned aperture semantics and the out-of-range LDS/scratch behaviour are listed as inferred in TechnicalDebt.md, with the scalar operands 235-238 (src_shared_base, src_private_base, ...) that stay unmodelled.

Tested

Windows 11 Pro, AMD Ryzen 5 7600, NVIDIA GeForce RTX 5080 (driver 32.0.16.1074), 32 GB, MinGW-w64 GCC 15.2.0.

• GTA V PPSA04264 (01.005) ran with the first version of this change (dword loads and stores) on the combined gta-v/testing branch (Performance mode, the prologue); not re-run with the routed atomics and byte-wise accesses yet.
• On this branch: builds; agc_driver_flat_apertures passes (83 of its 121 rows fail on the previous version), and the FLAT, global, LDS, BDA, atomic, scratch and recompiler tests pass. Full ctest: all pass except the relinker's Python tests (my local Python does not find their helper modules) and guest_gameupdate.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: feat(agc): compute dispatches with scratch memory #1182 (scratch memory for the private aperture)
• AI-assisted: yes. Written with Claude (Anthropic) under my direction; I reviewed and tested it.

🤖 Generated with Claude Code

[Celegans12]

Yōtei's ray-traversal kernels (see my note on #1182) are a real-world case for this one: they push the traversal stack with flat_store_dword to generic addresses whose high dword is 0x80000000 (LDS) or 0x70000000 (scratch), chosen per lane, and pop it with ds_read / scratch_load.

Replaying the seven programs on #1182 + #1190, the two big kernels recompile (their next stop is LDS over the device limit), and the results match what I got with my own version. Two gaps showed up:

• 0x8000356b00 still stops at unsupported scalar source operand code 235. It reads src_shared_base / src_private_base (235/237) as 64-bit values to build its aperture pointers, so those two need decoding too.
• Sub-dword flat stores into an aperture go through the BDA path. A variant of my test that pushes bytes to an LDS address faults with BDA access failed: address=0x8000000000000401, while the dword stores land in LDS as expected. Byte and short loads/stores into both apertures would need the same routing as the dword ones.

Tested on an RTX 3080 (Windows 11) and on lavapipe. I can share the execution test, which covers per-lane LDS/scratch/global mixes, x2/x4 and byte/short accesses, if you'd like it for this PR.

[adriadam10]

Thanks, the approach (routing on the high dword in the compute FLAT path) looks reasonable. Things to fix before this can land:

• Rebase: the branch carries a2fb99fb7 (the #1182 commit), which is already on main as 20e055c24; it now conflicts in tests/Graphics.cpp. Please rebase so only the aperture commit remains (the serialization version should then be the next free version after what has landed).
• Exact or throw: only 32-bit loads and stores are routed. Sub-dword stores (SpirvMemoryInstructions.cpp:852, "still stored through BDA") and flat atomics (EmitBdaAtomic path, ~line 1224) whose address is in the shared/private aperture go to BDA with a bogus address and silently do the wrong thing. Either route them too or throw when the aperture can't be excluded; a plausible wrong result is worse than the previous failure.
• Tests: FlatApertures.cpp covers only dword load/store. Please add rows for the paths above (route or throw), a lane-varying case mixing LDS/scratch/global addresses in one wave, and a wave64 run.
• Source: the 0x8000/0x7000 bases are described as "as seen in GTA V's traversal addresses", not verified. Please cite where they come from (SH_MEM_BASES values, or a reference); we can't measure the console's apertures on our Linux oracle.
• TechnicalDebt: list what stays unmodelled (e.g. src_shared_base/src_private_base, operand 235, reported by @Celegans12 on #1182) under "Unknown function info"/the matching section.

[MichonGoddijn231849]

Thanks. Rebased onto main, so only the aperture commit remains (the #1182 commit is gone). This push:

• Serialization: unchanged. Without the duplicate scratch commit nothing new is serialized, so version 12 stays and the PR no longer touches it.
• Exact or throw: sub-dword stores and unaligned dword accesses to an aperture now go byte-wise to LDS or scratch, and 32- and 64-bit FLAT atomics route to LDS (64-bit ones under the workgroup lock), scratch (read-modify-write, since SPIR-V has no atomics on Function storage) or BDA. I tried throwing for 64-bit FLAT atomics first, but that broke the existing agc_driver_global_atomics_float (a 64-bit float max on a global address), so they route too.
• Tests: FlatApertures.cpp is now a 121-row table: dword, sub-dword (aligned and crossing) and unaligned accesses, 32- and 64-bit atomics on shared, private and guest addresses, a wave mixing LDS, scratch and guest addresses, and wave64. On the previous version 83 rows fail; guest rows pass on both. Byte readbacks also caught an address truncation the old byte path had.
• Source: I found no public source that confirms 0x8000/0x7000 for gfx10 or the console (the amdkfd layouts I could read differ), so the description now says where they come from and TechnicalDebt.md lists them as inferred.
• TechnicalDebt: operands 235-238 (src_shared_base, src_private_base, ...) stay unmodelled and throw at decode; listed together with the inferred unaligned and out-of-range aperture behaviour, a few lines up from the end of the section.

[Celegans12]

I ran Ghost of Yōtei (PPSA26344 v1.008) for an hour with this PR at bb70ded, the version before your last push, and the two big ray-traversal kernels run now. 0x8000173b00 and 0x800014e300 push their traversal stacks through your aperture path and ran on queue 0x51, about 47 dispatches and 2.0 s a frame, with no device fault, no BDA fault and no GPU hang. The SCRATCH_EN skips from the run before (13.9 a frame, now covered by #1182) are gone too.

The LDS half is now #1776, which doesn't depend on this one. At bb70ded your shared-aperture arm went through the same LDS element pointer, so it followed the device-memory LDS without changes; I haven't checked your new byte-wise and atomic routes against it yet. The cost has moved to pipeline creation: each data variant of these kernels is about 1.06M SPIR-V words and took 215-243 s to create, which has nothing to do with your change.

The three small kernels (0x8000356b00 and the two that stop at a runtime formatted V#) didn't come up in this run, so I couldn't try the operand 235-238 handling in the game. I'll put your current head f36e64c into the next run and report back.

Details

Windows 11, RTX 3080 Laptop, our integration tree on main 4b005b2 (which has #1182) + this PR at bb70ded + #1776 (as 1a4c9b07, before its rebase onto #639) + our other open PRs; a 1,200 s capture pass, then 3,600 s.

Kernel	LDS	Pipeline creations in the run	Each
0x8000173b00	60.5 KiB	2	235.7 s and 242.6 s
0x800014e300	63.5 KiB	4	214.8-234.3 s

[dispatch] pipeline took 235667 ms (1057733 words, program 0x8000173b00)
[dispatch] pipeline took 242623 ms (1057733 words, program 0x8000173b00)
[dispatch] pipeline took 234326 ms (1058445 words, program 0x800014e300)

All nine watchdog "stuck" lines in the run were pipeline compiles and cleared; 0x8000117800 and 0x8000167500 did not run.

[MichonGoddijn231849]

@Celegans12 thanks for the hour-long run. Good to see the traversal stacks of both kernels going through the apertures on Yōtei too.

[CouchRiv]

@MichonGoddijn231849 the new agc_driver_flat_apertures fails on current main (CachyOS Linux; RTX 4070 Ti, Radeon 610M RADV and lavapipe 26.2.4 alike), tested at f36e64c rebased on a8b032b:

AGC graphics: flat_store8 shared offset 0 wave32: lane 0 read 0xdeadbeef back, expected 0x000000be

Since #639 the buffer path follows OOB_SELECT in V# dword3. The test's 0x01016fac is mode 0 (structured), and with stride 0 every table load and output store is out of bounds, so the lane addresses read as 0 and the first flat store faults at BDA address 0. fa2df2e moved the memory and wave execution tests to 0x31016fac; the same one-line change in BufferDescriptor (FlatApertures.cpp:99) makes it pass on all three drivers. The routing change itself needed nothing. CI did not see this because it ran on the PR's old base, before #639.

It is still too slow for CI: 21 s on NVIDIA, 14 s on RADV, about 300 s on lavapipe (the CI run timed out at 120 s). Nearly all of it is lavapipe compiling each 64-bit atomic kernel once, 5-38 s each (16 kernels, about 210-250 s in total). The shared/private/guest rows reuse the kernel and cost about 0.2 s each. Fewer distinct 64-bit ops, or splitting the file (each tests/execution/*.cpp is its own ctest test), should get it under the 30 s in CONTRIBUTING.md.

[danzaio]

Ran the new test on real hardware since CI Windows skips it: RTX 4050 laptop, Windows, MinGW 15.2 build of f36e64c, ANYPS5_GPU=4050.

First run after the build timed out at 60s (same picture as the Linux job: alive past device setup, then silence after the write-watch line, no result rows). Every run after that passes in about a second, five in a row, plus one 14s pass with APS5_TRACE_DISPATCH_IO on.

So I could not reproduce a logic hang. This looks like cold Vulkan pipeline compilation on the first run rather than an infinite loop in the routing: a fresh CI runner pays the same cost, and llvmpipe pays it slower, which fits the 120s Linux timeout. If it keeps biting CI, a longer timeout for this test or a shared pipeline cache would tell the two apart for good.

[MichonGoddijn231849]

@danzaio thanks, that fits a cold pipeline compile. The Linux job's 120 s timeout on lavapipe is probably the same thing.

@CouchRiv your report is a different failure, and I had missed it: flat_store8 shared offset 0 wave32 reads 0xdeadbeef back. That is a wrong value, not a timeout, and on three drivers. So the 8-bit FLAT store into the shared aperture does not reach LDS at all on those drivers. I will reproduce it on lavapipe tomorrow and fix it. While doing that I'll also do the rebase @adriadam10 asked for: only the aperture commit, on the next free serialization version.

[frankea]

Fourth driver for @CouchRiv's failure. Head f36e64cc merged onto current main c72d5ce1, Windows 11 10.0.26200, RTX 4090 (NVIDIA 616.92), MinGW-w64 GCC 15.2.0 r7, Release:

[gpu] deferred write-back failed: BDA access failed: address=0x0 instruction=0x18 bytes=1 stage=4 reason=1
AGC graphics: flat_store8 shared offset 0 wave32: lane 0 read 0xdeadbeef back, expected 0x000000be

It fails in 0.25 s, so this is not the cold pipeline compile @danzaio saw; the BDA fault at address 0 fits the OOB_SELECT reading of the V# (0x01016fac) after #639. The rest of the suite in that build is 573 of 573 (3 environment skips). I can rerun the test with 0x31016fac in BufferDescriptor on this GPU once the next push is up.

Tested with AI assistance (Claude Code).

[danzaio]

Verified the fix commit on real hardware: RTX 4050 laptop, Windows, MinGW 15.2 build of f36e64c, ANYPS5_GPU=4050. agc_driver_flat_apertures passes in about 1.5s, all 121 rows including the shared/private aperture routing and the sub-dword store cases from the report. Nothing else in the run looks off. AI-assisted: yes.

[MichonGoddijn231849]

Rebased onto main (af1a8ed), head 5122e721. @frankea was right about the cause. Thanks, and thanks @CouchRiv and @danzaio for the runs.

• The reported failure: the test's V# word 3 was 0x01016fac, OOB_SELECT 0. After perf(agc): baseline shader recompilation on registration #639 that made its output buffer read as out of range, which produced the BDA fault at address 0 and the 0xdeadbeef read-back. It is now 0x31016fac (OOB_SELECT 3, raw), as in main's other execution tests.
• A second failure the rebase exposed: flat_atomic64_swap shared returned 0xdeadbeef. Main's StoreAddressU32x2..x4 (b958425, one BDA lookup per wide store) bypassed the aperture routing, so the flat_store_dwordx2 that sets up LDS went to BDA at 0x8000000000000000. Wide FLAT stores now store each component through the aperture, like the 32-bit store and the wide load. Outside the apertures they keep the single BDA lookup.

Tested on Windows, RTX 5080:

• agc_driver_flat_apertures passes all rows.
• All 21 other agc_driver_flat_*, agc_driver_global_*, agc_driver_bda_* and agc_driver_masked_global_store tests and agc_shader_memory pass.
• check_conventions.py passes.

@adriadam10 the branch holds only the two aperture commits now, and it no longer changes the request serialization version.

[adriadam10]

Thanks for the fixes and to everyone who ran it. I measured the routed semantics on real RDNA2 (gfx1035, HSA, LDS of 4096 bytes, Linux's unaligned mode) with flat accesses into the shared aperture. Unaligned in-range loads and stores (byte-wise, little-endian), lo >= 0x10000 reading 0 with no wrap, flat_atomic_add/_x2 and an out-of-range atomic returning 0 all match the PR. Two things don't:

• The aperture is a 4 GB window, not a 16-bit tag (SpirvMemoryInstructions.cpp:355-356). On gfx1035, src_shared_base/src_shared_limit read 0x00010000_00000000/0x00010000_ffffffff, and src_private_* read 0x00020000_00000000/0x00020000_ffffffff. That is the gfx9+ layout in Linux's kfd_flat_memory.c (MAKE_LDS_APP_BASE_V9() = 0x1 << 48, MAKE_LDS_APP_LIMIT(base) = base | 0xFFFFFFFF): the SH_MEM_BASES field becomes bits 63:48 and the window covers 4 GB from there. SplitAperture only compares the top 16 bits, so a high dword of 0x8000_0001 reaches LDS here while the hardware sends it to global memory. Comparing the whole high dword with 0x80000000 / 0x70000000 fixes it. That also makes the TechnicalDebt line at :330 inaccurate: there is a public source for the layout, and only the PS5's base values are inferred.

• An access that runs past the end of LDS is dropped as a whole, not byte by byte (:397-405, :442-447). If any byte is out of range, the hardware reads 0 or writes nothing for that dword access. Multi-dword accesses are handled per dword. The rows that differ (LDS pre-filled so byte a holds a & 0xff, store data 0xaabbccdd, 0x11223344):

  access	lo	hardware	PR
  flat_load_dword	0xffd	0x00000000	0x00fffefd
  flat_load_dword	0xfff	0x00000000	0x000000ff
  flat_load_ushort	0xfff	0x0000	0x00ff
  flat_store_dword	0xffe	dword 0xffc unchanged 0xfffefdfc	0xccddfdfc
  flat_store_short	0xfff	dword 0xffc unchanged 0xfffefdfc	0xddfefdfc
  flat_store_dwordx2	0xffa	first dword written, second dropped: dword 0xffc = 0xfffeaabb	0x3344aabb

  Checking that the element's last byte is in range before the byte loop (and skipping the whole element otherwise) would match these. Rows for them in FlatApertures.cpp would pin the behaviour.

I couldn't measure scratch reached through flat addresses or the console's alignment mode (SH_MEM_CONFIG). The TechnicalDebt line already lists both as inferred, which is fine.

[MichonGoddijn231849]

Thanks @adriadam10, both measurements are now in 4ab7fb7a.

• 4 GB windows: SplitAperture compares the whole high dword with 0x80000000 (shared) and 0x70000000 (private), so 0x8000_0001 goes to global memory. The TechnicalDebt line now names kfd_flat_memory.c and your gfx1035 values as the layout's source, and keeps only the PS5's two base values as inferred.
• Accesses past the end of LDS: a dword or short load or store is checked against the end of LDS by its last byte first. If that byte is out of range, the load reads 0 and the store writes nothing, for the whole access. Multi-dword accesses go per dword, so a dwordx2 at 0xffa writes its first dword and drops its second, as in your table.

New rows in FlatApertures.cpp, with the test's 1 KiB LDS:

• a dword whose last lane straddles the end, at offsets 773 and 774;
• a dword entirely past the end, at 776;
• a short straddling it, at 775;
• a byte at the last valid address, at 775, which still writes.

With the old byte-wise behaviour the dword row fails ("lane 31 read 0x00b5392f back from an access past the end of LDS"); with the change it passes. The other 21 flat/global/BDA tests, agc_shader_memory and check_conventions.py pass on Windows (RTX 5080).

[orishamir]

4ab7fb7a merges cleanly onto current main (4a747c48) but agc_driver_flat_apertures_tests does not build there: tests/execution/FlatApertures.cpp:81 calls GuestAllocations::Mutation().Add(block, GuestBytes, true, true), and since c11a9cb5 Add takes a fifth gpu argument (no matching function for call to ... Add(uint8_t*&, const std::size_t&, bool, bool)). #374 and #1619 made the same one-argument adaptation.

With that call changed to Add(block, GuestBytes, true, true, true) locally, on llvmpipe: agc_driver_flat_apertures and the other 21 flat/global/BDA tests pass (flat_apertures takes about 90 s on llvmpipe), and the new rows pin 4ab7fb7a: with SpirvMemoryInstructions.cpp at 5122e721, the test fails with "flat_store32 shared offset 773 wave32: lane 31 read 0x00b5392f back from an access past the end of LDS, expected 0 (the store is dropped whole)". SplitAperture compares the whole high dword against 0x80000000 / 0x70000000. The hardware side rests on your gfx1035 measurements; nothing here checks it.

AI-assisted: yes (Claude Code Fable 5.1 xhigh)

[adriadam10]

Thanks for the fixes and for everyone's runs. Rebased onto current main and merging:

• Rebased: the only conflict was docs/dev/TechnicalDebt.md (adjacent lines), both kept.
• Added a commit: FlatApertures.cpp:81 now passes the fifth gpu argument to GuestAllocations::Mutation().Add, as @orishamir reported; the test did not compile on main.
• Checked on RADV after the rebase: agc_driver_flat_apertures (121 rows) and the FLAT, global, BDA, LDS and scratch tests pass; the test fails with the recompiler changes reverted.
• The 4 GB windows and the drop-whole-access-past-LDS behaviour match the gfx1035 measurements from the earlier review.