Skip to content

fix(shader): keep a shader's selected image-table records within the image and pair limits - #7527

Merged
oneandonlydean merged 3 commits into
boykopovar:mainfrom
EagleBearerYS:fix/shader-selected-records-image-pair-limits
Oct 18, 2026
Merged

oneandonlydean merged 3 commits into
boykopovar:mainfrom
EagleBearerYS:fix/shader-selected-records-image-pair-limits

Conversation

@EagleBearerYS

@EagleBearerYS EagleBearerYS commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

What

A regression from our #6243. On main 1c523bd, Death Stranding 2: On the Beach (PPSA02015) gets five pixel shaders refused at ResolveShaderAbi, before the first frames of its pre-logo scene (TripyFox's A/B):

  • 0xc280a65500: "sampled image/sampler pair limit exceeded";
  • 0xc280a9e900, 0xc282a85000, 0xc282ab9f00 and 0xc282ad0b00: "image resource limit exceeded".

TripyFox's build logs a refused shader instead of aborting; plain main aborts the guest thread there. With #6243's merge (0e94c5d) reverted, a cold run went past that point and refused nothing. Their replay of the five registration requests gives, without #6243, 2 or 3 images (1 or 2 through image tables, 1 direct), 4-6 samplers and 5-7 pairs. With it, every one stops at 64 images and 64 pairs, all of them sampled only.

How it happens (main 9caecfd):

  1. perf(shader): select the record of a T# whose offset the CPU evaluates #6243 gives a table read whose key the CPU evaluates a table of its own, with only the selected record. TryMakeTableColumn (ResourceTracker.cpp) sets selected and interns a selection from that read's own offset, and TableColumn::operator== (ControlFlowInfo.hpp) compares both.
  2. InternSource dedupes sources by their column, so N reads of one heap at N such keys become N column sources, where main before perf(shader): select the record of a T# whose offset the CPU evaluates #6243 had one.
  3. Collect adds an image for each, and AssignTables a table. Past 64 images, AddImage refuses with "image resource limit exceeded"; past 64 image/sampler pairs, AddSampledPair refuses with "sampled image/sampler pair limit exceeded".

The change: #6243's split stays where it fits. Its own tests require two keys of one heap to plan as two one-record tables.

  • PlanTableColumns now estimates the images and sampled pairs the program would add, direct images and samplers included, keyed the way AddImage and AddSampledPair key them.
  • Only if either would pass its limit, it plans again with the heap that has the most selected columns unselected, so that heap's reads share one table as they did before perf(shader): select the record of a T# whose offset the CPU evaluates #6243. It repeats until both fit or no such heap is left.
  • The estimate counts images exactly and can only overcount pairs. So it errs toward sharing a table, never toward a refusal that main before perf(shader): select the record of a T# whose offset the CPU evaluates #6243 wouldn't have had.
  • AddImage's image key is factored out (ImageKey, SameImageKey), so the estimate and Collect key images the same way.
  • ARCHITECTURE.md and the shader debt entry say so.

What stays as it is:

  • Shaders within both limits plan exactly as now (see Tested).
  • The disk-cache format doesn't change. Only shaders the limits refused plan differently, and a refusal is never cached: ShaderDiskCache::StorePlan runs only after ResourceProgram returns, and the failure memo is per process.
  • A heap that shares one table again resolves its keys as before perf(shader): select the record of a T# whose offset the CPU evaluates #6243, so those shaders plan, at the cost they had then.

Related: #5424 (Death Stranding 2: On the Beach bring-up; the A/B, the replay and the refused shader list are TripyFox's).

Tested

  • GitHub's Linux (lavapipe, Mesa 25.2.8) and Windows runners (verify run):
    • On both, the four new tests fail 3 of 3 runs with only ResourceTracker.cpp as on main. With the fix, they pass 10 of 10, along with the other agc_shader_memory_tests ctests and agc_driver_image_table, which skips on Windows (no Vulkan device).
    • libc and libkernel pass on Windows.
    • On Linux, libc's precise_wait failed once in the suite run with the fix and passed with main's code. It's a timing test that doesn't link the recompiler, and an earlier run of the first two commits failed it on both sides.
  • Windows, the full CI build (fork Build): 0 of 1,478 tests fail, and the four new ones pass. The 563 skips are all ones check_skips expects on a runner with no Vulkan driver or display.
  • Linux, a run that mirrors CI's Linux job (a cloud machine, Ubuntu 24.04, GCC 13.3, Mesa lavapipe 25.2.8 with LLVM 20.1.2), on the fix and docs commits: 1,445 tests against 1,443 on main 9caecfd. The only differences are the fix's two tests, absent on main and passing here. The 5 failures on both are that machine's IPv6 tests. The third commit only adds two more agc_shader_memory_tests cases (see Repeats).
  • Same plans for every existing test: on that machine, a build with temporary instrumentation (not committed) logged a fingerprint of every plan (sources, table columns, images, samplers, pairs) and a hash of every emitted SPIR-V module. Over the full suite, all 423 test command lines that exist on both main and this branch plan and emit identically.
  • New tests in tests/ShaderMemory.cpp. The shader samples a heap of 128 distinct T#s, each read keyed from its own SRT word, and optionally T#s loaded straight from the SRT (direct images):
    • agc_shader_selected_records_image_limit:
      • 64 reads keep perf(shader): select the record of a T# whose offset the CPU evaluates #6243's split: 64 selected tables and 64 pairs.
      • 65 reads plan as one unselected table with 65 key domains, in which keys 2, 34 and 66 bind their T#s and keys 1 and 67 are outside.
      • Two heaps with 40 and 30 reads (70 images) plan as 31: the 40 share one table, and the 30 keep their selected tables.
    • agc_shader_selected_records_pair_limit: 32 reads with 2 samplers keep 32 tables and 64 pairs. 33 reads plan as one table and 2 pairs.
    • agc_shader_selected_records_direct_image_limit: 63 reads and 1 direct image (64 images) keep the split: 63 selected tables beside the direct image. With 2 direct images (65 images, over the limit only because of them), the 63 reads share one table: 3 images and 3 pairs, in which keys 2, 33 and 64 bind their T#s.
    • agc_shader_selected_records_direct_pair_limit: 31 reads and 1 direct image, each with 2 samplers (64 pairs), keep the split. A third sampler on the direct image (65 pairs) makes the reads share one table: 2 images and 5 pairs, in which keys 2, 17 and 32 bind.
  • Red: with only ResourceTracker.cpp as on main, the four tests fail, the image ones with "image resource limit exceeded" and the pair ones with "sampled image/sampler pair limit exceeded". The direct tests' cases within the limits pass there too, as boundary cases should; they fail at their case over the limit.
  • Repeats: on Linux, all 14 agc_shader_memory_tests ctests and agc_driver_image_table pass 10 of 10.
  • Not tested: Death Stranding 2: On the Beach itself. TripyFox offered to replay the five requests and do a cold run on this branch.

Checklist

…image and pair limits

Since boykopovar#6243, each read whose record offset the CPU evaluates gets its own
selected table column, so a shader that reads one heap at many such keys
plans one image per key. Past 64 keys that is refused with "image resource
limit exceeded", or with "sampled image/sampler pair limit exceeded" when
each read is sampled twice, where the shader planned within both limits
before.

After planning the table columns, the tracker now estimates the images
and sampled pairs that collecting the program would add, keyed as
AddImage and AddSampledPair key them, from the planned column sources
and the direct handles. If either is over its limit, it plans again with
the columns of one heap unselected, the heap with the most selected
columns first, until both fit or no heap is left. Those reads then share
one keyed column, as before boykopovar#6243: the union of their key domains and the
largest key. A shader within both limits plans exactly as before, from
one planning pass.

The estimate counts images exactly. A direct sampler is told apart by its
handle's words and whether the sample adjusts, before the canonicalization
collecting it applies, so the pair count can only be too high and errs
toward merging.
…ge limit

The selected-records limit tests only read through image tables, so
nothing showed that the estimate counts direct images and samplers too.
TableManyKeyCode can now also load T#s straight from the SRT and sample
them, with the same samplers as the table reads and optionally a third
sampler on the last one.

agc_shader_selected_records_direct_image_limit:
- 63 selected reads and 1 direct image, 64 images: the per-key split is
  kept (63 selected tables and the direct image).
- 63 selected reads and 2 direct images, 65 images: over the limit only
  because of the direct images. Heap A's reads merge into one unselected
  table beside the 2 direct images (3 images, 3 pairs), and keys 2, 33
  and 64 still bind their records.

agc_shader_selected_records_direct_pair_limit:
- 31 selected reads and 1 direct image, each with 2 samplers, 64 pairs:
  the split is kept.
- The same with a third sampler on the direct image, 65 pairs: over the
  pair limit only because of the direct sampler. The reads merge into
  one table (2 images, 5 pairs), and keys 2, 17 and 32 still bind.

With ResourceTracker.cpp as on main 9caecfd, the over-limit cases fail
with "image resource limit exceeded" and "sampled image/sampler pair
limit exceeded", and the cases within the limits pass.
@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 7381 core/shader/recompiler/Optimization/src/ResourceTracker.cpp L1593-1595
PR 7388 docs/dev/debt/shader.md L36

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1341 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
PR 1368 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 1395 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 2434 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 4650 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 4891 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5021 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/shader.md
PR 5371 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
docs/dev/debt/shader.md
PR 5426 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/shader.md
PR 5499 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5500 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
docs/dev/debt/shader.md
PR 5925 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/ARCHITECTURE.md
PR 5941 docs/dev/debt/shader.md
PR 6009 docs/dev/debt/shader.md
PR 6044 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/ARCHITECTURE.md
PR 6176 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6177 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
PR 6185 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6186 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6188 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6192 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6232 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/Optimization/src/ResourceTracker.cpp
PR 6235 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
docs/dev/debt/shader.md
PR 6259 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6280 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/shader/recompiler/Optimization/src/ResourceTracker.cpp
docs/dev/debt/shader.md
PR 6300 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6320 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6360 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6412 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
docs/dev/ARCHITECTURE.md
docs/dev/debt/shader.md
PR 6481 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6488 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/Optimization/src/ResourceTracker.cpp
docs/dev/debt/shader.md
PR 6490 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6499 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6501 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6503 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
docs/dev/ARCHITECTURE.md
PR 6535 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6645 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
PR 6741 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6747 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6763 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/ARCHITECTURE.md
PR 6767 docs/dev/debt/shader.md
PR 6788 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6808 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6868 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6905 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6931 docs/dev/ARCHITECTURE.md
PR 6937 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6977 docs/dev/debt/shader.md
PR 7002 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
PR 7085 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/shader/recompiler/Optimization/src/ResourceTracker.cpp
docs/dev/debt/shader.md
PR 7201 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7223 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/shader/recompiler/Optimization/src/ResourceTracker.cpp
docs/dev/debt/shader.md
PR 7332 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
docs/dev/debt/shader.md
PR 7426 docs/dev/debt/shader.md
PR 7464 core/libs/prx/libSceAgcDriver/CMakeLists.txt

@github-actions github-actions Bot added build shader core/shader docs agc libSceAgc* labels Oct 18, 2026
@oneandonlydean
oneandonlydean merged commit e3c4694 into boykopovar:main Oct 18, 2026
11 checks passed
adriadam10 pushed a commit that referenced this pull request Oct 19, 2026
#7527's selected-record limit test reads a record's descriptor words through the
table snapshot's shared words list; this branch keeps a snapshot's words per column,
so the test reads them from its column instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build docs shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants