Skip to content

fix(recompiler): bound cache keys before padded offset arithmetic - #6858

Open
JJenkx wants to merge 4 commits into
boykopovar:mainfrom
JJenkx:fix/cache-entry-header-lengths
Open

JJenkx wants to merge 4 commits into
boykopovar:mainfrom
JJenkx:fix/cache-entry-header-lengths

Conversation

@JJenkx

@JJenkx JJenkx commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

What

Follow-up to #6296: reject an encoded cache key length before converting it to size_t and calculating its eight-byte padded payload offset. That PR's shared parseEntryHeader currently accepts a 48-byte header declaring keyBytes = UINT64_MAX - 47 when the payload length matches the file length: the offset calculation wraps to zero. Its full-entry readers then receive an invalid key span. Current main has an earlier key/body check; this regression is in the proposed #6296 refactor, not a reproduced game crash on main.

Check the key against both the declared file body and the largest representable padded offset. Keep entry format, valid-entry behavior and shared validation unchanged. Reuse one alignment constant. The regression checks header-only validation, EntryKey and DecodeEntry, plus a header whose reported file length is UINT64_MAX, where checking only key/body fit would still let alignment wrap. No large allocation is needed.

Depends on #6296, exact head 721af439c96abf6d9e061a75cb2e99eb1e8498eb. Its three commits are carried with original authorship, rebased onto main 61d3d3080c372dd0b6adcf4dc13247108495d763, followed by one correction commit. After the dependency merges, retain only the correction. #6032 consumes these helpers; this does not include its archive tool.

Tested

Refresh at 36b5b8e1711aa4b86e5450d23660f5eeb5cb5e32 on main 61d3d3080c372dd0b6adcf4dc13247108495d763: retained main's cache format 38 alongside the dependency's 48-byte header constant. The three dependency commits retain their authorship; the correction is unchanged. Release build of full libs, agc_shader_disk_cache_tests and shader_cache_file_read_tests passes with bundled dependencies/SPIRV-Tools. Both focused tests pass, 2/2 in 2.19 s. Conventions and whitespace checks pass. The earlier baseline/negative-control results below were recorded before this refresh. No native-title or speed claim.

Arch Linux 7.2.9-arch1-1, GCC 16.2.1, Release, bundled dependencies and SPIRV-Tools enabled. Ryzen 9 7950X.

  • Main plus unchanged feat(recompiler): add an uncached compile entry point and entry key helpers for offline tools #6296: agc_shader_disk_cache and shader_cache_file_read pass, 2/2 in 2.07 s.
  • Add the regression without the correction: agc_shader_disk_cache fails immediately with an entry header with an overflowing key length is recognized.
  • Correction: full libs, agc_shader_disk_cache_tests and shader_cache_file_read_tests build; both focused tests pass, 2/2 in 2.08 s. Existing cache round trips, corrupt-file rejection, prebuilt-pack checks, uncached-compilation determinism and untouched disk-cache totals remain covered.
  • python3 tools/check_conventions.py --base origin/main and git diff --check pass.

No native game or GPU execution test, full CTest suite, macOS test or loading/FPS improvement is claimed. This is a cache-file validation correction.

Additional contributor validation: keravyn-ops reports agc_shader_disk_cache passing in 3.79 s at head 1d45d6ba0, on Windows 11 Home 10.0.26100, WinLibs GCC 15.2.0 r7, Release, Radeon 780M (RDNA 3), AMD driver 25.10.20.01. This is a CPU cache test on that machine; no title or GPU execution coverage is established.

Checklist

  • Based on current main; dependency is explicit
  • One topic per PR
  • No comments in code except technical debt
  • Unimplemented paths throw; no silent stubs added
  • No investigation reports, images or game assets added to the repository
  • New third-party code is a submodule built from source (none added)
  • Generic behaviour, not specific to one title
  • Depends on: feat(recompiler): add an uncached compile entry point and entry key helpers for offline tools #6296
  • Unblocks: none
  • GPUs tested: none; CPU cache serialization/file-read checks only, Arch Linux
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 6470 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 6924 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 6947 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 7395 core/shader/recompiler/ShaderDiskCache.hpp L16

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 3247 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 4072 core/shader/recompiler/Recompiler.cpp
PR 4650 core/shader/recompiler/Recompiler.cpp
PR 5021 core/shader/recompiler/Recompiler.cpp
PR 5500 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 5609 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Optimization/include/Optimization/ResourceProgram.hpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 5925 core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 6177 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
PR 6235 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
PR 6280 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 6412 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 6486 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 6488 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 6503 core/shader/recompiler/Recompiler.cpp
PR 6539 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Optimization/include/Optimization/ResourceProgram.hpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 6683 core/shader/recompiler/ShaderDiskCache.hpp
PR 6879 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 6941 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 6970 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 6977 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 7201 core/shader/recompiler/Recompiler.cpp
PR 7223 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 7332 core/shader/recompiler/Recompiler.cpp
PR 7381 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 7390 core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 7642 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/Optimization/include/Optimization/ResourceProgram.hpp
core/shader/recompiler/Recompiler.cpp
core/shader/recompiler/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.hpp

@keravyn-ops

Copy link
Copy Markdown

AMD / Windows run of this PR at 1d45d6b: Radeon 780M (RDNA 3, 0x15bf), AMD driver 25.10.20.01 (32.0.21020.1007), Windows 11 Home 10.0.26100, WinLibs GCC 15.2.0 r7, Release.

Test suite only, no title run. Every ctest entry built from the test targets this PR changes passes (1):

  • agc_shader_disk_cache (3.79 s): shader disk cache tests passed

AI-assisted: yes (runs made on the device above).

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 18, 2026
@JJenkx
JJenkx force-pushed the fix/cache-entry-header-lengths branch from 1d45d6b to 626226e Compare October 18, 2026 02:20
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 18, 2026
CompileShaderUncached compiles one request without the process-lifetime
source cache or a resource snapshot, so batch tools can compile shader
archives with bounded worker lifetimes. Optional frontend and backend
timings report where each compilation spent its time.
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 18, 2026
SaloQT and others added 3 commits October 18, 2026 14:28
EntryKey returns the validated key stored in an encoded cache entry
and IsCurrentEntryHeader checks that a file header belongs to this
compiler source and format, so offline tooling can group and screen
existing cache files without decoding them. Both validate against the
aligned payload layout.
EntryKey and IsCurrentEntryHeader round-trip an encoded entry and
reject truncated or damaged headers. CompileShaderUncached produces
the same encoded entry twice without touching the disk-cache totals.
@JJenkx
JJenkx force-pushed the fix/cache-entry-header-lengths branch from 626226e to 36b5b8e Compare October 18, 2026 18:31
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants