Skip to content

fix(agc): keep the Vulkan pipeline cache file with its shader cache version - #6771

Open
XairRrR wants to merge 2 commits into
boykopovar:mainfrom
XairRrR:fix/pipeline-cache-version
Open

XairRrR wants to merge 2 commits into
boykopovar:mainfrom
XairRrR:fix/pipeline-cache-version

Conversation

@XairRrR

@XairRrR XairRrR commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

What

The Vulkan pipeline cache file lived in the shader cache root, and every run merged its pipelines into it, so pipelines built from SPIR-V of earlier recompiler versions were never dropped. The file only grew: 965 MB for Uncharted: Legacy of Thieves Collection (PPSA05684, #4001) after a few recompiler changes, all of it loaded into host memory when the device is created (874 MiB of heap at startup).

The file now sits in the directory of the current shader cache source version (ShaderCacheDirectory()/<source version>/vk-pipelines-*.bin), next to the variants it was built from, so it is dropped with them when that version's directory ages out. The USAGE.md sentence on the pipeline cache says where the file is.

Related: #6958 (Pipodi) edits PipelineCache.cpp and the same Recorder.cpp test file (periodic save), different lines; whichever merges second may need a trivial rebase. #5875 (merged) journals a run's pipelines and already lists ShaderDiskCache::SourceVersion among the reviewed names; #6568 edits the same USAGE.md paragraph. Neither moves the pipeline cache file, which main still writes to the shader cache root.

Tested

  • Rebased onto aff725b after perf(agc): journal the pipelines of a run and build them ahead on the next one #5875 merged; main at that commit does not build here (PipelineJournal.cpp static assertions), so the tests below ran on the previous base 3f171d0 with the same change.
  • Linux (Arch, kernel 7.2.9), RTX 4070 Ti SUPER, driver 615.78.08.
  • agc_pipeline_cache and agc_pipeline_cache_file: the file is written inside the current source version's directory and no file is left in the root; other version directories are tolerated. Re-run with the review fix on main 29a546d, also agc_pipeline_journal and agc_driver_pipeline_journal. Negative controls: PipelineCache.cpp from main fails, a file under another version fails.
  • Uncharted: Legacy of Thieves Collection (PPSA05684, Uncharted: Legacy of Thieves Collection (PPSA05684) on Linux / NVIDIA #4001): after a recompiler change the run starts a new, empty pipeline cache file in the new version directory instead of loading the 965 MB root file.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: none
  • GPUs tested: NVIDIA RTX 4070 Ti SUPER, driver 615.78.08, Linux
  • AI-assisted: yes

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 16, 2026
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 7731 docs/user/USAGE.md L273

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1368 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 1395 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 5021 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
docs/user/USAGE.md
PR 5925 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
docs/user/USAGE.md
PR 6171 docs/user/USAGE.md
PR 6185 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6186 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6187 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6188 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6192 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6209 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6683 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6747 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6788 docs/user/USAGE.md
PR 6868 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6924 docs/user/USAGE.md
PR 6946 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
docs/user/USAGE.md
PR 6947 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
docs/user/USAGE.md
PR 6970 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6971 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6973 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6987 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6989 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6995 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 7002 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 7048 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 7332 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
docs/user/USAGE.md
PR 7417 docs/user/USAGE.md
PR 7427 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 7457 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 7505 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 7604 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 7622 core/libs/prx/libSceAgcDriver/Graphics/src/PipelineCache.cpp
PR 7642 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
docs/user/USAGE.md
PR 7685 docs/user/USAGE.md
PR 7800 docs/user/USAGE.md

@XairRrR
XairRrR force-pushed the fix/pipeline-cache-version branch from c98c999 to ac7f69f Compare October 16, 2026 11:33
@github-actions github-actions Bot added docs agc libSceAgc* and removed needs rebase Conflicts with main, rebase needed labels Oct 16, 2026

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The version-scoped cache path and existing Vulkan UUID/device checks look sound. One regression-test assumption looks brittle: pipelineCacheFileTests calls remove_all(directory) only after the lambda is exercised, then the new cacheFile helper requires the cache root to contain exactly one entry. A prior failed/interrupted run can leave an older version subdirectory in this unique temp directory, making the successful cache test fail even though the current version contains exactly one expected file. Could the helper select the current ShaderDiskCache::SourceVersion() subdirectory (or assert the current version directory and file while tolerating other versions)? That would keep the test focused on the new path without assuming cleanup from a previous test process.

@XairRrR

XairRrR commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX100 thanks for the review. Agreed, the helper should look up the current version's directory instead of assuming it is the only entry. Pushed 7c6fcf2, cacheFile now:

  • requires every entry of the cache root to be a directory (no vk-pipelines-*.bin left in the root, other version directories tolerated),
  • opens <root>/<source version as %016llx> (ShaderDiskCache::Generated::SourceVersion, the value SourceVersion() returns) and requires it to exist,
  • requires exactly one vk-pipelines-* file in that directory.

For the record, a previous process could not leak into this run before either: the test root is aps5-pipeline-cache-file-test-<random>, removed at the start of each loop iteration and by RemoveDirectory on exit. The new helper is still the better check, because it ties the file to the current source version instead of to whichever directory happens to be there.

Tested with the two commits on current main 29a546d, Linux, RTX 4070 Ti SUPER, driver 615.78.08: agc_pipeline_cache, agc_pipeline_cache_file, agc_pipeline_journal, agc_driver_pipeline_journal, agc_driver_compute_pipeline_outside_gpu_mutex pass. Negative controls on agc_pipeline_cache_file:

  • PipelineCache.cpp from main (file in the root): fails with "a pipeline cache file lives in the shader cache root".
  • file written under SourceVersion() + 1: fails with "the current shader cache version has no directory" (the previous helper accepted this).

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 17, 2026
@blawesom

Copy link
Copy Markdown
Contributor

Rebase handoff — rebased onto current main (authorship unchanged). One conflict in docs/user/USAGE.md: your pipeline-cache-with-its-shader-subdirectory wording merged with main's newer no-size-limit sentence, both kept in one paragraph.

Verified on Linux (CachyOS, kernel 7.2.9, GCC 16.2.1, Ninja, Release, BUILD_TESTING=ON, AMD RX 9070 XT / RADV):

To pick it up:

git fetch https://github.com/blawesom/AnyPS5 rebase/6771
git checkout <your branch>
git merge --ff-only FETCH_HEAD

Head is d07e5926f.

@XairRrR

XairRrR commented Oct 17, 2026

Copy link
Copy Markdown
Contributor Author

@blawesom thanks for the rebase. I rebased onto the newer main 864729d myself and resolved docs/user/USAGE.md the same way, keeping both sentences in one paragraph. git range-diff shows both commits identical to your 3857c5a12 and d07e5926f. Pushed as dc4d6d2; tools/check_conventions.py --base origin/main passes.

…ersion

The pipeline cache file lived in the shader cache root and every run
merged its pipelines into it, so pipelines built from SPIR-V of earlier
recompiler versions were never dropped. The file only grew (965 MB for
PPSA05684 after a few recompiler changes) and the driver loads all of
it into host memory at device creation.

The file now sits in the directory of the current shader cache source
version, next to the variants it was built from.

AI-assisted: yes
…ctory

The test helper took the only entry of the shader cache root as the
version directory. It now opens the directory named after the current
shader cache source version, requires exactly one pipeline cache file
there, and only requires the root to hold no files, so other version
directories do not fail it.

AI-assisted: yes
@XairRrR
XairRrR force-pushed the fix/pipeline-cache-version branch from dc4d6d2 to ca42107 Compare October 19, 2026 01:58
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 19, 2026
@adriadam10 adriadam10 self-assigned this Oct 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants