Repository navigation
Conversation
CompileShaderUncached compiles one request without the process-lifetime source cache or a resource snapshot, so batch tools can compile shader archives with bounded worker lifetimes. Optional frontend and backend timings report where each compilation spent its time.
Read-only packs of encoded cache entries are mapped once at startup from the shader cache prebuilt directory (plus ANYPS5_PREBUILT_SHADER_CACHE, opt out with ANYPS5_NO_PREBUILT_SHADERS). Disk loads try packs and loose prebuilt folders first and fall through to the writable cache and the compiler on a miss. Keys, source version, format and payload checksums are validated exactly as for writable entries, and interval logs report prebuilt hits, misses, rejections and load timings. Depends on the uncached compile entry point only through the offline tool that produces these packs; the loader itself is independent.
…lt packs agc_shader_archive extracts native shader programs from IoStore containers, builds static AGC requests, compiles them through the production recompiler with bounded workers, validates the SPIR-V, can compile pipelines on a Vulkan device, and publishes the results as prebuilt packs for the runtime loader. Captured requests can be replayed offline with their actual targets, and probe reports exact pack hit coverage and load timings. Oodle Mermaid streams are decoded by a new decoder in Oodle.cpp whose bitstream layout follows the MIT-licensed oozextract 0.5.5 decoder; unsupported codecs and entropy forms throw and are recorded in technical debt. The target is excluded from the default build. Extraction of one measured title reproduces all 4,803 shader groups byte-for-byte. Replaying eleven captured requests publishes an eleven-entry pack, and probe then loads eleven of eleven from the pack with an empty writable cache. Depends on the uncached compile entry point and the prebuilt pack loader commits.
New Markdown outside docs/ is rejected by the conventions check, so the tool guide lives with the other developer documents.
github-actions
Bot
added
the
open PR limit
|
Thank you for your contribution! Authors with no pull requests merged into |
7 tasks done
11 tasks done
JJenkx
added a commit
to JJenkx/AnyPS5
that referenced
this pull request
Oct 14, 2026
…cache Integrate SaloQT's upstream PRs boykopovar#5792 and boykopovar#5793 against main 3242151. Preserve mapped writable entries and resource-plan caching, fix Linux tool linking and SPIR-V span access, and accept padded cache payload envelopes. Add pack corruption, lifetime, cross-process and alignment regressions. Co-authored-by: SaloQT <123214229+SaloQT@users.noreply.github.com>
JJenkx
added a commit
to JJenkx/AnyPS5
that referenced
this pull request
Oct 14, 2026
…cache Integrate SaloQT's upstream PRs boykopovar#5792 and boykopovar#5793 against main 3242151. Preserve mapped writable entries and resource-plan caching, fix Linux tool linking and SPIR-V span access, and accept padded cache payload envelopes. Add pack corruption, lifetime, cross-process and alignment regressions. Co-authored-by: SaloQT <123214229+SaloQT@users.noreply.github.com>
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
An offline tool that extracts shader programs from a game's archives, recompiles them ahead of time, and publishes the results as a prebuilt shader pack for the loader in #5792 (merged).
tools/shader-archive(new): reads Unreal IoStore v8.utoc/.ucaspairs, decodes the Oodle Mermaid quanta those archives use with its own clean-room decoder (no vendored Oodle), scans shader libraries, and builds shader variants fully offline through the existing recompiler. Output is published atomically as a*.packfile (the format feat(recompiler): load prebuilt shader packs before the writable cache #5792 loads) plus a manifest.docs/dev/TechnicalDebt.md; the tool fails on archives it cannot decode instead of guessing.Scope, honestly: extraction covers Unreal IoStore v8 only today; other engines' archives (Insomniac, RAGE) need their own readers. Keys are exact, so statically built variants only hit when the runtime request matches exactly — the verified end-to-end numbers are for replayed captured requests, and a real-game hit-rate measurement with a statically built pack is still open (also in TechnicalDebt). The tool also writes
pipeline-cache.bin, but nothing loads it at runtime yet; whether offline pipeline compiles remove driver compile cost is unmeasured, so this PR claims no pipeline-compile saving.Tested
Windows 11: rebuilt on current main;
shader_archive,agc_shader_disk_cacheandshader_cache_file_readpass. Full real-archive run: 4,803/4,803 IoStore groups decoded byte-identical to the licensed-SDK reference, 80,794 shader hashes matching. Offline build + publish + probe against captured requests: 10/10 exact-key hits. Not tested on Linux locally (no Linux box here); CI covers it.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)