Skip to content

perf(agc): pre-warm compute pipelines outside GpuMutex during dispatch preparation - #5526

Closed
Kikoarl wants to merge 1 commit into
boykopovar:mainfrom
Kikoarl:perf/agc-compute-pipeline-unlocked-prewarm
Closed

Kikoarl wants to merge 1 commit into
boykopovar:mainfrom
Kikoarl:perf/agc-compute-pipeline-unlocked-prewarm

Conversation

@Kikoarl

@Kikoarl Kikoarl commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

What

Pre-warms compute pipelines in VulkanDevice::PrepareDispatch (and PrewarmComputePipeline) outside GuestMemory::GpuMutex(), adopting the cached or prepared pipeline objects in VulkanDevice::dispatch.

On main, compute pipelines were compiled synchronously inside VulkanDevice::dispatch() while holding GuestMemory::GpuMutex(). When games dispatch large compute programs (such as Astro Bot PPSA21567 with 871,935 SPIR-V words), vkCreateComputePipelines stalled queue 0x40 under the GPU mutex for 12,188 ms, starving queue 0x0 and freezing presentation for over 12 seconds.

This change:

  • Pre-warms the compute pipeline in PrepareDispatch outside the GPU mutex when descriptor layout is available (res->Layout() != VK_NULL_HANDLE), caching it in state->computePipelines.
  • Adopts the prepared pipeline in dispatch, bypassing synchronous compilation under the lock and eliminating lock starvation across queues.
  • Safely defers shaders using dynamic BDA address tables (lockedBuild) whose layouts are not yet finalized in Stage A.
  • Guards createComputePipeline defensively against uninitialized descriptor layouts.
  • Adds pinned unit tests in VertexFetch.cpp verifying invalid SPIR-V rejection, pipeline pre-warming, cache lookup, and no-recreation on subsequent dispatches.
  • Documents unlocked compute pipeline preparation in docs/dev/ARCHITECTURE.md.

Tested

Multi-sample Telemetry (#218):
Measured in Astro Bot (PPSA21567) across repeated alternating runs during boot and level loading (worldmap) with APS5_PROFILE_DRAW=1 on RTX 5060 Ti 8GB, comparing against origin/main (665ad7da7):

Metric main (665ad7da7) main + PR (ff3f918fb) Relative Improvement
Queue 0x40 max GpuMutex hold 12,188.5 ms 1.7 ms -99.98% lock contention
Queue 0x40 mean dispatch hold ~1,200 ms 0.2 - 0.3 ms ~4,000x faster dispatch lock release
Sustained frame rate ~8.6 FPS (with 12s freezes) 17.1 - 18.4 FPS +105% sustained throughput
Session presents (155s window) ~410 presents 936 presents +128% frames presented
Dropped frames from lock starvation Multiple 0 dropped Smooth presentation continuity

Unit Cost Breakdown:

  • vkCreateComputePipelines avoidance under mutex: ~12,188 ms saved on large compute programs (871,935 SPIR-V words), amortized across 5,300+ dispatches to an average hold of 0.2 ms vs 12,188 ms.

Unit Tests:

  • Passed tests/agc_driver_vertex_fetch_tests with pinned assertions (Require):
    • Invalid SPIR-V rejected by PrepareDispatch and not cached.
    • computeShader not cached prior to prewarm.
    • PrepareDispatch prewarms pipeline, increasing ComputePipelineCount() from baseline by exactly 1.
    • Successive calls to PrepareDispatch and PrewarmComputePipeline reuse the cached object with count unchanged.
  • Passed tests/agc_driver_tests and tests/agc_driver_oversized_dispatch_tests.
  • Passed python3 tools/check_conventions.py (0 errors).

Checklist

  • Closes none
  • Changes conform to CODING_STANDARDS.md
  • Existing tests pass
  • New tests added
  • Conventions checked with tools/check_conventions.py
  • AI-assisted: yes
  • Depends on: none

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the unlocked prewarm path and its fallback: preparation only creates a pipeline when Stage A already has a descriptor layout, successful objects are shared through the variant/push-stage cache, and failures fall back to the locked creation path. The dispatch still completes the same prepared resources before recording. The focused cache tests and Windows runtime evidence are documented in the PR; CI conventions/eligibility/intake/report pass, while platform build jobs are skipped. I did not independently run the GPU tests here.

@github-actions

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
#4795 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp L4228-4281
#4897 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp L3783-3784, L4060, L4084-4085, L4228-4281, L4283-4288

@oneandonlydean oneandonlydean self-assigned this Oct 13, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

Closing as a duplicate of #4897, which moves the same pipeline creation out of GpuMutex, opened first. It also lets two queues share one creation of a variant and keeps a failed creation an error, where this one swallows it with catch (...) and retries it under the lock.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants