Repository navigation
perf(agc): stage raw buffer elements a dispatch sweeps - #6486
Open
karanvora2599 wants to merge 1 commit into
Open
karanvora2599 wants to merge 1 commit into
karanvora2599 wants to merge 1 commit into
Conversation
|
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
Open
11 tasks done
Contributor
|
Rebase handoff — rebased onto current Verified on Linux (CachyOS, kernel 7.2.9, GCC 16.2.1, Ninja, Release,
To pick it up: Head is |
github-actions
Bot
added
needs rebase
Since 2688b76 a read and written V# element is staged in device memory up to 16 MiB when the direct dispatch's threads times the V# stride cover at least half of it. A raw buffer has stride 0, so it never passed and kept the 2 MiB written window: GL storage buffers compiled by Mesa stay bound in place in the host import however hard a dispatch sweeps them. The PS5 OpenGL Showcase's particle update (262,144 threads, each loading and storing its own 32-byte record of an 8 MiB SSBO) took about 53 ms of GPU per frame on an RTX 4060 bound in place; staged it takes about 0.1 ms plus 2.2 ms of bulk copies. The recompiler already records the furthest byte each buffer resource's accesses reach past their base (maxByteExtent). The binding plan now carries it per guest buffer element (DescriptorBinding::bufferExtent, shader disk cache format 37), and a raw element uses it as its record size in the sweep test. Strided elements are unchanged.
karanvora2599
force-pushed
the
perf/agc-stage-swept-raw-buffers
branch
from
October 18, 2026 00:57
486809d to
bc15be9
Compare
Contributor
Author
|
Rebased onto
The traces for both the original table and this one are attached below. |
Contributor
Author
|
Traces for the Showcase runs (PPSA99005, RTX 4060 Laptop, driver 591.86, The table in the description (10cb276d1, this change on main 92cdc7b):
The rebased head bc15be9 against main 51cee90 (my comment above):
Each file's summary line is in the description or in that comment, in the same order. |
Contributor
Author
|
A data point for the case where #6202 lands first, since both change the PS5 OpenGL Showcase (PPSA99005). Same laptop and scene as the description: RTX 4060 Laptop, driver 591.86, With #6202 in, this PR still takes the frame from about 97 ms to 46 ms, and the GPU time per present from about 84 ms to 34 ms. The title's own counter agrees: about 5.0 fps on With both, Thanks @MiguelAdolfsen for #6202. |
11 tasks done
12 tasks done
github-actions
Bot
added
needs rebase
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A compute dispatch that reads and writes most of a raw (stride 0) guest buffer now gets the device-local staging that @oneandonlydean's rule from 2688b76 already gives strided buffers, up to 16 MiB. This extends their rule; it doesn't replace it.
Title. PPSA99005, the PS5 OpenGL Showcase (Game bring-up #6289). @MiguelAdolfsen's #6189 reports its slow scene, and their #6202 addresses the storage round trip described there. This PR addresses the other large cost in the same frame: the particle dispatch.
Background. Since 2688b76, a written V# element that the shader also reads (
bufferRead) is staged in device memory up to 16 MiB when the direct dispatch's threads × V# stride cover at least half of it (ShaderResources::addGuestBuffer). A raw buffer has stride 0, so it never passes the test and keeps the 2 MiB written window. GL and Vulkan-style storage buffers compiled by Mesa are raw buffers, so they stay bound in place in the host import, however hard the dispatch sweeps them.The PS5 OpenGL Showcase (PPSA99005) is such a case.
particle_csruns 262,144 threads over an 8 MiB SSBO; each thread loads and stores its own 32-byte record. Bound in place, that dispatch takes about 53 ms of GPU per frame on an RTX 4060 Laptop. Staged, it takes about 0.1 ms plus 2.2 ms of bulk copies. WithAPS5_NO_HOST_IMPORT=1it still takes about 60 ms, so the cost is the shader's own loads and stores to host memory, not the import.How.
BufferResource::maxByteExtent: immediate offset plus access width).DescriptorBindingBuildernow copies it into a newDescriptorBinding::bufferExtent, one entry per guest buffer element, likebufferRead.addGuestBufferuses it as the record size when the V# stride is 0: swept meansread && dispatchThreads != 0 && recordBytes != 0 && dispatchThreads × recordBytes × 2 ≥ size. Strided elements are unchanged.bufferRead(format 24). Users' shader caches are rebuilt once after the update. The binding plan is what the driver reads at bind time; the IR resource list it comes from is not kept with it.Why results don't change. Staging is the existing path: copy in before the dispatch, bind the device-local buffer in the import's place, copy the whole element back after it. Only the decision to take it changes, for raw elements a dispatch reads, writes and covers.
Where it matters. Raw storage buffers a dispatch sweeps, between 2 and 16 MiB. A kernel that touches few records of a large raw pool stays in place, as the rule intends: 1,024 threads × 32 bytes against 4 MiB is not swept. A raw element whose accesses carry a large immediate offset (up to 4,095 bytes) overstates its record size; the worst case is a needless copy of at most 16 MiB in and out.
Tested
Windows 11, MinGW-w64 GCC 15.2.0, Release.
main's newguestBufferByteshelper next toaddGuestBuffer, and both are kept. Disk-cache format 37 is still free there. Rebuilt on that base:agc_read_written_stagingandagc_shader_disk_cachepass on all three devices;main51cee90 (warm-up of each, then main / PR / PR / main) repeats the result. GPU busy per present goes from 121.6 / 119.2 ms to 71.7 / 71.6 ms, frame p50 from 199 / 198 ms to 150 / 152 ms, and presents per 10 s from 51 / 51 to 67 / 66. These four traces are in the trace comment below.PS5 OpenGL Showcase (PPSA99005, v1.0.2), RTX 4060 Laptop (Ada), driver 591.86, Windows 11. Measured in the default scene with
APS5_TRACE_SECONDS=15:10andAPS5_SPIRV_OPT=none, alternating main / this PR / this PR / main, after a warm-up run of each. Summary lines (frame_ms,gpu_busy_ms_per_present) are below; the four traces are in the trace comment below, with the rebased runs.With
APS5_PROFILE_GPU=1 APS5_TRACE_STAGING=1:particle_cs) takes about 53 ms per dispatch. Only the 384 KiB and 40-byte elements are staged.The rest of the frame is the storage write-backs #6202 addresses, and draws.
Summary lines (main ab1, PR ab1, PR ab2, main ab2) and [gputime] excerpts
APS5_PROFILE_GPU=1, main, then this PR (by-program part):On main,
APS5_NO_HOST_IMPORT=1leaves the dispatch at about 60 ms, so the cost is the shader's access to host memory, not the import.agc_read_written_staging(new ctest entry runningreadWrittenStagingTests, which only ran in the no-argument mode ofagc_driver_recorder_tests, so CI never ran it). New raw cases:It passes on the RTX 4060, the Intel UHD (run with
VK_LOADER_DRIVERS_DISABLE=*nv-vk*, since this test picks the discrete GPU) and lavapipe.Negative check: with the sweep test back to
descriptor.Stride()alone, as on main, it fails witha dispatch sweeping a read and written 4 MiB raw element bound it in place.agc_shader_disk_cacheround-tripsbufferExtent.Every
agc_test (600), one at a time, at this PR:agc_driver_store_over_untrackedfails (the first result was not stored). It only stores, so this change doesn't reach it. fix(recompiler): defer the TYPE check of a guarded read-only static V# #6412 reports the same failure on main, starting at af4e074; on my machine it passes with a build from before af4e074.VK_LOADER_DRIVERS_DISABLE=*nv-vk*, so the recorder tests run on it too): passes or skips except the following, none of which goes through buffer staging. I haven't compared these two against main on this device:agc_driver_ds_swizzle, and the image-table tests (bindless_images,image_sint8,image_table,image_table_cache_bounds,image_table_reuse): these also fail on main e858dbd on this device;agc_import_watch, at the Windows import probe's settle queue, andagc_sample_dumps, a segfault;agc_driver_store_over_untracked.agc_import_watchandstore_over_untracked, plus segfaults indepth_fast_clear_submit,depth_storage_dispatch,sample_mask_depth_stencil,stencil_ref_export_unused,stencil_replayandstencil_replay_no_gpl. The stencil ones pass when run alone; the others are the known intermittent lavapipe crashes.The no-argument run of
agc_driver_recorder_testsstops atkeyProofTests((9) a scan under a pending write was kept), also in a build of main 760ffa6, before reachingreadWrittenStagingTests. That's why the new entry runs the staging test on its own.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)DescriptorBindingfield, extendsaddGuestBufferand takes disk-cache format 37), as perf(agc): journal the pipelines of a run and build them ahead on the next one #5875, feat(tools): add an offline shader archive tool that publishes prebuilt packs #6032 and feat(recompiler): add an uncached compile entry point and entry key helpers for offline tools #6296 do for the format line. Whichever lands second rebases and takes the next format number.