Repository navigation
perf(agc): copy draw inputs on NVIDIA's Windows driver instead of binding host imports in place - #6763
perf(agc): copy draw inputs on NVIDIA's Windows driver instead of binding host imports in place#6763SP4C3B4R-8 wants to merge 2 commits into
Conversation
|
Rebased onto |
643afa8 to
bf1bae5
Compare
|
Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
|
The A/B you asked for, on AMD: same scene, ASTRO BOT (PPSA21564 01.007.000) Host: Windows 11 Pro 26200, Ryzen 7 5800X3D, Radeon RX 9070 XT (AMD proprietary driver 32.0.23033.1002), 32 GB, WinLibs GCC 15.2.0 r7, Release. Build.
GPU time with
So on this AMD driver the GPU reads host imports of guest memory about as fast as driver-allocated buffers, unlike the 164 vs 45 ms you measured on NVIDIA. Copying only adds CPU work, and this scene is limited by the pipelined draw committer here, so the frame rate drops about 3 %. That points at the NVIDIA driver rather than Windows as the case to copy for. Gating the default on AI-assisted (Claude Code); the builds and runs are on my machine. |
|
Data point from a CPU-bound title on Windows: Metal Gear Solid 4 (PPSA18194), gameplay from a save, i9-10850K, RTX 3080 10 GB (610.74), MinGW GCC 15.2. Build: a local stack (#6188, #6187/#6192, #6273, #6077) + this PR, one binary,
Here the frame is held by the draw thread (~140 ms per present, GPU busy ~30 ms), so the copies cost CPU on that thread (vertex time about 2x) and the GPU saving does not show. Frame rate is about the same (scenes are not identical across the two runs; the vertex phase is the clear difference). Not asking for a change; noting that the gain depends on whether the title is GPU- or draw-thread-bound, in case a size threshold or per-buffer heuristic is considered later. Tested with AI assistance (Claude Code). |
|
A/B on a Windows NVIDIA card in a second title: RTX 3060 12 GB (NVIDIA 617.42), i5-12400F, Windows 10 19045, MinGW GCC Release,
The draw class drops by a quarter (-8 ms per present) and GPU busy by the same amount, so the in-place reads cost the same here as on your 5070 Ti. The frame rate doesn't move because this scene is CPU-bound on this machine: the freed time shows up as idle gaps (+8 ms). 0 fatal in all six rows. AI-assisted: yes (Claude Code) |
|
One test-isolation edge on Windows: |
bf1bae5 to
3dfcf22
Compare
SP4C3B4R-8
changed the title
|
Thanks all for the A/B runs. @SoRadGaming good call: the slow in-place reads are NVIDIA's driver, not Windows. With parl0v's RTX 3060 numbers (draws 32.3 against 24.3 ms per frame in place) and yours on the RX 9070 XT (in place as fast as copied), the default now copies only with @MarcZX100 fixed: the copied-only test no longer depends on the environment. It sets @ps54all noted. On a frame held by the draw thread the copies cost CPU. A size threshold could come later, once there is a scene where it changes the frame rate. Rebased onto main ( |
|
Another NVIDIA data point, in case it helps: RTX 5090, driver 617.42, Windows 11, Ryzen 9 9950X3D. Setup:
The build was main 47a266c with #6060's settle floor lowered to 64 KiB locally ( |
3dfcf22 to
dc86eb9
Compare
|
I am preparing a 007 First Light Windows cross-test of dc86eb9 on RTX 5070 Ti / 7800X3D / NVIDIA 617.42. The active fresh-main baseline is still completing cold pipeline compilation, so there is no result for this PR yet. The integration is 87f94c6 on main e5c2b3c, with the required public boot/resource PRs and an opt-in 16 MiB import window. Merge preview has one VulkanDevice.cpp conflict adjacent to that window option; your driver policy can be preserved independently. I will compare your in-place/copy switches in the same build, scene and warm cache and report official trace summaries plus draw GPU time. No duplicate implementation from me. Raw title data stays local. AI-assisted: Codex. |
|
Completed a Windows 007 First Light menu pair on integration Same binary, warm caches, 3840x2160, Performance mode, offline main menu with its animated background, no input during each sample. Only
This single pair shows no clear gain in this scene (about 1.3% between arms, similar to movement/run variation). Recent fully-menu GPU counter reports put the draw class near 3 ms/present in both; compute dominates the GPU time. This does not contradict the draw-heavy Astro Bot result, and it does not measure level gameplay. The first copied warm-up had a 43.455 s transition stall and resumed before the owned stop; I did not use it as a menu sample. Similar WAIT_REG_MEM_64 diagnostics occur in the previous baseline and imported control, and the copied repeat passed the transition, so there is no demonstrated copy-specific hang or regression. Official import8bd-b: p99 ms ##@@@% 0-888.6 copy8bd-a2: p99 ms %%%@@@ 0-752.8 No title patches or time scaling. Raw title code/assets/descriptors/requests/logs remain local. AI-assisted: Codex. |
github-actions
Bot
added
needs rebase
dc86eb9 to
ff0ee09
Compare
|
This pull request is labeled |
…ding host imports in place NVIDIA's Windows driver reads a host import of guest memory far slower than a buffer it allocated. Since draw vertex and index ranges are bound in place from host imports (boykopovar#2253, measured on Linux), Astro Bot's Crash Site on Windows 11 with an RTX 5070 Ti spends 164 ms of every frame in draws on the GPU, and 45 ms with the ranges copied as before; with the same build, 4.71 fps in place against 7.52 copied. An RTX 3060 shows the same (draws 32.3 against 24.3 ms per frame), while AMD's Windows driver reads them in place about as fast as a copy (RX 9070 XT: 69.8 against 73.1 ms), so copying there only costs CPU. The device decides once (DrawInputsInPlaceUsable, Context::drawInputsInPlace): ranges are copied with VK_DRIVER_ID_NVIDIA_PROPRIETARY on Windows and bound in place everywhere else. APS5_DRAW_INPUTS_IN_PLACE=1 binds them in place and APS5_NO_DRAW_INPUTS_IN_PLACE=1 copies them on any device, the second winning. agc_draw_inputs_in_place_driver checks the decision per driver and host; agc_draw_inputs_copied checks that an aligned imported range is copied and its read not noted when the context copies, whatever the inherited environment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GN8FJFTedayRXLHythXkZ4
ff0ee09 to
6df86af
Compare
|
A/B on a CPU-bound UE5 title: Sword of the Sea (PPSA25344), Windows 11 26200, RTX 4070 12 GB, NVIDIA 617.42, Ryzen 9 7900, 3840x2160. Build:
Both runs used
The PR does what it says on this title: draw GPU time halves on the title screen and drops about 5× in game. Frame time barely moves, because the title is CPU-bound on the queue worker:
|
| Metric | Baseline | Candidate | Change | Change (%) |
|---|---|---|---|---|
| frame_ms.p50 | 431.261 | 413.158 | -18.103 | -4.198 |
| frame_ms.p90 | 455.783 | 424.865 | -30.918 | -6.783 |
| frame_ms.p99 | 487.533 | 447.400 | -40.133 | -8.232 |
| mean_frame_ms | 431.141 | 413.047 | -18.093 | -4.197 |
| gpu_busy_ms_per_present | 252.340 | 173.341 | -78.999 | -31.307 |
| Driver.Packet | 753.306 | 721.972 | -31.334 | -4.160 |
| Driver.Packet.dispatch | 169.215 | 137.014 | -32.201 | -19.030 |
| Driver.Packet.draw | 188.195 | 178.386 | -9.809 | -5.212 |
| Driver.Packet.memory_wait | 255.096 | 267.239 | +12.143 | +4.760 |
{"commit":"2ccad8739d02","dirty":false,"env":{"APS5_DRAW_INPUTS_IN_PLACE":"1","APS5_PROFILE_DRAW":"1","APS5_PROFILE_GPU":"1","APS5_THREAD_PRIORITY":"high","APS5_TRACE_DIR":"<path>","APS5_TRACE_SUMMARY_EVERY":"10"},"gpu":"NVIDIA GeForce RTX 4070","driver":"NVIDIA 617.42","os":"Windows 10.0.26200","resolution":"3840x2160","window":{"spec":"summary_every 10","first_present":1137,"ms":10347.373},"interval_start_s":294.871,"interval_end_s":305.218,"presents":24,"frame_ms":{"p50":431.261,"p90":455.783,"p99":487.533,"max":487.533},"gpu_busy_ms_per_present":252.34,"worker_busy_pct":{"queue worker 0x0":98.101,"queue worker 0x20":76.811},"top_phases_ms_per_frame":{"Driver.Packet":753.306,"Driver.Packet.memory_wait":255.096,"Driver.Packet.draw":188.195,"Driver.Draw":186.668,"Driver.Packet.dispatch":169.215,"Graphics.Draw":155.603,"Graphics.Draw.draw_and_resource_release":141.098,"Driver.Dispatch":120.981},"trace_dropped":false,"trace":""}
{"commit":"2ccad8739d02","dirty":false,"env":{"APS5_PROFILE_DRAW":"1","APS5_PROFILE_GPU":"1","APS5_THREAD_PRIORITY":"high","APS5_TRACE_DIR":"<path>","APS5_TRACE_SUMMARY_EVERY":"10"},"gpu":"NVIDIA GeForce RTX 4070","driver":"NVIDIA 617.42","os":"Windows 10.0.26200","resolution":"3840x2160","window":{"spec":"summary_every 10","first_present":1148,"ms":10326.181},"interval_start_s":505.701,"interval_end_s":516.027,"presents":25,"frame_ms":{"p50":413.158,"p90":424.865,"p99":447.4,"max":447.4},"gpu_busy_ms_per_present":173.341,"worker_busy_pct":{"queue worker 0x0":98.089,"queue worker 0x20":76.951},"top_phases_ms_per_frame":{"Driver.Packet":721.972,"Driver.Packet.memory_wait":267.239,"Driver.Packet.draw":178.386,"Driver.Draw":176.905,"Graphics.Draw":166.423,"Graphics.Draw.draw_and_resource_release":141.328,"Driver.Packet.dispatch":137.014,"Driver.Dispatch":110.59},"trace_dropped":false,"trace":""}Limits:
- There was one run per side on the full path. On the title screen I have two runs per side: 77.8–80.1 ms in place against 74.1–76.7 ms with the PR.
- Between identical runs of this title I measured about ±6 % of noise in the cinematic.
- In game, the character falls through the floor shortly after getting control in both variants (this is unrelated to the PR), so the in-game scene only roughly matches between runs.
- I saw no visual regression. In game there seemed to be slightly more colour with the PR, but I have no screenshots, so take that as anecdotal.
AI-assisted: yes (Claude Code). The build, the runs and this comment were done and written by Claude Code on my machine.
This branch has not been deployed
9 participants
What
Since #2253, a draw binds its vertex and index ranges in place from the host import that holds them. That was measured on Linux. NVIDIA's Windows driver reads a host import of guest memory far slower than a buffer it allocated, and the draws pay for it on the GPU.
Astro Bot (PPSA21564),
-lvl hub_crashsite, standing at the start position, same build both times,APS5_PROFILE_GPU=1:main)drawGPU time per present ([gputime])Same number of draws (about 220 timed ranges per present); compute dispatches cost the same in both. The frames match (shadows, foliage, sky checked on dumped frames).
Others' A/B runs in this thread: an RTX 3060 shows the same (draws 32.3 against 24.3 ms per frame in place, parl0v), while AMD's Windows driver reads in place about as fast as a copy (RX 9070 XT: 69.8 against 73.1 ms, SoRadGaming), so copying there only costs CPU.
DrawInputsInPlaceUsable,Context::drawInputsInPlace): ranges are copied withVK_DRIVER_ID_NVIDIA_PROPRIETARYon Windows, as before perf(agc): read draw vertex and index buffers in place from host imports #2253, and bound in place everywhere else.APS5_DRAW_INPUTS_IN_PLACE=1binds them in place andAPS5_NO_DRAW_INPUTS_IN_PLACE=1copies them on any device; the second wins.docs/dev/ARCHITECTURE.mdgets the line.Copies in device-local host-visible memory (ReBAR) cut the draw time further, to 39 ms, but the frame rate stayed the same in this scene (CPU-bound there after this change), so this PR keeps the copies where they were.
Tested
Windows 11 Pro (26300), MinGW-w64 GCC 15.2.0, Release, Ryzen 7 7800X3D, RTX 5070 Ti (16 GB).
agc_draw_inputs_in_place_driver: the decision per driver and host (NVIDIA proprietary on Windows copies; NVIDIA elsewhere, AMD, Intel, RADV and llvmpipe bind in place).agc_draw_inputs_copied(agc_driver_recorder_tests --draw-inputs-copied-only): withContext::drawInputsInPlaceoff, an aligned imported range is not bound in place and its read is not noted. WithInPlaceDrawInputignoring the flag it fails ("an imported draw input was bound in place where draw inputs are copied"). It passes withAPS5_DRAW_INPUTS_IN_PLACE=1exported, andagc_draw_inputs_in_placepasses withAPS5_NO_DRAW_INPUTS_IN_PLACE=1exported: neither depends on the environment.agc_pipeline_link_optimizationstill passes;libsbuilds.tools/check_conventions.py --base origin/mainpasses.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)🤖 Generated with Claude Code
https://claude.ai/code/session_01GN8FJFTedayRXLHythXkZ4