Skip to content

perf(libc): collect resident writes directly into the output buffer - #7471

Open
Dripl1x wants to merge 2 commits into
boykopovar:mainfrom
Dripl1x:codex/direct-resident-write-collection
Open

Dripl1x wants to merge 2 commits into
boykopovar:mainfrom
Dripl1x:codex/direct-resident-write-collection

Conversation

@Dripl1x

@Dripl1x Dripl1x commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

What

This PR is part of a series of graphics and memory-management performance improvements for AnyPS5, with SuperTuxKart as the main profiling workload. This is item 08/10; the UI Lab startup/cache fix follows as a separate eleventh PR. Each topic's measurements are distinguished from the combined result.

Collect writes from a stable resident mapping directly into the caller's output buffer. Preserve capacity, fresh-page accounting, unaligned starts and reset behavior while avoiding two temporary collections.

Tested

Pinned benchmark base: 6447ed1ed7fd11c7eef60df336e8d66fac271188. Tested topic snapshot: a648e12340ca77dd950f7d6c9d7b372732aeb479. Windows 11 Pro build 26300, Ryzen 7 5800H, RTX 3070 Laptop, NVIDIA 617.42. Frozen runtime module hashes and raw traces were verified. No opt-in optimization flags are required.

3/3 memory/write-watch cases passed. The behavioral test also passes on main. Coverage includes capacities zero and one, output sentinels, unaligned starts, clear=false, retention of unreported dirty pages and a 2-MiB resident-run boundary.

SuperTuxKart tutorial/menu, 1920x1080, warm caches, seconds 20-28:

Configuration FPS p50 ms p90 ms p99 ms
main control 9.509 104.114 110.231 113.153
This topic 9.844 101.295 103.657 106.906

Observed difference: +3.52% in one capture per configuration; this does not establish a repeatable isolated gain.

Trace summary lines
{"commit":"6447ed1ed7fd","dirty":false,"env":{"APS5_TRACE_SECONDS":"20:8"},"gpu":"NVIDIA GeForce RTX 3070 Laptop GPU","driver":"NVIDIA 617.42","os":"Windows 10.0.26300","resolution":"1920x1080","window":{"spec":"seconds 20:8","first_present":44,"ms":8097.757},"presents":77,"frame_ms":{"p50":104.114,"p90":110.231,"p99":113.153,"max":113.153},"gpu_busy_ms_per_present":5.352,"worker_busy_pct":{"queue worker 0x0":3.181},"top_phases_ms_per_frame":{"Driver.Packet":3.294,"Driver.Packet.draw":2.701,"Driver.Draw":2.646,"Driver.Draw.shader_compile_and_link":1.561,"Shader.DrawStage":0.838,"Driver.Present":0.824,"Driver.Present.queue_present":0.584,"Vulkan.QueuePresent":0.583},"trace_dropped":false,"trace":"aps5-trace-6447ed1ed7fd-20276.json.gz"}
{"commit":"6447ed1ed7fd","dirty":false,"env":{"APS5_TRACE_SECONDS":"20:8"},"gpu":"NVIDIA GeForce RTX 3070 Laptop GPU","driver":"NVIDIA 617.42","os":"Windows 10.0.26300","resolution":"1920x1080","window":{"spec":"seconds 20:8","first_present":58,"ms":8025.344},"presents":79,"frame_ms":{"p50":101.295,"p90":103.657,"p99":106.906,"max":106.906},"gpu_busy_ms_per_present":5.292,"worker_busy_pct":{"queue worker 0x0":2.864},"top_phases_ms_per_frame":{"Driver.Packet":2.864,"Driver.Packet.draw":2.324,"Driver.Draw":2.275,"Driver.Draw.shader_compile_and_link":1.37,"Driver.Present":0.852,"Shader.DrawStage":0.744,"Driver.Present.queue_present":0.602,"Vulkan.QueuePresent":0.598},"trace_dropped":false,"trace":"aps5-trace-6447ed1ed7fd-8760.json.gz"}

Raw trace names and SHA-256 values are recorded in the series evidence index. The common evidence link below contains those files.

Other homebrew checks

Workload Control FPS This topic FPS Scope
vkQuake 54.590 56.628 Moving demo; small differences need matched camera repeats
Syobon Action 30.276 30.280 Approximately 30-FPS cap
Middle Ages Ping Pong 59.940 59.939 Approximately 60-FPS cap

UI Lab: the 21 default concepts were rendered and visually reviewed in a navigation cycle for this topic. Artwork was present; this covers default navigation, not all themes or every widget interaction.

Known baseline rendering issue: vkQuake still has intermittent flickering and an intermittently black floor on main and the tested performance branches. These changes do not fix it. Yamagi Quake II gameplay stops at unsupported last-provoking-vertex flat interpolation, and UZDoom at unsupported clip/cull-distance handling on the baseline; neither supplies a valid gameplay FPS comparison.

Dependencies, series and overlap

#7448 adds a physical-release lifetime test at the same GuestMemory.cpp insertion sites. Keep both independent tests when rebasing. Series item 10 builds on this direct-output collector.

The project overlap scan compared the local topics against 785 open PRs and checked merges into main e5c2b3c325e485ba7fd417c47c46f48aa2c89115. This topic merges cleanly in that snapshot. Some other PRs already conflict with main, so the tool cannot compare all their shared-file hunks; the full report is retained with the common series evidence package. No exact duplicate was identified in the inspected implementations. This is not a certification that every conflicting foreign branch is composition-tested.

Conventions and whitespace checks passed for the tested topic range. The benchmark base remains pinned for comparability; this is not a fresh-current-main build/test claim. Measurements and focused tests refer to the pinned source revisions above; no full-suite CI run is claimed.

Combined series result

Same pinned main, machine, 1920x1080 SuperTuxKart tutorial/menu scene, warm caches, maximized window and muted audio. APS5_TRACE_SECONDS=20:8 only; no application-output redirection. FPS is presents divided by the recorded capture interval; frame quantiles are computed independently. Rows are sorted by increasing FPS. Each row is one capture.

Configuration FPS p50 ms p90 ms p99 ms
Pinned main 9.509 104.114 110.231 113.153
main + #6658 + item 01 14.248 68.284 78.652 95.172
main + #6658 15.493 63.782 68.742 78.478
Historical ten-topic stack + #6658, before item-10 import correction 26.124 37.622 41.458 44.996
Corrected ten-topic stack + #6658 + tested #6678 revision 32.323 30.874 32.059 33.862

The corrected combination is 3.40x the pinned-main throughput, with p50 reduced by 70.3%. This is a combined result, not the gain of each PR. The earlier 32.646-FPS combination is superseded because item 10 needed its resident-import correctness repair. Screen-counter peaks near 35 FPS are not an average.

The combination includes #6658 (local integration 6baa697b2f6d6154fe7f144a77eb0af1ce37f173) and #6678 at tested revision 5f822a50086fd880d6b4346709b70f8901431f43. #6658 is open. #6678 merged on 2026-10-18 with later head fd301c78a5d625d6eba8b8a32e5746a044fc235e; that newer revision and current main were not benchmarked here. Their implementations and gains are credited separately. Related earlier SuperTuxKart work #6718 and Windows tracking work #6701 are separate PRs and are not relabeled as items in this series.

Checklist

  • Based on pinned main 6447ed1ed7fd; clean merge checked against main e5c2b3c325e4; related overlaps and duplicate-review limits are listed above
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a (library-only change; game test hardware above)
  • AI-assisted: yes

The published head adds publication metadata only; its complete source tree is identical to the tested snapshot above.

Series index

These are the ten performance topics from the same profiling effort, followed by a separate UI Lab cache-preservation fix. Membership in the series does not imply a dependency.

Item Pull request Required dependency
01 #7464: perf(agc): keep unimported label stores on the CPU #6658
02 #7465: perf(agc): defer small host imports until first GPU use none
03 #7466: perf(agc): validate host imports through their registered owner none
04 #7467: perf(libc): avoid repeated Windows path component scans none
05 #7468: perf(libkernel): find direct-memory aliases through the physical index none
06 #7469: perf(libkernel): find free direct-memory ranges in allocation order none
07 #7470: perf(libc): release exact resident mappings without repeated queries none
08 #7471: perf(libc): collect resident writes directly into the output buffer none
09 #7472: perf(libc): reuse saved direct-memory buffers when unreferenced none
10 #7473: perf(libc): skip write-watch queries for proven clean resident ranges #7471
11 #7474: fix(agc): preserve pipeline caches on immediate guest exit none

Reproduction evidence

Series traces, summary lines, module hashes, focused-test logs and UI contact sheets. Game measurements use the pinned base and source revisions described above; assets contain no game binaries.

Complete the publication checklist and series cross-links for boykopovar#7471.
No source or test changes.
@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 7448 core/libs/tests/GuestMemory.cpp L1569, L2838-2839

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 4449 core/libs/tests/GuestMemory.cpp
PR 5021 core/libs/tests/GuestMemory.cpp
PR 6044 core/libs/tests/GuestMemory.cpp
PR 6469 core/libs/tests/GuestMemory.cpp
PR 7105 core/libs/tests/GuestMemory.cpp
PR 7332 core/libs/tests/GuestMemory.cpp

@github-actions github-actions Bot added the libc label Oct 18, 2026
@wachi35676

Copy link
Copy Markdown
Contributor

This is equivalent by inspection: when run.fresh == 0, the old loop reports exactly the GetWriteWatch pages with max(at, from) and settles nothing. The 462 related tests pass on Windows.

A clean 2 MiB resident collect takes 1.68–1.90 µs on main and 1.39–2.12 µs with this PR, which is within noise on this machine.

AI-assisted: yes (Claude Code); results above were run on this machine.

@Dripl1x

Dripl1x commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

@wachi35676 Thanks for checking the equivalence and running the related Windows tests. Agreed that the clean 2 MiB measurements are within noise on your machine; the change removes the intermediate copy, but those timings do not establish a repeatable isolated speedup.

@github-actions github-actions Bot added perf Performance change perf numbers missing perf pull request without a before and after summary line labels Oct 18, 2026
@github-actions

Copy link
Copy Markdown

This pull request is labeled perf, so it needs the summary line of a capture on main and one with the change, from the same title, scene and machine, as the Performance report form asks. Please paste both aps5-summary.jsonl lines, attach the two traces and add the output of tools/trace_digest.py.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

libc perf numbers missing perf pull request without a before and after summary line perf Performance change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants