Skip to content

perf(libkernel): find direct-memory aliases through the physical index - #7468

Open
Dripl1x wants to merge 2 commits into
boykopovar:mainfrom
Dripl1x:codex/direct-memory-alias-index
Open

Dripl1x wants to merge 2 commits into
boykopovar:mainfrom
Dripl1x:codex/direct-memory-alias-index

Conversation

@Dripl1x

@Dripl1x Dripl1x commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

What

This PR is part of a series of graphics and memory-management performance improvements for AnyPS5, with SuperTuxKart as the main profiling workload. This is item 05/10; the UI Lab startup/cache fix follows as a separate eleventh PR. Each topic's measurements are distinguished from the combined result.

Use the existing physical-span index to select direct-memory aliases that must be unpinned before a mapping changes. Collect overlapping virtual mappings under the existing lock and wait for their leases outside that lock.

Tested

Pinned benchmark base: 6447ed1ed7fd11c7eef60df336e8d66fac271188. Tested topic snapshot: d2ca227e8f3230cb0b733608ab3fd84cdd9336a9. Windows 11 Pro build 26300, Ryzen 7 5800H, RTX 3070 Laptop, NVIDIA 617.42. Frozen runtime module hashes and raw traces were verified. No opt-in optimization flags are required.

2/2 targeted memory cases passed. Coverage includes a live lease delaying physical alias creation, coherence of overlapping aliases, partial removal, release across 48 reverse-ordered pieces and concurrent access while resident runs split.

SuperTuxKart tutorial/menu, 1920x1080, warm caches, seconds 20-28:

Configuration FPS p50 ms p90 ms p99 ms
main control 9.858 100.715 104.093 114.401
This topic 10.030 99.103 101.810 115.256

Observed difference: +1.75% in one capture per configuration; this does not establish a repeatable isolated gain.

Trace summary lines
{"commit":"6447ed1ed7fd","dirty":false,"env":{"APS5_TRACE_SECONDS":"20:8"},"gpu":"NVIDIA GeForce RTX 3070 Laptop GPU","driver":"NVIDIA 617.42","os":"Windows 10.0.26300","resolution":"1920x1080","window":{"spec":"seconds 20:8","first_present":62,"ms":8013.754},"presents":79,"frame_ms":{"p50":100.715,"p90":104.093,"p99":114.401,"max":114.401},"gpu_busy_ms_per_present":5.278,"worker_busy_pct":{"queue worker 0x0":3.074},"top_phases_ms_per_frame":{"Driver.Packet":3.07,"Driver.Packet.draw":2.495,"Driver.Draw":2.445,"Driver.Draw.shader_compile_and_link":1.48,"Driver.Present":0.846,"Shader.DrawStage":0.795,"Driver.Present.queue_present":0.58,"Vulkan.QueuePresent":0.579},"trace_dropped":false,"trace":"aps5-trace-6447ed1ed7fd-37076.json.gz"}
{"commit":"6447ed1ed7fd","dirty":false,"env":{"APS5_TRACE_SECONDS":"20:8"},"gpu":"NVIDIA GeForce RTX 3070 Laptop GPU","driver":"NVIDIA 617.42","os":"Windows 10.0.26300","resolution":"1920x1080","window":{"spec":"seconds 20:8","first_present":65,"ms":8075.401},"presents":81,"frame_ms":{"p50":99.103,"p90":101.81,"p99":115.256,"max":115.256},"gpu_busy_ms_per_present":5.288,"worker_busy_pct":{"queue worker 0x0":3.052},"top_phases_ms_per_frame":{"Driver.Packet":2.999,"Driver.Packet.draw":2.435,"Driver.Draw":2.387,"Driver.Draw.shader_compile_and_link":1.448,"Driver.Present":0.841,"Shader.DrawStage":0.827,"Driver.Present.queue_present":0.536,"Vulkan.QueuePresent":0.535},"trace_dropped":false,"trace":"aps5-trace-6447ed1ed7fd-20672.json.gz"}

Raw trace names and SHA-256 values are recorded in the series evidence index. The common evidence link below contains those files.

Other homebrew checks

Workload Control FPS This topic FPS Scope
vkQuake N/A N/A Earlier compatibility capture; no matched FPS claim here
Syobon Action N/A N/A Earlier compatibility capture; no matched FPS claim here
Middle Ages Ping Pong N/A N/A Earlier compatibility capture; no matched FPS claim here

UI Lab: the 21 default concepts were rendered and visually reviewed in a navigation cycle for this topic. Artwork was present; this covers default navigation, not all themes or every widget interaction.

Known baseline rendering issue: vkQuake still has intermittent flickering and an intermittently black floor on main and the tested performance branches. These changes do not fix it. Yamagi Quake II gameplay stops at unsupported last-provoking-vertex flat interpolation, and UZDoom at unsupported clip/cull-distance handling on the baseline; neither supplies a valid gameplay FPS comparison.

Dependencies, series and overlap

#7448 changes release ordering to unmap views before returning physical pages. This topic only selects aliases through the physical index; it does not fix that separate lifetime race.

The project overlap scan compared the local topics against 785 open PRs and checked merges into main e5c2b3c325e485ba7fd417c47c46f48aa2c89115. This topic merges cleanly in that snapshot. Some other PRs already conflict with main, so the tool cannot compare all their shared-file hunks; the full report is retained with the common series evidence package. No exact duplicate was identified in the inspected implementations. This is not a certification that every conflicting foreign branch is composition-tested.

Conventions and whitespace checks passed for the tested topic range. The benchmark base remains pinned for comparability; this is not a fresh-current-main build/test claim. Measurements and focused tests refer to the pinned source revisions above; no full-suite CI run is claimed.

Combined series result

Same pinned main, machine, 1920x1080 SuperTuxKart tutorial/menu scene, warm caches, maximized window and muted audio. APS5_TRACE_SECONDS=20:8 only; no application-output redirection. FPS is presents divided by the recorded capture interval; frame quantiles are computed independently. Rows are sorted by increasing FPS. Each row is one capture.

Configuration FPS p50 ms p90 ms p99 ms
Pinned main 9.509 104.114 110.231 113.153
main + #6658 + item 01 14.248 68.284 78.652 95.172
main + #6658 15.493 63.782 68.742 78.478
Historical ten-topic stack + #6658, before item-10 import correction 26.124 37.622 41.458 44.996
Corrected ten-topic stack + #6658 + tested #6678 revision 32.323 30.874 32.059 33.862

The corrected combination is 3.40x the pinned-main throughput, with p50 reduced by 70.3%. This is a combined result, not the gain of each PR. The earlier 32.646-FPS combination is superseded because item 10 needed its resident-import correctness repair. Screen-counter peaks near 35 FPS are not an average.

The combination includes #6658 (local integration 6baa697b2f6d6154fe7f144a77eb0af1ce37f173) and #6678 at tested revision 5f822a50086fd880d6b4346709b70f8901431f43. #6658 is open. #6678 merged on 2026-10-18 with later head fd301c78a5d625d6eba8b8a32e5746a044fc235e; that newer revision and current main were not benchmarked here. Their implementations and gains are credited separately. Related earlier SuperTuxKart work #6718 and Windows tracking work #6701 are separate PRs and are not relabeled as items in this series.

Checklist

  • Based on pinned main 6447ed1ed7fd; clean merge checked against main e5c2b3c325e4; related overlaps and duplicate-review limits are listed above
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a (library-only change; game test hardware above)
  • AI-assisted: yes

The published head adds publication metadata only; its complete source tree is identical to the tested snapshot above.

Series index

These are the ten performance topics from the same profiling effort, followed by a separate UI Lab cache-preservation fix. Membership in the series does not imply a dependency.

Item Pull request Required dependency
01 #7464: perf(agc): keep unimported label stores on the CPU #6658
02 #7465: perf(agc): defer small host imports until first GPU use none
03 #7466: perf(agc): validate host imports through their registered owner none
04 #7467: perf(libc): avoid repeated Windows path component scans none
05 #7468: perf(libkernel): find direct-memory aliases through the physical index none
06 #7469: perf(libkernel): find free direct-memory ranges in allocation order none
07 #7470: perf(libc): release exact resident mappings without repeated queries none
08 #7471: perf(libc): collect resident writes directly into the output buffer none
09 #7472: perf(libc): reuse saved direct-memory buffers when unreferenced none
10 #7473: perf(libc): skip write-watch queries for proven clean resident ranges #7471
11 #7474: fix(agc): preserve pipeline caches on immediate guest exit none

Reproduction evidence

Series traces, summary lines, module hashes, focused-test logs and UI contact sheets. Game measurements use the pinned base and source revisions described above; assets contain no game binaries.

Complete the publication checklist and series cross-links for boykopovar#7468.
No source or test changes.
@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 4449 core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp
PR 5021 core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp
PR 6879 core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp
PR 7105 core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp
PR 7149 core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp
PR 7201 core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp

@wachi35676

Copy link
Copy Markdown
Contributor

Differential check: I put main's g_directMappings and physical-span code in one harness, with random inserts, virtual erases and physical erases. Comparing the old full scan with the new indexed RequireAliasesUnpinned selection over 3 × 400k operations gives 0 mismatches in about 360k alias queries (about 320k non-empty), in the same order.

With 20,000 mappings, a lookup takes 128–147 µs with the old code and 0.22–0.24 µs with the new. The 462 guest_/libkernel/libc tests pass on Windows (GCC 15.2).

AI-assisted: yes (Claude Code); results above were run on this machine.

@Dripl1x

Dripl1x commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

@wachi35676 Thanks for the independent differential test and the Windows coverage. The zero-mismatch result across the alias queries, including their order, is useful confirmation of the indexed selection. I will keep the lookup microbenchmark separate from any whole-game FPS claim.

@github-actions github-actions Bot added perf Performance change perf numbers missing perf pull request without a before and after summary line labels Oct 18, 2026
@github-actions

Copy link
Copy Markdown

This pull request is labeled perf, so it needs the summary line of a capture on main and one with the change, from the same title, scene and machine, as the Performance report form asks. Please paste both aps5-summary.jsonl lines, attach the two traces and add the output of tools/trace_digest.py.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

libkernel perf numbers missing perf pull request without a before and after summary line perf Performance change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants