perf(agc): compare an import window's pages when the window closes

[oneandonlydean]

What

A host import that leaves its range write-watched can resolve the write protection of the pages it imports without storing anything. NVIDIA's Linux driver does that for every imported page (4 of 4 ... after the import in the probe line). GuestMemory::ImportWatched stamped such pages as a possible CPU store (BlockMaybeWritten), and the only code that could clear the stamp was StorageTexture::Refresh for an image tracked by 64 KiB unit, by comparing with a copy of the guest bytes the image was uploaded from (#2984). So that this compare has its copy, a fast-cleared upload reads its whole surface from guest memory wherever such an import is possible (#3372). That is the read #2699 removed for watched memory (measured there on PPSA14632); on NVIDIA it is back. Nothing else uses the copy on watched memory, and a surface that is cleared every frame pays for it every frame: in a title we test with, 139 MiB of guest memory were read per frame in the menu, almost all of it here.

The compare now happens once, where the stamp is made, and only for an import whose range stays watched. ImportWatched takes that as a parameter (staysWatched). The caller knows it before the import is made:

• Linux host-pointer import: the range stays watched when the device's probe saw no page written after a GPU read or write (unwatchImports is false).
• Linux dma-buf import: the range stays watched when the dma-buf probe says so (unwatchDmaBufImports is false).
• Windows: only an import through a separate shared alias stays watched. A direct import leaves the watch.

For an import that stays watched, ImportWatched copies the range, collects the stores made so far (stamped as CPU stores, as before), runs the import, and then compares each page whose write protection was resolved in the window with the copy:

• A page with other bytes is stamped as a CPU store.
• A page with the same bytes keeps only the CPU-side stamp that UnchangedSinceCollected and StoredOver read, as before. Its block stamp doesn't move, so UnchangedSince and ChangedBlocks hold for it.

A store between the copy and the first collect is in that collect and stamped as a CPU store. A store after it is in the window and is compared with bytes copied before it. The collect re-arms a page before the page is compared, so a store after the compare is in the next collect.

For an import that leaves the watch, nothing is copied or compared. The stores made so far are collected as before, the pages resolved in the window get the CPU-side stamp as before, and the range is taken out of the watch before the tracker is unlocked (the caller did that after the call before). Two things differ from main there:

• The window no longer moves the block stamp of the pages it resolved. Those pages have left the watch, so this is visible only to watched pages that share a 64 KiB block with the first or last page of the import: they now read unchanged where main read them as possibly written.
• Where the two import kinds differ in whether they stay watched (lavapipe: host-pointer imports are compared, dma-buf imports stay watched), a failed host-pointer import and the dma-buf import that follows it are two windows instead of one, so that each window knows its kind before it starts. The pages the failed import resolved are collected by the second window as CPU stores, as they are after any failed import on main. Where the kinds agree it is one window, as before.

This replaces code from three merged PRs. Their cases stay covered:

• fix(agc): ignore unchanged imported texture blocks #2984 (unit compare in Refresh): a pending image whose units an import window resolved without new bytes must not be uploaded again. The window no longer stamps those units, for any consumer, so the compare in Refresh has nothing left to resolve. agc_driver_dcc_keys_first_write (an unchanged pending image was uploaded again) and its import-window part pass unchanged.
• fix(agc): keep a fast-clear copy where an import window can stamp watched memory #3372 (WatchedImportsStampPages, the two HostImports flags, the fast-clear copy where a window can stamp): the copy existed only for fix(agc): ignore unchanged imported texture blocks #2984's compare. The test fix(agc): keep a fast-clear copy where an import window can stamp watched memory #3372 added stays as it is and passes: after a window that stored nothing new a pending image is not uploaded again, and a store of a new byte inside a window is seen.
• perf(agc): read a fast-cleared surface's guest bytes only when the import is compared #2699 (no guest-byte copy for a fast-cleared upload of watched memory): this is its behaviour again on every device. Its check in agc_driver_dcc_keys_first_write is back to GuestSnapshotValid() == !stillWatched.

Also removed: BlockMaybeWritten and the tracker's written-stamp table (it now always equals the block stamp). A store of new bytes inside a window now reads as BlockWritten at a write-back as well, where BlockMaybeWritten was stored over. The technical-debt entry for the import window (now in docs/dev/debt/libSceAgcDriver.md) is shorter: the store of identical bytes inside a window remains, the layer-tracked images are gone from it.

Cost: a range the tracker covers is copied once before its import when the import leaves it watched, and only then. The copy is held while the import is made and the tracker mutex is held as before.

• Copied: host-pointer imports on NVIDIA's Linux driver, dma-buf imports on a device whose dma-buf probe keeps them watched, Windows imports through a shared alias, and every import under APS5_WRITE_WATCH_IMPORTS=watch. In the title above on NVIDIA that was 62 imports and 14.6 GiB over one run, the largest 2.2 GiB, measured on the previous revision of this branch, which copied the same ranges on this device; it was not measured again. The copy is now made in one pass (assign instead of resize and memcpy).
• Not copied: host-pointer imports on a device where they are compared (lavapipe here; AMD and Intel if their probe reports writes after a GPU read or write), Windows direct imports, every import under APS5_WRITE_WATCH_IMPORTS=unwatch, and ranges the tracker doesn't cover.

Copying only the pages a collect has armed is not done here. ImportWatched throws for a range it tracks, is asked to keep watched and can't read; its caller checks that the range is accessible first. The check inside is CheckRange, so on Windows it answers from the table of committed arena pages that the registry's invalidator keeps current, as every other CheckRange does.

Tested

Linux (CachyOS, kernel 7.2.3, GCC 16.2.1, Release with CI's configure flags and ANYPS5_ENABLE_SPIRV_TOOLS=ON), RTX 4090, NVIDIA 610.57.04 (host imports stay watched; 4 after the import; dma-buf imports unavailable). Two commits, rebased onto main d539606: 406c72d, and 1d3f642, which changes only the setup of the second new test case (two lines in tests/WriteTracking.cpp). The rebase conflicted only in docs/dev/TechnicalDebt.md, whose entries moved to docs/dev/debt/; tools/debt_split.py resolve moved this entry to docs/dev/debt/libSceAgcDriver.md. The code applied unchanged on top of #4385, #4291, #4108, #2802, #1357, #4221 and #4186, and nothing they added reads BlockMaybeWritten, the written-stamp table or WatchedImportsStampPages. An import parked for another device (#3976, #4291, #4385) is not touched here: its range keeps the watch state it had, as on main, and a new import of that range is a window of its own. The head merges into main 682ef0c without conflicts. The table and the lines marked d539606 are from this head. The lines marked a11c8f7 and the profiled runs were measured on earlier bases with the same change and were not run again.

test	main d539606 with only the test changes	this branch
agc_write_tracking (two new cases, no device)	doesn't build: the second case passes the new parameter. With the first case only it fails: a page an import window resolved without changing its bytes reads as written	passes
agc_driver_dcc_keys_first_write	fails: a fast-cleared upload of write-watched memory read its guest bytes	passes
agc_driver_recorder_tests, full run (not in ctest)	fails: (w) an import that stored nothing reads as a change to a cache taken before it	passes

• The first new agc_write_tracking case: an import window that stays watched, in which one page gets the byte it already held and a page in another block gets a new byte. The first block must read unchanged and keep its CPU-side stamp, the second must read written, a third untouched, and a store after the window must be seen.
• The second new case: a range with one page made unreadable. An import that stays watched must throw before the import is made (it reads the range). An import that leaves the watch must not throw, so it took no copy; its range must be out of the watch afterwards, the block after it must be untouched, and on Linux the watched pages that share a block with it must read unchanged, keep the CPU-side stamp and still see a later store. With the copy made unconditionally this case fails with guest memory is not readable; with the window stamping stores it fails with a store to a page that left the watch reads as a store to the watched pages of its block. The case changes the page's protection with VirtualProtect or mprotect and reports the change with GuestAllocationsInvalidate, as the registry does for a guest protection change and as tests/BdaResources.cpp does.
• d539606: agc_write_tracking, agc_driver_dcc_keys_first_write, agc_unit_shadow, agc_host_import_unmap, agc_unchanged_cpu_stamps, agc_storage_baseline_settled and agc_driver_depth_fast_clear_submit 5 times each, 35 of 35 pass; the full recorder run 3 of 3.
• d539606: full ctest (--parallel 4, ANYPS5_REQUIRE_VULKAN=1), 979 tests, on main d539606 and on this head: 971 pass, 7 skip (agc_driver_stencil_ref_export, video_out_flip_lifetime, _reopen, _controls, _overlay, _unopened, video_out_frame_dump_queue), and the same one fails on both, guest_kernel_fcntl (fcntl: descriptor 0 is neither a file, a directory, a pipe nor a character device; the run had no terminal on its standard input).
• d539606: python3 tools/check_conventions.py --base origin/main and python3 tools/check_test_registration.py build pass.
• d539606: a second title we test with (a 2D title), one scripted 220 s run on this head, frames dumped on the CPU: all seven frame checks pass, the menu at 19.6 fps and gameplay at 38.2 fps with every frame dumped, 0 skipped draws, 0 fatal lines, 0 kernel Xid lines. main d539606 was not run for comparison.
• a11c8f7: the AGC, shader and write-tracking tests (ctest -R "agc|shader|write", 391 tests) on lavapipe (Mesa 26.2.3, LLVM 22.1.8): the same two fail on main a11c8f7 and on 101db8e (agc_driver_emulated_color_compare, agc_driver_image_plane_address_on_line_implicit_lod), the same 28 skip. There host-pointer imports are compared and dma-buf imports stay watched (0 of 4 scratch pages written after a GPU read, 4 after a GPU write; dma-buf imports keep write protection: yes), so the import-window part of agc_driver_dcc_keys_first_write prints that it is not tested.
• a11c8f7: copies counted with a log line that is not in the commit:

run	windows	not copied	copied
lavapipe, full recorder run	83	69 under the device's own decision (host-pointer imports compared) and 9 after the test forced the compare	5 (1.6 MiB), all after the test forced the watch with SetImportWatch
lavapipe, `ctest -R "agc	shader	write"`	105
NVIDIA, `ctest -R "agc	shader	write"`	110
NVIDIA, full recorder run	84	41 that the tracker doesn't cover and 4 that leave the watch (forced compare)	39 (35.7 MiB)

• The first title above was profiled on an earlier revision of this branch (f70b1db) on an earlier base, not on this head. On NVIDIA this head copies and compares the same ranges. Scripted from boot through its title screen and menu into the first level, warm caches, APS5_PROFILE_DRAW=1, one run each. The base is main 2220eff with feat(agc): depth planes sampled as 2D arrays, cubes and integer views #425, fix(libSceAgcDriver): retire a depth surface once another writer covers its memory #454, perf(agc): keep unchanged guest layers of a depth plane copy #2490 and perf(agc): skip the depth plane copy when no slice changed #2491, which the title needs to get past its intro; the change is that revision on top. It could not be measured on main d539606: with fix(libc): round mspace block sizes to the 32-byte alignment #4407 on top the title runs its intro and then terminates with storage image access to depth/stencil surface ... is not implemented, before the title screen, so there is nothing to compare there.

scene	base	base + previous revision	guest memory read per frame
title screen	18.5 fps, 54.1 ms	22.8 fps, 43.9 ms
menu	15.75 fps, 63.5 ms	20.6 fps, 48.5 ms	138.6 -> 2.4 MiB
first level	18.9 fps, 53.0 ms	21.9 fps, 45.6 ms

The target lookup per draw went from 37.7 to 8.5 us at the title screen. Guest memory read over the whole run: 323 GiB before, 26 GiB after. Two more runs of the same sequence with one more fix on top gave 22.9 / 20.75 / 22.3 and 22.7 / 21.0 / 22.2 fps, 0 kernel Xid lines, and frames that match the base by eye. The title screen was reached at 32.0 s, against 34.9 s on the base, with the copies included.

Not tested:

• The lavapipe run and the copy counts on this head; they are the lines marked a11c8f7.
• Windows. Not compiled or run here; it needs the Windows CI build. CI built and ran 101db8e: it compiled, the first new agc_write_tracking case passed (the Windows walk's page compare), and the second crashed in the copy instead of throwing. The cause was the test: it made the page unreadable with VirtualProtect alone, the first case had already checked the same allocation, so CheckRange answered from its page table that the page was readable. The second commit (now 1d3f642) reports the change; it was not run on Windows here. A title doesn't reach this: importAllocation queries every page of the range with VirtualQuery before it calls ImportWatched and makes no import over a page that is not committed, PAGE_NOACCESS or PAGE_GUARD, and a guest protection change goes through the registry. Still not run on Windows: an import through a shared alias (the alias condition entry.alias != nullptr || !state.unwatchImports), and the second half of the second case, which collects a range that holds a PAGE_NOACCESS page.
• AMD and Intel. Whether their host-pointer imports are compared is their probe's result, which was not seen here; if they are, they now take no copy, as lavapipe does. Nothing was run on either.
• A dma-buf import that stays watched, and the two-window path before it. lavapipe reports that for dma-buf, but its host-pointer imports succeed, so no test here reaches a dma-buf import of guest memory. Whether a failed host-pointer import resolves write protection on any driver is not known; if one does, those pages read as CPU stores.
• A guest thread storing during a window in a running title was reasoned about, not provoked by a test: the new cases store from the import callback.
• The 2D title on main d539606 without the change.
• The profiled run of the first title on this head or on current main (see above).

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Unblocks: none
• AI-assisted: yes (Claude Code)

[kr4y4]

A Windows run after the merge, since the description lists Windows as not compiled or run here.

Setup: Windows 11 (26200), WinLibs GCC 15.2.0 r7, Release, CI configure flags, an RTX 4070 Laptop GPU and an Intel Raptor Lake-S iGPU, main ea2bbef.

• agc_write_tracking passes on both GPUs with --repeat until-fail:3. That includes CheckUnwatchedImport, with the VirtualProtect and GuestAllocationsInvalidate setup from 1d3f642: the import that stays watched throws before the import, and the one that leaves the watch doesn't read its range. Its block-neighbour checks are #ifndef _WIN32, so they didn't run.
• agc_driver_dcc_keys_first_write and the other 16 entries the changed test files feed pass on both GPUs with --repeat until-fail:3.
• The full recorder run (agc_driver_recorder_tests with no arguments) passes on both GPUs.
• Full suite (ctest --timeout 120 --parallel 4): Intel 1038 of 1038. NVIDIA fails only the device loss reported on fix(shader): split a BDA span read's in-place loads from its fallback #4759.

Not reached: an import through a shared alias. APS5_WRITE_WATCH_IMPORTS=watch can't force one on Windows, because importWatchRequest is under #ifndef _WIN32 and decideImportWatch sets unwatchImports = true there. With it set, agc_driver_dcc_keys_first_write still prints host imports are compared, not watched: the write-watched block runs as unwatched.

Written with AI assistance.