fix(agc): compare Windows host imports after late driver writes

[ToyuAU]

What

Addresses #1434: Windows drivers can mark imported pages dirty after the synchronous import window. Treating those marks as CPU stores makes storage-image write-back discard newer GPU results. Windows host-pointer imports use the existing byte-comparison paths instead.

GuestMemory::Unwatch excludes overlapping 64 KiB tracker blocks and invalidates collection memos. A new private allocation that fully replaces an excluded block restores tracking, stamps a new generation, removes old driver-store pieces and invalidates collection memos. Protection-only commits, shared remappings and partly replaced blocks retain the exclusion. A later import excludes the replacement again.

WindowsMappings::Commit reports only the private allocations it creates from placeholders, coalescing adjacent fresh spans. GuestArena notifies the tracker after the mapping mutex is released, avoiding the inverse lock order of write collection. Commit generations are recorded with exclusions so a delayed notification cannot re-enable tracking for an import made after that commit. Generation records are stored sparsely for excluded blocks. The VirtualAlloc2 documentation distinguishes placeholder replacement with a private allocation from committing already committed memory.

This addresses the address-reuse failure reported in the recorder executable without weakening AllocateWatched's assertion. The fresh-backing observer and Windows integration test still need a Windows build and hardware retest; this remains a draft.

The test target now links libc, configures Windows unwind and supplies the Windows runtime library path, correcting the reported MinGW __gxx_personality_v0 link failure.

Related: #1046 changes the Linux probe and includes the same recorder snapshot-reuse guard. #1310 overlaps the write-tracking tests. Neither is a dependency. The Linux probe is unchanged here. The alternative linked in the Windows report keeps imports watched and classifies an all-pages dirty sweep; this PR continues using byte comparisons instead of that heuristic.

TechnicalDebt records that snapshot reuse and unit shadows remain unavailable on excluded backing, that only complete private-backing replacement restores coverage, and that restoration on import retirement and comparison performance remain future work.

Tested

macOS arm64, AppleClang 17, C++20, rebased onto main 6b573862:

• agc_write_watch_coverage passes with -Wall -Wextra -Werror and AddressSanitizer/UndefinedBehaviorSanitizer. It covers exclusions, fresh-backing restoration, partial replacements, clipping, adjacent blocks, repeated imports and stale/same-generation mapping notifications.
• python3 tools/check_conventions.py --base origin/main and git diff origin/main...HEAD --check pass.
• Earlier portable CMake/Ninja target verification and tools/tests/test_progress.py passed. These are not full Windows build results.

A new Windows-only agc_write_tracking case uses the real arena and tracker: a protection-only commit retains the exclusion; resetting/releasing and allocating the same address with fresh private backing restores it, invalidates old stamps, detects a subsequent CPU edit and excludes a second import again. This case was not run locally.

Windows results reported by popcoder565, for main bc1fbe81 merged with the original bab05aaf and the suggested two linker lines: Windows 10 (19045), RTX 3060 Laptop, driver 616.92, MinGW-w64 GCC 15.2.0, Release.

• agc_driver_dcc_keys_first_write and agc_driver_srgb8_color_target: 20/20 each, versus approximately one-third and three-quarters failures on main.
• agc_write_watch_coverage and agc_write_tracking: pass.
• Full registered CTest suite: 419/419, two rounds.
• The separate agc_driver_recorder_tests executable failed 3/3 at AllocateWatched; main passed 10/10. Those observations predate this PR's fresh-backing follow-up and do not establish that the latest revision fixes the hardware regression.

Draft pending a supported Windows rebuild and reruns of agc_write_tracking, the two original GPU regressions, full CTest and the separately built agc_driver_recorder_tests executable. No full Linux build or title run is claimed locally.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
• No notes, investigation .md files or images added to the repository
• New third-party code is a submodule built from source (none added)
• Generic behaviour, not specific to one title
• Depends on: none
• AI-assisted: yes (OpenAI Codex)

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
#1310	core/libs/prx/libSceAgcDriver/tests/WriteTracking.cpp L119, L129

[popcoder565]

Ran this on Windows, which the description says couldn't be run: Windows 10 (19045), RTX 3060 Laptop (driver 616.92), MinGW-w64 GCC 15.2.0, Release, main bc1fbe8 merged with bab05aa.

1. The build fails at agc_write_watch_coverage_tests: undefined reference to __gxx_personality_v0. The target needs what the other test targets have:

   target_link_libraries(agc_write_watch_coverage_tests PRIVATE libc)
   configure_windows_unwind(agc_write_watch_coverage_tests)

2. With those two lines:
   • agc_driver_dcc_keys_first_write and agc_driver_srgb8_color_target pass 20 of 20 runs each. On main they fail in about one run in three and three in four.
   • agc_write_watch_coverage and agc_write_tracking pass.
   • Full ctest passes 419 of 419, two rounds.
3. agc_driver_recorder_tests, which is not in ctest but which fix(agc): compare host imports when device writes reach the write watch #1046 registers, fails in 3 of 3 runs with the watched block is not watched. On main it passes 10 of 10. The check is AllocateWatched (tests/Recorder.cpp:62). A new arena block at an address an earlier import excluded stays unwatched, which is the trade-off this PR's TechnicalDebt entry describes.

For comparison, I tried keeping imports watched instead: popcoder565/AnyPS5@fix/agc-import-pin-marking. The driver's second marking covers every page of the import about 1-2 ms after the window, so ImportWatched remembers the range. The first walk that finds all of its pages written stamps them as ImportWindow and forgets it, re-probing for up to 4 ms when it sees a marking in progress.

• The two tests pass 80 of 80 and 40 of 40 runs, and full ctest passes 417 of 417 in three rounds.
• agc_driver_recorder_tests keeps passing.
• agc_write_tracking gets a GPU-free check of the rule.

I haven't opened it as a PR, since this one already targets #1434. Whichever approach the maintainers prefer is fine.

[gunerky]

Independent Windows retest of the fresh-private-backing follow-up.

Tested PR head 2b2636a0e75b9cbc217639d5d2727995b0d8fe2f, with matching main 6b57386237a652f11ecdd2c420f43376adb972e6 in separate, unmodified detached worktrees.

These results cover the two pinned revisions above. Newer main, rebased revisions and the eventual merged tree have not been tested.

Windows 11 Pro (26300), desktop RTX 3060, driver 617.42; WinLibs MinGW-w64 GCC 15.2.0 UCRT/posix/SEH, Ninja Release, BUILD_TESTING=ON, SPIRV-Tools=OFF. Vulkan was required, the physical RTX 3060 was selected, and shader caching was disabled. No import-policy override, artificial delay, game input or driver change.

Normal Windows host results, with workspace-local TEMP/TMP:

Test	Main	PR
agc_driver_dcc_keys_first_write	30/30	30/30
agc_driver_srgb8_color_target	30/30	30/30
agc_write_tracking	30/30	30/30
agc_write_watch_coverage	Not present	30/30
Separate agc_driver_recorder_tests	10/10	10/10
Full registered CTest	Not run	420/420, 0 skipped

The PR's default build, patched libs target, local conventions check and diff whitespace check passed without a local source correction. The Windows private-address-reuse case executed in all 30 write-tracking runs, including protection-only exclusion, fresh-backing restoration, stale-generation invalidation, a later CPU edit and re-import.

The recorder is still unregistered in CTest, so it was explicitly built and run. No the watched block is not watched assertion occurred. Draw snapshot reuse and unit shadows are explicitly not tested under the comparison policy; watched storage refresh runs through its unwatched path. The import-policy assertion passed. The range-leaving-watch and import-window cases remain Linux-only.

Main's intermittent GPU failure did not reproduce here. These results therefore corroborate the latest Windows build, fresh-backing integration and recorder follow-up, but do not establish a before/after driver fix on this configuration. A rerun on the original Windows 10 / RTX 3060 Laptop / 616.92 configuration remains useful. No performance or game-compatibility conclusion is claimed.

The initial restricted AppContainer full run had 16 failures; its disk-cache write failure also occurred on main and disappeared with workspace TEMP/TMP. All 420 passed in the normal host run. Initial restricted failures are retained locally and are not claimed as PR regressions.

AI-assisted: OpenAI Codex assisted with preparation, execution and analysis of this validation and report. No production code change was authored.

[ArjenS-LA]

Independent Windows retest of the fresh-private-backing follow-up.

Tested PR head 2b2636a0e75b9cbc217639d5d2727995b0d8fe2f in an isolated detached worktree, with main 7fccb7ff5ff7043c044b7c820cbb63a4bd4c48f6 as the baseline. Both source checkouts were unmodified. The baseline is newer than the PR’s base, 6b57386237a652f11ecdd2c420f43376adb972e6, so this is not an exact parent-versus-PR comparison.

These results cover those pinned revisions. Newer revisions and the eventual merged tree have not been tested.

Windows 11 Enterprise (10.0.26200), NVIDIA GeForce RTX 3050 Ti Laptop GPU, driver 591.55; WinLibs MinGW-w64 GCC 15.2.0 UCRT/posix/SEH r2, Ninja Release, BUILD_TESTING=ON, SPIRV-Tools=OFF. The project documents WinLibs r7; this run used the installed r2 toolchain.

The targeted execution tests selected NVIDIA, required Vulkan, and disabled shader caching. No import-policy override, artificial delay, game input, or driver change was used.

Dedicated validation-harness results, excluding additional CTest runs:

Test	Main	PR
agc_driver_dcc_keys_first_write	20/20	30/30
agc_driver_srgb8_color_target	20/20	30/30
agc_write_tracking	1/1	1/1
agc_write_watch_coverage	Not present	1/1
Separate agc_driver_recorder_tests	3/3	10/10
Full registered CTest	Not run	420/420, 0 skipped

The PR’s full build, patched libs target, conventions check, and diff whitespace check passed without a local source correction. Both tracking tests also passed in a separate targeted CTest run and in the full suite. The Windows private-address-reuse case passed, covering protection-only exclusion, fresh-backing restoration, old-generation invalidation, a subsequent CPU edit, and re-import.

The recorder remains unregistered in CTest, so it was explicitly built and run. No the watched block is not watched assertion occurred. Draw snapshot reuse and unit shadows are explicitly not tested under the comparison policy; watched storage refresh runs through its unwatched path. The Windows import-policy assertion passed.

Main’s intermittent GPU failure did not reproduce here. These results therefore support the latest Windows build, fresh-backing integration, and recorder follow-up, but do not establish a before/after driver fix on this configuration. A rerun on the original Windows 10 / RTX 3060 Laptop / 616.92 configuration remains useful. No performance or game-compatibility conclusion is claimed.

Initial sandbox build attempts encountered Git ownership/dependency-check problems and a stalled Ninja process. The build completed after resuming outside the sandbox, and the full CTest run passed in the normal Windows host environment. Those build-environment problems are not claimed as PR regressions.

AI-assisted: OpenAI Codex assisted with preparation, execution, and analysis of this validation and report. No production code change was authored.

[adriadam10]

Checked on Linux: build and full ctest (459/459) on top of current main; agc_write_tracking and agc_write_watch_coverage pass. The Windows-only reuse case can't run here, so I'm relying on the Windows runs above (popcoder565, gunerky, ArjenS-LA: both GPU regressions, agc_write_tracking, the separately built recorder and full ctest pass on the current head).

Added one commit of ours:

• Moved the TechnicalDebt line from "Silent stubs" (end of section) to "Functional": it records a behaviour gap, not a silent stub.

Notes, not blocking:

• tests/Recorder.cpp:1175: the new Watched() guard also skips the snapshot-reuse test on Linux when imports end up unwatched, where it used to fail. Fine while the recorder isn't in ctest, worth remembering when fix(agc): compare host imports when device writes reach the write watch #1046 registers it.

Merging once CI is green.

[Celegans12]

I ran Ghost of Yōtei (PPSA26344 v1.008) on plain main with this PR to check a start-up abort we hit in our own tree. Plain main doesn't abort, but both what it shows and our abort come from the gap your TechnicalDebt line already mentions: once a host import is created on Windows, its 64 KiB blocks leave write tracking for the rest of the process.

Yōtei creates its Vulkan device twice. On main 4b005b2 the 3.3 GiB range at 0x5000000000 is imported on the first device, and when the device is recreated at swapchain creation (27 s) main imports it again in one piece, so imports go from 4,932 MiB back to 4,935 MiB and nothing needs a mirror. Our tree carries #290's 64 MiB import windows and a heap-mirror change of ours that serves such a range from a heap mirror until it can be imported whole. There the range wasn't imported again after the recreation (imports fell from 4,933 MiB to 206 MiB), no heap mirror could be made because the range was no longer watched, and the next address-based build fell back to copying all 3,380 MiB and stopped at the 64 MiB APS5_ADDRESS_COPY_MAX_MIB limit. So that abort is ours to deal with, though I'd expect the same on main whenever a large range that was imported once later has its import refused and needs a mirror.

What plain main does show is the cost of the comparison. Storage images inside imported ranges get no write stamps, so every refresh uploads them again: about 20 GiB of "untracked" uploads per 10 s, and 42 presents in 240 s. With only the Windows part reverted (unwatchLocked a no-op, unwatchImports = false), the same build had no untracked uploads and 188 presents. Do you have a plan for putting the watch back when an import is retired, or for keeping stamps on imported storage images? I'm happy to test a change against this case.

Details

Windows 11, RTX 3080 Laptop, the same post-gamma save, APS5_HOST_IMPORT_MIB=6144. Plain main 4b005b2 (which has this PR) stops at its first present on a descriptor read that an open PR fixes, so these runs add one local env-gated probe that skips a dispatch that throws (APS5_FMT_PROBE_TOLERATE=2); 240 s each, no FATAL.

[gpu] host import of 0x5000000000+0xd3400000 ok (1 live, 3380.0 MiB live, 3380.0 MiB ever)    first device
[gpu] host import of 0x7ff708b56000+0x15000 ok (1013 live, 4932.5 MiB live, 4932.5 MiB ever)    last before the recreation
[gpu] host import of 0x5000000000+0xd3400000 ok (1 live, 3380.0 MiB live, 8312.5 MiB ever)    first on the new device
[gpu] host import of 0x8001800000+0x200000 ok (1014 live, 4934.5 MiB live, 9867.0 MiB ever)
[buffers] image mirrors: 8 ranges (29.8 MiB, 0 writable, 0 heap 0.0 MiB), 214 address-based builds served, ... unwatched heaps 0
[storage] uploads by path (count/MiB, 10 s): clear 28/739.8 direct 1784/21609.3 ...; by reason: keys 12/272.9 untracked 1800/22076.2

With the revert: 284 direct storage uploads (2,594 MiB) per 10 s on average against 1,659 (19,676 MiB), and 0.98 against 0.21 fps over the last 100 s. The revert run came second and started with the 104 shader cache entries the first run wrote (27 for the first); both created 21 pipelines.

Our tree (integration on main 4b005b2):

FATAL: an address-based build copies 3380 MiB of 1 registered ranges, past 64 MiB (APS5_ADDRESS_COPY_MAX_MIB); the largest: 0x5000000000+0xd3400000 (3380.0 MiB committed, outside the write-watched arena)
FATAL: abort() (SIGABRT) on thread 23628

The run before this PR mirrored the same range and later released the mirror for an import. With the revert, a 1,200 s capture pass and a 3,600 s run in our tree had no FATAL and 0 unwatched heaps.

[JAB-dev]

On Windows this change costs PPSA22520 (Animal Well) most of its frame rate: about 59 FPS before it, 7.5 FPS after (Windows 11, RTX 4070 SUPER). A git bisect --first-parent names this merge (7a047bc) as the first bad commit; its parent fe9beae is good. With every import compared, [rescache] revalidate goes from 0 to 10,725 full walks per 10 s, 6.8 s of every 10 s.

#1847 keeps imports watched and tells the late driver marks apart from CPU stores by comparing a copy of each import's bytes for a settle period, so #1434 stays fixed. @ToyuAU @adriadam10, a review would be welcome.

[ps54all]

Regression report for PPSA12544 (Hollow Knight: Silksong) on Windows after this change landed (838d02a8), with a measurement of what it costs.

Setup: Windows 11, MinGW GCC 15.2 Release, NVIDIA RTX 3080 (driver reports Vulkan 1.4), origin/main 4b6ab6bb plus #846 for the bring-up of this title. Same converted game, same build, only the import-watch decision differs.

Before (host imports stay write-watched): the title reaches its main menu at 59.94 FPS (window-title counter, 2848 frames at the 60 s mark). The player also played past the menu into gameplay on the pre-#1441 code at well over 30 FPS (reported by the user; I only measured the menu).

After (host imports compared, as merged): the same menu runs at 9.9 to 10.3 FPS (655 frames at 66 s, 733 at 75 s, warm shader cache). APS5_PROFILE_DRAW=1 over a 10 s window shows:

• [storage] uploads by path: direct 6929 to 8191 uploads, 25.4 to 25.9 GiB; by reason untracked 7214 to 8461 uploads, 27 to 28 GiB. Every refresh of a resident target counts as untracked and re-uploads it.
• [dcc]: 49871 scans / 1017 MiB scanned per 10 s, key proofs 42514 of 42515 unstable.
• [resources]: image preparation 8034 ms of 8000 builds, [recorder] 3452 syncs waited 12.1 s for the GPU.

Experiment: I locally restored the previous behaviour on Windows with an environment switch in decideImportWatch (state.unwatchImports = false when set), nothing else changed. Same title, same machine: 59.94 FPS, [storage] uploads by path drops to clear 1154 / 5.4 GiB, direct 0.

So the compare path is correct for the late-driver-write case in #1434, but on this GPU it makes every Silksong frame re-upload its render targets. Two questions for the author and maintainers:

1. Is the late-write behaviour from AGC: on Windows (NVIDIA) a new host import's pages are marked written after ImportWatched's window, so write-backs drop GPU results #1434 specific to some drivers or GPUs? On Linux decideImportWatch probes it (ProbeImportWriteProtection) and compares only when the probe says writes arrive after the import. A Windows probe, or an opt-out, would keep the fix where it is needed and the speed where it is not.
2. If the comparison must stay, can a target be skipped when no CPU store was observed after the last GPU write (the watched-page path), instead of re-uploading the whole image on every refresh?

I can test any change on this title and report the numbers. I have not changed upstream code for this.

AI assistance: this report was prepared with Claude Code; all numbers above are from my own runs.

[JAB-dev]

@ps54all @Celegans12 my PR #1847 does what you're both describing: imports stay write-watched on Windows, and only the driver's late marks right after an import are checked against a copy of the import's bytes, so #1434 stays fixed. APS5_WRITE_WATCH_IMPORTS=unwatch keeps today's comparisons and =watch plain watching. On Animal Well it brings back 59 FPS, and TheAggie measured ccleste going from 15.5 back to 29.2 FPS. Since you both offered, it'd be great to see how Silksong and Ghost of Yōtei do with it.

[ps54all]

Follow-up: #1847 was closed, and #2096 (merged 10-09) fixes this for PPSA12544. Same machine (Windows 11, RTX 3080), main 6cc33583 plus a local integration branch, no import-watch override: the main menu holds 59.94 FPS again (4176 frames at 90 s), against 9.9-10.3 FPS before. Thanks for the quick turnaround.

Written with AI assistance.