fix(agc-driver): keep a live-backed snapshot mismatch from failing the dispatch

[ee6aba54]

What

GuestBufferMemory::AddSnapshot compares a captured word with the registered memory that backs it and throws guest snapshot differs from registered memory on a mismatch. For a region backed by a live host import or mirror the mismatch only means the capture raced a guest write: the GPU reads the live bytes either way, as the hardware would, so the dispatch is sound.

A mismatch on a live-backed region is now counted and reported at most every 10 s with the snapshot, the region and the first differing offset, and the dispatch continues. APS5_SNAPSHOT_CHECK_FATAL=1 restores the throw. A snapshot-backed region still fails, and its message now names the snapshot, the region and the first differing offset too.

Tested

• Ubuntu 24.04, Linux 7.0, GCC 13.3, Release, based on 0aa4cb1c2. agc_driver_graphics and agc_driver_bda_device pass. No new test: the race needs a guest write between the capture and the compare.
• PPSA15304 (engine: Unreal Engine 5): on main the throw ended the title 65 to 91 s after the title screen in three runs (a 4 byte snapshot of a live import). With the change the mismatches are rare (7 in a 15 minute run through menu and cutscene, each reported) and the runs reached the first cutscene with the expected image.
• Windows not built or tested.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Unblocks: PPSA15304 (abort shortly after the title screen)
• GPUs tested: NVIDIA GeForce RTX 5080, driver 595.99.02, Ubuntu 24.04
• AI-assisted: yes

🤖 Generated with Claude Code

[fisherjoey]

Demon's Souls (PPSA01341 v1.07) hits the same throw, guest snapshot differs from registered memory (GuestBufferMemory.cpp:2769 on main 6546a33), during the tutorial level load. It killed 2 of 4 level loads here (Linux, RTX 5070 Ti, NVIDIA 595.91). My build is main plus #758, #5181, ThatsLeo's perf/shader-spirv-on-disk and a local stencil-export workaround, because the title doesn't reach this dispatch on main alone. The check and the window it guards are the same on main.

What changes the bytes, from a run with every mismatch logged:

• The title's CPU job threads refill per-frame tables while a compute capture is in flight. 6678 of 6679 mismatches were CPU writes made after the capture started. None had a pending GPU write or a label behind it.
• In the fatal case, queue 0x50 dispatches a ~318k-word shader. The capture is fresh, inline, and 15.4 ms old at the check, with no lock wait. 23 whole pages of a 120 MiB writable host import (0x39f300000) differ, by 13 to 223 of 1024 dwords per page. Both versions look valid (pointer/size pairs, matrices), so this is a title refilling its tables, not corruption.
• The window is between ShaderMemory::Capture (Dispatch.cpp:255) and ShaderResources::Complete() under the GPU mutex.
• One case in 6679 was a GPU write: a region read raw on write evidence before a queue-0 write landed.

This PR stops the throw, but the dispatch then reads the new bytes. On hardware the dispatch reads memory when it runs, and the capture is what stands for that moment here. Could the regions that differ be bound from the captured bytes instead? That keeps the capture's view and doesn't fail the dispatch. #1787 named gate-time snapshots as the missing piece for a related case.

I can test the PR on this stack and post the results here.

AI-assisted: yes

[ee6aba54]

Thanks for the trace, that matches what PPSA15304 does: its job threads refill per-frame tables while a capture is in flight, and the mismatch is the title's own later write, not corruption.

Binding the differing regions from the captured bytes is the right follow-up: it keeps the capture's view of the memory, which is what stands for the moment the dispatch runs on hardware. It needs a snapshot buffer per differing region and a path that prefers it over the import, so it is a change of its own; this PR only stops the dispatch from failing, as the title cannot continue past the throw at all. I would keep the two apart: this one first, the captured-bytes binding as a follow-up PR that can also take the gate-time snapshots #1787 asked for.

A run of this PR on your stack would help, yes. If the load survives, the one mismatch with a GPU write behind it (the region read raw on write evidence) is the case I would look at first, since that one is not covered by reading the new bytes.

[fisherjoey]

Run of this PR (67973cd) on the Demon's Souls stack from my comment above: main 6546a33 + #758 + #5181 + #4806 + #5747 + the local stencil workaround. Linux, RTX 5070 Ti, NVIDIA 595.91, warm cache. A local diagnostic on top logs each mismatch with its writer and keeps this PR's rule unchanged.

The level load survived. The level started at about t+155 to t+215 and ran for 240 to 300 s, until an NVIDIA Xid 31 at t+451 (MMU fault, copy engine CE0, PTE write). I saw the same Xid once before on a build without this PR, so I can't tie it to this change.

Mismatches: 11607 were logged and none were thrown. All of them were in group-adopted dispatches, none inline. The inline queue 0x50 case that killed my earlier runs didn't come up in this one.

writer	count
CPU write after the capture, no GPU writer	8586
no CPU write seen, no noted writer (mostly region 0x3d0800000+0x7800000, queues 0x20/0x50; not classified yet)	2991
noted GPU writer	30

The GPU-write case you asked about occurred 12 times, in four bursts between t+215 and t+397:

• Readers: queue 0x50 shaders 0x249250100, 0x24e361e00 and 0x24e36d500, all group-adopted.
• Region: 0x24ed7fd70+0x10 in the writable host import at 0x200300000. Capture age 48 to 75 ms, nothing pending at the check, no CPU write.
• Writer: queue-0 dispatch 0x248881200, which writes 0x24ed7fcb0+0x2d0. Write-evidence streak 4 to 6, changed 0.
• Captured 00000004 00000005 00000006 00000007 (3 of the 12 had d9fe1a98 00007f9b 00000001 00000001). At the check, memory held 08007492 00fff000 0af00000 40000000, which looks like a buffer descriptor.
• After it, each dispatch went on with the live bytes. Those shaders logged no error and frames kept advancing. The last burst was more than 55 s before the Xid.

So here the raw read on evidence returned bytes from before queue 0's dispatch wrote the region. The capture holds the stale values, and the live region holds the descriptor the writer produced.

The other 18 writer-backed mismatches weren't raw reads on evidence (streak 0, the dwords were never sampled). They were queue 0x30/0x20 reads near 0x24ed7fdd0/0x24ed7fd60 with the same writer 0x248881200, and queue 0x50 shaders 0x2497f1300/0x2497f8300 over 0x39b904eb0 with writer shader 0x24afe2c00.

AI-assisted: yes

[ee6aba54]

Thanks for running it. The 12 writer-backed cases settle the question for this PR: the raw read on write evidence had sampled the region before queue 0's dispatch stored the descriptor there, so the capture held stale values and the live bytes were the right ones; the dispatches that went on with them raised nothing. Reading the live bytes is what the hardware would have seen, since the writer ran before these readers in the title's own order.

That also narrows the follow-up: binding from the captured bytes is only right when the capture is newer than every writer of the region, which needs the writer's completion to be part of the decision; a snapshot taken before a pending GPU writer landed must not win over the import. I would build the follow-up around that check rather than preferring the capture as such.

The Xid 31 is a separate thing (PTE write on the copy engine); if it repeats on main without this PR, it deserves its own issue with the kernel line.

AI-assisted: yes

[simonbronk]

Windows run with Onimusha: Way of the Sword (PPSA27836, RE Engine), where current main stops on this message.

Setup: Windows 11 Pro 26200, WinLibs GCC 15.2.0 r7, Release, RTX 5070 (driver 617.42). main ed835aa plus the PRs the title still needs (#5769), with and without this PR.

• Without this PR, two runs stopped at presentation: [videoout] presentation failed: AGC graphics: guest snapshot differs from registered memory. One run had perf(agc): import resident direct memory through a section alias on Windows #5851 and one didn't, so that PR isn't the cause.
• With this PR (head 67973cd89), the same point logs one mismatch and the run continues:
  guest snapshot differs from registered memory: snapshot 0x2112c00280+0x20 (live region 0x2112c00000+0x10000, writable, host-backed) differs first at +0x0; the live region serves the GPU (1 mismatches so far; ...)
  • Over 20 minutes there was only that one mismatch. The title then compiled about 6,000 new shaders, which it never reached before.
• A later run without this PR (it also left out fix(agc): skip unused pixel stages with retained export formats #5631) went through 20 minutes without the mismatch, so the race doesn't happen on every run.

The PR merges cleanly into main ed835aa and builds on Windows. The title still shows a black screen after this point; that is a separate problem, and I'm following it on #5769.

[adriadam10]

Checked the diff (only AddSnapshot changes; live-backed mismatches are counted and reported at most every 10 s, snapshot-backed regions and APS5_SNAPSHOT_CHECK_FATAL=1 still throw, with the region and first differing offset in the message), the scoped build and agc_driver_* ctest on the PR merged onto main (the only failure, agc_driver_tessellation SEGFAULT, fails the same without this change), and the runs reported by @fisherjoey (Linux, Demon's Souls) and @simonbronk (Windows, Onimusha). Merging once the full Build is green.