Repository navigation
fix(libc): track writes to singly mapped direct memory without faults on Windows - #5596
Conversation
… on Windows A direct memory page mapped at one guest address now lives in private MEM_WRITE_WATCH memory at that address, and its writes are collected with GetWriteWatch like flexible memory: no fault is raised. A page mapped at a second address moves its whole resident run into the section, behind a read-only fence while it is copied, and every address becomes a view of it, tracked by write protection. Unmapped resident pages that are not blank are kept in a private reservation until they are mapped again, not in the section. Ported onto main's 4 KiB subpage tracking of shared views: resident runs keep a per-page collected bit and enter the fresh ranges, so Fresh and Armed collections select them like shared views; a demoted run carries its collected bits into the views and reports every page once. MapAlias throws through refuse() when a resident run extends outside the aliased range, and the caller keeps main's fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V1W4jdNSKTV7xwuS6xQPJV
Map made every singly mapped section page private resident memory. A JIT or shm_open section is file backed and committed: its bytes live in the file, and its alias descriptors map duplicated handles, which the physical page slots do not match, so a resident copy diverged from the file, and committing its view on a demote failed with access denied (guest_jit_shared_memory). Only direct memory backings, mapped through the new GuestArenaMapDirect, now become resident; other sections keep main's shared views. Shared pages are committed only while their section page is reserved, as on main. The socket host write tests map their direct memory a second time, so the write-protected shared path they check is still the one they exercise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V1W4jdNSKTV7xwuS6xQPJV
CheckBreakingAResidentRunKeepsConcurrentAccesses and CheckRemappedDirectMemoryIsChargedOnce passed on main, where a singly mapped direct memory page is a shared view and no resident run exists. Both now require their first mapping to be private committed memory, so on main they fail there instead of passing without exercising a run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V1W4jdNSKTV7xwuS6xQPJV
The entry links WindowsMappings.hpp, so the debt-file convention puts it in docs/dev/debt/libc.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V1W4jdNSKTV7xwuS6xQPJV
|
Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.
|
|
@tibzejoker thanks, this covers the 10-11 points. Since it changes the Windows memory model, could you run GTA V (PPSA04264) on this head alone on top of |
|
@oneandonlydean GTA V (PPSA04264) on this head alone. Windows 11, i9-13900K, RTX 4090 (NVIDIA 616.92), MinGW-w64 GCC 15.2.0, Release. Write watch on, no other PR and no local change, one run each:
So, on its own, this head removes the write-watch crash and nothing new shows up before the next known stop. With #448 (fdfawef324's port) on top, the Story prologue ran for 150 s with correct rendering, as posted in #5024. AI-assisted (Claude Code). |
…range leaves the write watch On Windows, singly mapped direct memory is private since #5596, so its host import is a direct import that leaves the write watch instead of a shared alias that stays watched. A storage image whose results were pending when its memory was imported had no saved guest bytes, and storing them threw. GuestMemory now calls a hook before Unwatch and before an ImportWatched that does not stay watched. Every image with results pending over the range saves the guest bytes of its pending units that have none, then collects and records the pieces the keep rule leaves to the CPU for the writes the watch saw. The next store and Refresh leave those pieces to the CPU and compare the rest with the saved bytes. A map-time import made without the GPU lock is deferred to the locked retry when pending results overlap the range.
|
A correction to my run above: "removes the write-watch crash" was based on one run per tree. Since then I've run GTA more often with this change in (main ee9037d + #757 and the other Story PRs, write watch on). In 1 run of 6, the same crash came back 15 s into Story: So this change makes the crash rare in GTA, but it doesn't remove it. One possible reason is that only direct memory goes resident, so writes to other shared views can still fault. I haven't traced that one crash, so I can't tell whether it was red-zone damage or something else at the same site. AI-assisted (Claude Code). |
|
Resident Evil Village (PPSA01556, 01.210.000) aborts at boot since this PR, 19 s in and before any dialog: RTX 3070 Laptop GPU (8 GB), driver 617.42, Windows 11 Home 10.0.26300, Ryzen 9 5900HS, 15.4 GB RAM. That range is a 419 MiB direct-memory heap mapped once. Before this PR it was shared views, served by a heap mirror, and Village booted to gameplay on our older branch. With this PR it's resident private memory. The address-based build's 64 MiB window import leaves the write watch for private memory, and the heap mirror refuses ranges with unwatched blocks, so the whole range would be copied on each build, past the 64 MiB guard. Raising the guard ( #5688 fixes it: heap mirrors keep such ranges and refresh the blocks outside the write watch. With it the abort is gone and the range is host-imported: |
…indows Since boykopovar#5596 a direct memory page mapped at one guest address is private write-watched memory. The AGC driver imported such ranges as plain read-write memory, without an alias, and Windows host imports without an alias leave guest write tracking: every use of the range was compared and refilled on the CPU. A range that holds resident direct memory now takes the alias path: MapAlias moves every resident run the range touches into the section (as a second mapping does) instead of refusing runs that extend past the range, and the import stays write-tracked through the guest views. Direct memory the GPU never imports keeps fault-free write watching. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…indows Since boykopovar#5596 a direct memory page mapped at one guest address is private write-watched memory. The AGC driver imported such ranges as plain read-write memory, without an alias, and Windows host imports without an alias leave guest write tracking: every use of the range was compared and refilled on the CPU. A range that holds resident direct memory now takes the alias path: MapAlias moves every resident run the range touches into the section (as a second mapping does) instead of refusing runs that extend past the range, and the import stays write-tracked through the guest views. Direct memory the GPU never imports keeps fault-free write watching. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…indows Since boykopovar#5596 a direct memory page mapped at one guest address is private write-watched memory. The AGC driver imported such ranges as plain read-write memory, without an alias, and Windows host imports without an alias leave guest write tracking: every use of the range was compared and refilled on the CPU. A range that holds resident direct memory now takes the alias path: MapAlias moves every resident run the range touches into the section (as a second mapping does) instead of refusing runs that extend past the range, and the import stays write-tracked through the guest views. Direct memory the GPU never imports keeps fault-free write watching. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Linux widens a collect of at most one 64 KiB block to the whole block within an epoch (boykopovar#1310), so the memo serves the block's other ranges. Windows kept the requested range because it was not tested there. On Windows with boykopovar#5596's resident direct memory every walk is a GetWriteWatch call, and ASTRO BOT's hub walks about 50k ranges of at most 64 KiB per 10 s on queue 0. Windows coverage does not know which pages of a block are committed or inaccessible, and a walk refuses such a range before it collects anything. So a widened walk that fails is retried over the requested pages; without that, a collect beside uncommitted pages of its block fails. agc_driver_image_table_reuse expected one more table resolution on Windows than on Linux, from the narrower collects; it now expects Linux's count on both. The new uncommitted-pages test resets its range before releasing it, so the arena does not hand the next test an address inside memory that is still committed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
4 participants
What
Port of @DotDebian's #637 onto current
main, as asked on #637. DotDebian stays the author of the main commit, and our changes are separate commits.On Windows, AnyPS5 tracked CPU writes to direct memory by arming its section views read-only and taking a write fault. The exception frame Windows builds for such a fault, with live AVX state, overwrites the System V red zone below RSP, so guest leaf functions that keep values there read back garbage (#361). GTA V's zlib
inflate_fastwas one of them.With this change, a direct memory page mapped at a single guest address is private
MEM_WRITE_WATCHmemory at that address, and its writes are collected withGetWriteWatch, like flexible memory: no fault. A page mapped at a second address moves its whole resident run into the section behind a read-only fence while it is copied, and every address then becomes a view, tracked by write protection as before.Commits:
fix(libc): track writes to singly mapped direct memory without faults on Windows(author DotDebian): fix(libc): track writes to singly mapped direct memory without faults on Windows #637's two commits squashed and merged by hand ontomain's 4 KiB subpage tracking of shared views. Resident runs keep a collected bit per page and enter the fresh ranges, soFreshandArmedcollections select them like shared views. A demoted run hands its collected bits to the new views and reports every page once. The fixes from the 2026-10-07 review are kept: fenced copies in demote and evict, demoting the whole run, unmapped pages kept out of the section, and thewatchResidentguard inCollect.fix(libc): keep resident mappings to direct memory backings on Windows: only direct memory goes resident, throughGuestArenaMapDirect. Other single-mapped sections (file-backed JIT and shm memory, whose aliases use duplicated handles) keepmain's shared-view path; making them resident brokeguest_jit_shared_memory.test(libc): require resident direct memory in the resident run tests:CheckBreakingAResidentRunKeepsConcurrentAccessesandCheckRemappedDirectMemoryIsChargedOncenow check that the first mapping isMEM_PRIVATE, so they fail onmaininstead of passing there.docs(libc): move the Windows direct memory debt entry to libc.md.The 2026-10-11 review points:
MapAliasthrows throughrefuse(...)when a resident run extends outside the aliased range.importAllocationkeepsmain's caller and fallback, with noentry.alias == nullptrbranch.tests/Recorder.cppkeepsmain's switches and adds--host-import-second-mapping-only.mainnow fail there, as above.Tested
Windows 11, i9-13900K, RTX 4090, MinGW-w64 GCC 15.2.0 (WinLibs r7), Release,
BUILD_TESTING=ON, onmaince5c839.guest_memory: passes (ctest -R guest_memory, 2/2). Onmain, the resident run tests fail at the newMEM_PRIVATEcheck, and fix(libc): track writes to singly mapped direct memory without faults on Windows #637's owng_accessViolations == 0check fails as well.ctest: 1112/1112 pass.tools/check_conventions.py --base upstream/mainpasses.mainwithout this change it crashes in the RAGE main thread right after the save loads (access violation at eboot 0x28aacee). With it, the prologue ran for 150 s with correct rendering, onmain8d98e88 plus the other open PRs GTA needs to reach Story (feat(agc): storage image access to depth planes through 2D array views #5003, feat(agc): single-sample sample masks as a per-quad-pixel discard #448's port, feat(agc): sample depth plane arrays from a later base slice #1186, fix(agc): CB resolve into any tiled color target #1189).Not tested on Linux: the change is in the Windows mapping code.
Checklist
main; no other open PR implements the same functions (supersedes fix(libc): track writes to singly mapped direct memory without faults on Windows #637; fix(libkernel): map direct memory as write-watched private pages until it is aliased #2054 overlaps and is stale)NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)🤖 Generated with Claude Code
https://claude.ai/code/session_01V1W4jdNSKTV7xwuS6xQPJV