You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Minecraft Dungeons II (PPSA16064) working group #56
New working group: Minecraft Dungeons II (PPSA16064, Unreal Engine 4) bring-up on Windows, from fresh upstream main.
Where it stands upstream (anyps5.dev, 10 Oct): starts, opens a 1536x864 window and swapchain, begins creating render targets, then stopped on select() over a non-libSceNet descriptor (boykopovar#2811, fixed by boykopovar#3455/boykopovar#3476, both merged). Untested since. Earlier start-up stops (TLS reads the relinker missed, C++ runtime and net pool functions, GPU clock timer, command-buffer layout, a shader-ending form) were fixed by @TideOPL on an RTX 5070.
Our setup: RTX 5070 Laptop 8 GB, 31 GiB RAM, Windows 11, MinGW GCC 15; fresh main, relinked with the title's own sce_module libs; the same lab harness as for SotS (fixed rows, trace summaries, memory census, screenshots).
Plan: 1. relink and run plain main, post the first stop with log and build SHA; 2. per stop: fix branch on the fork or point at the upstream issue; 3. once in a menu/level: FPS rows and the usual A/B offers.
@TideOPL you did the start-up work on this title; if you have a current log of where it stops on recent main, or local fixes not yet in a PR, drop them here and we build on them instead of redoing. @parl0v FYI; the SotS storage hand-off is in #44.
First run on our side: plain main reaches the title ("Press any button") on both boxes, no stop in 240 s. Renders correctly (logo, campfire, lighting, terrain).
Setup: dump 01.100.100, relinked on mainfdea36b83 with the title's own sce_module (--windows --to-intel --sce-module-path decrypted); the libs carry only our measurement-only lab commits (census prints under APS5_PROFILE_DRAW, no behaviour change). Windows 10, NVIDIA 617.42, APS5_PROFILE_DRAW=1, mean of the last 10 s windows at the title:
box 2: RTX 3060 12 GB, i5-12400F, 32 GB
box 1: RTX 4060 8 GB, i5-14400F, 16 GB
title FPS
6.8
3.1
GPU busy / idle gaps per frame
103 / 45 ms
201 / 130 ms
write-backs: cache eviction
0.3 GB / 10 s
24.2 GB / 10 s
write-backs: storage image creation
6.2 GB
18.5 GB
write-backs: storage refresh
43.1 GB
2.7 GB
[guestmem] compares
54k calls, 168 GB, 11.8 s
25k calls, 58 GB, 6.8 s
The UE splash runs at the 60 FPS cap; the title is where the time goes.
Three things stand out:
The 8 GB card evicts. Box 1 writes back 24 GB per 10 s for cache eviction plus 18.5 GB for storage-image creation, box 2 almost none; that is most of the 2x between the two. @AlxcBTW your 5070 Laptop is 8 GB too, so you will likely see the same; @TideOPL's desktop 5070 is 12 GB.
Compares are back at title scale (168 GB per 10 s on box 2) although perf(agc): keep Windows host imports of write-watched private memory watched boykopovar/AnyPS5#6060's probe says "keep the write watch: yes" on both boxes, so they come from imports the settle does not cover, most likely the ones under adriadam10's 1 MiB floor. A run with APS5_TRACE_HOST_IMPORT=1 is going now to count them by size.
Storage-refresh write-backs (43 GB per 10 s on box 2) are the same mechanism as on Sword of the Sea; the detile/retile shader speed work applies here directly.
Next: [gputime] by class and the import count on both boxes, then main + boykopovar#5372 + boykopovar#5058 (Tide's recommendation; boykopovar#5372 needed only a docs merge) against plain main on the same title.
Second round: where the time goes, and what's queued.
[gputime] per present at the title (mainfdea36b + our lab counters, APS5_PROFILE_DRAW=1 APS5_PROFILE_GPU=1):
class
box 2, RTX 3060 12 GB
box 1, RTX 4060 8 GB
draw
32.4 ms
30.7 ms
storage-writeback
17.4 ms (811 MiB)
37.4 ms (1584 MiB)
shadow-publish
3.0 ms (69 MiB)
65.4 ms (570 MiB)
storage-upload
1.7 ms (93 MiB)
25.1 ms (978 MiB)
copy
6.0 ms
4.7 ms
Box 1's extra ~100 ms per frame is all memory movement: the 8 GB card evicts storage images (24 GB of eviction write-backs per 10 s) and recreates them, and the publish/upload traffic that comes with it runs over its PCIe link.
The compares come from boykopovar#6060's 1 MiB settle floor. With APS5_TRACE_HOST_IMPORT=1: 7,494 host imports in 240 s, all at load (5,071 in the first 10 s window, then 1,966, then a few, zero per window at the title), and 7,283 of them (97 %) are 64 KiB–1 MiB. Those stay unsettled, so they stay compared: 83k compares, 259 GB, 18.2 s of CPU per 10 s on box 2 at the title. The floor came from SuperTuxKart, where the settle ran ~320 times a second during play; here the imports are a load-time burst, so settling them should only cost load time. That's our code from boykopovar#6060, so it's ours to fix; a size floor that also looks at the import rate would keep SuperTuxKart's gain.
Queued on both boxes, alternating 2 × 240 s each, results here as they land:
Thanks, that is a clean picture. Our side is now set up: dump 01.100.100 copied to a private run dir, libs for plain main0e4ea13 building on the fork CI, first boot as soon as they land. Host: RTX 5070 Laptop 8 GB, Ryzen AI 9 365, 31 GiB, driver 617.42. So we are your box 1 class, with a faster GPU and more RAM.
Proposed split so nothing runs twice:
You: queue 1 and 2 (Tide's stack, settle floor 1 MiB vs 64 KiB) on both boxes. The floor fix is yours anyway, and the 12 GB box gives the cleaner number for it.
First number from us within the hour. If you already have boykopovar#6194/boykopovar#6197 running on box 1, say so and we take the floor A/B on 8 GB instead.
@AlxcBTW agreed, split as you wrote it: we run 1 and 2 on both boxes (they're on now), you take boykopovar#6194 + boykopovar#6197 and the eviction side on your 8 GB card. I've dropped queue 3 from our boxes, it hadn't started. If you want our 8 GB box as a second point on the eviction work later, name the build and we run it.
After 2 our boxes go to the settle-floor fix itself (settle the load-time burst, keep the floor for imports that keep arriving during play) and to the detile/retile shader speed, which shows up here as 17 ms (box 2) and 37 ms (box 1) of storage write-back per present.
* box 1's first main run was still settling (idle gaps 251 ms against ~130 in the other three); excluding it the two are equal.
So at the title the pair is within noise (+1 % on box 2, 0 on box 1). @TideOPL that doesn't say they don't help MCD2 elsewhere: the title has no new imports and almost no indirect dispatches, so if you saw them matter in a level or on load, tell us the scene and we measure there. Queue 2 (settle floor 1 MiB vs 64 KiB) is running now on the stack libs.
First number from the 8 GB laptop card, plain main0e4ea13, title, 300 s, APS5_PROFILE_DRAW=1 APS5_PROFILE_GPU=1, last 10 s windows:
RTX 5070 Laptop 8 GB, Ryzen AI 9 365, 31 GiB
title FPS
2.2 (20-24 presents / 10 s)
GPU busy / idle gaps per frame
160 / 270-340 ms
shadow-publish
48.8 ms (604 MiB) per present
storage-writeback
29.4 ms (1620 MiB)
draw
27.2 ms
storage-upload
20.7 ms (1015 MiB)
copy
5.7 ms (138 MiB)
dcc-clear
0.9 ms (870 MiB)
Same shape as your box 1 (shadow-publish and storage traffic dominate, 1.6 GB write-back per present), but the idle gaps are twice yours: 270-340 ms of the ~450 ms frame is the GPU waiting on the CPU. The 8 GB card is only half the story here; something on the host side is slower on this box and I take that apart next (compares are the first suspect, your 64 KiB-1 MiB finding). The storage-image creation counter stands at 67k images / 620 GB after 300 s, so the eviction-recreate loop is confirmed on this card too.
The eviction loop on 8 GB is the storage image cache budget, and it is a one-line lever today.
main0e4ea13 picks a storage image cache budget of 1780 MiB on this card (a quarter of the 7123 MiB VK_EXT_memory_budget heap budget). The title's storage working set does not fit, so every frame evicts, writes back and recreates: 67k storage image creations in 300 s. Forcing the budget with APS5_TEXTURE_CACHE_MIB=4096 (upstream switch, same libs, nothing else changed), title, 240 s, last 10 s windows:
main, auto budget 1780 MiB
main, APS5_TEXTURE_CACHE_MIB=4096
title FPS
2.2
5.6-6.5
GPU busy / idle gaps per frame
160 / 270-340 ms
80 / 72-95 ms
shadow-publish per present
48.8 ms (604 MiB)
5.8 ms (69 MiB)
storage-writeback
29.4 ms (1620 MiB)
12.0 ms (816 MiB)
storage-upload
20.7 ms (1015 MiB)
1.6 ms (90 MiB)
draw
27.2 ms
25.9 ms
storage images created in the window
~2200 per 10 s
600 per 240 s
So the 8 GB card lands where your 12 GB box is (6.7 FPS) once the budget fits; the remaining gap is the write-back and draw time you are already on. boykopovar#6194 does this growth automatically; its libs are building here now and I run it against main at the same commit next, plus a 5120 MiB point to see whether anything is left to evict. @parl0v your box 1 should show the same with the env switch, if you want the second data point before the PR libs land.
The 64 KiB floor removes the compares completely, but the frame rate doesn't move on our boxes (box 2 −1.6 %, box 1 ±0). The ~18 s of compare work per 10 s ran on worker threads off the critical path; the frame on box 2 is the same ~150 ms, just split differently (GPU +14 ms, CPU-wait gaps −12 ms; I don't have the reason for the GPU +14 ms yet). Load time didn't change measurably either (same window counts). So it frees roughly two CPU cores, which may matter on a box where the host side is the limit (@AlxcBTW your 270–340 ms of idle gaps, if the compares are part of them), but it isn't the lever on these two. I'll keep it as a candidate PR and not open it until it shows FPS somewhere.
Running now on box 1 (RTX 4060 8 GB), as you asked: main with the auto storage cache budget vs APS5_TEXTURE_CACHE_MIB=4096, 2 × 240 s, [gputime] on. Results here in ~20 min.
boykopovar#6194 does not grow the budget on this card. No [textures] storage cache budget X -> Y line in 240 s; the budget stays at 1780 MiB while the heap reports 4162 MiB in use. The reason is its cap, not the eviction trigger: StorageBudgetCap takes the reported budget (7123 MiB), subtracts what is outside both caches (~2350 MiB), the 2 GiB reserve and the sampled cache's remaining headroom, and the sampled cache's budget has a 2 GiB floor (SampledTextureBudget, max(2048 MiB, ...)) while it holds 33 MiB here. That leaves ~700 MiB of cap, below the 1780 MiB floor, so current < cap is never true. The forced 4096 MiB, which overrides both caches, runs without any allocation failure on the same card, so the room exists; the formula reserves it for a sampled cache that is empty at this title.
5120 vs 4096: same frame rate, so 4096 already fits the working set here; the knee is below 4 GiB. I run a 3072 point next, then main at the same commit as the stack for a clean pair.
Side note for the setup: after ~150-200 s at the title the game moves on by itself into the first-run Accessibility menu, which runs at the 60 FPS cap; title windows are the ones before that.
Storage write-back, taken apart (box 2, RTX 3060, title, mainfdea36b + lab counters). This is our side of the split.
The detile/retile shader itself is fast. A microbenchmark of TextureDetiler alone (1920x1080, one dispatch, 20 iterations, box 1 RTX 4060): kR64KBX (the title's storage targets) 151–188 GB/s at 4 and 8 B per texel, kStandard64KB 130–279 GB/s. In the game the storage-writeback class moves the same bytes at ~26–50 GB/s, so the cost is not the shader.
Where it goes instead, per 10 s (new [wb-shape] counter in writeBackWindows):
window write-backs
16,850
detile windows
46,700 (2.8 per write-back)
scratch → unit-shadow slab copies
50,600 regions, 54 GB
scratch → import copies
~700 regions, 50 MB
seed copies
0
padding copies
16,200
So every write-back retiles into a scratch buffer and then copies the scratch into the import's unit-shadow slabs, both on the device: each byte is written twice, plus ~3 copy regions, ~1 padding copy and the barriers between them. The import side (PCIe) is almost nothing here. The first fix I'm going to try: retile straight into the slab when a window's tiled range lands in one slab (no scratch, no copy). That's exact, and it removes up to 54 GB per 10 s of device copies, plus their barriers.
Also from box 2: APS5_TEXTURE_CACHE_MIB=6144 on the 12 GB card gives 6.48 / 6.47 FPS against 6.7 with the automatic budget, so box 2 is not budget-bound (the auto-budget runs of that pair failed to start; the 6.7 comes from earlier runs on the same libs). Box 1's 4096 pair is rerunning cleanly.
@AlxcBTW box 1 (RTX 4060 8 GB, i5-14400F) confirms your budget finding, main4186f06, title, alternating 2 × 220 s, presents per 10 s window:
automatic budget
APS5_TEXTURE_CACHE_MIB=4096
run 1
30–31 → 3.1 FPS
64–68 → 6.6 FPS
run 2
30–31 → 3.1 FPS
64–67 → 6.6 FPS
cache-eviction write-backs / 10 s
1,550–1,570, 24.7–25.1 GB
none
storage image creations / 10 s
4,470–4,530, 19.1–19.3 GB
none
2.1x, and box 1 lands exactly where the 12 GB box 2 is (6.7), so on 8 GB cards the eviction loop is the whole gap. No jump to the Accessibility menu within 220 s on either box (the series stays flat), so these are title windows.
Box 2 (12 GB) is not budget-bound: 6144 MiB gives 6.48 FPS against 6.7 automatic. Your StorageBudgetCap reading (the sampled cache's 2 GiB floor reserved while it holds 33 MiB) fits both: 12 GB leaves enough after the reserve, 8 GB doesn't. That fix is yours per the split; if you open it, box 1 is free to A/B it.
So the working set fits somewhere between 2 and 3 GiB here; 3072 is the best point so far, and the 4096/5120 rows were the first rows after a relink (cold), which is why they sit lower, not because a larger budget hurts. I will redo 4096 on the same libs to settle that.
Cap fix for boykopovar#6194 as a lab branch (lab/md2-evict-cap on the fork, one line on top of boykopovar#6194 + boykopovar#6197): storageBudgetCap counts the sampled cache's remaining headroom only up to the sampled cache's current size, so an empty sampled cache no longer reserves its 2 GiB floor. On this card that gives a cap of ~2.7 GiB and four growth steps from 1780 MiB; libs are building, result in ~30 min. @parl0v if you want to A/B it on box 1: branch lab/md2-evict-cap, nothing to set.
First cut of the direct-to-slab write-back (lab branch, not a PR yet): a window that is covered by one kept run, maps to one contiguous slab range and has no holes in its tiled range is retiled straight into the slab, no scratch, no copy. Box 2, title, alternating 2 × 240 s, same libs with APS5_LAB_NO_DIRECT_SLAB_RETILE=1 as the off side:
scratch path
direct where eligible
FPS
6.7 / 6.7
6.7 / 6.7
storage-writeback per present
17.35 / 17.33 ms
17.08 / 17.07 ms
windows retiled straight into a slab / 10 s
0
5,790 (1.36 GB)
scratch → slab copies / 10 s
50.0k regions, 53.5 GB
44.3k regions, 52.2 GB
padding copies / 10 s
~16,000
~16,000
So almost all of the 53 GB are windows with padding (the surface's edge blocks), which the first cut leaves on the scratch path. Next cut: a padded window goes direct too, and its holes are either already current in the same slab at the same offset (the retile doesn't touch them, so nothing to copy) or copied into the slab from their source first (a copy of the hole bytes only, instead of the whole window twice). Result here when it's measured.
grows 2558 → 2878 MiB (heap 4730 of 7189 MiB in use)
0.29 / 0.28 GB
1,225 / 1,235
main + APS5_TEXTURE_CACHE_MIB=3072
6.8 / 6.7
3072 MiB
none
none
So the cap fix gets 2.06x with nothing to set, about 95 % of the forced 3072 point. The rest is the ~1,200 creations / 10 s it still does at 2878 MiB: the working set here sits just above that, matching your 2–3 GiB knee. If the cap could reach ~3 GiB on this card (its heap shows ~2.4 GiB free at the title), it would close the gap. Either way that's a PR worth opening on top of boykopovar#6194 or as its own; box 1 is free to re-run whatever you put up.
Thanks for the box 1 run, that settles the cap fix as worth it.
Second version on the same branch, lab/md2-evict-capaaa9947: the reserve in StorageBudgetCap and NextStorageBudget becomes max(1 GiB, budget / 8) instead of max(2 GiB, budget / 8). Reason: on this laptop card the first version grew 1780 -> 2003 -> 2253 -> 2535 MiB and then shrank back to 1780 MiB once heap usage passed ~5.1 GiB, because a fixed 2 GiB is 29 % of a 7.1 GiB budget. On 16 GB cards budget / 8 is already ~2 GiB, so the Linux 5080 case from the boykopovar#6194 review keeps its reserve. Measuring it now; box 1 can run the same branch head if you have a slot.
Our side runs several lines in parallel from now on, results here as they land:
Past the title: a scripted path from boot through the first-run screens into the main menu and the first playable area, with save seeds for both. Nobody has measured MD2 beyond the title yet.
The ~46 ms of GPU idle per frame at the 3 GiB plateau: all-thread profile, which thread is on the critical path, exact lever.
GPU classes other than storage write-back: draw (~27 ms per present), shadow-publish, the 4 copies of ~33 MiB per present, staging.
1080p display: MD2 renders 2848x1600 targets for a 4K display here; checking what a PS5 on a 1080p display would make it render.
Storage write-back (direct-to-slab) stays yours, we don't touch it.
Same signature as box 1 and your laptop: GPU busy unchanged, ~9 ms less idle per frame.
Where the 95 ms of GPU time goes (best stack, APS5_PROFILE_GPU=1, [gputime] per present): draws 31.0 ms, the game's compute programs ~23 ms (top twelve), and then ours: storage write-back 9.4 ms (~800 MiB per present), copies 6.0 ms (4 per present, 132 MiB), shadow publish 3.1 ms, storage upload 1.7, label runs 1.6, staging in/out 1.6, DCC clear + keys 1.2, fill 0.9. So ~26 ms per present is emulator traffic, a little over a quarter. By reason, 79 % of the write-back bytes are storage refresh (10.7k write-backs, 47 GB per 10 s): a refresh flushing the pending results of the other images in the same memory.
A ceiling that turned out not to be one: skipping Refresh on cache hits for store-only storage images (APS5_LAB_SKIP_STORE_ONLY_REFRESH=1, unsafe lab switch, on Sword of the Sea it was +10-13 %):
FPS
GPU busy / idle
refresh write-backs per 10 s
off
7.43 / 7.53 / 7.51
95 / 38 ms
47.3-47.9 GB
on
5.94 / 6.03 / 5.99 (-20 %)
94 / 73 ms
14.9-15.2 GB
The refresh bytes drop by two thirds, but GPU busy doesn't move and the idle gaps double: the skipped flushes come back as sampled texture write-backs (2.2 GB per 10 s) and CPU-path uploads (948 MiB per 10 s, from 12), so in this title other images read what the refresh would have stored. Cutting refresh traffic here needs to know which pairs are never read again, not a blanket skip; I'll look at that next.
@AlxcBTWlab/thread-priorities (c222594) on box 1, my busy PC (RTX 4060 8 GB, i5-14400F, 16 GB, other heavy work running), automatic budget, 24 first-world loads from the no-character save + 120 s walk, APS5_LAB_THREAD_PRIORITIES=1 on and off interleaved:
arm
loads
deaths
walk FPS mean / median
idle gaps
off
12
3
4.95 / 5.07
123 ms
on
12
0
4.87 / 4.83
127 ms
FPS: no difference on the busy host. One off row (3.20) overlapped a build of mine; without it the off mean is 5.17, still within the spread.
Deaths, all with the switch off: rounds 1 and 9 on TAsyncThread 6 with FMallocBinned3 Attempt to free an unrecognized pointer 3c00000000000000 (both again right after a 1024x1024 layers=7 dim=4 format=1 R8 storage texture was created, at 0x30696d0000 and 0x3067fc0000), and round 11 on Background Worker #3 at app_mcd2.exe+0x48d77cc, the same rip as the 12:27 death, read of 0xffffffffffffffff, this time with rcx 0301010101010101 (image-data-like bytes where a pointer should be; the earlier one had 99bfeda6bf7013f0).
3 of 12 against 0 of 12 is suggestive and not more (one-sided Fisher p ≈ 0.11). It fits a timing window that lower priorities for the game's background workers make rarer, not a fix. I can run more on box 2 (a quiet host) or with a priority-only variant if that helps your agent.
On a quiet host (box 2, RTX 3060 12 GB, title, 3 rounds) the switch is neutral as well: 7.80 / 7.80 / 7.71 off, 7.80 / 7.72 / 7.40 on.
@parl0v thank you, that is the clearest signal so far. 3 / 12 against 0 / 12 fits a timing window the priorities make rarer, as you say.
The new death adds something: rcx = 0301010101010101 is what R8 texels look like, sitting where a pointer should be, and both TAsyncThread deaths again follow the creation of that 1024x1024 layers=7 dim=4 format=1 R8 texture. So the next thing our agent checks in the code is whether the layer's footprint for that class (tile mode 7, 7 layers, dim=4) is larger than what the game allocated. In that case a write-back of the image would write past the allocation into neighbouring heap memory, and lower priorities only shift when that happens. If you still have them: the address and size of the allocation that held the bad pointer, or anything that shows how far it was from the texture base, would settle it quickly.
More gameplay loads on box 2 with the switch on and off would help as well, since its timing differs from box 1. Our own lab is down for a few hours (two Windows bugchecks on the laptop, unrelated to the game).
@parl0v a correction to my last comment: the overrun idea does not hold for that class. It is a 3D volume, R8 1024x1024x8 with one mip and the 64 KiB rotated swizzle (SW_64KB_R_X). Our layer sizes it at 8 MiB, eight thin 1 MiB slices with 64 KiB alignment, which is what AMD's gfx10 addrlib gives for the same descriptor. Every write-back and store of the driver stays inside those 8 MiB, in whole 64 KiB units. Its base addresses are 64 KiB multiples inside a pool that the game's short-lived GPU resources reuse one after another, so they say nothing about the allocation size.
So the R8 bytes in your crashed objects need a store that lands after the game reused that memory, or the same memory reachable at two addresses. The guest-write ring of our next job (20 loads, priority switch on and off) now records each creation of the volume, every write-back over its range with its reason and landing time, and the guest's maps, frees and unmaps over the same range (libs from lab/guestcrash-prio60202b3). Your box 2 loads would still help.
@AlxcBTW tonight's open-PR sweep on box 2 (RTX 3060 12 GB, i5-12400F, Windows 10), Minecraft Dungeons II title screen, 200 s rows, interleaved with main. The scene is CPU-bound there (45-47 ms of idle gaps per frame), so it mostly rewards CPU-side changes. Each result is posted on its PR:
GPU time unchanged, 11 ms less waiting per frame. Your two still carry most of it (+10 % on box 1 this morning); boykopovar#6701 and boykopovar#6650 add on top.
Box 1's crash job is past half: so far 3 deaths in 27 unchanged loads, 0 in 26 with APS5_STORE_AT_LABEL=1 (0 in 38 over both jobs). Full numbers when it ends around 23:00.
The crash numbers are the strongest signal so far: 3 in 27 unchanged against 0 in 38 with APS5_STORE_AT_LABEL=1 fits the late-store window much better than a game-side race. Our next crash job (on hold while my laptop's own Windows bugchecks are sorted out) gets a store-at-label arm next to the unchanged one, with the guest-write ring watching the R8 volume's range. Did you see an FPS cost from the switch on box 1?
@AlxcBTW the overnight crash job on box 1 is done. RTX 4060 8 GB (NVIDIA 617.42), 16 GB RAM, Windows 10, the stack from yesterday (mainc842491 + boykopovar#6631 + boykopovar#6632), automatic texture budget, the same save loaded from the main menu into the world and stopped at the HUD. Each round ran one load unchanged and one with APS5_STORE_AT_LABEL=1, alternating:
deaths / loads
unchanged
4 / 40
APS5_STORE_AT_LABEL=1
0 / 40
With the 12-load job from this morning that's 5/52 against 0/52 (Fisher exact one-sided p = 0.028). Every death is on TAsyncThread 6 during the world load:
3 of 4: FMallocBinned3 Attempt to free an unrecognized pointer 3c00000000000000, faulting at libc.prx.guest.prx+0x14b60 under app_mcd2.exe+0x1f2c8.
1 of 4: a read through a garbage pointer at app_mcd2.exe+0x48d77cc (rcx 847e1f007f7e1800, read of 0xffffffffffffffff). The other garbage pointers from earlier runs (0301010101010101, 99bfeda6bf7013f0) look the same: texel bytes, not addresses.
The last storage image created before the death was an R8 3D volume (dim=4 tile=7 format=1, 1024x1024x7 or 512x512x27) in 3 of 4, and a 1024x1024 2D (format=56) in the other.
This matches the open item in docs/dev/debt/libSceAgcDriver.md (pending storage-image results): results are stored when something reads their memory, not when the work that wrote them completes, so a title that waits for the work, frees the image and reuses the memory can get its new data overwritten by the late store. My reading: the game's allocator hands the image's memory back out during the load, a late write-back puts texels over its block headers, and the next free trips on them. APS5_STORE_AT_LABEL=1 flushes every pending image before each GPU label, which closes the window.
APS5_STORE_AT_LABEL=1 is the workaround for 8 GB cards in the meantime.
Next: box 1 is now running the same load against current main, boykopovar#6914 (orders deferred write-backs after queued draws' index reads) and boykopovar#6937 (rejects pending writes before tracked snapshot reuse), 25 rounds each, to see whether either closes it without the flag. If neither does, I'll look at storing only the images the waited-for work wrote at the label, instead of every pending one.
@parl0v 5/52 against 0/52 settles it for me, and the debt entry explains the mechanism. The targeted fix (store at the label only what the waited-for work wrote) is yours; we won't start a parallel one.
What we add tonight from my laptop: our crash job runs three interleaved arms (unchanged, APS5_STORE_AT_LABEL=1, thread priorities) with the guest-write ring over the R8 volume's range. The ring marks every write-back that lands after the game freed or unmapped that memory, so it should show the late store directly, and the rows give the switch's FPS cost on a second 8 GB card. I'll post both when it has run.
@AlxcBTW a correction to my last post: APS5_STORE_AT_LABEL=1 is not a usable workaround. I measured its cost on box 2 (RTX 3060 12 GB, mainbc3c0fc, title screen, 200 s rows, arms alternating):
FPS
GPU busy
idle gaps
unchanged
6.69 / 6.66
104 / 105 ms
46 / 46 ms
APS5_STORE_AT_LABEL=1
1.90 / 1.90
396 / 397 ms
133 / 129 ms
-71 %: it writes every pending image back at every label. That also weakens the crash result: with the frame time almost four times longer, the 0/52 may come from the changed timing as much as from the ordering. The flag is a diagnostic, not a fix.
Two other corrections from reading the code: the "guest memory changed while GPU results were pending" line is capped at 8 prints per run, so "exactly 8 per load" was the cap, not a count; what holds is that every unchanged load logs it and no flagged load does. And the R8 volumes are thick 3D images, which StorageTexture tracks as one unit for the whole image instead of 64 KiB blocks (blockUnits excludes geometry.thick), so a 55 MiB volume's pending results are stored all or nothing over memory the game reuses. That's where I'm looking for a targeted fix.
@parl0v thanks for measuring the switch's cost, that changes how the 0/52 reads.
On the "one unit" point, our reading of the code (lab/lead-best8 and origin/main40255d7) differs for these particular volumes: they are all tile 7 (SW_64KB_R_X), and DescribeSurface sends a 3D surface in the R_X/Z_X swizzles down the thin path, so geometry.thick stays false; only the S, S_X, D_X and standard 3D modes are thick, as in AMD's addrlib. With a 64 KiB-aligned base blockUnits is true, so the 1024x1024x55 volume (56 MiB) has 896 units.
The all-or-nothing comes from the marking instead: a dispatch that writes the storage image calls MarkDirty, which marks every unit pending, so any store over the volume (eviction, a flush of everything, a reader of part of its range) writes back all units except the 64 KiB blocks the CPU wrote since, including units the shader never touched. Over memory the game has reused, that is the same effect you describe, but the fix would be on the marking side (pending only for what the dispatch wrote, or a store limited to what the waited-for work wrote), not on unit granularity.
Tonight's job here (11 unchanged loads, 10 with priorities, store-at-label dropped) records when each volume became pending and lists, per store, the guest frees and unmaps over the stored bytes between pending and landing.
@parl0v our crash job from tonight: 21 loads on the RTX 5070 Laptop 8 GB (lab stack on main + boykopovar#6631/boykopovar#6632 and our pending fixes, automatic budget), unchanged against thread priorities, with the guest write ring on. No guest corruption death in either arm (one device loss at the title before any load, ours).
The ring shows the pieces of your late-store reading:
The R8 3D volumes (13-48 per row, all SW_64KB_R_X, so thin and tracked in 64 KiB units, but a GPU write marks every unit pending) stay pending for tens of seconds: 14.8 s median before the store decision, 41 s max.
They are stored whole (5-31 whole-volume stores per row) by evictions and idle flushes, and the bytes reach guest memory through a unit-shadow publish up to 67 s after the decision (whole-volume stores: median 0.64 s, 90th percentile 63.5 s).
In 2 of 21 loads CPU stores hit 64 KiB blocks of a publish that was still in flight: about 3,000 blocks in the slowest load, among them blocks of two 63x63x62 R8 volumes; 34 blocks covering a 2048x1088 R8 image whose memory also held a 1024x1024 RGBA8 array (your format 56 class) in another.
We saw no guest free or unmap over a volume's bytes before a store. MD2 reuses that memory inside its own pool: in about half of the volume write-backs another surface had already been created over the same bytes.
Thread priorities: 0 deaths in either arm, gameplay FPS -3 % (noise). So on our side the switch is not needed; the window you describe is there either way.
@AlxcBTW a correction to my post above: the R8 volumes are not thick images. tile=7 in the StorageTexture log line is the driver's own enum value for SW_64KB_R_X, which the driver lays out as one 2D surface per depth slice, and the 1024x1024 volume (56 slices; the log prints depth minus one) at a 64 KiB-aligned base is already tracked in 64 KiB units, 896 of them. The "one unit for the whole image" part was wrong, and thick-image tracking wouldn't change anything for these volumes. The death counts and the APS5_STORE_AT_LABEL result stand.
GPU busy per frame drops from 101 to 89 ms. Sword of the Sea title on the same box, 3 rounds: 10.03 → 10.36 (+3.3 %).
In gameplay on my RTX 4060 8 GB none of the three change much: there the frame is set by storage-cache evictions (about 200 storage images re-created per frame at the automatic budget).
Stage data on plain main46f2762 (no PRs on top), RTX 4060 8 GB (NVIDIA 617.42), Windows 10, automatic texture cache: loading my save gets into gameplay. At 415 s the HUD, the minimap and the "Escape from Squid Coast" objective are up, the character then walks in all four directions, and the process is still running at the end of the row. It runs at 3.5 FPS on this card in that spot.
The first-world-load stop listed as boykopovar#6301 did not happen in this run. The world load does still crash in some loads on this box. That's a different failure (timing-dependent, it goes away with tracing), not boykopovar#6301's refusal.
Activity
parl0v commentedon Oct 15, 2026
First run on our side: plain
mainreaches the title ("Press any button") on both boxes, no stop in 240 s. Renders correctly (logo, campfire, lighting, terrain).Setup: dump 01.100.100, relinked on
mainfdea36b83with the title's ownsce_module(--windows --to-intel --sce-module-path decrypted); the libs carry only our measurement-only lab commits (census prints underAPS5_PROFILE_DRAW, no behaviour change). Windows 10, NVIDIA 617.42,APS5_PROFILE_DRAW=1, mean of the last 10 s windows at the title:[guestmem]comparesThe UE splash runs at the 60 FPS cap; the title is where the time goes.
Three things stand out:
APS5_TRACE_HOST_IMPORT=1is going now to count them by size.Next:
[gputime]by class and the import count on both boxes, thenmain+ boykopovar#5372 + boykopovar#5058 (Tide's recommendation; boykopovar#5372 needed only a docs merge) against plainmainon the same title.AI-assisted: yes (Claude Code)
parl0v commentedon Oct 15, 2026
Second round: where the time goes, and what's queued.
[gputime]per present at the title (mainfdea36b + our lab counters,APS5_PROFILE_DRAW=1 APS5_PROFILE_GPU=1):Box 1's extra ~100 ms per frame is all memory movement: the 8 GB card evicts storage images (24 GB of eviction write-backs per 10 s) and recreates them, and the publish/upload traffic that comes with it runs over its PCIe link.
The compares come from boykopovar#6060's 1 MiB settle floor. With
APS5_TRACE_HOST_IMPORT=1: 7,494 host imports in 240 s, all at load (5,071 in the first 10 s window, then 1,966, then a few, zero per window at the title), and 7,283 of them (97 %) are 64 KiB–1 MiB. Those stay unsettled, so they stay compared: 83k compares, 259 GB, 18.2 s of CPU per 10 s on box 2 at the title. The floor came from SuperTuxKart, where the settle ran ~320 times a second during play; here the imports are a load-time burst, so settling them should only cost load time. That's our code from boykopovar#6060, so it's ours to fix; a size floor that also looks at the import rate would keep SuperTuxKart's gain.Queued on both boxes, alternating 2 × 240 s each, results here as they land:
main4186f06 vsmain+ perf(agc-driver): join every range a bound buffer crosses boykopovar/AnyPS5#5372 + [HOLD] perf(agc): pre-check memory coverage and flush pending images in indirect dispatch boykopovar/AnyPS5#5058 (Tide's recommendation; perf(agc-driver): join every range a bound buffer crosses boykopovar/AnyPS5#5372 needed only a docs merge).APS5_LAB_SETTLE_FLOOR, a lab switch, same libs): title FPS, compares, load time.AI-assisted: yes (Claude Code)
AlxcBTW commentedon Oct 15, 2026
Thanks, that is a clean picture. Our side is now set up: dump 01.100.100 copied to a private run dir, libs for plain
main0e4ea13 building on the fork CI, first boot as soon as they land. Host: RTX 5070 Laptop 8 GB, Ryzen AI 9 365, 31 GiB, driver 617.42. So we are your box 1 class, with a faster GPU and more RAM.Proposed split so nothing runs twice:
First number from us within the hour. If you already have boykopovar#6194/boykopovar#6197 running on box 1, say so and we take the floor A/B on 8 GB instead.
parl0v commentedon Oct 15, 2026
@AlxcBTW agreed, split as you wrote it: we run 1 and 2 on both boxes (they're on now), you take boykopovar#6194 + boykopovar#6197 and the eviction side on your 8 GB card. I've dropped queue 3 from our boxes, it hadn't started. If you want our 8 GB box as a second point on the eviction work later, name the build and we run it.
After 2 our boxes go to the settle-floor fix itself (settle the load-time burst, keep the floor for imports that keep arriving during play) and to the detile/retile shader speed, which shows up here as 17 ms (box 2) and 37 ms (box 1) of storage write-back per present.
AI-assisted: yes (Claude Code)
parl0v commentedon Oct 15, 2026
Queue 1 done:
main4186f06 vsmain+ boykopovar#5372 + boykopovar#5058, title, alternating 2 × 240 s, mean of the last six 10 s windows:* box 1's first
mainrun was still settling (idle gaps 251 ms against ~130 in the other three); excluding it the two are equal.So at the title the pair is within noise (+1 % on box 2, 0 on box 1). @TideOPL that doesn't say they don't help MCD2 elsewhere: the title has no new imports and almost no indirect dispatches, so if you saw them matter in a level or on load, tell us the scene and we measure there. Queue 2 (settle floor 1 MiB vs 64 KiB) is running now on the stack libs.
AI-assisted: yes (Claude Code)
AlxcBTW commentedon Oct 15, 2026
First number from the 8 GB laptop card, plain
main0e4ea13, title, 300 s,APS5_PROFILE_DRAW=1 APS5_PROFILE_GPU=1, last 10 s windows:Same shape as your box 1 (shadow-publish and storage traffic dominate, 1.6 GB write-back per present), but the idle gaps are twice yours: 270-340 ms of the ~450 ms frame is the GPU waiting on the CPU. The 8 GB card is only half the story here; something on the host side is slower on this box and I take that apart next (compares are the first suspect, your 64 KiB-1 MiB finding). The storage-image creation counter stands at 67k images / 620 GB after 300 s, so the eviction-recreate loop is confirmed on this card too.
Running next:
main+ boykopovar#6194 + boykopovar#6197 vsmainat the same commit (636b1b9), both building now.Screenshot: title renders correctly here as well.
AlxcBTW commentedon Oct 15, 2026
The eviction loop on 8 GB is the storage image cache budget, and it is a one-line lever today.
main0e4ea13 picks a storage image cache budget of 1780 MiB on this card (a quarter of the 7123 MiB VK_EXT_memory_budget heap budget). The title's storage working set does not fit, so every frame evicts, writes back and recreates: 67k storage image creations in 300 s. Forcing the budget withAPS5_TEXTURE_CACHE_MIB=4096(upstream switch, same libs, nothing else changed), title, 240 s, last 10 s windows:APS5_TEXTURE_CACHE_MIB=4096So the 8 GB card lands where your 12 GB box is (6.7 FPS) once the budget fits; the remaining gap is the write-back and draw time you are already on. boykopovar#6194 does this growth automatically; its libs are building here now and I run it against
mainat the same commit next, plus a 5120 MiB point to see whether anything is left to evict. @parl0v your box 1 should show the same with the env switch, if you want the second data point before the PR libs land.parl0v commentedon Oct 15, 2026
Queue 2 done: settle floor 1 MiB (as merged) vs 64 KiB (
APS5_LAB_SETTLE_FLOOR=0x10000, lab switch, same libs:main4186f06 + boykopovar#5372 + boykopovar#5058), title, alternating 2 × 240 s:The 64 KiB floor removes the compares completely, but the frame rate doesn't move on our boxes (box 2 −1.6 %, box 1 ±0). The ~18 s of compare work per 10 s ran on worker threads off the critical path; the frame on box 2 is the same ~150 ms, just split differently (GPU +14 ms, CPU-wait gaps −12 ms; I don't have the reason for the GPU +14 ms yet). Load time didn't change measurably either (same window counts). So it frees roughly two CPU cores, which may matter on a box where the host side is the limit (@AlxcBTW your 270–340 ms of idle gaps, if the compares are part of them), but it isn't the lever on these two. I'll keep it as a candidate PR and not open it until it shows FPS somewhere.
Running now on box 1 (RTX 4060 8 GB), as you asked:
mainwith the auto storage cache budget vsAPS5_TEXTURE_CACHE_MIB=4096, 2 × 240 s,[gputime]on. Results here in ~20 min.AI-assisted: yes (Claude Code)
AlxcBTW commentedon Oct 15, 2026
Two more rows on the 8 GB laptop card, title, 240 s each:
main+APS5_TEXTURE_CACHE_MIB=5120main636b1b9 + boykopovar#6194 + boykopovar#6197, automatic budgetboykopovar#6194 does not grow the budget on this card. No
[textures] storage cache budget X -> Yline in 240 s; the budget stays at 1780 MiB while the heap reports 4162 MiB in use. The reason is its cap, not the eviction trigger:StorageBudgetCaptakes the reported budget (7123 MiB), subtracts what is outside both caches (~2350 MiB), the 2 GiB reserve and the sampled cache's remaining headroom, and the sampled cache's budget has a 2 GiB floor (SampledTextureBudget,max(2048 MiB, ...)) while it holds 33 MiB here. That leaves ~700 MiB of cap, below the 1780 MiB floor, socurrent < capis never true. The forced 4096 MiB, which overrides both caches, runs without any allocation failure on the same card, so the room exists; the formula reserves it for a sampled cache that is empty at this title.5120 vs 4096: same frame rate, so 4096 already fits the working set here; the knee is below 4 GiB. I run a 3072 point next, then
mainat the same commit as the stack for a clean pair.Side note for the setup: after ~150-200 s at the title the game moves on by itself into the first-run Accessibility menu, which runs at the 60 FPS cap; title windows are the ones before that.
parl0v commentedon Oct 15, 2026
Storage write-back, taken apart (box 2, RTX 3060, title,
mainfdea36b + lab counters). This is our side of the split.The detile/retile shader itself is fast. A microbenchmark of
TextureDetileralone (1920x1080, one dispatch, 20 iterations, box 1 RTX 4060):kR64KBX(the title's storage targets) 151–188 GB/s at 4 and 8 B per texel,kStandard64KB130–279 GB/s. In the game the storage-writeback class moves the same bytes at ~26–50 GB/s, so the cost is not the shader.Where it goes instead, per 10 s (new
[wb-shape]counter inwriteBackWindows):So every write-back retiles into a scratch buffer and then copies the scratch into the import's unit-shadow slabs, both on the device: each byte is written twice, plus ~3 copy regions, ~1 padding copy and the barriers between them. The import side (PCIe) is almost nothing here. The first fix I'm going to try: retile straight into the slab when a window's tiled range lands in one slab (no scratch, no copy). That's exact, and it removes up to 54 GB per 10 s of device copies, plus their barriers.
Also from box 2:
APS5_TEXTURE_CACHE_MIB=6144on the 12 GB card gives 6.48 / 6.47 FPS against 6.7 with the automatic budget, so box 2 is not budget-bound (the auto-budget runs of that pair failed to start; the 6.7 comes from earlier runs on the same libs). Box 1's 4096 pair is rerunning cleanly.AI-assisted: yes (Claude Code)
parl0v commentedon Oct 15, 2026
@AlxcBTW box 1 (RTX 4060 8 GB, i5-14400F) confirms your budget finding,
main4186f06, title, alternating 2 × 220 s, presents per 10 s window:APS5_TEXTURE_CACHE_MIB=40962.1x, and box 1 lands exactly where the 12 GB box 2 is (6.7), so on 8 GB cards the eviction loop is the whole gap. No jump to the Accessibility menu within 220 s on either box (the series stays flat), so these are title windows.
Box 2 (12 GB) is not budget-bound: 6144 MiB gives 6.48 FPS against 6.7 automatic. Your
StorageBudgetCapreading (the sampled cache's 2 GiB floor reserved while it holds 33 MiB) fits both: 12 GB leaves enough after the reserve, 8 GB doesn't. That fix is yours per the split; if you open it, box 1 is free to A/B it.AI-assisted: yes (Claude Code)
AlxcBTW commentedon Oct 15, 2026
Budget ladder on the 8 GB laptop card,
main636b1b9, title, 200 s rows, presents per 10 s in the steady title windows:So the working set fits somewhere between 2 and 3 GiB here; 3072 is the best point so far, and the 4096/5120 rows were the first rows after a relink (cold), which is why they sit lower, not because a larger budget hurts. I will redo 4096 on the same libs to settle that.
Cap fix for boykopovar#6194 as a lab branch (
lab/md2-evict-capon the fork, one line on top of boykopovar#6194 + boykopovar#6197):storageBudgetCapcounts the sampled cache's remaining headroom only up to the sampled cache's current size, so an empty sampled cache no longer reserves its 2 GiB floor. On this card that gives a cap of ~2.7 GiB and four growth steps from 1780 MiB; libs are building, result in ~30 min. @parl0v if you want to A/B it on box 1: branchlab/md2-evict-cap, nothing to set.parl0v commentedon Oct 15, 2026
First cut of the direct-to-slab write-back (lab branch, not a PR yet): a window that is covered by one kept run, maps to one contiguous slab range and has no holes in its tiled range is retiled straight into the slab, no scratch, no copy. Box 2, title, alternating 2 × 240 s, same libs with
APS5_LAB_NO_DIRECT_SLAB_RETILE=1as the off side:So almost all of the 53 GB are windows with padding (the surface's edge blocks), which the first cut leaves on the scratch path. Next cut: a padded window goes direct too, and its holes are either already current in the same slab at the same offset (the retile doesn't touch them, so nothing to copy) or copied into the slab from their source first (a copy of the hole bytes only, instead of the whole window twice). Result here when it's measured.
AI-assisted: yes (Claude Code)
parl0v commentedon Oct 15, 2026
@AlxcBTW
lab/md2-evict-cap(b8463e6) on box 1, RTX 4060 8 GB, title, alternating 2 × 220 s, presents per 10 s in the steady windows:main636b1b9lab/md2-evict-capmain+APS5_TEXTURE_CACHE_MIB=3072So the cap fix gets 2.06x with nothing to set, about 95 % of the forced 3072 point. The rest is the ~1,200 creations / 10 s it still does at 2878 MiB: the working set here sits just above that, matching your 2–3 GiB knee. If the cap could reach ~3 GiB on this card (its heap shows ~2.4 GiB free at the title), it would close the gap. Either way that's a PR worth opening on top of boykopovar#6194 or as its own; box 1 is free to re-run whatever you put up.
AI-assisted: yes (Claude Code)
AlxcBTW commentedon Oct 15, 2026
Thanks for the box 1 run, that settles the cap fix as worth it.
Second version on the same branch,
lab/md2-evict-capaaa9947: the reserve inStorageBudgetCapandNextStorageBudgetbecomes max(1 GiB, budget / 8) instead of max(2 GiB, budget / 8). Reason: on this laptop card the first version grew 1780 -> 2003 -> 2253 -> 2535 MiB and then shrank back to 1780 MiB once heap usage passed ~5.1 GiB, because a fixed 2 GiB is 29 % of a 7.1 GiB budget. On 16 GB cards budget / 8 is already ~2 GiB, so the Linux 5080 case from the boykopovar#6194 review keeps its reserve. Measuring it now; box 1 can run the same branch head if you have a slot.Our side runs several lines in parallel from now on, results here as they land:
Storage write-back (direct-to-slab) stays yours, we don't touch it.
50 remaining items
data: Minecraft Dungeons II reaches Menus on plain main and stops at …
parl0v commentedon Oct 16, 2026
@AlxcBTW box 2 is back (RTX 3060 12 GB, a quiet host), title screen, automatic budget, 200 s rows:
Best stack vs the gameplay base, alternating, 3 rounds each:
main+ boykopovar#6308 + boykopovar#5372 + boykopovar#6490Same signature as box 1 and your laptop: GPU busy unchanged, ~9 ms less idle per frame.
Where the 95 ms of GPU time goes (best stack,
APS5_PROFILE_GPU=1,[gputime]per present): draws 31.0 ms, the game's compute programs ~23 ms (top twelve), and then ours: storage write-back 9.4 ms (~800 MiB per present), copies 6.0 ms (4 per present, 132 MiB), shadow publish 3.1 ms, storage upload 1.7, label runs 1.6, staging in/out 1.6, DCC clear + keys 1.2, fill 0.9. So ~26 ms per present is emulator traffic, a little over a quarter. By reason, 79 % of the write-back bytes arestorage refresh(10.7k write-backs, 47 GB per 10 s): a refresh flushing the pending results of the other images in the same memory.A ceiling that turned out not to be one: skipping
Refreshon cache hits for store-only storage images (APS5_LAB_SKIP_STORE_ONLY_REFRESH=1, unsafe lab switch, on Sword of the Sea it was +10-13 %):The refresh bytes drop by two thirds, but GPU busy doesn't move and the idle gaps double: the skipped flushes come back as
sampled texturewrite-backs (2.2 GB per 10 s) and CPU-path uploads (948 MiB per 10 s, from 12), so in this title other images read what the refresh would have stored. Cutting refresh traffic here needs to know which pairs are never read again, not a blanket skip; I'll look at that next.AI-assisted: yes (Claude Code)
parl0v commentedon Oct 16, 2026
@AlxcBTW
lab/thread-priorities(c222594) on box 1, my busy PC (RTX 4060 8 GB, i5-14400F, 16 GB, other heavy work running), automatic budget, 24 first-world loads from the no-character save + 120 s walk,APS5_LAB_THREAD_PRIORITIES=1on and off interleaved:TAsyncThread 6withFMallocBinned3 Attempt to free an unrecognized pointer 3c00000000000000(both again right after a1024x1024 layers=7 dim=4 format=1R8 storage texture was created, at0x30696d0000and0x3067fc0000), and round 11 onBackground Worker #3atapp_mcd2.exe+0x48d77cc, the sameripas the 12:27 death, read of0xffffffffffffffff, this time with rcx0301010101010101(image-data-like bytes where a pointer should be; the earlier one had99bfeda6bf7013f0).On a quiet host (box 2, RTX 3060 12 GB, title, 3 rounds) the switch is neutral as well: 7.80 / 7.80 / 7.71 off, 7.80 / 7.72 / 7.40 on.
AI-assisted: yes (Claude Code)
AlxcBTW commentedon Oct 16, 2026
@parl0v thank you, that is the clearest signal so far. 3 / 12 against 0 / 12 fits a timing window the priorities make rarer, as you say.
The new death adds something:
rcx = 0301010101010101is what R8 texels look like, sitting where a pointer should be, and both TAsyncThread deaths again follow the creation of that1024x1024 layers=7 dim=4 format=1R8 texture. So the next thing our agent checks in the code is whether the layer's footprint for that class (tile mode 7, 7 layers,dim=4) is larger than what the game allocated. In that case a write-back of the image would write past the allocation into neighbouring heap memory, and lower priorities only shift when that happens. If you still have them: the address and size of the allocation that held the bad pointer, or anything that shows how far it was from the texture base, would settle it quickly.More gameplay loads on box 2 with the switch on and off would help as well, since its timing differs from box 1. Our own lab is down for a few hours (two Windows bugchecks on the laptop, unrelated to the game).
AI-assisted: yes (Claude Code)
AlxcBTW commentedon Oct 16, 2026
@parl0v a correction to my last comment: the overrun idea does not hold for that class. It is a 3D volume, R8 1024x1024x8 with one mip and the 64 KiB rotated swizzle (
SW_64KB_R_X). Our layer sizes it at 8 MiB, eight thin 1 MiB slices with 64 KiB alignment, which is what AMD's gfx10 addrlib gives for the same descriptor. Every write-back and store of the driver stays inside those 8 MiB, in whole 64 KiB units. Its base addresses are 64 KiB multiples inside a pool that the game's short-lived GPU resources reuse one after another, so they say nothing about the allocation size.So the R8 bytes in your crashed objects need a store that lands after the game reused that memory, or the same memory reachable at two addresses. The guest-write ring of our next job (20 loads, priority switch on and off) now records each creation of the volume, every write-back over its range with its reason and landing time, and the guest's maps, frees and unmaps over the same range (libs from
lab/guestcrash-prio60202b3). Your box 2 loads would still help.AI-assisted: yes (Claude Code)
parl0v commentedon Oct 16, 2026
@AlxcBTW tonight's open-PR sweep on box 2 (RTX 3060 12 GB, i5-12400F, Windows 10), Minecraft Dungeons II title screen, 200 s rows, interleaved with
main. The scene is CPU-bound there (45-47 ms of idle gaps per frame), so it mostly rewards CPU-side changes. Each result is posted on its PR:maindrawclass 32.3 -> 24.3 ms per present (-25 %), freed time becomes idlemainboth roundsmainboth roundsThen the four that help this scene, stacked on
mainc842491 (boykopovar#6631 2a15be0 + boykopovar#6632 23a804e + boykopovar#6701 fe95848 + boykopovar#6650 fc25ed9, all merge cleanly), three rounds:mainGPU time unchanged, 11 ms less waiting per frame. Your two still carry most of it (+10 % on box 1 this morning); boykopovar#6701 and boykopovar#6650 add on top.
Box 1's crash job is past half: so far 3 deaths in 27 unchanged loads, 0 in 26 with
APS5_STORE_AT_LABEL=1(0 in 38 over both jobs). Full numbers when it ends around 23:00.AI-assisted: yes (Claude Code)
AlxcBTW commentedon Oct 16, 2026
@parl0v thank you for the sweep, that is a lot of rows. Good to see boykopovar#6631 and boykopovar#6632 hold on a second machine, and boykopovar#6701 / boykopovar#6650 stacking cleanly on top; we will put those two into our next lab stack.
The crash numbers are the strongest signal so far: 3 in 27 unchanged against 0 in 38 with
APS5_STORE_AT_LABEL=1fits the late-store window much better than a game-side race. Our next crash job (on hold while my laptop's own Windows bugchecks are sorted out) gets a store-at-label arm next to the unchanged one, with the guest-write ring watching the R8 volume's range. Did you see an FPS cost from the switch on box 1?AI-assisted: yes (Claude Code)
parl0v commentedon Oct 16, 2026
@AlxcBTW the overnight crash job on box 1 is done. RTX 4060 8 GB (NVIDIA 617.42), 16 GB RAM, Windows 10, the stack from yesterday (
mainc842491 + boykopovar#6631 + boykopovar#6632), automatic texture budget, the same save loaded from the main menu into the world and stopped at the HUD. Each round ran one load unchanged and one withAPS5_STORE_AT_LABEL=1, alternating:APS5_STORE_AT_LABEL=1With the 12-load job from this morning that's 5/52 against 0/52 (Fisher exact one-sided p = 0.028). Every death is on
TAsyncThread 6during the world load:FMallocBinned3 Attempt to free an unrecognized pointer 3c00000000000000, faulting atlibc.prx.guest.prx+0x14b60underapp_mcd2.exe+0x1f2c8.app_mcd2.exe+0x48d77cc(rcx 847e1f007f7e1800, read of0xffffffffffffffff). The other garbage pointers from earlier runs (0301010101010101,99bfeda6bf7013f0) look the same: texel bytes, not addresses.dim=4 tile=7 format=1, 1024x1024x7 or 512x512x27) in 3 of 4, and a 1024x1024 2D (format=56) in the other.This matches the open item in
docs/dev/debt/libSceAgcDriver.md(pending storage-image results): results are stored when something reads their memory, not when the work that wrote them completes, so a title that waits for the work, frees the image and reuses the memory can get its new data overwritten by the late store. My reading: the game's allocator hands the image's memory back out during the load, a late write-back puts texels over its block headers, and the next free trips on them.APS5_STORE_AT_LABEL=1flushes every pending image before each GPU label, which closes the window.APS5_STORE_AT_LABEL=1is the workaround for 8 GB cards in the meantime.Next: box 1 is now running the same load against current
main, boykopovar#6914 (orders deferred write-backs after queued draws' index reads) and boykopovar#6937 (rejects pending writes before tracked snapshot reuse), 25 rounds each, to see whether either closes it without the flag. If neither does, I'll look at storing only the images the waited-for work wrote at the label, instead of every pending one.AI-assisted: yes (Claude Code)
AlxcBTW commentedon Oct 16, 2026
@parl0v 5/52 against 0/52 settles it for me, and the debt entry explains the mechanism. The targeted fix (store at the label only what the waited-for work wrote) is yours; we won't start a parallel one.
What we add tonight from my laptop: our crash job runs three interleaved arms (unchanged,
APS5_STORE_AT_LABEL=1, thread priorities) with the guest-write ring over the R8 volume's range. The ring marks every write-back that lands after the game freed or unmapped that memory, so it should show the late store directly, and the rows give the switch's FPS cost on a second 8 GB card. I'll post both when it has run.AI-assisted: yes (Claude Code)
parl0v commentedon Oct 16, 2026
@AlxcBTW a correction to my last post:
APS5_STORE_AT_LABEL=1is not a usable workaround. I measured its cost on box 2 (RTX 3060 12 GB,mainbc3c0fc, title screen, 200 s rows, arms alternating):APS5_STORE_AT_LABEL=1-71 %: it writes every pending image back at every label. That also weakens the crash result: with the frame time almost four times longer, the 0/52 may come from the changed timing as much as from the ordering. The flag is a diagnostic, not a fix.
Two other corrections from reading the code: the "guest memory changed while GPU results were pending" line is capped at 8 prints per run, so "exactly 8 per load" was the cap, not a count; what holds is that every unchanged load logs it and no flagged load does. And the R8 volumes are thick 3D images, which
StorageTexturetracks as one unit for the whole image instead of 64 KiB blocks (blockUnitsexcludesgeometry.thick), so a 55 MiB volume's pending results are stored all or nothing over memory the game reuses. That's where I'm looking for a targeted fix.AI-assisted: yes (Claude Code)
AlxcBTW commentedon Oct 16, 2026
@parl0v thanks for measuring the switch's cost, that changes how the 0/52 reads.
On the "one unit" point, our reading of the code (lab/lead-best8 and
origin/main40255d7) differs for these particular volumes: they are all tile 7 (SW_64KB_R_X), andDescribeSurfacesends a 3D surface in theR_X/Z_Xswizzles down the thin path, sogeometry.thickstays false; only the S, S_X, D_X and standard 3D modes are thick, as in AMD's addrlib. With a 64 KiB-aligned baseblockUnitsis true, so the 1024x1024x55 volume (56 MiB) has 896 units.The all-or-nothing comes from the marking instead: a dispatch that writes the storage image calls
MarkDirty, which marks every unit pending, so any store over the volume (eviction, a flush of everything, a reader of part of its range) writes back all units except the 64 KiB blocks the CPU wrote since, including units the shader never touched. Over memory the game has reused, that is the same effect you describe, but the fix would be on the marking side (pending only for what the dispatch wrote, or a store limited to what the waited-for work wrote), not on unit granularity.Tonight's job here (11 unchanged loads, 10 with priorities, store-at-label dropped) records when each volume became pending and lists, per store, the guest frees and unmaps over the stored bytes between pending and landing.
AI-assisted: yes (Claude Code)
AlxcBTW commentedon Oct 17, 2026
@parl0v our crash job from tonight: 21 loads on the RTX 5070 Laptop 8 GB (lab stack on
main+ boykopovar#6631/boykopovar#6632 and our pending fixes, automatic budget), unchanged against thread priorities, with the guest write ring on. No guest corruption death in either arm (one device loss at the title before any load, ours).The ring shows the pieces of your late-store reading:
SW_64KB_R_X, so thin and tracked in 64 KiB units, but a GPU write marks every unit pending) stay pending for tens of seconds: 14.8 s median before the store decision, 41 s max.Thread priorities: 0 deaths in either arm, gameplay FPS -3 % (noise). So on our side the switch is not needed; the window you describe is there either way.
AI-assisted: yes (Claude Code)
parl0v commentedon Oct 18, 2026
@AlxcBTW a correction to my post above: the R8 volumes are not thick images.
tile=7in theStorageTexturelog line is the driver's own enum value for SW_64KB_R_X, which the driver lays out as one 2D surface per depth slice, and the 1024x1024 volume (56 slices; the log prints depth minus one) at a 64 KiB-aligned base is already tracked in 64 KiB units, 896 of them. The "one unit for the whole image" part was wrong, and thick-image tracking wouldn't change anything for these volumes. The death counts and theAPS5_STORE_AT_LABELresult stand.AI-assisted: yes (Claude Code)
parl0v commentedon Oct 18, 2026
@AlxcBTW an update on the MCD2 title. I have three perf PRs open:
Stacked on
maina0f3dc4, box 2 (RTX 3060 12 GB, i5-12400F, NVIDIA 617.42), default settings, 215 s title rows alternating:mainmain+ boykopovar#7427 + boykopovar#7461 + boykopovar#7560GPU busy per frame drops from 101 to 89 ms. Sword of the Sea title on the same box, 3 rounds: 10.03 → 10.36 (+3.3 %).
In gameplay on my RTX 4060 8 GB none of the three change much: there the frame is set by storage-cache evictions (about 200 storage images re-created per frame at the automatic budget).
parl0v commentedon Oct 19, 2026
Stage data on plain
main46f2762 (no PRs on top), RTX 4060 8 GB (NVIDIA 617.42), Windows 10, automatic texture cache: loading my save gets into gameplay. At 415 s the HUD, the minimap and the "Escape from Squid Coast" objective are up, the character then walks in all four directions, and the process is still running at the end of the row. It runs at 3.5 FPS on this card in that spot.The first-world-load stop listed as boykopovar#6301 did not happen in this run. The world load does still crash in some loads on this box. That's a different failure (timing-dependent, it goes away with tracing), not boykopovar#6301's refusal.