fix(agc): reclaim idle buffers on allocation OOM

[Sanjaysrinivas]

What

Supersedes closed #2828.

AGC buffer allocations can report Vulkan out-of-memory while the pool still holds idle buffers or empty slab blocks. Release that idle memory and retry the allocation once, only when actual VkDeviceMemory was freed.

Dedicated host buffers, device buffers and new host slab blocks share this recovery path. Live allocations remain retained. Other Vulkan errors, persistent OOM and failures with no reclaimable memory still throw. Failed allocation output handles are discarded. No new dependencies; the remaining memory limits are recorded in docs/dev/TechnicalDebt.md.

Rebased onto main a11c8f7f70a0ed4e71504a89de69e8cdae687817. The final rebase includes #3511, which removes an unused ImGui gitlink that broke hosted checkout; tested renderer and fixture blobs are unchanged. Refreshed the upstream implementation and searched existing PRs before publishing: no equivalent idle-buffer allocation recovery was found. Related buffer sizing/accounting work (#2233 and #2283) changes nearby code without implementing this policy. Depends on: none.

The overlap bot flagged #3564 because both branches inserted different technical-debt entries at the same location. Moved the buffer allocation entry beside the other AGC limits and verified that the two branches merge cleanly with both entries retained. #3564 implements libc ftrylockfile; this PR remains independent. Production and test files are unchanged from the validated head 6cc7d65afe3c7d79dd14dc86089a133bbc864e5a.

Tested

Windows, MinGW GCC 15.2, Release, BUILD_TESTING=ON. Vulkan execution used the RTX 4070 Laptop GPU with Khronos core validation enabled and Vulkan required.

• The extended pool regression fails with "idle memory did not permit recovery" using main's production allocation files at 1e8fff68c8f6d3b67e38be584e15706a70c9d8bf; it passes after restoring and rebuilding this PR.
• Coverage includes host/device OOM, dedicated allocations, new and empty slabs, partially live slabs with mapped data preserved, persistent failure, no reclaimable memory, device loss, first-attempt success and invalid failed-allocation handles.
• Final selected CTest run: 26/29 passed (29.03 seconds). No selected checks skipped; no Vulkan validation errors were reported.
• The three failures are agc_driver_prepared_shaders, _indirect and _uncached, each reporting "shader failure capture was incomplete". Rebuilt main's production allocation files in the same build/environment: all three fail identically. The recovery regression also fails on that baseline, while test registration passes. Restored this PR's files byte for byte, rebuilt, and repeated the selected checks.
• Fresh native build, tools/check_conventions.py --base origin/main and git diff --check origin/main...HEAD pass. Linux was not tested locally.

The private game integration, runtime binaries and title assets were not changed by this publication. This generic fix does not establish gameplay, stable FPS or a bound on live images and other retained resources.

Checklist

• Based on current main; no other open PR implements the same recovery policy
• One topic per PR; follow-ups go in a new PR
• No new comments in code
• Unimplemented paths throw; allocation errors still throw when recovery fails
• No notes, investigation .md files or images added to the repository
• No new third-party code
• Generic behaviour, not specific to one title
• Depends on: none
• AI-assisted: yes

[MarcZX10]

The conventions and relinker checks are currently failing during checkout, before they reach this PR. The logs show fatal: No url found for submodule path '3rdparty/imgui' in .gitmodules on base 1e8fff68; this appears to be the shared main-branch checkout issue tracked by #3511. The PR-specific register-context and write-watch checks are passing. Sharing this in case it helps distinguish the infrastructure failure from the buffer-pool changes.

[MarcZX10]

Update: #3511 is now merged and main no longer has the orphan 3rdparty/imgui gitlink. The current failed conventions/relinker runs still check out base 1e8fff68 and stop before building this PR. Rebasing this branch onto current main should let those checks evaluate the patch.

[MarcZX10]

Update: I see the branch has a new head, and conventions plus all relinker jobs now pass. That clears the earlier checkout failure; thanks for refreshing the branch.

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
#290	core/libs/prx/libSceAgcDriver/Graphics/src/BufferPool.cpp L162 core/libs/prx/libSceAgcDriver/Graphics/src/Resources.cpp L106, L185

[adriadam10]

Checked on current main: agc_buffer_pool and the rest of the libSceAgcDriver ctest scope pass (384/384); the new regression fails when the production changes are reverted. Lock use in trim() is sound (no pool mutex held across the Vulkan allocation) and a merge-tree with #3564 is clean. Merging.