fix(libkernel): release flexible accounting after fixed mmap

[MarcZX10]

What

On a successful MAP_FIXED anonymous or file-backed mmap, the host replaces the old guest pages. If those pages were charged in g_flexibleRanges, leaving the old interval recorded makes sceKernelAvailableFlexibleMemorySize under-report available memory. Release the replaced interval from flexible accounting only after the mapping succeeds; failed mappings leave the charge unchanged.

The regression replaces the middle page of a three-page flexible mapping, checks that neighboring bytes remain intact and the replaced page is zeroed or contains the mapped file data, verifies the flexible budget decreases by exactly one page, then confirms the full budget is restored after unmapping. The file-backed case runs on Linux, where regular file mappings are supported by this path.

This addresses only the confirmed fixed-replacement behavior in #7120. It does not change the issue’s separate, unconfirmed question about whether ordinary anonymous mmap consumes the flexible budget.

Tested

• Linux x86-64, GCC 12.2, Ninja Release: built libkernel and guest_memory_tests.
• guest_mmap_fixed_anonymous passed, including the anonymous and Linux file-backed accounting regressions. This host (Intel Xeon E5-1620 v2) lacks CPU features required by the test runtime, so the focused test used a temporary local-only bypass of the startup CPU feature guard; the guard and all unrelated source were restored. This is not a standard unmodified-host runtime result.
• python3 tools/check_conventions.py --base upstream/main passed.
• git diff --check passed.
• tools/pr_overlap.py --branch codex/fix-mmap-flexible-replacement-7120 reported no overlap with open PRs.
• No Windows or PS5 run; no full build claimed.

Checklist

• Based on current main; no other open PR implements the same POSIX mmap behavior
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file; n/a
• No notes, investigation .md files or images added to the repository
• New third-party code: none
• Generic behavior, not title-specific
• Depends on: none
• Fixes: none; libkernel: MAP_FIXED over flexible memory keeps its flexible charge, and anonymous mmap is not charged as flexible #7120 has a separate unconfirmed accounting question
• Unblocks: none
• GPUs tested: n/a
• AI-assisted: yes

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
PR 6534	core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp L120-121

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 4449	core/libs/tests/GuestMemory.cpp
PR 5021	core/libs/prx/libkernel/DirectMemory/Export.cpp core/libs/tests/GuestMemory.cpp
PR 6044	core/libs/tests/GuestMemory.cpp
PR 6222	core/libs/prx/libkernel/DirectMemory/Export.cpp core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp
PR 6565	core/libs/tests/GuestMemory.cpp
PR 7201	core/libs/prx/libkernel/DirectMemory/Export.cpp

[MarcZX10]

Overlap follow-up: #6534 independently changes the same DoMapAnon call in PosixMemory.cpp to pass fixedAnonymous, as part of Windows stack replacement. That is the same call-site hunk this PR changes to release flexible-memory accounting after success. The behavior is distinct, but the hunk will need a mechanical rebase/merge if either PR lands first; this PR does not depend on #6534 at runtime. I will preserve both behaviors when resolving the overlap. #7201 also touches Export.cpp, but its current changes are in separate macOS query/locking paths; the ReleaseFlexibleMappingRange wrapper here is additive.

[MarcZX10]

Manual overlap audit on current main 922dc89: #4449 adds separate shared-backing tests; #5021 changes the Export.cpp VirtualQuery/lock paths and GuestMemory.cpp test gates; #6044 is a rename of the memory test file; #6222 adds macOS VirtualQuery and file-descriptor pinning paths; #6565 adds separate lease tests. Their current diffs do not change this PR’s flexible-range release wrapper, fixed mmap success paths, or focused tests. #7201 is likewise separate macOS query/locking work. The only same-callsite behavior overlap is #6534, already described above; it passes fixedAnonymous into DoMapAnon for Windows stack replacement, while this PR releases flexible accounting after success.

[MarcZX10]

I rebased this PR onto current main 922dc8999 because its original parent was six commits behind. The rebased commit has the same patch ID as the prior head, preserves the MarcZX100 author, and the diff and conventions checks pass on the new base. The earlier Linux Release build/focused regression run applies to the unchanged patch before this rebase; a fresh local build could not be configured here because the worktree’s other required submodules are not initialized. Hosted CI reran conventions and eligibility successfully; Linux/Windows and other platform build jobs are skipped by path policy.