feat(libc): reserve the guest arena on macOS

[yripper]

What

The guest arena never came up on macOS, so every guest allocation threw: the arena seeds its used ranges from /proc/self/maps and reserves the rest with MAP_FIXED_NOREPLACE, and macOS has neither. This keeps the same Arena class and constants and swaps only those two host primitives under __APPLE__:

• Host mappings: a mach_vm_region walk from ArenaStart to ApplicationAreaEnd.
• Reservation without replacing: mach_vm_map with VM_FLAGS_FIXED, VM_PROT_NONE and max protection VM_PROT_ALL, in the same 1 GiB chunks. A fixed mach_vm_map fails instead of replacing, and KERN_NO_SPACE maps to EEXIST, so the loop that skips occupied chunks is unchanged.

The POSIX commit and protection paths (PlaceAt, GuestArenaSetProtection) are enabled for __APPLE__ as well. On Linux the code is split into HostMappings and ReserveNoReplace with the same calls and error mapping.

The ranges macOS never lets a process map (0x10_0000_0000–0x70_0000_0000), and the dyld and Rosetta runtime regions inside the application area, are recorded as debt in docs/dev/debt/libc.md.

This is the arena step agreed in #1, so @mugurc's direct-memory branch can be based on it.

Tested

• New standalone guest_arena test (core/libs/tests/guest_arena, macOS only), 4 modes: available, unmappable, host, commit. All pass on macOS 26 on an M1 Pro, both as x86-64 under Rosetta 2 and as arm64.
• On my macOS branch, which carries the rest of the port, God of War: Sons of Sparta (PPSA28997) boots into gameplay with this arena.
• Linux and Windows: not built here. The Linux path keeps the same /proc/self/maps parsing and mmap(MAP_FIXED_NOREPLACE) handling; Windows is untouched.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Unblocks: none
• GPUs tested: n/a (no GPU code)
• AI-assisted: yes

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
PR 6222			core/libs/tests/GuestArenaHost.cpp core/libs/tests/guest_arena/CMakeLists.txt

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 1835	core/libs/prx/libc/src/GuestArena.cpp
PR 5021	core/libs/CMakeLists.txt core/libs/prx/libc/src/GuestArena.cpp docs/dev/debt/libc.md
PR 5303	docs/dev/debt/libc.md
PR 6044	core/libs/CMakeLists.txt
PR 6222	core/libs/CMakeLists.txt core/libs/prx/libc/src/GuestArena.cpp core/libs/tests/GuestArenaHost.cpp core/libs/tests/guest_arena/CMakeLists.txt docs/dev/debt/libc.md
PR 6226	docs/dev/debt/libc.md
PR 6334	docs/dev/debt/libc.md
PR 6483	docs/dev/debt/libc.md
PR 6565	docs/dev/debt/libc.md
PR 6879	core/libs/prx/libc/src/GuestArena.cpp
PR 7201	core/libs/CMakeLists.txt core/libs/prx/libc/src/GuestArena.cpp
PR 7447	docs/dev/debt/libc.md

[adriadam10]

Checked the Linux side: HostMappings/ReserveNoReplace keep the same /proc/self/maps parsing, MAP_FIXED_NOREPLACE call and error mapping; build and the libs ctest scope are green on the PR merged onto main. The macOS path (mach_vm_region walk, fixed mach_vm_map, the guest_arena tests) can't be run here, so that part rests on your M1 Pro runs; the full Build will compile it on the macOS job. Debt line is in docs/dev/debt/libc.md. Merging.

[xsyetopz]

@yripper I have an Intel Mac running i7-4820HQ and macOS Monterey (v12). If you could set up the things I need to do to test it on an Intel Mac, do so here. Thanks!

[Alexius0815]

Tested #6124 (head 913adca) on an Apple M4 (16 GB), macOS 27.2 (26B5112a). Standalone guest_arena project, Release, built per architecture: available, unmappable, host (12 / 33 protected host regions), commit all pass, both as x86-64 under Rosetta 2 and native arm64; ctest 4/4. So the arena also comes up on macOS 27 and M4.

ccleste (PPSA59322, catalog 1.00, sha256 checked, no sce_module) on mugurc/ci/macos-series-2 + #6124 + #6448 + #6580 (without it libc does not compile with the macOS SDK) + #6928 + #6668 + #6667 + #6235, with the three DirectMemory.cpp arena guards widened to APPLE as @DavidSamir described: it creates the MoltenVK 1.4.2 device and swapchain, opens audio, loads its sounds and songs and dispatches compute work, then stops at a pixel shader that reads barycentrics without VK_KHR_fragment_shader_barycentric needs a vertex shader before it. The draw is a rect list whose pixel shader reads raw per-vertex parameters, so there is no path for it on macOS (no geometry shaders, barycentric extension disabled on MoltenVK).

AI-assisted: yes

[imellag]

Tested on an Apple M4 Pro (MacBook Pro, 24 GB), macOS 27.0.1 (26A434), Rosetta 2, Apple clang 21.0.0, CMake 4.4.4, Ninja 1.13.2.

PR head 913adca3a merged into current main (385ed71f2), standalone core/libs/tests/guest_arena project, Release and Debug, built per architecture:

Build	available	unmappable	host	commit
x86_64 under Rosetta	pass	pass	pass (11 or 12 host regions, depending on where dyld lands)	pass
native arm64	pass	pass	pass (24 host regions)	pass

--repeat until-fail:20 on both Release builds: no failures.

Address space on this host. A separate program that walks mach_vm_region from 0x2_0000_0000 to 0xFC_0000_0000 at startup, six launches per architecture:

• The range the kernel never maps starts 1 GiB lower than the new debt line says: there are two entries with maximum protection none, 0xF_C000_0000-0x10_0000_0000 and 0x10_0000_0000-0x70_0000_0000, on both architectures. A MAP_FIXED mapping at 0xF_BFFF_0000 succeeds and one at 0xF_C000_0000 fails with EACCES; 0x6F_FFFF_0000 fails and 0x70_0000_0000 succeeds. The extra GiB lies inside SystemReservedStart-SystemReservedEnd, so placement is unaffected; only the range in the debt line could start at 0xF_C000_0000.
• Under Rosetta, dyld starts between 0x2_0024_C000 and 0x2_04A6_C000 and the Rosetta runtime region between 0x3_0059_C000 and 0x3_094C_6000 across launches, consistent with "near 0x2_0200_0000 and 0x3_0400_0000".
• As native arm64, 0x1_FA00_0000-0xF_C000_0000 is entirely covered by host regions, mostly read-only. This only matters for a native arm64 host, not for titles under Rosetta.

What the tests do not cover. With the mach_vm_region walk disabled (HostMappings starting at ApplicationAreaEnd, so it returns no regions), all four tests still pass on both architectures: TryReserve records every 1 GiB chunk it cannot reserve as occupied, so host regions are still refused, at chunk granularity. The walk makes a difference for free space next to a host region. A check linked against this PR's GuestArena.cpp, with the host map captured in a static initializer that runs before g_reserved:

	fixed mapping over dyld (0x2_02ED_B000)	fixed 64 KiB mapping at 0x2_0000_0000, free, same GiB as dyld
this PR	rejected	accepted
walk disabled	rejected	rejected (overlaps a host region in the guest arena)

Native arm64 gives the same pattern with a host region at 0x74_AC00_0000 and a free gap at 0x74_8000_0000. Without the walk, the guest loses the whole GiB around dyld and the one around the Rosetta runtime. A test case like this one would keep the walk from regressing unnoticed.

Check source

Built with xcrun clang++ -std=c++20 -O1 -arch x86_64 -I core/libs -o gap arena_gap.cpp core/libs/prx/libc/src/GuestArena.cpp core/libs/prx/libc/src/GuestWriteWatch.cpp (this file first, so its initializer runs before the arena's), and with -arch arm64.

#include "prx/libc/include/GuestArena.hpp"

#include <mach/mach.h>
#include <mach/mach_vm.h>

#include <algorithm>
#include <cstdint>
#include <cstdio>
#include <stdexcept>
#include <vector>

using namespace GuestArena;

namespace {

constexpr std::uintptr_t ArenaStart = 0x0000000200000000ull;
constexpr std::uintptr_t ApplicationAreaEnd = 0x000000FC00000000ull;
constexpr std::uintptr_t Chunk = 1ull << 30;
constexpr std::size_t Bytes = 0x10000;

struct Region {
    std::uintptr_t start;
    std::uintptr_t end;
};

std::vector<Region> HostRegions() {
    std::vector<Region> regions;
    mach_vm_address_t address = ArenaStart;
    while (address < ApplicationAreaEnd) {
        mach_vm_size_t size = 0;
        vm_region_basic_info_data_64_t info{};
        mach_msg_type_number_t count = VM_REGION_BASIC_INFO_COUNT_64;
        mach_port_t object = MACH_PORT_NULL;
        if (mach_vm_region(mach_task_self(), &address, &size, VM_REGION_BASIC_INFO_64, reinterpret_cast<vm_region_info_t>(&info), &count, &object) != KERN_SUCCESS) break;
        if (address >= ApplicationAreaEnd) break;
        regions.push_back({static_cast<std::uintptr_t>(address), static_cast<std::uintptr_t>(address + size)});
        address += size;
    }
    return regions;
}

bool Overlaps(const std::vector<Region>& regions, std::uintptr_t start, std::uintptr_t end) {
    for (const auto& region : regions)
        if (region.start < end && start < region.end) return true;
    return false;
}

const std::vector<Region> RegionsBeforeArena = HostRegions();

}

int main() {
    const auto& regions = RegionsBeforeArena;
    std::printf("regions before the arena: %zu, after: %zu\n", regions.size(), HostRegions().size());
    for (const auto& host : regions) {
        const auto chunk = host.start & ~(Chunk - 1);
        for (auto gap = std::max(chunk, ArenaStart); gap + Bytes <= chunk + Chunk; gap += Bytes) {
            if (Overlaps(regions, gap, gap + Bytes)) continue;
            std::printf("host region 0x%llx-0x%llx, free gap 0x%llx in the same 1 GiB chunk 0x%llx\n",
                static_cast<unsigned long long>(host.start), static_cast<unsigned long long>(host.end),
                static_cast<unsigned long long>(gap), static_cast<unsigned long long>(chunk));
            try {
                GuestArenaMarkUsed_nid_postfix(reinterpret_cast<const void*>(host.start), host.end - host.start);
                std::puts("fixed guest mapping over the host region: accepted");
            } catch (const std::exception& error) {
                std::printf("fixed guest mapping over the host region: rejected (%s)\n", error.what());
            }
            try {
                GuestArenaMarkUsed_nid_postfix(reinterpret_cast<const void*>(gap), Bytes);
                std::puts("fixed guest mapping in the free gap: accepted");
                GuestArenaRelease_nid_postfix(reinterpret_cast<const void*>(gap), Bytes);
            } catch (const std::exception& error) {
                std::printf("fixed guest mapping in the free gap: rejected (%s)\n", error.what());
            }
            return 0;
        }
    }
    std::puts("no free gap shares a 1 GiB chunk with a host region");
    return 0;
}

Not checked: Linux and Windows, the full macOS libc build (it does not link on main yet) and any title.

AI-assisted: yes (Claude Code). I ran all of the above on my machine.