Repository navigation
fix(libkernel): report the precommitted stack of an adopted thread on Windows - #5918
Merged
adriadam10 merged 1 commit intoOct 18, 2026
Merged
Conversation
Contributor
|
This overlaps in effect with #5703, which is about the same Windows host-stack layout from the other side. #5703 makes The difference a title can see is the reported stack bottom itself. Here it moves up by up to 64 KiB, so a collector or guard-page logic that uses |
12 tasks done
Contributor
|
Our case on Resident Evil Village (PPSA01556), to make the comparison concrete. The protect that stops is on thread 23012's own stack. A local trace of It's a default-size host thread, not one AnyPS5 creates: its lowest 0xd000 bytes were still only reserved, and threads AnyPS5 creates are precommitted since 999a723. So this PR's rule should apply to it, and Village would then protect only committed pages. We'll boot Village with this PR in place of #5703 and post whether it passes that stop. @karimibrahim07, could you try PPSA99869 with #5703 in place of this PR? On fidelity: on FreeBSD a thread's reported stack is all mapped, so a title can read or protect any part of it. On Windows the bottom of a host stack is only reserved, so neither holds there today. This PR reports only the committed part; #5703 keeps the reported range and commits what a protect needs. If one of them alone covers both titles, keep that one. If each covers only its own, keep both. |
Contributor
Author
|
@EagleBearerYS here is PPSA99869 with #5703 in place of this PR, on
@CnRJay, the part only this PR changes is what the reported range contains. On a relinked executable's main thread and on default-size host threads, the lowest 64 KiB of the reported stack are 0xd000 reserved bytes and 0x3000 bytes of Windows guard pages. With #5703 alone they stay that way until something protects them, so reading the reserved part faults. With this PR, the stack that The two PRs don't conflict and cover different cases, so I'd keep both. If the maintainers would rather take one, #5703 alone fixes this title. If both land, I'll update this PR's debt line, because |
Contributor
|
Thanks for running it both ways. Agreed, they don't conflict and cover different cases:
So both can land. Our Village boot with this PR in place of #5703 is still queued; I'll post whether it gets past that stop. |
Contributor
|
The Village boot: Resident Evil Village (PPSA01556) with this PR in place of #5703 gets past that stop. Setup:
What changed:
So each PR alone covers both titles, PPSA99869 and Village. |
11 tasks done
karimibrahim07
force-pushed
the
fix/libkernel-adopted-stack
branch
from
October 14, 2026 20:52
2696360 to
3ee93b0
Compare
This was referenced Oct 15, 2026
11 tasks done
|
Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
This was referenced Oct 15, 2026
Closed
Merged
12 tasks done
Contributor
|
I reproduced the remaining The release's The patch permits fixed anonymous replacement within one owned native Windows stack, holding the stack ownership lock through validation and replacement. It preserves the reservation, rejects foreign/cross-boundary/native PAGE_GUARD ranges before mutation, and decommits/recommits the target pages. It also preserves protection tracking on both sides of partial replacements so thread cleanup restores them correctly. Only POSIX fixed anonymous mmap opts into this path; other mapping APIs keep their previous behavior. Arena host-allocation holes are excluded from arena ownership. Current-stack replacements must end at least 64 KiB below active host frames; peer-stack callers must coordinate with the peer. No additional mmap flags are enabled. Windows 11 / WinLibs GCC 15.2.0 results:
Baseline: Patched: The patch is based on main Release ZIP SHA-256: Patch against maindiff --git a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp
index 4dbce761d..ca0423e49 100644
--- a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp
+++ b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp
@@ -794,15 +794,15 @@ void* MapPlaced(void* addr, size_t len, int prot, int flags, size_t alignment) {
return aligned;
}
-void EraseProtections(std::uintptr_t start, std::uintptr_t end) {
- auto it = g_protections.lower_bound(start);
- if (it != g_protections.begin() && std::prev(it)->second.end > start) --it;
- while (it != g_protections.end() && it->first < end) {
+void EraseProtections(std::uintptr_t start, std::uintptr_t end, std::map<std::uintptr_t, ProtectedRange>& protections = g_protections) {
+ auto it = protections.lower_bound(start);
+ if (it != protections.begin() && std::prev(it)->second.end > start) --it;
+ while (it != protections.end() && it->first < end) {
const auto rangeStart = it->first;
const auto range = it->second;
- it = g_protections.erase(it);
- if (rangeStart < start) g_protections.emplace(rangeStart, ProtectedRange{start, range.prot});
- if (range.end > end) it = g_protections.emplace(end, ProtectedRange{range.end, range.prot}).first;
+ it = protections.erase(it);
+ if (rangeStart < start) protections.emplace(rangeStart, ProtectedRange{start, range.prot});
+ if (range.end > end) it = protections.emplace(end, ProtectedRange{range.end, range.prot}).first;
}
}
@@ -937,9 +937,56 @@ int DoMapDirect(void** addr, size_t len, int prot, int flags, int64_t physStart,
return 0;
}
-int DoMapAnon(void** addr, size_t len, int prot, int flags, size_t alignment, bool exclusive) {
+int DoMapAnon(void** addr, size_t len, int prot, int flags, size_t alignment, bool exclusive, [[maybe_unused]] bool allowStackReplacement) {
ValidateOutput(addr);
if (len == 0 || (len & (PS5_PAGE_SIZE - 1)) != 0) return SCE_KERNEL_ERROR_EINVAL;
+#ifdef _WIN32
+ if (allowStackReplacement && (flags & GuestMapFixedFlag) != 0 && !KernelArena::Get().Owns(*addr, len)) {
+ constexpr int GuestMapNoOverwrite = 0x80;
+ if ((prot & ~7) != 0 || (flags & ~(GuestMapFixedFlag | GuestMapNoOverwrite)) != 0)
+ return SCE_KERNEL_ERROR_EINVAL;
+ ValidateRange(*addr, len, ValidateAlignment(alignment));
+ const auto start = reinterpret_cast<std::uintptr_t>(*addr);
+ int result = SCE_KERNEL_ERROR_EINVAL;
+ ReplaceGuestThreadStack(start, len, (prot & 3) == 3, [&] {
+ if (exclusive || (flags & GuestMapNoOverwrite) != 0) {
+ result = SCE_KERNEL_ERROR_ENOMEM;
+ return false;
+ }
+ void* allocation = nullptr;
+ for (auto cursor = start; cursor < start + len;) {
+ MEMORY_BASIC_INFORMATION memory{};
+ if (VirtualQuery(reinterpret_cast<void*>(cursor), &memory, sizeof(memory)) != sizeof(memory) ||
+ (memory.State != MEM_COMMIT && memory.State != MEM_RESERVE) || memory.Type != MEM_PRIVATE ||
+ (memory.Protect & PAGE_GUARD) != 0 ||
+ memory.RegionSize == 0 || reinterpret_cast<std::uintptr_t>(memory.BaseAddress) > cursor ||
+ memory.RegionSize > std::numeric_limits<std::uintptr_t>::max() - reinterpret_cast<std::uintptr_t>(memory.BaseAddress))
+ return false;
+ if (allocation == nullptr) allocation = memory.AllocationBase;
+ if (memory.AllocationBase != allocation) return false;
+ const auto next = reinterpret_cast<std::uintptr_t>(memory.BaseAddress) + memory.RegionSize;
+ if (next <= cursor) return false;
+ cursor = std::min(start + len, next);
+ }
+ std::lock_guard lock(g_protectionLock);
+ auto replacement = g_protections;
+ EraseProtections(start, start + len, replacement);
+ const auto recorded = replacement.emplace(start, ProtectedRange{start + len, prot}).first;
+ result = SCE_KERNEL_ERROR_ENOMEM;
+ if (!VirtualFree(*addr, len, MEM_DECOMMIT)) return false;
+ if (VirtualAlloc(*addr, len, MEM_COMMIT, WinProtFromPosix(LinuxProtFromSce(prot))) != *addr) {
+ const auto error = GetLastError();
+ recorded->second.prot = 0;
+ g_protections.swap(replacement);
+ throw std::system_error(static_cast<int>(error), std::system_category(), "Guest stack replacement commit failed");
+ }
+ g_protections.swap(replacement);
+ result = 0;
+ return true;
+ });
+ return result;
+ }
+#endif
GuestAllocations::Mutation mutation;
if (exclusive && mutation.Overlaps(*addr, len)) return SCE_KERNEL_ERROR_ENOMEM;
flags = PlaceHintInReservation(*addr, len, flags);
diff --git a/core/libs/prx/libkernel/DirectMemory/DirectMemory.hpp b/core/libs/prx/libkernel/DirectMemory/DirectMemory.hpp
index a17307030..65028ac7a 100644
--- a/core/libs/prx/libkernel/DirectMemory/DirectMemory.hpp
+++ b/core/libs/prx/libkernel/DirectMemory/DirectMemory.hpp
@@ -35,7 +35,7 @@ void DirectMemoryRetype(int64_t start, size_t len, int memoryType);
bool DirectMemoryLargestFree(int64_t searchStart, int64_t searchEnd, size_t alignment, int64_t* startOut, size_t* sizeOut);
size_t DirectMemoryFreeRun(uint64_t offset, uint64_t limit);
int DoMapDirect(void** addr, size_t len, int prot, int flags, int64_t physStart, size_t alignment);
-int DoMapAnon(void** addr, size_t len, int prot, int flags, size_t alignment = PS5_PAGE_SIZE, bool exclusive = false);
+int DoMapAnon(void** addr, size_t len, int prot, int flags, size_t alignment = PS5_PAGE_SIZE, bool exclusive = false, bool allowStackReplacement = false);
int DoMapFileBacked_nid_no_patch(void** addr, size_t len, int prot, int flags, int descriptor, std::uint64_t offset, size_t alignment);
int DoMapShared(void** addr, size_t len, int prot, HostSharedMemory object);
int DoMprotect(const void* addr, size_t len, int prot);
@@ -49,4 +49,4 @@ void ForgetJitSharedMemory(int descriptor);
void DuplicateJitSharedMemory(int from, int to);
void GuestPageTableUsage(std::uint64_t granularity, std::uint64_t* cpu_entries, std::uint64_t* gpu_entries);
-#endif
\ No newline at end of file
+#endif
diff --git a/core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp b/core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp
index 6f2c99760..d1426843f 100644
--- a/core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp
+++ b/core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp
@@ -109,15 +109,17 @@ void* APS5_VABI mmap_nid_postfix(void* address, std::size_t length, int protecti
if (descriptor != -1 || offset != 0) return failed(GuestInvalid);
const auto start = reinterpret_cast<std::uintptr_t>(address);
if (fixedAnonymous && ((start & (PS5_PAGE_SIZE - 1)) != 0 ||
- rounded > std::numeric_limits<std::uintptr_t>::max() - start ||
- !GuestArena::GuestArenaContains_nid_postfix(address, rounded))) return failed(GuestInvalid);
+ rounded > std::numeric_limits<std::uintptr_t>::max() - start)) return failed(GuestInvalid);
+#ifndef _WIN32
+ if (fixedAnonymous && !GuestArena::GuestArenaContains_nid_postfix(address, rounded)) return failed(GuestInvalid);
+#endif
const bool exclusive = fixedAnonymous && (mappingFlags & GuestMapExclusive) != 0;
const int placement = fixedAnonymous ? GuestMapFixed | (exclusive ? GuestMapNoOverwrite : 0) : 0;
void* mapped = fixedAnonymous ? address : nullptr;
const auto alignment = !fixedAnonymous && (flags & GuestAlignedSuper) != 0 ? SuperpageSize : PS5_PAGE_SIZE;
try {
- const auto result = DoMapAnon(&mapped, rounded, protection, placement, alignment, exclusive);
- if (result != 0) return failed(GuestNoMemory);
+ const auto result = DoMapAnon(&mapped, rounded, protection, placement, alignment, exclusive, fixedAnonymous);
+ if (result != 0) return failed(result == SCE_KERNEL_ERROR_EINVAL ? GuestInvalid : GuestNoMemory);
return mapped;
} catch (const std::invalid_argument&) {
return failed(fixedAnonymous ? GuestInvalid : GuestNoMemory);
diff --git a/core/libs/prx/libkernel/Pthread/include/Pthread.hpp b/core/libs/prx/libkernel/Pthread/include/Pthread.hpp
index 1252760c5..562fb3005 100644
--- a/core/libs/prx/libkernel/Pthread/include/Pthread.hpp
+++ b/core/libs/prx/libkernel/Pthread/include/Pthread.hpp
@@ -10,6 +10,7 @@
#include <cstddef>
#include <cstdint>
#include <ctime>
+#include <functional>
#include <mutex>
#include <string>
#include <thread>
@@ -155,6 +156,9 @@ struct PthreadPrivate {
bool GuestThreadStack(std::uintptr_t address, std::uintptr_t* start, std::uintptr_t* end);
bool GuestThreadStackHostRangeOverlaps(std::uintptr_t start, std::uintptr_t end);
void NoteGuestStackProtection(std::uintptr_t start, std::size_t bytes, bool readWrite);
+#ifdef _WIN32
+bool ReplaceGuestThreadStack(std::uintptr_t start, std::size_t bytes, bool readWrite, const std::function<bool()>& replace);
+#endif
int GuestThreadCpuClockId(const PthreadPrivate* thread);
bool GuestThreadCpuNanos(int cpuClockThread, std::uint64_t* nanos);
void SetLiveGuestThreadsAffinity(KernelCpumask mask);
diff --git a/core/libs/prx/libkernel/Pthread/src/Thread.cpp b/core/libs/prx/libkernel/Pthread/src/Thread.cpp
index b7e180b1c..be2088ef2 100644
--- a/core/libs/prx/libkernel/Pthread/src/Thread.cpp
+++ b/core/libs/prx/libkernel/Pthread/src/Thread.cpp
@@ -223,20 +223,65 @@ static void RegisterStack(PthreadPrivate* self) {
static void RestoreStackPages(std::uintptr_t start, std::size_t bytes) noexcept {
#ifdef _WIN32
DWORD previous = 0;
- const bool restored = VirtualProtect(reinterpret_cast<void*>(start), bytes, PAGE_READWRITE, &previous) != 0;
+ const bool restored = VirtualAlloc(reinterpret_cast<void*>(start), bytes, MEM_COMMIT, PAGE_READWRITE) != nullptr &&
+ VirtualProtect(reinterpret_cast<void*>(start), bytes, PAGE_READWRITE, &previous) != 0;
#else
const bool restored = ::mprotect(reinterpret_cast<void*>(start), bytes, PROT_READ | PROT_WRITE) == 0;
#endif
if (!restored) std::fprintf(stderr, "[thread] cannot restore the protection of guest stack pages 0x%llx+0x%zx at thread exit\n", static_cast<unsigned long long>(start), bytes);
}
+static void ReplaceStackProtection(std::map<std::uintptr_t, std::size_t>& ranges, std::uintptr_t start, std::size_t bytes, bool readWrite) {
+ const auto end = start + bytes;
+ auto it = ranges.upper_bound(start);
+ if (it != ranges.begin() && std::prev(it)->first + std::prev(it)->second > start) --it;
+ while (it != ranges.end() && it->first < end) {
+ const auto previousStart = it->first;
+ const auto previousEnd = previousStart + it->second;
+ it = ranges.erase(it);
+ if (previousStart < start) ranges.emplace(previousStart, start - previousStart);
+ if (previousEnd > end) {
+ ranges.emplace(end, previousEnd - end);
+ break;
+ }
+ }
+ if (!readWrite) ranges[start] = bytes;
+}
+
void NoteGuestStackProtection(std::uintptr_t start, std::size_t bytes, bool readWrite) {
std::lock_guard lock(stackLock);
- auto it = protectedStackPages.upper_bound(start);
- if (it != protectedStackPages.begin() && std::prev(it)->first + std::prev(it)->second > start) --it;
- while (it != protectedStackPages.end() && it->first < start + bytes) it = protectedStackPages.erase(it);
- if (!readWrite) protectedStackPages[start] = bytes;
+ auto replacement = protectedStackPages;
+ ReplaceStackProtection(replacement, start, bytes, readWrite);
+ protectedStackPages.swap(replacement);
+}
+
+#ifdef _WIN32
+bool ReplaceGuestThreadStack(std::uintptr_t start, std::size_t bytes, bool readWrite, const std::function<bool()>& replace) {
+ if (bytes == 0 || bytes > std::numeric_limits<std::uintptr_t>::max() - start) return false;
+ std::uintptr_t low = 0;
+ std::uintptr_t high = 0;
+ const bool current = CurrentStack(&low, &high) && start >= low && start + bytes <= high;
+ constexpr std::uintptr_t HostFrameReserve = 64 * 1024;
+ const auto frame = reinterpret_cast<std::uintptr_t>(&low);
+ if (current && (frame < HostFrameReserve || start + bytes > frame - HostFrameReserve)) return false;
+ std::lock_guard lock(stackLock);
+ if (!current && std::none_of(liveStacks.begin(), liveStacks.end(), [&](const auto& entry) {
+ return start >= entry.second.first && start + bytes <= entry.second.second;
+ })) return false;
+ auto replacement = protectedStackPages;
+ ReplaceStackProtection(replacement, start, bytes, readWrite);
+ auto failure = protectedStackPages;
+ ReplaceStackProtection(failure, start, bytes, false);
+ try {
+ if (!replace()) return false;
+ } catch (...) {
+ protectedStackPages.swap(failure);
+ throw;
+ }
+ protectedStackPages.swap(replacement);
+ return true;
}
+#endif
static void UnregisterStack(PthreadPrivate* self) {
std::lock_guard lock(stackLock);
diff --git a/core/libs/tests/GuestKernelStackProtect.cpp b/core/libs/tests/GuestKernelStackProtect.cpp
index 8df5bb638..f136070ac 100644
--- a/core/libs/tests/GuestKernelStackProtect.cpp
+++ b/core/libs/tests/GuestKernelStackProtect.cpp
@@ -1,5 +1,6 @@
#include "SceTypes.hpp"
#include <atomic>
+#include <cstdio>
#include <cstdint>
#include <cstdlib>
#ifdef _WIN32
@@ -8,15 +9,19 @@
#endif
#include <windows.h>
#else
-#include <cstdio>
#include <sys/mman.h>
#endif
extern "C" {
int APS5_VABI scePthreadCreate(Pthread* thread, const PthreadAttr* attr, PthreadEntry entry, void* arg, const char* name);
int APS5_VABI scePthreadJoin(Pthread thread, void** retval);
+int APS5_VABI scePthreadKeyCreate(PthreadKey* key, pthread_key_destructor_func_t destructor);
+int APS5_VABI scePthreadKeyDelete(PthreadKey key);
+int APS5_VABI scePthreadSetspecific(PthreadKey key, void* value);
int APS5_VABI sceKernelIsStack(void* addr, void** start, void** end);
+void* APS5_VABI mmap_nid_postfix(void* address, std::size_t length, int protection, int flags, int descriptor, std::int64_t offset) noexcept;
int APS5_VABI mprotect_nid_postfix(void* address, std::size_t length, int protection) noexcept;
+int* APS5_VABI __error_nid_postfix();
}
static constexpr int SCE_OK = 0;
@@ -73,6 +78,134 @@ static void UnmapHostPages(void* pages) {
#endif
}
+#ifdef _WIN32
+static void CheckFixedStackRemap(std::uintptr_t low, void* start) {
+ constexpr int FixedPrivateAnonymous = 0x1012;
+ auto* const before = reinterpret_cast<volatile unsigned char*>(low);
+ auto* const page = reinterpret_cast<volatile unsigned char*>(low + Page);
+ auto* const after = reinterpret_cast<volatile unsigned char*>(low + 2 * Page);
+ auto* const address = const_cast<unsigned char*>(page);
+ for (std::size_t index = 0; index < Page; ++index) {
+ before[index] = 0x35;
+ page[index] = 0xa7;
+ after[index] = 0x59;
+ }
+ const auto checkBytes = [&](unsigned char expected) {
+ for (std::size_t index = 0; index < Page; ++index) {
+ Require(before[index] == 0x35);
+ Require(page[index] == expected);
+ Require(after[index] == 0x59);
+ }
+ };
+ const auto reject = [&](void* candidate, std::size_t bytes, int flags, int descriptor, std::int64_t offset, int error) {
+ *__error_nid_postfix() = 0;
+ Require(mmap_nid_postfix(candidate, bytes, 0, flags, descriptor, offset) == reinterpret_cast<void*>(static_cast<std::uintptr_t>(-1)));
+ if (error != 0) Require(*__error_nid_postfix() == error);
+ checkBytes(0xa7);
+ };
+ reject(address + 1, Page, FixedPrivateAnonymous, -1, 0, 22);
+ reject(address, Page, FixedPrivateAnonymous, 0, 0, 22);
+ reject(address, Page, FixedPrivateAnonymous, -1, Page, 22);
+ reject(address, Page, FixedPrivateAnonymous | 0x4000, -1, 0, 0);
+ int active = 0;
+ const auto activePage = reinterpret_cast<std::uintptr_t>(&active) & ~(Page - 1);
+ void* activeStart = nullptr;
+ void* activeEnd = nullptr;
+ Require(sceKernelIsStack(&active, &activeStart, &activeEnd) == SCE_OK);
+ Require(activePage >= reinterpret_cast<std::uintptr_t>(activeStart));
+ Require(activePage + Page <= reinterpret_cast<std::uintptr_t>(activeEnd));
+ const auto activeBefore = QueryPage(activePage).raw;
+ const auto activeEndBefore = QueryPage(activePage + Page - 1).raw;
+ reject(reinterpret_cast<void*>(activePage), Page, FixedPrivateAnonymous, -1, 0, 22);
+ Require(QueryPage(activePage).raw == activeBefore);
+ Require(QueryPage(activePage + Page - 1).raw == activeEndBefore);
+ const auto boundary = reinterpret_cast<std::uintptr_t>(start) & ~(Page - 1);
+ Require(boundary >= Page);
+ const auto outside = boundary - Page;
+ const auto outsideBefore = QueryPage(outside).raw;
+ reject(reinterpret_cast<void*>(outside), 2 * Page, FixedPrivateAnonymous, -1, 0, 22);
+ Require(QueryPage(outside).raw == outsideBefore);
+
+ void* const host = MapHostPages();
+ Require(host != nullptr);
+ auto* const hostBytes = static_cast<volatile unsigned char*>(host);
+ hostBytes[0] = 0x63;
+ hostBytes[HostBytes - 1] = 0x94;
+ const auto hostBefore = QueryPage(reinterpret_cast<std::uintptr_t>(host)).raw;
+ reject(host, Page, FixedPrivateAnonymous, -1, 0, 22);
+ Require(QueryPage(reinterpret_cast<std::uintptr_t>(host)).raw == hostBefore);
+ Require(hostBytes[0] == 0x63 && hostBytes[HostBytes - 1] == 0x94);
+ UnmapHostPages(host);
+
+ SYSTEM_INFO system{};
+ GetSystemInfo(&system);
+ const auto hostPage = static_cast<std::size_t>(system.dwPageSize);
+ Require(hostPage != 0 && 2 * hostPage <= Page);
+ auto* const guarded = address + hostPage;
+ DWORD originalProtection = 0;
+ Require(VirtualProtect(guarded, hostPage, PAGE_READWRITE | PAGE_GUARD, &originalProtection) != 0);
+ Require(originalProtection == PAGE_READWRITE);
+ *__error_nid_postfix() = 0;
+ Require(mmap_nid_postfix(address, Page, 0, FixedPrivateAnonymous, -1, 0) == reinterpret_cast<void*>(static_cast<std::uintptr_t>(-1)));
+ Require(*__error_nid_postfix() == 22);
+ MEMORY_BASIC_INFORMATION guardedMemory{};
+ Require(VirtualQuery(guarded, &guardedMemory, sizeof(guardedMemory)) == sizeof(guardedMemory));
+ Require(guardedMemory.State == MEM_COMMIT && guardedMemory.Protect == (PAGE_READWRITE | PAGE_GUARD));
+ DWORD replacedProtection = 0;
+ Require(VirtualProtect(guarded, hostPage, originalProtection, &replacedProtection) != 0);
+ Require(replacedProtection == (PAGE_READWRITE | PAGE_GUARD));
+ checkBytes(0xa7);
+
+ Require(mmap_nid_postfix(address, Page, 0, FixedPrivateAnonymous, -1, 0) == address);
+ MEMORY_BASIC_INFORMATION memory{};
+ Require(VirtualQuery(address, &memory, sizeof(memory)) == sizeof(memory));
+ Require(memory.State == MEM_COMMIT && memory.Protect == PAGE_NOACCESS);
+ Require(reinterpret_cast<std::uintptr_t>(memory.BaseAddress) + memory.RegionSize >= low + 2 * Page);
+ Require(mprotect_nid_postfix(address, Page, 3) == 0);
+ checkBytes(0);
+ for (std::size_t index = 0; index < Page; ++index) page[index] = 0xc8;
+ Require(mmap_nid_postfix(address, Page, 3, FixedPrivateAnonymous, -1, 0) == address);
+ Require(QueryPage(low + Page).writable);
+ checkBytes(0);
+ page[0] = 0x76;
+ page[Page - 1] = 0x82;
+ Require(page[0] == 0x76 && page[Page - 1] == 0x82);
+}
+
+struct PeerStack {
+ std::atomic<int> phase{0};
+ void* start = nullptr;
+ std::uintptr_t low = 0;
+};
+
+static void* APS5_VABI WaitForPeerRemap(void* arg) {
+ auto& context = *static_cast<PeerStack*>(arg);
+ int local = 0;
+ void* end = nullptr;
+ Require(sceKernelIsStack(&local, &context.start, &end) == SCE_OK);
+ context.low = (reinterpret_cast<std::uintptr_t>(context.start) + Page - 1) & ~(Page - 1);
+ Require(context.low + GuardBytes + Page < reinterpret_cast<std::uintptr_t>(&local));
+ Require(mprotect_nid_postfix(reinterpret_cast<void*>(context.low), GuardBytes, 3) == 0);
+ context.phase.store(1);
+ context.phase.notify_one();
+ context.phase.wait(1);
+ Require(context.phase.load() == 2);
+ return nullptr;
+}
+
+static void CheckPeerStackRemap() {
+ PeerStack context;
+ Pthread thread = nullptr;
+ Require(scePthreadCreate(&thread, nullptr, WaitForPeerRemap, &context, nullptr) == SCE_OK);
+ context.phase.wait(0);
+ Require(context.phase.load() == 1);
+ CheckFixedStackRemap(context.low, context.start);
+ context.phase.store(2);
+ context.phase.notify_one();
+ Require(scePthreadJoin(thread, nullptr) == SCE_OK);
+}
+#endif
+
static void* APS5_VABI Worker(void* arg) {
auto& passed = *static_cast<std::atomic<bool>*>(arg);
int local = 0;
@@ -119,6 +252,9 @@ static void* APS5_VABI Worker(void* arg) {
Require(QueryPage(low).writable);
static_cast<volatile unsigned char*>(start)[0] = 4;
Require(mprotect_nid_postfix(start, stackBytes, 3) == 0);
+#ifdef _WIN32
+ CheckFixedStackRemap(low, start);
+#endif
void* host = MapHostPages();
Require(host != nullptr);
Require(mprotect_nid_postfix(host, HostBytes, 1) == -1);
@@ -129,31 +265,68 @@ static void* APS5_VABI Worker(void* arg) {
return nullptr;
}
-static std::atomic<std::uintptr_t> leftGuard{0};
+enum class GuardMode { Protect, Remap, Split };
-static void* APS5_VABI LeaveGuard(void*) {
- int local = 0;
- void* start = nullptr;
- void* end = nullptr;
- Require(sceKernelIsStack(&local, &start, &end) == SCE_OK);
- const auto low = (reinterpret_cast<std::uintptr_t>(start) + Page - 1) & ~(Page - 1);
- Require(mprotect_nid_postfix(reinterpret_cast<void*>(low), GuardBytes, 0) == 0);
- leftGuard.store(low);
- return nullptr;
+struct GuardContext {
+ GuardMode mode;
+ PthreadKey key = -1;
+ std::uintptr_t low = 0;
+ std::atomic<unsigned> calls{0};
+};
+
+static void APS5_VABI CheckRestoredGuard(void* arg) {
+ auto& context = *static_cast<GuardContext*>(arg);
+ Require(context.low != 0);
+ auto* const bytes = reinterpret_cast<volatile unsigned char*>(context.low);
+ for (std::uintptr_t offset = 0; offset < GuardBytes; offset += Page) {
+ Require(QueryPage(context.low + offset).writable);
+ bytes[offset] = 4;
+ bytes[offset + Page - 1] = 5;
+ Require(bytes[offset] == 4 && bytes[offset + Page - 1] == 5);
+ }
+ context.calls.fetch_add(1);
}
-static void* APS5_VABI WriteLeftGuard(void*) {
+static void* APS5_VABI LeaveGuard(void* arg) {
+ auto& context = *static_cast<GuardContext*>(arg);
int local = 0;
void* start = nullptr;
void* end = nullptr;
Require(sceKernelIsStack(&local, &start, &end) == SCE_OK);
- const auto guard = leftGuard.load();
- if (guard >= reinterpret_cast<std::uintptr_t>(start) && guard + GuardBytes <= reinterpret_cast<std::uintptr_t>(&local)) {
- reinterpret_cast<volatile unsigned char*>(guard)[0] = 4;
+ const auto low = (reinterpret_cast<std::uintptr_t>(start) + Page - 1) & ~(Page - 1);
+ Require(low + GuardBytes + Page < reinterpret_cast<std::uintptr_t>(&local));
+ context.low = low;
+ Require(scePthreadSetspecific(context.key, &context) == SCE_OK);
+ const auto mode = context.mode;
+ if (mode == GuardMode::Remap) {
+ Require(mmap_nid_postfix(reinterpret_cast<void*>(low), Page, 0, 0x1012, -1, 0) == reinterpret_cast<void*>(low));
+ } else {
+ Require(mprotect_nid_postfix(reinterpret_cast<void*>(low), GuardBytes, 0) == 0);
+ if (mode == GuardMode::Split) {
+ const auto leftBefore = QueryPage(low).raw;
+ const auto rightBefore = QueryPage(low + 2 * Page).raw;
+ const auto rightEndBefore = QueryPage(low + 3 * Page).raw;
+ auto* const middle = reinterpret_cast<void*>(low + Page);
+ Require(mmap_nid_postfix(middle, Page, 3, 0x1012, -1, 0) == middle);
+ Require(QueryPage(low + Page).writable);
+ Require(QueryPage(low).raw == leftBefore);
+ Require(QueryPage(low + 2 * Page).raw == rightBefore);
+ Require(QueryPage(low + 3 * Page).raw == rightEndBefore);
+ }
}
return nullptr;
}
+static void CheckGuardCleanup(GuardMode mode) {
+ GuardContext context{mode};
+ Require(scePthreadKeyCreate(&context.key, reinterpret_cast<pthread_key_destructor_func_t>(CheckRestoredGuard)) == SCE_OK);
+ Pthread thread = nullptr;
+ Require(scePthreadCreate(&thread, nullptr, LeaveGuard, &context, nullptr) == SCE_OK);
+ Require(scePthreadJoin(thread, nullptr) == SCE_OK);
+ Require(context.calls.load() == 1);
+ Require(scePthreadKeyDelete(context.key) == SCE_OK);
+}
+
#ifdef _WIN32
static void ProtectWholeStack() {
int local = 0;
@@ -191,16 +364,10 @@ int main() {
Require(scePthreadCreate(&thread, nullptr, Worker, &passed, nullptr) == SCE_OK);
Require(scePthreadJoin(thread, nullptr) == SCE_OK);
Require(passed.load());
- Require(scePthreadCreate(&thread, nullptr, LeaveGuard, nullptr, nullptr) == SCE_OK);
- Require(scePthreadJoin(thread, nullptr) == SCE_OK);
- for (std::uintptr_t offset = 0; offset < GuardBytes; offset += Page) {
- const auto left = QueryPage(leftGuard.load() + offset);
-#ifdef __linux__
- Require(!left.mapped || left.writable || left.raw == 1);
-#else
- Require(!left.mapped || left.writable);
+ CheckGuardCleanup(GuardMode::Protect);
+#ifdef _WIN32
+ CheckPeerStackRemap();
+ CheckGuardCleanup(GuardMode::Remap);
+ CheckGuardCleanup(GuardMode::Split);
#endif
- }
- Require(scePthreadCreate(&thread, nullptr, WriteLeftGuard, nullptr, nullptr) == SCE_OK);
- Require(scePthreadJoin(thread, nullptr) == SCE_OK);
}
diff --git a/docs/dev/debt/libkernel.md b/docs/dev/debt/libkernel.md
index 924649e64..82801640f 100644
--- a/docs/dev/debt/libkernel.md
+++ b/docs/dev/debt/libkernel.md
@@ -60,7 +60,7 @@
- [sceKernelMapFlexibleMemory / sceKernelMapNamedFlexibleMemoryInternal](../../../core/libs/prx/libkernel/DirectMemory/Export.cpp) (libkernel) - flag 0x8000 is accepted and ignored; its meaning is not established. KytyPS5 (`GUEST_MAP_UNKNOWN_8000`) and sharpemu accept it without reading it. Every known caller is an SDK 9 `libc.prx` heap, which passes it with read/write protection and uses the mapping as an older `libc.prx` uses `sceKernelMapNamedFlexibleMemory` with flags 0: through the Internal variant in Astro Bot PPSA21564, through `sceKernelMapFlexibleMemory` in Animal Well PPSA22520 01.000.011
- [sceKernelMtypeprotect](../../../core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp) (libkernel) - sets the memory type of the direct mappings in the page-rounded range and of the direct memory behind them, as shadPS4 and prosper do (KytyPS5 ignores the type); the type is not validated, flexible pages in the range only get the protection, and a later map of the retyped direct memory reports the type it was allocated with. `sceKernelBatchMap2` operation 4 takes the same path with the entry type
- [Map hints inside a reservation](../../../core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp) (libkernel) - `sceKernelMapDirectMemory` and the flexible and anonymous maps place a mapping without `MAP_FIXED` at its hint address when the whole range lies in a range still reserved by `sceKernelReserveVirtualRange`, as if `MAP_FIXED` were set. Taken from PPSA01474, which reserves 64 GiB and then maps its heaps inside it with hints and no `MAP_FIXED`, and uses the hint addresses afterwards; not measured on the console. Hints outside a reservation, or over a range that is partly mapped, stay hints
-- [Fixed anonymous mmap](../../../core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp) (libkernel) - `MAP_PRIVATE | MAP_ANON | MAP_FIXED` replaces the requested range through the existing guest mapping machinery and zeroes the new memory. `MAP_EXCL` is checked under the allocation-registry mutation lock, including virtual reservations, and returns `ENOMEM` without changing an occupied range; without `MAP_FIXED` it is `EINVAL`. Lengths are rounded to 16 KiB, while fixed addresses must be aligned and the complete rounded range must lie inside the guest arena. This follows FreeBSD 11.4 [argument validation and rounding](https://github.com/freebsd/freebsd-src/blob/8c639945a44588f6cfdddfb057bb5aad88735da0/sys/vm/vm_mmap.c#L226-L299), [fixed replacement](https://github.com/freebsd/freebsd-src/blob/8c639945a44588f6cfdddfb057bb5aad88735da0/sys/vm/vm_map.c#L1455-L1464), and [collision errno](https://github.com/freebsd/freebsd-src/blob/8c639945a44588f6cfdddfb057bb5aad88735da0/sys/vm/vm_mmap.c#L1604-L1613). The console behaviour has not been measured. AnyPS5 starts the guest arena at 0x200000000, so the fixed path rejects the low 0x400000 address requested by PICO-8 PPSA99808. The console address-space layout is unmeasured; executable placement outside the host arena is tracked in #5217. Static catalog disassembly finds `0x1012` in PPSA98001/PPSA99808 and `0x5012` in PPSA99808; these titles were not run.
+- [Fixed anonymous mmap](../../../core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp) (libkernel) - `MAP_PRIVATE | MAP_ANON | MAP_FIXED` replaces the requested range through the existing guest mapping machinery and zeroes the new memory. `MAP_EXCL` is checked under the allocation-registry mutation lock, including virtual reservations, and returns `ENOMEM` without changing an occupied range; without `MAP_FIXED` it is `EINVAL`. Lengths are rounded to 16 KiB and fixed addresses must be aligned. The complete rounded range must lie inside the guest arena or, for POSIX fixed anonymous `mmap` on Windows, one current or registered native thread stack; other mapping APIs do not opt into stack replacement. Native stack replacement keeps the reservation, decommits and recommits fresh pages, and rejects foreign allocations, ranges spanning reservations and host `PAGE_GUARD` pages before mutation. Current-stack ranges must end at least 64 KiB below the active host frame; callers replacing another thread's stack must coordinate with that thread. If recommitting fails, the call returns `ENOMEM` with the range still reserved; thread cleanup attempts to recommit it before restoring write access. This follows FreeBSD 11.4 [argument validation and rounding](https://github.com/freebsd/freebsd-src/blob/8c639945a44588f6cfdddfb057bb5aad88735da0/sys/vm/vm_mmap.c#L226-L299), [fixed replacement](https://github.com/freebsd/freebsd-src/blob/8c639945a44588f6cfdddfb057bb5aad88735da0/sys/vm/vm_map.c#L1455-L1464), and [collision errno](https://github.com/freebsd/freebsd-src/blob/8c639945a44588f6cfdddfb057bb5aad88735da0/sys/vm/vm_mmap.c#L1604-L1613). The console behaviour has not been measured. AnyPS5 starts the guest arena at 0x200000000, so the fixed path rejects the low 0x400000 address requested by PICO-8 PPSA99808. The console address-space layout is unmeasured; executable placement outside the host arena is tracked in #5217. Static catalog disassembly finds `0x1012` in PPSA98001/PPSA99808 and `0x5012` in PPSA99808; these titles were not run.
- [Mapping hints with no room above them](../../../core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp) (libkernel) - a mapping without `MAP_FIXED` whose hint lies above the guest address space, or has no free range at or above it, is placed from the bottom of the guest arena, as FreeBSD 11.4's [`vm_mmap_object`](https://github.com/freebsd/freebsd-src/blob/8c639945a44588f6cfdddfb057bb5aad88735da0/sys/vm/vm_mmap.c#L1569) retries through [`vm_map_find_min`](https://github.com/freebsd/freebsd-src/blob/8c639945a44588f6cfdddfb057bb5aad88735da0/sys/vm/vm_map.c#L1555-L1584) from its minimum address. FreeBSD's minimum is the end of the largest possible data segment rather than the start of the arena, and the console's address-space layout is not measured. PS5CEMU-HAR (PPSA99360) gives `sceKernelReserveVirtualRange` a hint near its own code, which is a host address here because the relinked executable lies outside the guest arena, so its JIT gets memory more than 2 GiB from its code and does not start
- [_sceKernelRtldThreadAtexitIncrement, _sceKernelRtldThreadAtexitDecrement](../../../core/libs/prx/libkernel/System/src/Rtld.cpp) (libkernel) - the argument is the `dso` address a title's `libc.prx` passes to `__cxa_thread_atexit` (PPSA14632), which fails when Increment returns nonzero. As in the [fpPS4](https://github.com/red-prig/fpPS4) reimplementation, Increment takes a reference on the loaded image containing the address and Decrement releases one; the variants without the leading underscore are assumed to match
- [tcgetattr, tcsetattr](../../../core/libs/prx/libkernel/File/src/Terminal.cpp) (libkernel) - every file and pipe descriptor is taken as not a terminal (`ENOTTY`), and a socket gets `EADDRNOTAVAIL` for `AF_INET` and `EOPNOTSUPP` otherwise, as FreeBSD's `in_control` and `pru_control_notsupp` answer a termios ioctl; what descriptors 0 to 2 of a title are on the console, and whether any of them is a terminal there, is unknown |
11 tasks done
karimibrahim07
force-pushed
the
fix/libkernel-adopted-stack
branch
from
October 16, 2026 02:58
3ee93b0 to
afa4740
Compare
11 tasks done
11 tasks done
Contributor
|
For coordination: I opened #6893 with the guard-size attribute-query fix. It shares |
Collaborator
|
Checked the diff, the shared committed-pages walk with |
Collaborator
|
Checked the diff, the shared committed-pages walk with |
Collaborator
|
Re-checked against current |
Contributor
Author
|
@MarcZX100 I checked the interaction on Windows and found nothing that needs an order. I merged this PR and #6893 (
The two changes touch different state: #6893 stores and reports the attribute's guard size, and this PR moves the reported bottom of an adopted host stack. Neither reads what the other writes. |
11 tasks done
12 tasks done
… Windows
An adopted thread (the main thread, or a host thread that calls
scePthreadSelf) reported its whole Windows stack reservation, and so
did sceKernelIsStack on a host thread before adoption. A relinked
executable reserves 0x210000 bytes for its main thread and commits the
top 0x200000, so the lowest 64 KiB of the reported stack hold only
reserved pages and Windows' guard pages. mprotect fails there, so
HotSpot in Java 25 LAN / DNS (PPSA99869) could not create the main
thread's stack guard pages ("Attempt to allocate stack guard pages
failed").
When everything above the bottom allocation granule is committed
read-write, the host stack is now reported from there, as created
guest threads report only their committed stack. Other host stacks
keep the whole range. The committed-pages walk is shared with
StartNativeThread and throws when VirtualQuery fails.
adriadam10
force-pushed
the
fix/libkernel-adopted-stack
branch
from
October 18, 2026 14:35
afa4740 to
267cb56
Compare
adriadam10
pushed a commit
that referenced
this pull request
Oct 19, 2026
…protect entries The entry now sits with the other sceKernelMprotect entries for DirectMemory.cpp, before the Windows guest module image protection one, instead of after the setcontext entry, where #5918 adds its own.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Java 25 LAN / DNS (PPSA99869, #5501) runs on
mainsince #5643, but its Java VM prints on screen:HotSpot (the PS5 port in mateo-cogeanu/ps5-java25-homebrew, patch
0019-ps5-preserve-native-reservations) commits the guard pages at the bottom of the main thread's stack withmprotect(bottom, 48 KiB, PROT_READ | PROT_WRITE), the bottom coming frompthread_attr_get_np. Its trace shows the call failing:The main thread is adopted (
scePthreadSelf), and on WindowsSetStackFromHostreports its whole host reservation, assceKernelIsStackdoes for a host thread before adoption. A relinked executable reserves 0x210000 bytes for that stack and commits the top 0x200000 (WindowsPeWriter.cpp), so the lowest 64 KiB of the reported stack hold only reserved pages and Windows' guard pages, and the hostmprotectinDoMprotect's guest stack branch fails there (EINVAL).Threads AnyPS5 creates already report only their committed stack (999a723a5).
HostStackLimitsnow does the same for a host thread when its stack is laid out that way: if everything above the bottom allocation granule is committed read-write, the stack starts there. Any other host stack keeps the whole range, so a stack that is not precommitted, like a test executable's 4 KiB commit, does not shrink. The top of the stack does not move, so collectors that scan from the top (Unity IL2CPP, #761) are unaffected. The committed-pages walk is shared withStartNativeThreadand throws whenVirtualQueryfails.The technical-debt entry records that host threads whose stack is not precommitted, and host threads on Linux, still report lower pages that
mprotectcannot change.Tested
Windows 11 Pro (26300), WinLibs GCC 15.2.0 r7, Release, on
mainf343489.guest_thread_selflinks with the relinker's stack sizes (--stack=0x210000,0x200000) and checks, on the main thread and on a default-size host thread (throughsceKernelIsStackbefore adoption andscePthreadAttrGetafter), that the reported stack is committed read-write throughout and that its lowest page can be made read-write, inaccessible and read-write again. A host thread that commits only 64 KiB must report exactlyGetCurrentThreadStackLimits. The test fails without the change (0xc0000409), with the trim applied only at adoption, and with the trim applied regardless of what is committed; it passes with the change (3 runs).ctest(--parallel 4 --timeout 120): 1191 of 1192 pass from either shell. Fromcmd,check_main_syncfails because the file name it prints is not decoded as UTF-8; from Git Bash,windows_trace_importsfails because Defender deletes its generated executable in Git Bash's temporary directory. Each passes in the other shell, and this PR touches neither.python tools/check_conventions.py --base origin/mainpasses;tools/pr_overlap.py --branchreports no overlap with open pull requests.PPSA99869-lan-dns-pass-01.000.000.zip, SHA-2563cfeed2f…11cc, as in Java 25 LAN / DNS (PPSA99869) #5501), relinked with--windows --to-intel --trap-sdk-syscall-stubs --registry:mainwith feat(libkernel): implement mincore #5643:FAIL: Native RW memory commit addr=0x400000 bytes=49152 mode=3 errno=22and the warning aboveDONE: Native RW memory commit addr=0x410000 bytes=49152 mode=3 errno=0, no warning,JNI_CreateJavaVM returned 0, andDestroyJavaVMcode 0. The LAN echo fails as before without its helper on a Mac; the DNS stage passed onmainwith feat(libkernel): implement mincore #5643 and was not rerun with this change.DestroyJavaVM, HotSpot now also removes the main thread's guard pages, and that fails as it already does for the VM's other threads:Attempt to deallocate stack guard pages failed (0x0000000000410000-0x000000000041c000). HotSpot's BSDos::pd_uncommit_memorymaps fresh pages there withmmap(MAP_FIXED | MAP_NORESERVE | MAP_ANONYMOUS, PROT_NONE), whichmmap_nid_postfixdoes not accept for these pages, so those 48 KiB stay inaccessible after the VM is destroyed. That is a separate fix.Linux not tested; its path is unchanged.
Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)