fix(relinker): read the PS4 needed-module tag in guest modules

[SandstromPL]

What

Until Dawn (CUSA02636) bundles libc.prx and libSceFios2.prx (01.00). Their conversion stops in GuestModuleBuilder with:

Cyclic guest initialization dependency: libSceFios2.prx

SuperCatElvis reported it on #3093 as stop 8 of the title's --windows conversion, on main d92a8da. He had the open #5138 applied and scratch fixes for his other stops: "these modules list their dependencies in 0x6100000f, not 0x61000045; falling back to it fixes the order".

GuestImageReader takes a guest module's import modules only from 0x61000045, the PS5 needed-module tag. PS4 modules use 0x6100000f (DT_SCE_NEEDED_MODULE) instead. Both tags have the same layout: module ID in bits 48-63, a version in bits 32-47 (not read here), name offset in the low 32 bits. Sources:

• Kyty names both tags (Elf.h L86-L87). It decodes them with one function (RuntimeLinker.cpp L300-L315, called at L1367-L1368).
• RPCSX decodes both in one branch (linker.cpp L688-L705).
• OpenOrbis create-fself writes one 0x6100000F per imported module (PS4Constants.go L25, OELFGenDynlibData.go L992-L997), in that layout (L882-L888).

The reader already accepts both forms of the export tag (0x6100000d and 0x61000043).

So no import of a PS4 module had a library, and each bound to any guest export with its NID. A libc.prx import declared from another module then bound to an export of libSceFios2.prx, which itself needs libc.prx, and the initialization order became a cycle.

The reader now also reads 0x6100000f, so a PS4 module's imports are tied to their declared modules, as a PS5 module's already are. A module that has both tags fails with Guest module mixes PS4 and PS5 needed-module tags. Kyty (RuntimeLinker.cpp L1363) and RPCSX (linker.cpp L803-L806) reject that too.

This changes how PS4 modules are converted, including modules built with OpenOrbis create-fself. Each change is what PS5 modules already get:

• An import declared from a module in its module's DT_NEEDED is no longer bound by NID across all modules.
  • From a guest module, it binds to that module's export.
  • From a host module, it stays a host import even when a guest module exports the NID. On Linux and macOS the dynamic linker resolves it from the host libraries.
  • On Windows the entry stub looks the import up in the declared library, then in the host libc.prx if the program's library list has it, and for libSceLibcInternal.prx also in a bundled libc.prx (WindowsEntryStubBuilder.cpp L155-L165). Before, every loaded library was searched.
• On Windows, a host library that a module imports from becomes a required load. If it is missing, the start stops with Failed to load module, where before it was skipped.
  • Guest-module imports are never weak, and guest modules need eager binding. So before this change, a NID with no provider already stopped the start with unresolved ELF import.
  • For CUSA02636 this changes only the message: no host library exports the two libSceDiscMap.prx functions that libSceFios2.prx imports. A separate PR will add that library.
• An import whose declared module is not in its module's DT_NEEDED still binds by NID alone, as before. On Windows, a host library with that module's name (up to the first dot, ignoring case) becomes a required load.
• The import names and needed-module entries of a PS4 module now get the checks a PS5 module's already get, so a malformed one fails the conversion:
  • Unknown import module ID: the ID has no needed-module entry.
  • Invalid qualified import, Invalid import module ID, Invalid import module name or Duplicate import module ID.
  • Ambiguous import module dependency: the declared module matches two DT_NEEDED names.

Not changed: executables. RelinkerPipeline still reads only 0x61000045, so the imports of a PS4 executable keep binding by NID alone; no title needs that yet.

Tested

Ubuntu 24.04, GCC 13.3, Release with BUILD_TESTING=ON, on main 97aac62. Locally the libusb submodule isn't checked out, so I removed its two lines from CMakeLists.txt. libSceUsbd then fails to compile, and ninja -k 0 built everything that does not depend on it.

guest_symbol_names (relinker/tests/test_guest_symbol_names.py) gains these cases:

Case	Result
a module whose import is declared from libSceLibcInternal.prx through 0x6100000f, next to a guest module exporting the same NID	the import stays unsuffixed (a host import), as with 0x61000045
sce_module/libc.prx exports X and imports Y from libkernel.prx; libSceFios2.prx exports Y and imports X from libc.prx (the shape of stop 8), with 0x6100000f, Linux and Windows	converts on both. On Linux, libc.prx's import of Y stays a host import, libSceFios2.prx's import of X binds to libc.prx, and libc.prx comes before libSceFios2.prx in the executable's DT_NEEDED
the same with 0x61000045 (control)	converts, as on main
a module with both tags	fails with Guest module mixes PS4 and PS5 needed-module tags
a module importing #A#C (module ID 2, not listed) through 0x6100000f	fails with Unknown import module ID
an import whose declared module (libFoo) is not in its DT_NEEDED (libFoo2.prx), through 0x6100000f	still binds by NID (shared#guest), as on main

test_windows_import_modules.py's consumer() gains a needed_module_tag parameter (default 0x61000045), so existing callers are unchanged.

On main:

• The first case gets shared#guest instead of shared.
• The stop-8 shape fails on both Linux and Windows with Cyclic guest initialization dependency: .../sce_module/libSceFios2.prx, the line in the report.
• The mixed module and the unknown-ID module convert.

With the change all cases pass, and so do windows_import_modules, guest_module_identity and guest_registry (which use consumer()), 5 runs out of 5. The Windows required-load behaviour comes from the existing code path for PS5 modules (WindowsEntryStubBuilder.cpp); no new test checks the load order.

Other checks:

• Full ctest on the RTX 3050 Ti: 1,253 of 1,257 pass, and all relinker tests pass.
  • 12 tests skip themselves: agc_driver_stencil_ref_export on this GPU, and the VideoOut flip and frame-dump tests, which need a display.
  • patched_libraries and guest_usbd are Not Run, because the libusb submodule isn't checked out.
  • Two AGC tests failed, in code this branch does not change. agc_pending_key_stores fails on main since perf(agc): clear a register fast clear without waiting for a known key store #6247, as reported there. agc_driver_store_over_untracked fails intermittently when run alone here.
• OpenOrbis samples: @Get-fix ran them with this PR on Windows 11 (comment), on main f633042 with fix(relinker): search libc.prx for libkernel-only Windows executables #5990 to fix(relinker): stop module syscall scans at the PLT and skip the PS4 loader path #5994, fix(relinker): accept section-relative guest module relocations #5138 and feat(libSceFreeType): implement the six FreeType functions OpenOrbis samples use #6376.
  • using_library still starts and calls into libExample.prx, a create-fself module with DT_NEEDED libkernel.prx and one 0x6100000f entry. Its import stubs now name libkernel.prx. __error, which AnyPS5 exports from libc.prx, still resolves through the libc.prx step.
  • The 13 other samples that start without this PR still start; none of them has an sce_module.
• macOS: @jotitadev ran it on an Apple M2 Max (review), on main b7cedd9, relinker-only builds for arm64 and for x86_64 under Rosetta.
  • The relinker suite passes on arm64. On x86_64 three Python 3.9 tests fail the same way on main.
  • guest_symbol_names fails on main with only the test changes, and passes with the fix.
  • With --macos, which the new test doesn't cover, main stops the 0x6100000f pair with the same cycle. With this PR both tags convert, the Mach-O load commands are identical for the two, and dyld initializes libc.prx first.
• Not run on the game here; we have no game files. The stop is SuperCatElvis's.
• Merges: the branch merges cleanly with main 4186f06. Six open PRs touch these files (fix(windows): enter guest startup on an arena-backed main stack #3965, fix(relinker): accept section-relative guest module relocations #5138, fix(relinker): preserve DF_SYMBOLIC on guest modules #5981, fix(relinker): run guest preinit arrays before the module initializer #6112, refactor(relinker): remove functions nothing calls #6325, fix(relinker): accept matching defined relative symbols #6357), and none of them gets a new conflict.
  • fix(relinker): run guest preinit arrays before the module initializer #6112 already conflicts with main, in core/relinker/CMakeLists.txt and docs/dev/debt/relinker.md, and has the same two conflicts with this PR merged. Its GuestImageReader.cpp line is in the lifecycle-array block, apart from this PR's hunks.
  • With those two resolved by keeping both sides, a relinker-only build of main 4186f06, this PR and fix(relinker): run guest preinit arrays before the module initializer #6112 passes all 182 relinker tests, including fix(relinker): run guest preinit arrays before the module initializer #6112's guest_preinit.
• tools/check_conventions.py from main passes with this title and body.

@Get-fix, thanks for running the OpenOrbis samples. @SuperCatElvis, could you check that this gets libc.prx and libSceFios2.prx past stop 8, and say which libc.prx import resolved to libSceFios2.prx?

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Fixes: none
• Unblocks: CUSA01664 LIMBO (CUSA01664) #6978 (Limbo: Get-fix's bring-up lists this PR in step 1, converting the bundled libc.prx and libSceFios2.prx); CUSA03312 Hasbro Family Fun Pack (CUSA03312) #7198 (Hasbro Family Fun Pack: stop 7 in savvaniss's conversion); CUSA02636 (no Game bring-up discussion): stop 8 of SuperCatElvis's --windows conversion on Relinker rejects PS4 executables: SCE dynamic tables and data in the code segment (two ready branches) + remaining PS4 work #3093, in the bundled libc.prx and libSceFios2.prx.
  • The title does not reach it on current main. A full conversion scans the eboot first, and stop 1 (its jump tables) still stops it there.
  • The modules are then read (stops 4-7; fix(relinker): accept section-relative guest module relocations #5138 covers 4, fix(relinker): accept matching defined relative symbols #6357 covers 6), and ordered (stop 8, this PR).
  • Their code analysis follows (stops 9 and 10, fix(relinker): find the PLT before the code in PS4 modules #6354; stop 11, fix(relinker): accept matching defined relative symbols #6357), and the eboot's Windows image comes last (stops 2 and 3).
  • Not rerun here.
• GPUs tested: n/a (relinker); full ctest on NVIDIA GeForce RTX 3050 Ti Laptop GPU, driver 580.178.04, Ubuntu 24.04
• AI-assisted: yes (Claude Code); I reviewed and tested every line

🤖 Generated with Claude Code

[Get-fix]

@SandstromPL Checked: using_library still starts with this PR.

libExample.prx (linked by create-fself) has DT_NEEDED libkernel.prx and one 0x6100000f entry, libkernel (module ID 1, version 0x101). I ran Windows 11 with WinLibs GCC 15.2, head 6f8d9a09d merged onto main f633042df together with #5990 to #5994, #5138, #6376 and a local change that answers the Linux TIOCGWINSZ with ENOTTY:

• The PR takes effect. The converted module is byte-identical with and without this PR, while the executable changes. Its import stubs for the module now name the declared library: unresolved ELF import 9BcDykPmo1I from libkernel.prx instead of unresolved ELF import 9BcDykPmo1I.
• The program starts and calls into the module:

  Opened exampleLib: 536870912 | Jumping to 0x7fffc2850020
  test string: Hi I'm from the library! You passed: 13371338 | rv = 0x00000000
  

  The module's imports that AnyPS5 exports from libc.prx rather than libkernel.prx, such as __error (9BcDykPmo1I), still resolve, through the lookup's host libc.prx step.
• No other change: the 13 other OpenOrbis samples that start without this PR still start; none of them has an sce_module. piglet stops at its Piglet imports, as before.
• Relinker-only ctest: 178/178 with this PR, fix(relinker): search libc.prx for libkernel-only Windows executables #5990, fix(relinker): stop module syscall scans at the PLT and skip the PS4 loader path #5994 and fix(relinker): accept section-relative guest module relocations #5138 on f633042df.

AI-assisted: yes (Claude Code). I ran the builds, conversions and samples myself on Windows 11.

🤖 Generated with Claude Code

[SandstromPL]

@Get-fix Thanks for running the samples! I've added your results to the description. That __error still resolves through the libc.prx step is good to see: that step only searches a libc.prx already in the program's library list, which your #5990 adds for a libkernel-only program like using_library.

On the bot's note: it couldn't compare GuestImageReader.cpp with #6112 because #6112 conflicts with main, so I checked it by hand.

• fix(relinker): run guest preinit arrays before the module initializer #6112 adds one line in the lifecycle-array block, apart from this PR's hunks. With this PR merged, it has the same two conflicts it has with main alone (core/relinker/CMakeLists.txt and docs/dev/debt/relinker.md).
• With those resolved by keeping both sides, main 4186f06 + this PR + fix(relinker): run guest preinit arrays before the module initializer #6112 passes all 182 relinker tests, including fix(relinker): run guest preinit arrays before the module initializer #6112's guest_preinit (relinker-only build, Linux, GCC 13.3). Reverting this PR's GuestImageReader.cpp change in that tree fails guest_symbol_names as on main, and guest_preinit still passes.

[jotitadev]

Ran this on macOS. The new test converts with the Linux and --windows targets only, so I also checked the --macos output.

Setup: Apple M2 Max, macOS 27.2, head 6f8d9a09d merged without a commit onto main b7cedd900 (clean). Relinker-only Release builds for arm64 and for x86_64 under Rosetta (/usr/bin/python3 3.9.6).

• Full relinker-only suite: arm64 173/173. x86_64 170/173; the 3 failures (pr_overlap_tools, release_package, linux_fixed_base) fail the same way on main (Python 3.9 issues unrelated to this PR).
• guest_symbol_names with only the test changes applied to main fails on both arches (the guest-declared-ps4 case still gets shared#guest); with the fix it passes.

--macos, using the test's libc.prx / libSceFios2.prx pair with each tag:

• main: 0x61000045 converts; 0x6100000f fails with Cyclic guest initialization dependency: .../libSceFios2.prx, the same stop as on Windows.
• This PR: both tags convert, and the Mach-O load commands are identical for the two tags. output.elf loads libc.prx.guest.prx before libSceFios2.prx.guest.prx, and libSceFios2.prx.guest.prx has @loader_path/libc.prx.guest.prx, so dyld initializes libc first.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

[SandstromPL]

@jotitadev Thanks for the macOS run, and especially for checking --macos, which the new test doesn't cover! Good to see the Mach-O load commands come out the same for both tags. I've added your results to the description.

[github-actions]

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 6977	core/relinker/relinker/src/guest/GuestImageReader.cpp core/relinker/relinker/tests/test_guest_symbol_names.py core/relinker/relinker/tests/test_windows_import_modules.py

[savvaniss]

Second-title data point: Hasbro Family Fun Pack (CUSA03312, my own dump) needs this PR, on Linux and Windows.

Relinker-only Release build, Windows 11 Pro 26300, WinLibs GCC 15.2.0 r7. Starting from main f3059421f, I merged, in this order: #7660, #6619 (a9fa0e8b0), #5138 (5f5641732), #6381 (6f8d9a09d), 3b158c1 from #6977, #6354 (81561867f), #6357 (32fe3aa8b), #6684 (ab9d21afc) and #6618 (bf06a8e5c). With that stack, the Linux and --windows conversions of the eboot and its 9 bundled modules complete. I then rebuilt the relinker once per member with only that member left out. Without this PR, both targets stop in the eboot's module graph:

FAIL: Cyclic guest initialization dependency: ...\sce_module\libc.prx

The other members each stop at their own step, so every one of them is needed for this title; the whole table is on #6977.

AI-assisted: yes (Claude Code)