fix(relinker): find the PLT before the code in PS4 modules

[SandstromPL]

What

Until Dawn (CUSA02636) bundles libSceFios2.prx and libc.prx (01.00). Their conversion stops in the code analysis, for libSceFios2.prx with:

PLT not found after the last unwind function (offset 0x3d033)

SuperCatElvis reported it on #3093 as stop 9 of the title's conversion, on main d92a8da. They reached it after #5138 and scratch fixes for the other stops before it. Both modules have their PLT at 0xc8, at the start of the text segment, before the code. In libSceFios2.prx the last unwind function ends at 0x3d033, and what follows is the module's DT_FINI routine at 0x3d040, 0x7d bytes without an unwind entry, then read-only data that includes the zlib CRC-32 table. libc.prx has the same layout: its DT_FINI routine at 0x828a0 follows its last unwind function [0x82890, 0x82896), and about 213 KiB of read-only data follow the routine.

#3142 (Get-fix's commit, carried by MarcZX100) stopped the pipeline's syscall and call-site scans of executables at the end of the PLT that follows the last unwind function, so that the read-only data after the code is not decoded as code. My #5195 moved that search into FindSceCodeEnd and applied the same end to the code analysis of the Windows and macOS outputs, for executables and guest modules. The search only looks after the last unwind function, so these two modules fail there.

When no PLT follows the last unwind function, FindSceCodeEnd now also looks before it:

• It looks for the same PLT0 (push [GOT+8]; jmp [GOT+16]) anywhere from the start of the segment up to the end of the last unwind function.
• That PLT (PLT0 plus 16 bytes per DT_SCE_PLTRELSZ entry) must not overlap an unwind function. Otherwise the conversion fails with PLT overlaps an unwind function.
• The extent then ends at the end of the last unwind function, so the data after it stays out. Without that, stop 10 follows: overlapping instruction boundaries (offset 0x3d1fe) inside the CRC-32 table.
• A PLT found in neither place fails with PLT not found before or after the unwind functions.

A PLT after the code, as in the executables converted so far, is found first, with the same result as before. That includes #5195's case of code without unwind entries between the last function and the PLT.

When the PLT comes first, there is no PLT after the code to bound the extent. The end of the last unwind function is then the only bound that doesn't guess the length of code without an unwind entry. #3093 ruled this bound out for the PLT-after layout because code without unwind entries can sit before the PLT, and that layout is unchanged. The cost: when the PLT comes before the code, code after the last unwind function that has no unwind entry is not analysed, and nothing reports it. An entry point, a branch target or an fs: access found there is dropped without an error; the debt entry records this.

• In both modules this is the DT_FINI routine, which the Windows and macOS outputs call when the program exits (WindowsGuestStartup::Finalize, and the fini stub that MacOsImage.cpp registers with __cxa_atexit). The code analysis starts from DT_FINI only inside the extent, so both routines are dropped, and an fs: access in one would run unchanged on the host.
• For these two modules this should lose nothing. SuperCatElvis reports that the libSceFios2.prx routine makes no fs: access, and that the libc.prx one reads the pointer at 0xbc880 and calls 0x2fa50 through it; I've asked whether that one has an fs: access. The code analysis rewrites no indirect call: the relocation sets the pointer, and 0x2fa50 is already a root as a function symbol (symbol 1411 in SuperCatElvis's table), and with fix(relinker): accept matching defined relative symbols #6357 also as a relocation target.
• With the same trimming, their run got past stops 9 and 10. Their full conversion then completed only after one more scratch change, for stop 11: libc.prx's RELATIVE relocation with a symbol, which MarcZX100's fix(relinker): accept matching defined relative symbols #6357 handles.
• Both PLTs end before the first unwind function, so the overlap check passes: [0xc8, 0x5b8) before 0x5c0 in libc.prx, and [0xc8, 0x5d8) before 0x5e0 in libSceFios2.prx, each followed by 8 zero bytes.

Tested

Ubuntu 24.04, GCC 13.3, Release with BUILD_TESTING=ON, on main 71bd287. Locally the libusb submodule isn't checked out, so I removed its two lines from CMakeLists.txt. libSceUsbd then fails to compile, and ninja -k 0 built everything that does not depend on it.

sce_code_extent (relinker/tests/test_sce_code_extent.py) can now put the PLT before the code: at 0x180, before the unwind function at 0x200, with the data right after the function. It can also add a second unwind function before the PLT, and put an unsupported TLS instruction in the function. As in #5195, the fixture is converted as the main executable (--skip-sce-module), so each case goes through both callers. No case converts an sce_module module. A module like libSceFios2.prx reaches the same FindSceCodeEnd only through the code analysis, which the Windows and macOS guest module writers run on the module's own bytes and program headers.

Case	Result
PLT before the code (Windows and macOS)	converts, with the import's call site in the PLT
the same, with a lea in the function pointing into the data after it, at fs-prefixed overlapping bytes (the shape of stop 10)	converts; the bytes after the function are not analysed
the same, with the ELF entry point right after the function, on an unsupported TLS instruction	converts; the entry point after the code is not analysed (the trade-off above)
the same, with another unwind function before the PLT	converts
the same, with an unsupported TLS instruction in the function after the PLT	throws Unsupported ... guest TLS instruction, so the function after the PLT is still analysed
a PLT ending where the function begins, and a PLT starting where an earlier function ends	convert (touching is not overlapping)
a PLT over the start of the function, and over its end	throws PLT overlaps an unwind function
no PLT	throws PLT not found before or after the unwind functions (renamed)

On main all 15 cases fail with PLT not found after the last unwind function. With the change they pass, 5 runs out of 5. The test also fails if:

• the overlap check is removed, or either of its comparisons is made inclusive;
• the backward search stops 16 bytes before the end of the last function, which misses a PLT straddling it;
• the extent runs to the end of the segment, so the data is scanned (Cannot decode instruction);
• the extent ends at the PLT, so the function after it is cut off (invalid function range).

The registry check in the test now compares call sites case-insensitively: the relinker writes 0x1F0, and earlier PLT addresses happened to have no hex letters.

Other checks:

• Full ctest on the RTX 3050 Ti: 1,243 of 1,248 pass, and all relinker tests pass.
  • 12 tests skip themselves: agc_driver_stencil_ref_export on this GPU, and the VideoOut flip and frame-dump tests, which need a display.
  • patched_libraries and guest_usbd are Not Run, because the libusb submodule isn't checked out.
  • Three tests failed, in code this branch does not change:
    • agc_pending_key_stores fails on main since perf(agc): clear a register fast clear without waiting for a known key store #6247, as reported there.
    • agc_driver_store_over_untracked fails 6 runs out of 10 when run alone here.
    • guest_kernel_fcntl aborted once under ctest -j10, and then passed 10 runs out of 10 when run alone.
• Not run on the game here; we have no game files. The stop is SuperCatElvis's.
• Merges: the branch merges cleanly with main 5d9dcc4. 28 open PRs touch these files, all through docs/dev/debt/relinker.md; 19 of them are yabanana's stacked AOT PRs, which already conflict with main in other files, and this branch adds no conflict to them. Only fix(relinker): stop module syscall scans at the PLT and skip the PS4 loader path #5994 (Get-fix) gets a new conflict, in the one debt line both rewrite.
  • fix(relinker): stop module syscall scans at the PLT and skip the PS4 loader path #5994 has since merged, so this branch now merges main f03a2da (8156186), resolving that hunk as planned. fix(relinker): stop module syscall scans at the PLT and skip the PS4 loader path #5994's sce_module syscall-scan line now says the scan stops "at the same end, or, when the PLT comes before the code, at the end of the last function of the unwind table", because its scan calls FindSceCodeEnd. The collector line keeps fix(relinker): stop module syscall scans at the PLT and skip the PS4 loader path #5994's /libexec/ld-elf.so.1 start with this PR's end.
  • On the merge, the relinker-only suite passes 199/199 (Ubuntu 24.04, GCC 13.3, Release), including fix(relinker): stop module syscall scans at the PLT and skip the PS4 loader path #5994's sce_module_code_extent and this PR's sce_code_extent.
• tools/check_conventions.py from main passes with this title and body.

@SuperCatElvis, thanks for the answers on #3093. If you can, could you swap this in for your stop-9 change and check that both modules get past stops 9 and 10?

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Fixes: none
• Unblocks: CUSA01664 LIMBO (CUSA01664) #6978 (Limbo: Get-fix's bring-up lists this PR in step 1, converting the bundled libc.prx and libSceFios2.prx); CUSA03312 Hasbro Family Fun Pack (CUSA03312) #7198 (Hasbro Family Fun Pack: stop 8 in savvaniss's conversion); CUSA02636 (no Game bring-up discussion): stops 9 and 10 of SuperCatElvis's conversion on Relinker rejects PS4 executables: SCE dynamic tables and data in the code segment (two ready branches) + remaining PS4 work #3093, in the bundled libSceFios2.prx and libc.prx. They were reached after fix(relinker): accept section-relative guest module relocations #5138 and scratch fixes for the other stops from 1 to 8; stop 1, the eboot's jump tables, still stops the title on main, and stop 11 (fix(relinker): accept matching defined relative symbols #6357) follows. Not rerun here.
• GPUs tested: n/a (relinker); full ctest on NVIDIA GeForce RTX 3050 Ti Laptop GPU, driver 580.178.04, Ubuntu 24.04
• AI-assisted: yes (Claude Code); I reviewed and tested every line

🤖 Generated with Claude Code

[github-actions]

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 5021	docs/dev/debt/relinker.md
PR 6328	docs/dev/debt/relinker.md
PR 6977	core/relinker/relinker/src/analysis/SceCodeExtent.cpp core/relinker/relinker/tests/test_sce_code_extent.py docs/dev/debt/relinker.md
PR 7173	docs/dev/debt/relinker.md
PR 7531	docs/dev/debt/relinker.md

[DProkos]

Windows check of 8156186 merged onto current main (48d61f53b, clean merge).

Windows 11 Pro 26200, WinLibs GCC 15.2.0 r7 UCRT, Ninja, Release, ANYPS5_RELINKER_ONLY=ON, BUILD_TESTING=ON, Python 3.11.0.

• sce_code_extent: passed.
• Full relinker suite: 196/196 passed.

Negative control: with SceCodeExtent.cpp restored to main and the updated test kept, sce_code_extent fails on sce-plt-missing (Windows output). main still reports the old message, PLT not found after the last unwind function (offset 0x210), where the test expects PLT not found before or after the unwind functions.

Not run: titles or homebrew.

AI-assisted: yes