Skip to content

feat(aot): convert whole x86-64 programs with guest-state lowering - #4599

Closed
yabanana wants to merge 256 commits into
boykopovar:mainfrom
yabanana:feature/aot-guest-lowering
Closed

yabanana wants to merge 256 commits into
boykopovar:mainfrom
yabanana:feature/aot-guest-lowering

Conversation

@yabanana

@yabanana yabanana commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

What

Add a second AOT lowering mode that converts a complete x86-64 position-independent ELF into native ARM64 code, without the per-function C signatures the existing compiler requires.

  • Each recovered unit (unwind ranges, entry, lifecycle, exports, direct-call and address-taken targets) becomes one ARM64 function over the guest register state and a separate guest stack, so internal functions keep their original calling convention.
  • The runtime maps the image's loadable bytes, applies RELATIVE/GLOB_DAT/JUMP_SLOT relocations, enforces segment and RELRO permissions per host page and binds imports to provider thunks. Missing imports fail at startup.
  • Calls push the guest return address and continue only when the callee returns to it; any other address propagates and traps at the top level. Indirect transfers check a finite entry table. Jump tables use a bound proven by cmp/ja, or an extent read from immutable data inside the unwind range that stays a candidate with a runtime switch and trap.
  • Conversion rejects unimplemented instructions, undecodable reachable code, flags read at unit entry or after a call, and TLS relocations. Calls to imports declared noreturn (glibc headers, LSB) end their path.
  • Semantics: integer, BMI1/BMI2, string and SSE/SSE2 forms with x86 behaviour where LLVM leaves results undefined (masked counts, preserved flags, divide traps, bit scans of zero, integer indefinite, x86 NaN propagation).
  • A glibc x86-64 provider for Darwin converts stat/dirent layouts, errno values, open/mmap/clock/*at constants, signal numbers, System V variadic arguments and guest callbacks (qsort, signal, atexit).

Locked/atomic instructions, x87, AVX, guest TLS and exception resume points are rejected for now; TechnicalDebt lists the limits.

Tested

Latest integration checkpoint (12 October): aligned with main 2f4cc8983, including the new component technical-debt layout. The final combined stack with the separate C++ follow-up #4718 passed 588 compiler/relinker/runtime checks and 26 native integrations on Apple Silicon. Rebuilt cube AOT/reference still match all eight RGB frames after 600 flips each; controller and typed module-bundle acceptance pass. These are local results; hosted CI is running on the updated head. Full Linux/Windows builds still require maintainer dispatch.

Apple Silicon/macOS, merged with current main and with the native AOT examples enabled: 576 compiler/relinker/runtime checks passed, including 17 new guest-lowering cases. Six of them compare 612 instruction forms (470 taken from the instruction shapes of zstd and yaml-cpp, 142 targeted) between the converted code and the same x86_64 code run under Rosetta, over random register, vector, flag and memory states. The others build small x86-64 programs and check variadic formatting from registers and stack, a guest qsort callback, traps on unknown targets and mismatched return addresses, startup failure on a missing import, proven and data-scanned jump tables, and the rejections above.

Locally, outside the repository: zstd 1.5.7 built as a glibc x86-64 PIE (Zig 0.14.1) and stripped of its section table converted in 18 s (1033 units, 190k instructions). 61 CLI cases (levels 1/3/19 and --ultra -22, block boundaries, streams, dictionaries, golden files from the zstd tree, corrupt/truncated input, unwritable output, SIGINT) produced the same exit codes, stdout, stderr and files as zstd built from the same source for arm64, and every compressed output decoded with an independent zstd. This is a source-equivalent comparison, not the same ELF run on Linux. No performance claim: the built-in benchmark only gives an indication. Hosted CI pending; full Linux/Windows builds need maintainer dispatch.

Checklist

Rebase the graphics proposal onto upstream b42ef24 without carrying the historical macOS merge chain. Includes the compiler/runtime prerequisite from PR boykopovar#1835 and the still-required macOS foundation from mugurc/macos-port at 21930ce; those platform changes remain until their upstream PRs land. Preserve upstream starting-quad folding and the qualified vertex lane rule. Keep x87-dependent Math.cpp outside the optional ARM64 graphics closure rather than weakening its ABI assertion.
Rebase the graphics proposal onto upstream b42ef24 without carrying the historical macOS merge chain. Includes the compiler/runtime prerequisite from PR boykopovar#1835 and the still-required macOS foundation from mugurc/macos-port at 21930ce; those platform changes remain until their upstream PRs land. Preserve upstream starting-quad folding and the qualified vertex lane rule. Keep x87-dependent Math.cpp outside the optional ARM64 graphics closure rather than weakening its ABI assertion.
This was referenced Oct 15, 2026
@yabanana

Copy link
Copy Markdown
Contributor Author

Superseded by #5021, which now contains the consolidated native ARM64 AOT implementation and verified consumers. #1835 remains the architecture discussion. Closing this stacked PR to keep one implementation review; its branch and review history are retained. This is not a merge into main.

@yabanana yabanana closed this Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build ci .github docs libc libkernel needs rebase Conflicts with main, rebase needed prx relinker shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants