feat(aot): discuss native ARM64 compiler architecture and integration

[yabanana]

What

Retain this PR as the architectural discussion and review record for the experimental offline x86-64 to native ARM64 compiler. The complete implementation is consolidated in #5021, including the runtime and runnable consumers. This draft preserves the original review context; it is not a second implementation to merge or a dependency that must land before #5021.

The architectural constraints remain native ARM64 AOT output, reuse of AnyPS5 platform services, explicit input/ABI/effect qualification and rejection of unsupported cases. The shipped executable contains no x86 interpreter, JIT or Rosetta dependency. The original proposal's typed native-call compiler and the broader guest-state compatibility backend have distinct qualified scopes; recovering native values and effects across caller chains remains the direction for further simplification and performance work.

The existing request for a runnable end-to-end consumer is retained in review history. The source-built hello_notify example, native provider startup and explicit Linux ARM64 target provide that reproduction path. They do not settle the maintainers' decision about architecture, dependency policy or in-tree acceptance. Further implementation review belongs in #5021; architecture and contributor coordination can remain here.

Tested

Discussion branch integrated with upstream 51acb79ac49eeabf2d0e1d377867f953eb1bcefa. All 466 local compiler/runtime tests and the conventions check passed on Apple M5 Max / macOS 27.2. Full combined consumer/runtime evidence is described on #5021 and must not be attributed to this narrower branch.

The source-built example and reproduction commands are in docs/dev/BUILD.md. Enable ANYPS5_RELINKER_ONLY, ANYPS5_NATIVE_AOT, ANYPS5_NATIVE_AOT_EXAMPLE and BUILD_TESTING. The bounded CI variant shortens the example delay; the runtime sleep behavior is unchanged. Full Linux/Windows builds still require maintainer dispatch; new-head hosted results are separate from local results and historical checks.

Checklist

• Current upstream base integrated; intentional implementation overlap with feat(aot): consolidate native ARM64 compiler, runtime and consumers #5021 is documented
• Draft retained for architecture discussion and prior review history
• Unsupported paths reject; no dynamic translation fallback
• No private workspace notes, logs, images or third-party binaries added
• Dependencies remain pinned source submodules
• Depends on: none (discussion only; implementation lives in feat(aot): consolidate native ARM64 compiler, runtime and consumers #5021)
• Unblocks: none
• GPUs tested: n/a for this compiler discussion branch
• AI-assisted: yes

[yabanana]

@boykopovar Could you review the architecture and integration of this initial compiler-only ARM64 AOT implementation, related to #1553? It is now ready for review at 7725ddba, rebased onto 6cc0503d. All 204 tests pass locally and the fresh native Apple ARM64 CI has passed: https://github.com/boykopovar/AnyPS5/actions/runs/37903536478. Full Linux/Windows and relinker validation is running separately. The pointer/ABI restrictions remain explicit in the description; the full runtime and complete game port are follow-up work.

[yabanana]

@boykopovar Current head a2f4f368 is rebased onto main 745b775e, including merged #2099 and #2151. Both previous upstream CI blockers (SPIRV headers and Windows Clang without AMDGPU support) are addressed in this base. All 207 local CTest cases pass; the initial compiler-only scope and pointer/ABI limitations remain explicit. Under the new policy, could you dispatch Build on main with input pr=1835 for full Linux/Windows validation, and review the current head? Skipped full-build jobs are not counted as passes.

[adriadam10]

Thanks for the work and for keeping the scope explicit (compiler-only, limits recorded in BUILD/TechnicalDebt).

This adds a new area rather than a fix, so it needs a direction call before a code review:

• ~5.4k lines of Python under core/aot/, two new third-party submodules (Capstone 5.0.6, pyelftools 0.32) and an opt-in ANYPS5_NATIVE_AOT CMake option.
• A new CI workflow (.github/workflows/native-aot.yml) that only runs on Apple Silicon runners; CONTRIBUTING has no rule for that.
• It is a parallel path to the macOS x86-64 port in macOS support #1, with no runtime consumer yet (no entry-point caller or import implementations).

@boykopovar @oneandonlydean: do you want this in-tree at this stage (new submodules, the Apple-only workflow, the long-term ARM64 direction from #1553), or should it live out of tree until a runtime exists? We can't build or run it on Linux, so the claims that depend on macOS/ARM64 (the 212 tests, the emitted code) remain unverified on our side.

[oneandonlydean]

I'd keep it out of tree until there's a runtime that runs something with it: we can't build or check it here, and it brings two submodules and an Apple-only workflow with no runnable output yet. The ARM64 direction (#1553) is the owner's call.

[yabanana]

Follow-up: the original PS5 DOOM executable now completes all three demos as ARM64 AOT, without Rosetta, with its original libc module converted and initialized alongside it.

The reproducible run uses lowbit/ps5-doom v1.1.1, PPSA99666, the frozen shareware WAD and the title's CPU scaler. Results: 5,026 / 3,836 / 2,134 gametics, six exact 1920×1080 RGB comparisons against the pinned source-built reference, normal title-requested exit, module finalization and Vulkan teardown. The executable is ARM64 Mach-O and reports translated=0. Audio, guest worker callbacks, file/stdio operations, Pad initialization and the capture connection run through native providers.

Four new commits separate the reused #3533 AudioOut shutdown fix, remaining VEX lowering, and the SCE services/consumer. The build and verification procedure and qualified binding manifest are included. No additional PR was opened; the branch builds on the existing ARM64 stack and still needs upstream review/integration.

Local validation: 711 compiler/runtime/relinker tests, 29 native tests, callback/varargs checks with ASan/UBSan (linked platform libraries uninstrumented), the controlled AudioOut shutdown regression, cube execution with 600 flips per implementation and eight exact RGB comparisons, Pad and the modular consumer. The Strings.cpp identity change was requalified after checking that the complete consumed memcpy/memset definitions and effects remained unchanged. The final static ABI guard also rejects host long double; both affected DOOM objects are byte-identical to the replayed objects. Hosted CI has not run on this branch.

Scope remains explicit: these are recorded demos with the CPU scaler, not AGC presentation, arbitrary WADs, controller hardware or a performance comparison. Scalar FMA currently rejects nonfinite operands; floating status and alternate rounding remain unqualified. VTEST exposed a Rosetta flag discrepancy, so all flags are checked against an independent integer model following the Intel specification. The compatibility compiler used here is separate from the ongoing native-quality compiler work.

Host: Apple M5 Max, macOS 27.2, Apple Clang 21, Vulkan/MoltenVK. AI-assisted: yes.