Skip to content

Onimusha: Way of the Sword (PPSA27836): Wwise thread AK::EventManager walks a corrupted list and faults on the EULA confirm #6046

Closed

Description

@simonbronk

Onimusha: Way of the Sword (PPSA27836, RE Engine, Wwise audio) on Windows stops with an access violation in the title's Wwise thread AK::EventManager every time the EULA is confirmed, probably when the confirm sound starts. Before that point the same crash happens at random during start-up in about half the runs. Resident Evil Village (PPSA01556, also RE Engine and Wwise) reports the same crash in #5169.

Setup: Windows 11 Pro 26200, RTX 5070 (driver 617.42), WinLibs GCC 15.2.0, Release. main 0b97f8b plus the title's PRs listed in #5769. The title is relinked with --windows --windows-fixed-base 0x400000000.

Crash sites (four runs, all in the title's own Wwise code, never in a host library):

FATAL: unhandled exception 0xc0000005 on thread 'AK::EventManager'
  rip app.exe+0x5915f1b  read of 0xffffffffffffffff   (twice)
  rip app.exe+0x58f7692  read of 0x00000000000000d0
  rip app.exe+0x58f93ca  write of 0x00000004058c3cc8

What the dumps show: each time, Wwise walks one of its own linked lists and the current node is not a valid node.

  • At +0x5915f1b the loop is mov 0x8(%rbx),%rax; cmpq $0,0x50(%rax), following 0x10(%rbx) as next. rax is 0x4000000006500000, and the node's qwords are 00fff00000000092 4000000006500000 0000000000000014 28000000209b5a7e ..., which look like packed data rather than pointers.
  • At +0x58f93ca the node pointer rsi is 0x4058c3c80, inside the title's code section (its qwords are x86 instructions). rdi points to a table of code addresses (0x4058befb0 0x4058bf280 0x4058c3c80 ...). So a vtable slot is being used as a list node, and the write then hits code.
  • Resident Evil Village (PPSA01556) #5169 describes the same pattern on Village as "a record with every qword shifted by 4 bytes".

These look like a freed or reused node still being on a list, or the list being changed by two threads at once. The usual cause would be a synchronization primitive that doesn't exclude or wake the way the title expects.

Ruled out:

  • Memory pressure: Village saw the crash only before raising its commit limit. Here, raising the commit limit from 49 GB to 63 GB changed nothing; the crash still happens on the EULA confirm.
  • Graphics PRs: it happens with and without fix(agc): skip unused pixel stages with retained export formats #5631, and appears in older runs on this branch too.

I haven't traced which libkernel / pthread calls the Wwise threads make around the confirm. Pointers on which primitives Wwise's PS5 build relies on (mutexes, condition variables, semaphores, event flags, sceKernelUsleep timing) would help narrow it down.

AI-assisted: yes (Claude Code)

Activity

  1. simonbronk commented on Oct 14, 2026

    @simonbronk
    ContributorAuthor

    Static checks so far, against main 0b97f8b and the title's Wwise code. None of them explains the crash:

    • Statically initialized mutexes: resolveMutex creates a lazily initialized mutex under a global lock, so two threads can't get separate mutexes.
    • sceLibcMspace*: every entry point holds arenaMutex. Split and coalesce keep the chunks contiguous, and in-place Realloc growth checks adjacency.
    • AJM sidebands: Wwise passes GetResampleInfo's result at voice+0x44, and the next sideband (DecodeSplit's) is at voice+0x78. That leaves 52 bytes, and the 48 bytes written fit.
    • AJM batch sizes: Wwise sizes each voice's batch from a per-job table at 0x40a1b7880: 0x30 0x30 0x48 0x80 0x40 0x30 0x40. AnyPS5's packed jobs fit within these: DecodeSplit with 4 inputs and 2 outputs is exactly 0x80, and Append bounds every job by info->size.
    • Event flags: the AND/OR and CLEAR_ALL/CLEAR_PAT handling and the predicate wait look correct.

    One thing to look at: libSceAcm is a no-op. sceAcmBatchStartBuffers hands back an ID, sceAcmBatchWait returns 0 at once, and the sceAcm_* builders write nothing. Wwise starts a batch at app.exe+0x58e139b with a count of 0 and a null batch_error. It then allocates a 16-byte pending-batch object (vtable 0x40a2c6058, batch ID at +8) and links it. If that object's lifetime depends on the batch actually running, a batch that completes instantly could free it early. That isn't verified.

    Next step on my side: one run with sync tracing through the EULA confirm.

  2. simonbronk commented on Oct 14, 2026

    @simonbronk
    ContributorAuthor

    New finding: the crash follows the title's settings save, not the EULA.

    • With a save present (SAVESERVICE-LINE-0-1Slot, written during an earlier run): AK::EventManager faults on the first menu confirm, already at the language selection, in 6 of 6 runs.
    • With the save directory moved aside: the language, EULA, brightness and gamma screens all go through. After the gamma confirm, the title writes the save (mount3 ... mode=0x21, setParam size=1328, umount2). The directory is stamped 22:57:23 and AK::EventManager faults at 22:57:25.

    So the trigger seems to be the title applying its stored settings after a save is loaded or written. The audio thread's last traced calls before the fault are only scePthreadMutexLock/Unlock, scePthreadSelf, sceKernelGetProcessTimeCounter, sceAudioOut2ContextGetQueueLevel and sceAudioOut2PortSetAttributes. Traced libraries: libkernel, libSceAcm, libSceAjm, libSceAudioOut.

    In one dump, the bad value is a local in the faulting frame on the audio thread's own stack. It reads 0x00000001309dd700 where the neighbouring slots hold 0x000000e8309dd700: the low half of a pointer is intact, and the high dword was replaced by 1.

    More things checked and ruled out:

    Next idea: find which Wwise structure is hit consistently, then catch the writer with APS5_WATCH_ADDR.

  3. simonbronk commented on Oct 14, 2026

    @simonbronk
    ContributorAuthor

    The faulting frame from the run in my last comment (app.exe+0x58cfa62) shows the object involved. The function walks an array of voice pointers (r13). For each voice it loads voice->[0x138], then ->[0x30], then reads ->[0x18]. At the fault, ->[0x30] is 0x3f3504f33f3504f3, which is the float pair 0.70710677, 0.70710677: an equal-power stereo pan gain. So the object voice+0x138 points to has been freed and its memory reused for a gain buffer, while the voice still references it. That fits the other dumps: list nodes that point at vtables, or contain packed data.

    One possible reason it only shows up here: libkernel records the guest's thread affinity and priority (scePthreadAttrSetaffinity, scePthreadSetaffinity, scePthreadAttrSetschedparam) but doesn't apply them to the host threads, so all of the title's threads run concurrently on every host core. The Wwise threads spin heavily: 60 to 88 million scePthreadYield calls in about 3 minutes. If Wwise's teardown of an output or bus on a settings change depends on the console's core pinning and priority ordering, unrestricted host scheduling could let the free and the use overlap. Not verified yet.

  4. simonbronk commented on Oct 14, 2026

    @simonbronk
    ContributorAuthor

    I tested the scheduling idea from my previous comment, and it doesn't fix the crash. With a local experiment that applies the guest affinity mask to each host thread (SetThreadAffinityMask), the Wwise threads are pinned as the title asks: AK::EventManager, AK::AudioOutThread, AK::IOThread, AK::JobMgr::Internal-AcmQueue, AK::BankManager and SimpleWwiseUpdateThread all use mask 0xf00, at guest priorities 256 or 700. AK::EventManager still faults on the first menu confirm.

    Mapping the guest priorities onto Windows thread priorities as well (256 → HIGHEST, 700 → LOWEST) doesn't change it either; the fault is the same. So host scheduling alone is not the cause.

  5. simonbronk commented on Oct 14, 2026

    @simonbronk
    ContributorAuthor

    Follow-up on scheduling: one run with the guest priorities mapped (256 → HIGHEST, 700 → LOWEST) got past the first confirm and faulted on the next one. A stricter mapping (256 → TIME_CRITICAL, priority boost disabled) faulted on the first confirm again. So that one pass was luck, and scheduling isn't the lever.

    Another symptom from the same runs: the title's audio stutters from the first sound onwards, on every run. That suggests the AudioOut2 pacing (sceAudioOut2ContextPush / sceAudioOut2ContextGetQueueLevel) doesn't match what Wwise expects. Wwise's output thread may be starving or running late. Whether that leads to the teardown race isn't known.

  6. simonbronk commented on Oct 14, 2026

    @simonbronk
    ContributorAuthor

    I measured the stutter with APS5_TRACE_AUDIOOUT2, APS5_TRACE_WAITS and APS5_TRACE_USLEEP, in a menu with no input.

    Output rate: the context uses 1024-sample grains (21.33 ms) at 48 kHz, queue depth 4, so real time needs about 47 pushes per second. The per-second summaries show 0.4 to 7.2 pushes/s, and the hardware queue is at 0/4 in every line. Nothing is dropped or rejected (queue-full rejects 0). So Wwise produces audio at well under a tenth of real time, and AudioOut2 starves.

    Where the time goes: the Wwise output thread runs this loop at app.exe+0x58e29c0:

    1. sceAudioOut2ContextPush(ctx, 1).
    2. Notify the renderer (0x4058d21f0).
    3. sceKernelWaitSema(sema, 1, NULL) at +0x58e29ea.

    The other side waits in sceKernelWaitEventFlag(flag, 1, OR|CLEAR_ALL, NULL, NULL) at +0x58e225a. APS5_TRACE_WAITS reports, per 10-second window:

    • sema exe+0x58e29ef: average 100 ms to 2.8 s per wait;
    • evf exe+0x58e225f: average 130 ms to 1.1 s per wait;
    • the title's general job event flag exe+0x8192a57: average 6 to 240 ms.

    So the handshake between the two Wwise threads takes tens of milliseconds to seconds per grain, where it should take microseconds, and both sides spend that time blocked. The usleep sites are not in the Wwise range.

    A run of minutes with the output this starved could plausibly reach the voice and bus teardown path behind the use-after-free, but that isn't shown. Next: measure the wake latency of sceKernelSetEventFlag→WaitEventFlag and sceKernelSignalSema→WaitSema ping-pong on Windows directly.

  7. simonbronk commented on Oct 14, 2026

    @simonbronk
    ContributorAuthor

    I measured the wake path directly to rule it out. This is a local guest test, not in the tree: two guest threads ping-pong exactly like the Wwise output and render threads. One calls sceKernelSetEventFlag(f, 1) then sceKernelWaitSema(s, 1, NULL); the other calls sceKernelWaitEventFlag(f, 1, OR|CLEAR_ALL, NULL, NULL) then sceKernelSignalSema(s, 1). Over 2000 rounds, 3 runs, on the same PC (Windows 11, Ryzen 7 9800X3D):

    run total p50 p90 p99 max
    1 9.3 ms 4.4 us 5.0 us 14.9 us 50.5 us
    2 10.1 ms 4.5 us 5.0 us 16.6 us 138 us
    3 9.9 ms 4.5 us 6.2 us 13.8 us 33 us

    So the event flag and semaphore wake-ups themselves are fast. In the title, the 100 ms to seconds per handshake must come from the render side being slow or waiting on something else. The title's job event flag exe+0x8192a57 waits 6 to 240 ms on average, and the trace counts 60 to 88 million scePthreadYield calls in about 3 minutes, so some threads spin. The next thing to find is what the Wwise render step waits on, or which spinning threads starve it.

  8. simonbronk commented on Oct 14, 2026

    @simonbronk
    ContributorAuthor

    The scePthreadYield spinning is not the starvation. I counted yields per call site and thread (local instrumentation, 150 s in a menu). 99.9% of them come from one site, exe+0xb6ae09, in ResourceManager::parallelProc. It's a pause loop: while (byte [obj+0xdc0] == 1) { lock dec [obj+0xdd0]; scePthreadYield(); ... }. The resource manager's parallel thread is parked by design and spins on one core, about 66k yields/s. Everything else is in the hundreds. On a 16-thread host that can't starve the Wwise render thread. The earlier 60 to 88 million counts were inflated by the call tracer.

    So the Wwise output starving to under 10% of real time is still unexplained. The wake primitives are fast (previous comment), and the CPU isn't saturated by spinners.

  9. simonbronk commented on Oct 14, 2026

    @simonbronk
    ContributorAuthor

    I mapped the Wwise output pump. It's driven by job chains, not by AudioOut2 timing, which explains the starvation pattern.

    Timed all-thread dump (APS5_HANG_DUMP_SECS=100) together with APS5_TRACE_WAITS. Over 100 s:

    thread wait count total avg
    AK::EventManager evf exe+0x58e225f 280 94.2 s 336 ms
    AK::AudioOutThread sema exe+0x58e29ef 187 85.3 s 456 ms

    Both sides are blocked most of the time; at the dump every AK::* thread is parked in ntdll.

    How the semaphore gets signalled:

    • AK::AudioOutThread (entry 0x4058d2990): Push(ctx, 1), set the render flag, then WaitSema.
    • The semaphore is only signalled by 0x4058d3080, which runs sceAudioOut2ContextAdvance and then sceKernelSignalSema(s, 1). Nothing calls it directly: it's the invoke slot of a 16-byte job object (vtable 0x40a2c60a8), built at 0x4058d2d40 and queued through 0x40581f3b0 behind a dependency.
    • The job it depends on is the ACM one (vtable 0x40a2c6058, invoke 0x4058d21a0). It's created right after sceAcmBatchStartBuffers and calls sceAcmBatchWait(ctx, id, 0xffffffff).

    So each grain goes: render → ACM batch → ACM-wait job → advance job → SignalSema → the output thread pushes. The ACM side completes at once in AnyPS5, so the pace is set by how soon queued jobs run. The engine's job workers (evf exe+0x8192a57) wait 6 to 240 ms on average per wake, and the observed 0.4 to 7 pushes/s matches the frame rate, not 47/s. So the jobs seem to run when workers wake for frame work, not when they're queued.

    Already ruled out: event flag Set wakes all waiters (NotifyAll), the SetEventFlag→Wait and SignalSema→Wait round trip is about 4.5 us, ContextAdvance returns 0, and GetQueueLevel's second output (Wwise's advance count) is depth - level.

    Next: find how 0x40581f3b0 queues a job and wakes a worker (which primitive, which bits). That's probably where a wake-up is lost or deferred.

  10. MarcZX10 commented on Oct 14, 2026

    @MarcZX10
    Contributor

    The semaphore instrumentation in #6123 may help check the AudioOutThread handshake: it logs create, signal, and wait begin/end with the semaphore address, name, and caller, so you can tell whether the specific semaphore is signaled and when. It does not trace event flags or job-queue wakeups, so it will not by itself identify why the ACM job is deferred. #6123 is open; its applicable checks pass, while platform builds were skipped by path filters.

  11. simonbronk commented on Oct 15, 2026

    @simonbronk
    ContributorAuthor

    Thanks @MarcZX100. The trace already shows the AudioOutThread semaphore is only signalled by the advance job (0x4058d3080), so the open question is when the job runs, not whether the semaphore is signalled. #6123 would confirm the timing, but it can't show the job-queue side, as you say.

    I followed the queue side in the disassembly:

    • Submit (0x40581f3b0 → 0x40581e180): the job goes into a per-type lock-free ring (0x40db68940). Then a per-type active-worker counter (0x40db689f0, capped by 0x40db68a30) is raised with a CAS. Only if it actually rose does it call sceKernelSetEventFlag(evf[type], 1) (handles at 0x40db68a50). If the optional worker-request callback at 0x40db68910 is set, it's called instead (fnRequestJobWorker-style, worker entry 0x40581d7a0).
    • Workers and helpers wait with sceKernelWaitEventFlag(evf, 1, OR|CLEAR_ALL, NULL, NULL). So one Set releases one waiter.
    • The crash stack is on this same path. In the AK::EventManager crash (frames #13 0x40581d99a, #14 0x40581f5b5, #15 0x4057dc5be, #16 0x4058d2248), the thread is inside 0x40581f440. That's a submit-and-help loop: it queues a job, then loops on WaitEventFlag(evf[0], 1, OR|CLEAR_ALL) and runs queued jobs itself (0x40581d850) until the slot state changes. The use-after-free happens while that thread runs a render job inline. The call trace just before it shows SetEventFlag from +0x582e62d (submit), then WaitEventFlag from +0x582f59f.

    So the stutter and the crash share Wwise's own job manager, and the wake-up depends on SetEventFlag + CLEAR_ALL with several waiters on one flag. AnyPS5's Set does NotifyAll and the first waiter to retake the lock clears the pattern. The others re-wait inside the host call. That matches the "one waiter released" model, but I haven't verified it against hardware. If the console releases every waiter that's satisfied at Set time, and clears only afterwards, this pool would wake differently.

    Does anyone know the exact release order of sceKernelSetEventFlag for CLEAR_ALL waiters on hardware? Next, I'll log the per-type counter and the Set/Wait pairs on evf[0] to see whether a wake-up goes to a worker that finds no job while the helper keeps sleeping.

  12. simonbronk commented on Oct 15, 2026

    @simonbronk
    ContributorAuthor

    Follow-up, negative result: the job manager's wake path is not where the stutter comes from.

    I traced every event flag operation that comes from the job manager range (0x40581d000–0x40581f700), logging the active-worker counters with each one. This was local, env-gated instrumentation, not in the tree. Two runs: 9 s and 60 s.

    • Worker caps are 0 for types 0–2, and 1 for type 3. They come from the title's job manager settings. Init at 0x40581ea30 copies the caps from the settings only when the settings' first field (the worker-request callback) is non-null. Here it's null, and the hook at 0x40db68910 is null too. So types 0–2 never wake a pool worker by design. Their jobs run on the submitting thread, which helps.
    • The submitting thread is on time. It does SetEventFlag(evf[0], 1) from +0x582e62d, then WaitEventFlag(evf[0], 1, OR|CLEAR_ALL) from +0x582f59f on the same thread. The wait returns in 0 µs (it consumes its own Set). It does this every 21.0 ms at the median (p90 21.3 ms), which is real-time pace for 1024-sample grains: 201 iterations in the 60 s run, all with the same thread and flag.
    • The type-3 worker waits on evf[1] from +0x582e95e for the whole run, and nobody sets it. Its counter reads 1 throughout, which matches its cap of 1.
    • sceAudioOut2ContextPush still runs at about 4 pushes/s in the same runs.

    So no wake-up is lost in the job manager. Event flag semantics with several CLEAR_ALL waiters don't matter here, because only one thread waits on evf[0]. Grains are rendered at real-time pace, but the output thread pushes about 4 per second. The gap is between the render and the AudioOutThread semaphore: the ACM-wait and advance jobs. Next, I'll find which thread runs those two jobs and when.

  13. simonbronk commented on Oct 15, 2026

    @simonbronk
    ContributorAuthor

    Update: on current main this crash no longer reproduces for me.

    Base: main 0afa6a6 with the title's PRs (#5907, #5776, #5770, #5703, #5841, later also #5884, #5915, #5815). Windows 11, RTX 5070. Over 9 runs, the title went through the language, EULA (held), brightness and gamma screens, the main menu, New Game, difficulty and control style, into the first scene load. None of the runs ended in AK::EventManager. Every stop was a GPU-side refusal further on.

    The old base (0b97f8b) hit the crash on the first confirm in almost every run. I haven't bisected which change in between removed it. I'll leave this open a little longer and close it if it stays gone.

  14. oneandonlydean commented on Oct 16, 2026

    @oneandonlydean
    Collaborator

    Closing on simonbronk's report that the AK::EventManager crash no longer reproduces on current main (9 runs, 10-15). Reopen if it comes back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions