fix(avplayer): correct audio-video playback timing and sync

[squirlyfoxy]

What

Fix audio-video playback timing and synchronization:

• Report the PS5 audio stream type and extended video frame rate.
• Keep video from running ahead of the audio clock.
• Emit loop warnings when playback reaches the next loop, rather than when read-ahead reaches EOF.
• Use per-port AudioOut locks so pacing one port does not block independent ports.
• Remove diagnostic traces and add regression tests.

Tested

Windows, Evil Inside (PPSA02508): both cutscenes play with synchronized audio and video, without the premature pause before scene transitions, afterward due to an unrelated or currently unidentified issue.

Tested also My Friend Peppa Pig (PPSA03706) with initial cutscene now playing well.

Full build and complete test suite have not been run. The AudioOut Ex test terminated while handling an expected exception; that verification remains incomplete.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw; silent stubs are listed in technical debt
• No notes, investigation .md files or images added to the repository
• New third-party code is a submodule built from source — none added
• Generic behaviour, not specific to one title
• Depends on: fix(avplayer): defer loop warnings until frame delivery #5846
• Unblocks: PPSA03706
• GPUs tested: RTX2060, Driver version 610.88
• AI-assisted: yes

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
PR 6922	core/libs/prx/libSceAudioOut/src/AudioOut.cpp L113, L354, L367, L390-391, L554

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 1601	core/libs/prx/libSceAvPlayer/src/Source.cpp core/libs/tests/GuestAvPlayer.cpp
PR 5021	core/libs/prx/libSceAudioOut/CMakeLists.txt
PR 6044	core/libs/prx/libSceAudioOut/CMakeLists.txt core/libs/tests/GuestAvPlayer.cpp
PR 6170	core/libs/SceTypes.hpp core/libs/prx/libSceAudioOut/CMakeLists.txt core/libs/prx/libSceAudioOut/src/AudioOut.cpp
PR 6879	core/libs/prx/libSceAudioOut/src/AudioOut.cpp

[jotitadev]

Linux run: head 83e7d26e0 merged onto main 44df21632 (clean merge). The description says the full build and suite were not run, so here they are.

Machine: Ubuntu 26.04, Intel i7-8700, GCC 15.2, full build, non-GPU tests (Release, BUILD_TESTING=ON).

• Build: full build passes.
• AudioOut/AvPlayer tests with the PR: all 18 audio_out* tests and guest_avplayer pass, audio_out_ex included (the description mentions it terminating on your side).
• Tests without the fix (this PR's two test files on main's SceTypes.hpp, AudioOut.cpp and AvPlayer sources):
  • audio_out_last_output_time aborts: "Independent port output waited 910 ms", "pacing one audio port must not block output on another port".
  • guest_avplayer fails: "PS5 audio stream must use type 2 so Unity enables it". It stops at the first check, so the frame-rate checks at 0x40/0x48 were not reached without the fix.
  • audio_out_ex passes either way.
• Non-GPU suite (ctest -E "^agc"): no regressions; the only failures are file_writer_binary and file_writer_text, which fail on main on this machine too.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

[JJenkx]

Independent Linux check of 83e7d26e0c06 on main 44df21632125: full libs build and all AudioOut/AvPlayer targets pass. With three added controlled port tests, 21/21 CTests pass, zero skips (Arch Linux, Ryzen 9 7950X; no title/speaker-quality or FPS claim).

The test-only commit exercises real public AudioOut calls with an interposed SDL queue and fixed guest clock, holding port A in backpressure until explicitly released. On main, output and close on independent port B fail to complete before A is released; the same-port close control passes. With this PR, all three pass. This supplements the virtual-port tests with controlled device-queue coverage. Removing only the same-port close lock makes the control fail on access to the reset device; restoring it passes. The fixture executable is byte-identical across these latter runs (SHA-256 39381fbb9f6e0218d4d86655e0a45e035437433b56f6ac4310fd869bb2b20917). This is a synchronization check, not a latency benchmark.

Separate no-sync contract concern: takeVideo now gates future frames whenever audioDriving is true, including sync mode 1. The public PS5 SharpProspero reference describes None as caller-paced delivery as frames decode. The added TestUnsyncedVideoDoesNotRunAheadOfAudio instead requires audio-clock gating in this mode.

I made a separate diagnostic probe: read one audio frame to activate the audio-backed clock, then request decoded video without consuming further audio and measure video lead over sceAvPlayerCurrentTime. With this PR, lead remains 0 ms and the caller-paced probe fails. Restoring only main's if (synced && timestamp > now) condition gives 129 ms lead and passes, with the same executable (SHA-256 8160cd759cfa6f1b24315a8fcb3b5442d50b7ef45cdfacf9bc26e1590180c362). This identifies the behavior change; the reference is public interop documentation, not a PS5 hardware measurement. The mode-1 contract needs clarification before treating the new gate as generally correct.

Full review source including the probe retains your original commit and intentionally exposes that probe failure. For comparison, the isolated AudioOut portion preserves your authorship, adds both test fixtures, and leaves main's AvPlayer behavior intact: full libs build and 22/22 checks pass, zero skips. The temporary close-lock fault and condition-revert probe patch are removed. Conventions, registration and whitespace pass. No downloaded game assets, captures, raw title logs or binaries are published.

[squirlyfoxy]

The no-sync concern is valid. I’ve restored if (synced && timestamp > now), replaced TestUnsyncedVideoDoesNotRunAheadOfAudio with your caller-paced probe, and retained the per-port AudioOut locks, including same-port close serialization.
The PS5 stream metadata and deferred loop notification fixes remain necessary independently of that gate. Testing My Friend Peppa Pig on Windows showed that reverting them caused skipped or prematurely terminated intros. With those fixes retained, the 73.567-second intro reaches its final frame at 73.533 seconds before closing.
The current local changes pass the full libs build and 18/18 Windows checks, including the no-sync probe. I also added your SDL-interposition fixture, but its three Linux-only cases were not run locally. These results support the synchronization fix and the observed title behavior; they do not establish PS5 hardware semantics or a latency improvement.

[jotitadev]

Linux run of the new head: c6b9c0498 merged onto main 5d9dcc42d (clean merge), in one tree with seven other library PRs that share no files with this one. The description says the three Linux-only SDL-interposition cases in AudioOutPortLocks.cpp were not run locally, so here they are.

Machine: Ubuntu 26.04, Intel i7-8700, GCC 15.2, full build, Release, BUILD_TESTING=ON.

• With the PR: audio_out_independent_output, audio_out_independent_close and audio_out_same_port_close pass, as do guest_avplayer and the new guest_avplayer_no_sync_lead. All 20 audio_out* tests plus both AvPlayer tests pass 5/5 with --repeat until-fail:5.
• Without the fix (the PR's tests, but SceTypes.hpp, AudioOut.cpp and the AvPlayer sources from main):
  • audio_out_independent_output and audio_out_independent_close abort with "AudioOut port locks: port output/close lock scope is wrong".
  • audio_out_same_port_close passes, as expected for the control case.
  • audio_out_last_output_time aborts: "Independent port output waited 909 ms".
  • guest_avplayer and guest_avplayer_no_sync_lead both fail at the first check, "PS5 audio stream must use type 2 so Unity enables it". So without the fix, --no-sync-lead-only does not get as far as the lead measurement.
• Non-GPU suite (ctest -E "^agc|gpu|vulkan|video_out"): no regressions. The only failures are file_writer_binary and file_writer_text, which also fail on main on this machine.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

[adriadam10]

Checked on Linux (merged onto current main): AudioOut/AvPlayer tests pass, and the new tests fail without the change (audio_out_independent_output, audio_out_independent_close, audio_out_last_output_time, guest_avplayer); audio_out_same_port_close is the control that passes either way. PS5 audio stream type 2 and the extended frame rate (details offset 48) match KytyPS5 and SharpProspero.

Added on rebase:

• Dropped the guest_avplayer_no_sync_lead ctest entry: --no-sync-lead-only is not read by main(), so it just re-ran the whole suite. The no-sync lead case already runs inside guest_avplayer.
• Resolved the CMake conflict with guest_avplayer_callback_close.

Note: the PR description no longer matches the diff (it covers sync/loop changes already merged via #5846). #6922 overlaps the per-port locking and will need a rebase after this lands.

Merging once CI is green.