Repository navigation
Conversation
|
Before deciding to merge this, I just want to call out a risk with this new render approach. Unless I'm reading it wrong, it passes the video through the effects pipeline. This is a problem because media3 uses 8-bit RGBA textures and output for SDR. For normal SDR content, this doesn't pose an issue, because SDR is usually 8-bit anyway. But in the anime scene, videos are compressed using 10-bit encoders, even on SDR sources. Passing that video through the effects pipeline results in pretty obvious and distracting banding. This is also a problem in libmpv. So the tradeoff for frame accurate subtitles is worse looking video when you're watching something encoded in 10-bit. For reference, that would be basically anything encoded with HEVC/x265 and AV1. |
|
You're right. I did not know that and I am surprised I missed an obvious detail like this one. |
| AssRenderType.OVERLAY_CANVAS | ||
| } else { | ||
| AssRenderType.OVERLAY_OPEN_GL | ||
| AssRenderType.EFFECTS_ATLAS |
There was a problem hiding this comment.
Actually, since videoeffects doesn't support hdr, we should detect if the video is hdr/dolbyvision or not.
If hdr, use OVERLAY_OPEN_GL, otherwhise, use EFFECTS_ATLAS.
There was a problem hiding this comment.
I agree with this. And should be straight forward too since this function is called from PlaybackViewModel.createPlayer which has a flag for HDR or not already.
Also, will need to add a check in PlaybackViewModel.createPlayer to recreate if HDR state changes though. Right now it only will if the backend changes.
There was a problem hiding this comment.
This point should be fixed.
I dont have a HDR TV, so I can't test it properly, but it should be fine.
|
Thanks for the PR! I was looking into this last night, but I haven't done a deep dive into the code. My initial thoughts are to default to Also, need to investigate if |
The version 0.6.0-beta01 added the EFFECTS_ATLAS render type (peerless2012/libass-android#90). In brief, it is more performant to OVERLAY_OPEN_GL. To quote the [doc](https://github.com/peerless2012/libass-android/tree/0b3617ace00b12aea552d0974978ca7e05a0c488/lib_ass_media#4-effects_atlas): ``` The fastest render type. It use OpenGL to render, but it use a atlas to push the libass bitmap to the GPU which makes it much more performant than any other mode. It have similar performance to mpv. ``` We will still use OVERLAY_OPEN_GL if the video is HDR because EFFECTS_ATLAS use [setVideoEffects](https://developer.android.com/reference/androidx/media3/exoplayer/ExoPlayer#setVideoEffects(java.util.List%3Candroidx.media3.common.Effect%3E)) which doesn't support HDR (it tonemap the video to SDR). I moved `assHandler?.init` because I was getting this exception: ``` Exception in coroutine java.lang.IllegalStateException: Player is accessed on the wrong thread. Current thread: 'DefaultDispatcher-worker-9' Expected thread: 'main' See https://developer.android.com/guide/topics/media/issuesplayer-accessed-on-wrong-thread at androidx.media3.exoplayer.ExoPlayerImpl.verifyApplicationThread(ExoPlayerImpl.java:3150) at androidx.media3.exoplayer.ExoPlayerImpl.setVideoEffects(ExoPlayerImpl.java:1433) at io.github.peerless2012.ass.media.render.AssOverlayManager.<init>(AssOverlayManager.kt:22) at io.github.peerless2012.ass.media.AssHandler.init(AssHandler.kt:142) at com.github.damontecres.wholphin.services.PlayerFactory.createVideoPlayer(PlayerFactory.kt:181) at com.github.damontecres.wholphin.services.PlayerFactory$createVideoPlayer$1.invokeSuspend(Unknown Source:15) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:34) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100) at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:124) at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:89) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:586) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:798) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:717) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:704) ```
384d813 to
2f98cb9
Compare
I'm not sure if that is a good idea. That's up to you, but I just wanted to make sure you know everything. |
|
I built this branch local and did some testing. I tested on my OnnPlus 4K Pro with a 4K HDR TV output via HDMI. For SDR samples, I confirmed the libass rendering in Exoplayer is more time accurate than the current Wholphin release. It also preformed very similar to mpv besides the below example. https://files.catbox.moe/ioxho8.mkv is an 11 second sample which contains large ASS subtitle at ~5 seconds. Mpv plays the file but there is some stuttering but no significnat audio and video desync. However for Exoplayer and libass at around ~5 seconds the video moves extremely slow and the audio continues onward at the normal rate. The audio finishes and then the video finishes ~30 seconds later. I don't know if this behavior is expected or not. https://files.catbox.moe/xkqvfm.mkv is a 15 second clip that I believe is triggering the HDR detection. When I play this file in Exoplayer, the bouncing subtitles on the book are clearly out of sync and look they they do in the current Wholphin release. This suggests it is using the I also checked with a sample I know is HDR with ASS subtitles, and it played and the player reports it is direct playing via "show debug info". However the subtitle are not complex, thus I don't expect any lagg and can't infer what render is being used. Is there an easy way to check in the debug app what render method is being used? "show debug info" does not list it. |
This is probably related to something in Wolphin (not libass-android) because I know that my phone drops around 30 frames with that file with libass-android demo and the audio seems fine (but maybe it's because it just drop 30 frames). Could you try with libass-android demo and see if you reproduce the desync with the audio? I still haven't checked why, but I do know that when mpv lags, it will always be worse with exoplayer/libass-android. If mpv doesn't lag, we also don't.
No. I added a note to myself to log it. |
| } else { | ||
| AssRenderType.OVERLAY_OPEN_GL | ||
| } | ||
| val renderType = assRenderType(isHdr) |
There was a problem hiding this comment.
We should log the renderType we use to be able to diagnostic some issue.
|
I grabbed an artifact build from https://github.com/peerless2012/libass-android/actions/runs/43117639949 and tested the file again with |
Description
The version 0.6.0-beta01 added the EFFECTS_ATLAS render type (peerless2012/libass-android#90).
In brief, it is more performant to OVERLAY_OPEN_GL.
To quote the doc:
I moved
assHandler?.initbecause I was getting this exception:Related issues
None
Testing
Emulator
Screenshots
Not relevant
AI or LLM usage
None