Skip to content

Update libass android to 0.6.0-beta01 and use EFFECTS_ATLAS - #2227

Open
moi15moi wants to merge 2 commits into
damontecres:mainfrom
moi15moi:Update-libass-android
Open

moi15moi wants to merge 2 commits into
damontecres:mainfrom
moi15moi:Update-libass-android

Conversation

@moi15moi

Copy link
Copy Markdown

Description

The version 0.6.0-beta01 added the EFFECTS_ATLAS render type (peerless2012/libass-android#90).

In brief, it is more performant to OVERLAY_OPEN_GL.
To quote the doc:

The fastest render type. It use OpenGL to render, but it use a atlas to push the libass bitmap to the GPU which makes it much more performant than any other mode. It have similar performance to mpv.

I moved assHandler?.init because I was getting this exception:

Exception in coroutine
java.lang.IllegalStateException: Player is accessed on the wrong thread.
Current thread: 'DefaultDispatcher-worker-9'
Expected thread: 'main'
See https://developer.android.com/guide/topics/media/issuesplayer-accessed-on-wrong-thread
	at androidx.media3.exoplayer.ExoPlayerImpl.verifyApplicationThread(ExoPlayerImpl.java:3150)
	at androidx.media3.exoplayer.ExoPlayerImpl.setVideoEffects(ExoPlayerImpl.java:1433)
	at io.github.peerless2012.ass.media.render.AssOverlayManager.<init>(AssOverlayManager.kt:22)
	at io.github.peerless2012.ass.media.AssHandler.init(AssHandler.kt:142)
	at com.github.damontecres.wholphin.services.PlayerFactory.createVideoPlayer(PlayerFactory.kt:181)
	at com.github.damontecres.wholphin.services.PlayerFactory$createVideoPlayer$1.invokeSuspend(Unknown Source:15)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:34)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100)
	at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:124)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:89)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:586)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:798)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:717)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:704)

Related issues

None

Testing

Emulator

Screenshots

Not relevant

AI or LLM usage

None

@YogiBear12

Copy link
Copy Markdown
Contributor

Before deciding to merge this, I just want to call out a risk with this new render approach.

Unless I'm reading it wrong, it passes the video through the effects pipeline. This is a problem because media3 uses 8-bit RGBA textures and output for SDR.

For normal SDR content, this doesn't pose an issue, because SDR is usually 8-bit anyway. But in the anime scene, videos are compressed using 10-bit encoders, even on SDR sources. Passing that video through the effects pipeline results in pretty obvious and distracting banding. This is also a problem in libmpv.

So the tradeoff for frame accurate subtitles is worse looking video when you're watching something encoded in 10-bit. For reference, that would be basically anything encoded with HEVC/x265 and AV1.

@moi15moi

Copy link
Copy Markdown
Author

You're right. I did not know that and I am surprised I missed an obvious detail like this one.
media3 only use a 10 bits texture if the video is hdr (source).
I don't know why they do that and don't use ColorInfo.lumaBitdepth to detect if the video is 10 bits or not.
This should be reported to media3.

AssRenderType.OVERLAY_CANVAS
} else {
AssRenderType.OVERLAY_OPEN_GL
AssRenderType.EFFECTS_ATLAS

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, since videoeffects doesn't support hdr, we should detect if the video is hdr/dolbyvision or not.
If hdr, use OVERLAY_OPEN_GL, otherwhise, use EFFECTS_ATLAS.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this. And should be straight forward too since this function is called from PlaybackViewModel.createPlayer which has a flag for HDR or not already.

Also, will need to add a check in PlaybackViewModel.createPlayer to recreate if HDR state changes though. Right now it only will if the backend changes.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This point should be fixed.
I dont have a HDR TV, so I can't test it properly, but it should be fine.

@damontecres

Copy link
Copy Markdown
Owner

Thanks for the PR! I was looking into this last night, but I haven't done a deep dive into the code.

My initial thoughts are to default to EFFECTS_ATLAS but switch to OVERLAY_OPEN_GL for HDR playback. Not sure if an override would be needed or not.

Also, need to investigate if EFFECTS_ATLAS can replace OVERLAY_CANVAS as well for the older Android versions.

The version 0.6.0-beta01 added the EFFECTS_ATLAS render type (peerless2012/libass-android#90).

In brief, it is more performant to OVERLAY_OPEN_GL.
To quote the [doc](https://github.com/peerless2012/libass-android/tree/0b3617ace00b12aea552d0974978ca7e05a0c488/lib_ass_media#4-effects_atlas):
```
The fastest render type. It use OpenGL to render, but it use a atlas to push the libass bitmap to the GPU which makes it much more performant than any other mode. It have similar performance to mpv.
```

We will still use OVERLAY_OPEN_GL if the video is HDR because EFFECTS_ATLAS use [setVideoEffects](https://developer.android.com/reference/androidx/media3/exoplayer/ExoPlayer#setVideoEffects(java.util.List%3Candroidx.media3.common.Effect%3E))
which doesn't support HDR (it tonemap the video to SDR).

I moved `assHandler?.init` because I was getting this exception:
```
Exception in coroutine
java.lang.IllegalStateException: Player is accessed on the wrong thread.
Current thread: 'DefaultDispatcher-worker-9'
Expected thread: 'main'
See https://developer.android.com/guide/topics/media/issuesplayer-accessed-on-wrong-thread
	at androidx.media3.exoplayer.ExoPlayerImpl.verifyApplicationThread(ExoPlayerImpl.java:3150)
	at androidx.media3.exoplayer.ExoPlayerImpl.setVideoEffects(ExoPlayerImpl.java:1433)
	at io.github.peerless2012.ass.media.render.AssOverlayManager.<init>(AssOverlayManager.kt:22)
	at io.github.peerless2012.ass.media.AssHandler.init(AssHandler.kt:142)
	at com.github.damontecres.wholphin.services.PlayerFactory.createVideoPlayer(PlayerFactory.kt:181)
	at com.github.damontecres.wholphin.services.PlayerFactory$createVideoPlayer$1.invokeSuspend(Unknown Source:15)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:34)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100)
	at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:124)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:89)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:586)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:798)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:717)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:704)
```
@moi15moi
moi15moi force-pushed the Update-libass-android branch from 384d813 to 2f98cb9 Compare November 13, 2026 00:16
@moi15moi

Copy link
Copy Markdown
Author

Also, need to investigate if EFFECTS_ATLAS can replace OVERLAY_CANVAS as well for the older Android versions.

I'm not sure if that is a good idea.
As someone said in the PR that added EFFECTS_ATLAS, with shitty TV, just using setVideoEffects (without any actual effects) cause frame drop.

That's up to you, but I just wanted to make sure you know everything.

@ethaldeman

Copy link
Copy Markdown

I built this branch local and did some testing. I tested on my OnnPlus 4K Pro with a 4K HDR TV output via HDMI. For SDR samples, I confirmed the libass rendering in Exoplayer is more time accurate than the current Wholphin release. It also preformed very similar to mpv besides the below example.

https://files.catbox.moe/ioxho8.mkv is an 11 second sample which contains large ASS subtitle at ~5 seconds. Mpv plays the file but there is some stuttering but no significnat audio and video desync. However for Exoplayer and libass at around ~5 seconds the video moves extremely slow and the audio continues onward at the normal rate. The audio finishes and then the video finishes ~30 seconds later. I don't know if this behavior is expected or not.

https://files.catbox.moe/xkqvfm.mkv is a 15 second clip that I believe is triggering the HDR detection. When I play this file in Exoplayer, the bouncing subtitles on the book are clearly out of sync and look they they do in the current Wholphin release. This suggests it is using the OVERLAY_OPEN_GL rendering. If i rencode the video to standard 264 and play that with Exoplayer, the subtitles are are in-sync. This suggest it is using the EFFECTS_ATLAS. MediaInfo does not list any HDR Format for the original video, so I am not sure exactly what is going on on why it may be triggering the HDR detection. The players reports it is direct playing via "show debug info".

I also checked with a sample I know is HDR with ASS subtitles, and it played and the player reports it is direct playing via "show debug info". However the subtitle are not complex, thus I don't expect any lagg and can't infer what render is being used. Is there an easy way to check in the debug app what render method is being used? "show debug info" does not list it.

@moi15moi

moi15moi commented Nov 13, 2026 •

Copy link
Copy Markdown
Author

Mpv plays the file but there is some stuttering but no significnat audio and video desync. However for Exoplayer and libass at around ~5 seconds the video moves extremely slow and the audio continues onward at the normal rate. The audio finishes and then the video finishes ~30 seconds later.

This is probably related to something in Wolphin (not libass-android) because I know that my phone drops around 30 frames with that file with libass-android demo and the audio seems fine (but maybe it's because it just drop 30 frames). Could you try with libass-android demo and see if you reproduce the desync with the audio?

I still haven't checked why, but I do know that when mpv lags, it will always be worse with exoplayer/libass-android. If mpv doesn't lag, we also don't.
I believe the way exoplayer drop frames is probably not efficient, so we continuously dropping frames instead of skipping them like mpv. This is a real issue, but for 99% .ass file, it should be fine.

Is there an easy way to check in the debug app what render method is being used?

No. I added a note to myself to log it.

} else {
AssRenderType.OVERLAY_OPEN_GL
}
val renderType = assRenderType(isHdr)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should log the renderType we use to be able to diagnostic some issue.

@ethaldeman

ethaldeman commented Nov 13, 2026 •

Copy link
Copy Markdown

I grabbed an artifact build from https://github.com/peerless2012/libass-android/actions/runs/43117639949 and tested the file again with EFFECTS_ATLAS. There was no video or audio sync issues but some stuttering. Playback was comparable to Wholphin mkv playback. I had 124 dropped frames.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants