Try not to access YT streams before they're available - #1028
Open
RJVB wants to merge 3 commits into
Open
Conversation
YouTube (now?) provides an `available_at` timestamp for streams, and it seems that accessing them before that time can result in a 403 Forbidden error. This commit introduces a simple fix that determines which of the streams of a given video is available last, and then does a non- blocking wait before attempting to open the selected streams. The wait is signalled via a statusbar message. There are no side-effects other than that the wait is imposed also when pasting a YT video link directly into the playlist, apparently because this operation involves starting playback. This should be avoidable but will require more work. Committed from host : Bmbl
zaps166
reviewed
Jul 18, 2026
Contributor
Author
|
On Saturday July 18 2026 15:14:58 Błażej Szczygieł wrote:
why divide by 1000 and cast to `int` and later multipy by 1000 as `int` - no sense + integer overflow possible
Goal is to convert (rounding up) to whole seconds. I see that in fact I should probably
Initially I just used `sleep(3)` but the simple, non-blocking wait function I picked up afterwards works with microseconds.
You're not wrong about the possibility of overflow, but that's not very likely to happen as long as `currentMSecsSinceEpoch()` returns a sensical value. But I now use a double inside the timer, since that's what s_wait() expects anyway.
BTW, I've tried to check for differences between this polite timer, and a simple, blocking call to usleep(), and couldn't find any evidence that the GUI freezes when I use the blocking timer.
How long it takes? Seconds/minutes/hours?
I think the longest I've seen is 7 seconds. YouTube have no incentive to make real users wait for noticeable amounts of time.
Does it properly abort on "stop" button or app close?
Bummer, apparently it does not. I probably have to check for some state variable, or do I need to set up a slot that gets connected to some signal in order to know if I need to interrupt the timer?
(I was expecting some function in QMPlay2Core to help here, but there isn't even an "isPlaying()" function?!)
Is there any path which can call it in GUI thread (sorry - don't remember 😅)? If yes - it can't be like this.
As I said, the reason I use a timer (based on QTest::wait())` is to avoid freezing the GUI.
GetYouTubeVideo() is called only from YouTube::convertAddress(), but how would I know if this can be called from the main thread?
What's supposed to happen if you quit the application while it's processing the paste of a YT video into the playlist?
|
Committed from host : Bmbl
zaps166
reviewed
Jul 25, 2026
zaps166
left a comment
zaps166
left a comment
Owner
There was a problem hiding this comment.
Recently I noticed forbidden error on YouTube, so I can test it. Unfortunately, it's not fully working:
Extensions/YouTube: url for format "139" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "249" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "140" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "251" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "160" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "278" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "394" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "133" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "242" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "395" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "134" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "18" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "243" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "396" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "135" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "244" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "397" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "298" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "302" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "398" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "299" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "303" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
Extensions/YouTube: url for format "399" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
[https @ 0x7f9adc74b7c0] HTTP error 403 Forbidden
[https @ 0x7f9adc764840] HTTP error 403 Forbidden
Contributor
Author
|
@zaps166 commented on this pull request.
Recently I noticed forbidden error on YouTube, so I can test it. Unfortunately, it's not fully working:
Extensions/YouTube: url for format "399" will be available at QDateTime(2026-07-25 12:24:32.000 CEST Qt::LocalTime) so in -0.432 s
[https @ 0x7f9adc74b7c0] HTTP error 403 Forbidden
[https @ 0x7f9adc764840] HTTP error 403 Forbidden
Oh damn. What's the link to that video? Does yt-dlp succeed?
Also `static_cast<int>((latest_available_at - QDateTime::currentMSecsSinceEpoch()) / 1000.0) + 1)`.
Is there a difference with a regular, readable int() cast??
> This interval can be <=0 if the stream is already available!
Should we wait in this case?
I would hope not, because how long should one wait in that case? Take the absolute value of the dt and round that up?
I have not yet had 403 errors with this patch in place, but I think we'll have to look at the yt-dlp code (possibly make it print info like above) to see what it does. But as far as I can say now, there are times when it doesn't say that it will wait.
Too much logs per every video. Put logs only if you need to wait (you did it already below).
OK for once we have this sorted out, but for now (and given your experience above) think it should at least log something for dt<=0.
I think it should go to stderr (`qCDebug`) + status bar as `statusBarMessage()`.
It's an info message, so probably rather qCInfo or qCWarning, no?
Well, logs in QMPlay2 are still bad.
No comment ;)
|
Owner
Video generally works, but fails once per few runs. It's not just one video (try https://www.youtube.com/watch?v=LXb3EKWsInQ). Same experience as without the patch.
No difference, only syntax 😅 |
Contributor
Author
|
On Saturday July 25 2026 05:42:04 Błażej Szczygieł wrote:
Video generally works, but fails once per few runs. It's not just one video (try https://www.youtube.com/watch?v=LXb3EKWsInQ). Same experience as without the patch.
Hmmm, I haven't managed to get it to fail for me, but I get a wait time of 4-5 seconds each time.
[25 Jul 2026 14:39:06.635] Waiting for 1s as required by YouTube
I only see negative wait times, so did you try my suggestion to use the absolute value of dt (rounded up)? If so, maybe increase the duration?
I wonder if this whole mechanism isn't a means to prevent too frequent accesses from a single client to a given video. At first I thought that part of the explanation is that they can't keep all formats of all videos available, but it doesn't make sense that you get the same wait time every time (or else it's because we don't play the entire video?). We may need to impose a minimum wait time (maybe even with an adaptive mechanism because there's clearly an effect of the location from which we connect).
|
Owner
|
Now hard coded 5 seconds ( Also you have a loop which doesn't check for |
Contributor
Author
|
zaps166 left a comment (zaps166/QMPlay2#1028)
Now hard coded 5 seconds (`Waiting for 5s as required by YouTube`) and still the same issue (once per few stop/play):
So there's really no difference in the incidence rate with or without the patch?
What about yt-dlp, does it run into the same issue on this video?
FWIW, the negative dt values are <-2s for me.
Also you have a loop which doesn't check for `youTubeDL.isAborted()`
Agreed on the probably no need for event checking here, but is youTubeDL running at this point? As far as I can tell, the wait currently comes after the call to youTubeDL.exec(), while some other thread is still waiting for the results of that.
Shouldn't there (also) be a call to isAborted() before doing all the json parsing work?
|
Contributor
Author
|
@zaps166 commented on this pull request.
Well, logs in QMPlay2 are still bad.
One thing you probably should do is to rename the Qt logging categories to something closer to the usual reverse URI domain notation. For instance, a line `org.qmplay2.extensions.youtube.debug=true` in qtlogging.ini would be easier to understand than `Extensions/YouTube.debug=true` ;)
|
Committed from host : Bmbl
Contributor
Author
|
René J.V. Bertin wrote on 20260725::16:35:15 re: "Re: [zaps166/QMPlay2] Try not to access YT streams before they're available (PR #1028)"
FWIW, the negative dt values are <-2s for me.
Systematically the case for formats 92-94 and 300, 301 for me (available only when I use my Firefox cookies). And when I tell yt-dlp to download those it doesn't impose a wait.
|
Owner
Maybe you have much slower machine. When I limit my CPU to 500 MHz, I also have ~-2s. It looks like time obtained by yt-dlp and time compared in QMPlay2 - ~2 seconds elapsed between point 1 and 2, so no need to wait at all. Anyway waiting doesn't help, I still can reproduce forbidden. Maybe you should check for cookies, referer, user agent (it's QMPlay2 by default, maybe youtube randomly rejects it?). |
Contributor
Author
|
On Thursday July 30 2026 16:02:05 Błażej Szczygieł wrote:
> FWIW, the negative dt values are <-2s for me.
Maybe you have much slower machine. When I limit my CPU to 500 MHz,
Probably much slower than yours, but not that much. It's an i7-6500U @2.50GHz, which should be fast enough (and I get the same wait times on a 2nd gen i7 @2.7Ghz that is noticeably slower).
Who knows what black magic that javascript challenge invokes that could introduce an effect of CPU speed on the delay with which videos are made available. And who knows what information is obtained from the browser cookies. I still think it's more likely that the difference in what we observe is linked to our geographic locations.
Maybe you should check for cookies, referer, user agent (it's QMPlay2 by default, maybe youtube randomly rejects it?).
I don't change anything to the referer or user agent. I do know that the use of firefox's cookies makes a difference in the stream formats that yt-dlp returns, but the rejection occurs afterwards.
In case you missed it: after a rejection I can reconstruct the list formula you use to open a video,audio pair directly with FFmpeg from the log. I have not yet managed to do this fast enough (= it always takes me a couple of seconds) and I have never had a rejection.
It's already a bit of a miracle that you can obtain stream URLs for formats that are available only if you have the right cookies set, and then access those streams without offering those same cookies.
Isn't there anything you can learn from the yt-dlp sources? They must have started to respect those `available_at` times for a reason (if they always did then only the notification about it is new, OR I simply never encountered videos for which a wait was required).
|
Owner
|
So what Maybe my occasional 403 reason is different than yours.
I mean that you may try to use standard user agent instead of QMPlay2 😄 and check results. |
Contributor
Author
|
On Friday July 31 2026 11:49:54 Błażej Szczygieł wrote:
So what `available_at` means? I think it returns time of my PC (in UTC) when temporary URL was generated.
I think it literally means "this stream will be available at UTC=x". Of course that should also be the time of your PC because evidently it is in perfect sync with the wall clock in our universe ;)
Maybe my occasional 403 reason is different than yours.
That's what I was hinting at. There can be other reasons why access is forbidden even if you wait the amount of time you're supposed to wait.
One evident possibility: too frequent connection attempts (since you said it happens only "once in a while" ... that kind of suggests you're trying again and again).
I mean that you may try to use standard user agent instead of QMPlay2 😄 and check results.
Remind me, that would be if I run into rejections, right?
|
Owner
|
There are more and more 403 errors, also in M3U8 HLS live streams (also in mpv and ffmpeg) - live stream plays for less than a minute and freezes with 403 errors. |
Contributor
Author
|
On Tuesday August 18 2026 03:53:19 Błażej Szczygieł wrote:
There are more and more 403 errors, also in M3U8 HLS live streams (also in mpv and ffmpeg) - live stream plays for less than a minute and freezes with 403 errors.
Yes, and "the page needs to be reloaded" errors start popping up all of a sudden yesterday evening, even on videos I had justed looked at before.
See yt-dlp/yt-dlp#17389, which contains a workaround.
|
Owner
|
🤔 I don't use cookies from browser |
Contributor
Author
|
On Tuesday August 18 2026 04:11:48 Błażej Szczygieł wrote:
zaps166 left a comment (zaps166/QMPlay2#1028)
:thinking: I don't use cookies from browser
From what I understand and see, that only makes matters worse. See the last comments in that yt-dlp ticket.
|
Owner
|
I see some posts there, but skimmed only. I tried nightly yt-dlp - same issue - downloads 39 seconds of live stream and then 403. However URL extracted using "Video Download Helper" works properly 🤔 |
Contributor
Author
|
On Tuesday August 18 2026 05:08:30 Błażej Szczygieł wrote:
zaps166 left a comment (zaps166/QMPlay2#1028)
I see some posts there, but skimmed only. I tried nightly yt-dlp - same issue - downloads 39 seconds of live stream and then 403.
I saw something about that on the ticket too.
However URL extracted using "Video Download Helper" works properly 🤔
Heh, I watch a lot of probably not very legal videos by storing the URLs extracted with that extension O:-)
You can hopefully just wait until that livestream is finished and then watch it "in the replay". I'm doing that right now with https://www.youtube.com/live/2DLSWibujiY (almost 5 minutes in).
I also got confirmation that not waiting until the `available_at` time is likely to throw 403 errors, and that update channel switches are permanent.
So a priori, anyone bitten by the current situation could just do a manual switch (e.g. via the extra arguments thingy). It would help if you'd add a 24h update timer to the update-on-startup, or an "update now" button, I've been meaning to bring that up a while now. ;)
|
Owner
Ok, but I want to test it; so far everything throws 403 xD
Understandable if the downloaded binary is from nightly channel, it'll download updates from that channel, too.
Instead of restarting QMPlay2? 24h timetout to re-check for updates seems to be good idea.
No if it's permanent live stream (TV). |
Contributor
Author
|
On Tuesday August 18 2026 05:47:55 Błażej Szczygieł wrote:
zaps166 left a comment (zaps166/QMPlay2#1028)
> I also got confirmation that not waiting until the `available_at` time is likely to throw 403 errors
Ok, but I want to test it; so far everything throws 403 xD
I think the nightly was just updated (2026.08.18.122307). I'm not seeing any 403s anymore, but if everything throws that error then you should probably be able to reproduce it when downloading with yt-dlp too, no?
In that case the additional output with `-vU` may contain some useful information aboud something different in your region ...
|
Owner
|
It's working so far 😄 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
YouTube (now?) provides an
available_attimestamp for streams, and it seems that accessing them before that time can result in a 403 Forbidden error.This commit introduces a simple fix that determines which of the streams of a given video is available last, and then does a non- blocking wait before attempting to open the selected streams. The wait is signalled via a statusbar message.
There are no side-effects other than that the wait is imposed also when pasting a YT video link directly into the playlist, apparently because this operation involves starting playback. This should be avoidable but will require more work.
#Closes: #1027