Skip to content

Leaked listening socket on httpd shutdown causes intermittent playback failures #1480

Open

Description

@chrishamm

First of all, thanks for this addon.

HTTPServer.server_close() in kodion/network/http_server.py replaces TCPServer.server_close() entirely and never closes the listening socket. Combined with allow_reuse_port = True on the same class, a shutdown can leave a bound socket behind that nobody accepts on, and the next start_httpd() binds a second socket next to it instead of failing with EADDRINUSE. The kernel then spreads incoming connections across both sockets, and everything that lands on the dead one hangs until the caller times out.

Context

  • Add-on Version: 7.4.4
  • Kodi Version: 21.3-Omega
  • Kodi GUI Language: English (en_gb)
  • Operating System: LibreELEC 12.2.1, aarch64, Python 3.11.13
  • Operating System Language: en_GB.UTF-8

Expected Behavior

shutdown_httpd() releases the proxy port. At most one socket is bound to it at any time.


Current Behavior

Two sockets listening on the proxy port, both owned by the same kodi.bin:

tcp        0      0 127.0.0.1:50152         0.0.0.0:*               LISTEN      1173/kodi.bin
tcp        6      0 127.0.0.1:50152         0.0.0.0:*               LISTEN      1173/kodi.bin

The second one sits at a permanently full accept backlog (6 = request_queue_size 5 + 1) and there were six connections in CLOSE_WAIT next to it. Ten requests to /ping from the box itself, %{http_code} and %{time_total}:

501/0.003470 501/0.002557 000/3.002385 501/0.002180 501/0.007498
501/0.005212 501/0.003338 501/0.002511 000/3.002294 501/0.002650

The 000 ones hit the dead socket and never get an answer. 501 is just because /ping is not a real path, the point is that the live socket answers in 3 ms.

Playback needs the manifest plus a stream of segment requests through the proxy, so in practice most attempts fail:

error <general>: CCurlFile::Open - <http://127.0.0.1:50152/youtube/manifest/dash?file=BnSTr7_NwtU.mpd> Failed with code 0:
error <general>: AddOnLog: inputstream.adaptive: Download failed, internal error: http://127.0.0.1:50152/youtube/manifest/dash?file=BnSTr7_NwtU.mpd
error <general>: CVideoPlayer::OpenInputStream - error opening [plugin://plugin.video.youtube/play/?video_id=BnSTr7_NwtU]
error <general>: Playlist Player: skipping unplayable item: 0, path [plugin://plugin.video.youtube/play/?video_id=BnSTr7_NwtU]

A Kodi restart fixes it and it comes back hours later. That matches how often the shutdown/restart cycle runs here: 10 HTTPServer: Starting lines in one kodi.log, 45 in the previous one.


Steps to Reproduce

  1. Let the addon run normally until the service monitor has gone through a number of idle shutdown/restart cycles. grep "HTTPServer: Shutting down" kodi.log to see them.
  2. netstat -tlnp | grep 50152. Once there are two LISTEN rows for the same PID, one of them with a non-zero Recv-Q, the box is in the broken state.
  3. for i in $(seq 1 10); do curl -s -m 3 -o /dev/null -w "%{http_code} " http://127.0.0.1:50152/ping; done gives a mix of 501 and 000.
  4. Play any video. It usually fails with the errors above, and occasionally works.

I could not pin down which cycle leaks. It is not every one, only those where a handler thread is still alive when the shutdown runs, which is why it takes hours to show up rather than happening on the first restart.


Log

I only have the ordinary event log for this, not a full debug log, since the box had already been in the broken state for days by the time I looked. The relevant lines are quoted above. The netstat and curl output is more conclusive than a debug log would be anyway, and both can be reproduced on any box in this state.


Additional Information

As far as I can tell the sequence is:

  1. shutdown_httpd() starts a thread for self.httpd.shutdown() and joins both threads with a 2 second timeout. If a request handler is still busy, the join times out and it carries on regardless.
  2. self.httpd.server_close() runs. It never calls super() and never touches self.socket, so the listening socket stays open. It also returns early on the AttributeError from self._threads.pop_all().
  3. self.httpd = None drops the reference, but the still-running handler thread holds self.server, so the socket is not garbage collected either. It stays bound and listening with nothing accepting on it.
  4. start_httpd() binds a new socket to the same address. allow_reuse_port = True makes that succeed silently.

What I run locally, on top of 7.4.4:

-    allow_reuse_port = True
+    allow_reuse_port = False

     def server_close(self):
+        self.socket.close()
+
         request_handler = self.RequestHandlerClass

The self.socket.close() is the actual fix. Turning off SO_REUSEPORT is not strictly needed, but with it off a leak that slips through shows up as a visible bind error instead of half the connections silently going nowhere, and I do not think a single server needs it. Since the patched shutdown the port is properly released on every idle shutdown here, and the video that had been failing for a week plays.

Both anxdpanic/master and MoojMidge/master and v7.4 still have the unpatched server_close as of today.

Activity

  1. Fludizz commented on Aug 8, 2026

    @Fludizz

    I have noticed the same happening on my Kodi box for I don't even remember how long. It doesn´t happen frequent enough to reliably reproduce but - it would lead to massive debug logs as it may take days before it happens, or just hours. Entirely random. My solution was always to just restart the entire system as soon as I see the Download Failed message in the logs after it fails to play a video.

    I have modified .kodi/addons/plugin.video.youtube/resources/lib/youtube_plugin/kodion/network/http_server.py as per what @chrishamm suggested and will report back in a few days.

  2. Fludizz commented on Aug 13, 2026

    @Fludizz

    So based on my experience across two different Kodi boxes, applying the fixes from @chrishamm actually significantly improved playback success - we have not had any video playback failures since patching my two Kodi boxes!

    I have checked both their logs and the HTTP errors we used to frequently see have not reappeared since changing this.

    No adverse effects were observed either - everything remains working as expected.

  3. Fludizz commented on Sep 2, 2026

    @Fludizz

    As final confirmation: I have had zero playback failures since applying this patch to both my kodi boxes.

    @MoojMidge - do you need a Pull request or can you include this in the next release from this ticket since it's such a tiny change?

  4. chrishamm commented on Sep 7, 2026

    @chrishamm
    Author

    Opened #1488 with the fix.

  5. Porozhniakov commented on Sep 22, 2026

    @Porozhniakov

    Confirming this on a different setup, same symptoms and same root cause.

    • Add-on Version: 7.4.4
    • Kodi Version: 21.3 (Debian package 3:21.3+dfsg-1+rpt2)
    • Operating System: Raspberry Pi OS, Raspberry Pi 4, aarch64, Python 3.13.5

    Two listening sockets on the proxy port, both owned by kodi.bin, the older one with a full backlog (request_queue_size = 5) and a queue of connections nobody ever accepted:

    LISTEN     0    5  127.0.0.1:50152  0.0.0.0:*        users:(("kodi.bin",pid=2280,fd=83))
    LISTEN     5    5  127.0.0.1:50152  0.0.0.0:*        users:(("kodi.bin",pid=2280,fd=82))
    CLOSE-WAIT 199  0  127.0.0.1:50152  127.0.0.1:51366
    CLOSE-WAIT 199  0  127.0.0.1:50152  127.0.0.1:60820
    

    The two oldest CLOSE-WAIT entries (lastrcv ≈ 121 h) are the inputstream.adaptive requests for a DASH manifest from a failed playback five days earlier. Nothing ever accepted them.

    One observation on the trigger that may help: the last healthy playback through the proxy was interrupted by a network outage. During it, proxied requests to www.youtube.com and *.googlevideo.com failed with ReadTimeoutError, and the proxy never recovered afterwards. That fits your theory that the leak happens when a handler thread is still alive at shutdown. A month earlier, during a period of unstable upstream connectivity, the same box had four listeners on 50152 at once.

    A side-effect-free way to check a box is ss -tanp '( sport = :50152 )'. Probing the port with curl only adds more entries to the dead socket's queue.

    I'm applying #1488 locally and will report back.

  6. Porozhniakov commented on Sep 26, 2026

    @Porozhniakov

    Follow-up: I've been running #1488 on a Raspberry Pi 4 (Raspberry Pi OS, Kodi 21.3, Python 3.13) since Sep 23.

    Checked on the running proxy:

    • only one listener on 127.0.0.1:50152, with an empty accept queue;
    • a test bind() with SO_REUSEPORT to the same port now fails with EADDRINUSE, so the patched code is the one running;
    • after playback stops, the idle shutdown releases the port completely, with no socket left on 50152;
    • playback resumes fine after a long pause.

    No playback failures so far. I haven't had a network outage since applying it, and that's what triggered the leak here before, so I can't say anything about that case yet.

  7. bigtime-cargo commented on Oct 29, 2026

    @bigtime-cargo

    Same issue on Android TV: TCL BeyondTV (Android 11, armeabi-v7a), Kodi 21.3 (kodinerds build), add-on 7.4.4. After a proxy restart netstat -tln showed two LISTEN sockets on 127.0.0.1:50152 with a non-empty accept queue, and playback failed with CCurlFile::Open … 127.0.0.1:50152/youtube/manifest/dash … Timeout was reached(28) until Kodi was restarted. We now run the equivalent of #1488 (TCPServer.server_close(self) in HTTPServer.server_close()) locally. +1 for merging #1488.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions