Skip to content

net: report accept queue overruns at most every few seconds - #2195

Open
AngeloRubens wants to merge 1 commit into
nanovms:masterfrom
AngeloRubens:net-accept-overrun-ratelimit
Open

AngeloRubens wants to merge 1 commit into
nanovms:masterfrom
AngeloRubens:net-accept-overrun-ratelimit

Conversation

@AngeloRubens

@AngeloRubens AngeloRubens commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

accept_tcp_from_lwip() prints one line for every connection it refuses because the queue of the listening socket is full. The console is synchronous: console_write() takes a global spinlock and the PL011 driver waits on the transmitter for every character. Under a burst of connections that means thousands of serial writes on the network input path, from every CPU, while the system is already overloaded, and the overload turns into a stall.

This counts the refusals and prints at most one line every five seconds, with the number refused since the previous one.

On an OCI aarch64 instance (2 vCPU) serving a Java application at 1,000-2,000 connections, consoles held 18,000-21,000 of these lines, and some 15-s load passes served almost nothing. With the report rate-limited (together with #2193), no pass stalled in 192 passes up to 2,000 connections. Details in #2194.

accept_tcp_from_lwip() prints one line for every connection it refuses because
the queue of the listening socket is full. The console is synchronous:
console_write() takes a global spinlock and the serial driver waits on the
transmitter for each character. Under a burst of connections that is thousands
of serial writes on the network input path, from every CPU, at the moment the
system is already overloaded, and it turns an overload into a stall.

Count the refusals and print at most one line every five seconds, with the
number refused since the previous one.
@AngeloRubens
AngeloRubens force-pushed the net-accept-overrun-ratelimit branch from 393f261 to ca2d574 Compare October 5, 2026 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant