Skip to content

feat(libkernel): support EVFILT_READ on the read end of a pipe - #5973

Merged
adriadam10 merged 2 commits into
boykopovar:mainfrom
pipe1os:feat/libkernel-kevent-pipe-read
Oct 17, 2026
Merged

adriadam10 merged 2 commits into
boykopovar:mainfrom
pipe1os:feat/libkernel-kevent-pipe-read

Conversation

@pipe1os

@pipe1os pipe1os commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

What

kevent threw kevent: EVFILT_READ on a non-socket descriptor for anything that isn't a guest socket. Java 25 Networking (PPSA99868) stops there on main, in its NIO channels and selector stage: the JDK's kqueue selector opens a pipe and registers its read end with EVFILT_READ / EV_ADD to wake itself up (KQueueSelectorImpl.java L84-L95).

EVFILT_READ now works on the read end of a pipe, following FreeBSD 11's filt_piperead: data is the number of buffered bytes (L1794), EV_EOF is set once the write end is closed (L1798-L1803), and the event is active while either holds (L1804). It uses the same host checks #3644 added for poll:

  • GuestSockets::PipeReadState (new, next to Ready) reads the buffered byte count and whether the write end is gone: NtQueryInformationFile on Windows, the query PipeReadiness makes; FIONREAD and POLLHUP on Linux. PipeReadiness and feat(libkernel): select on pipes on Windows #5013's PipeReady only report poll bits, so they don't give data; both are untouched.
  • GuestSockets::Ready, which the descriptor knote polls, falls back to it for a descriptor that isn't a socket.
  • GuestSockets::WaitAny puts the pipe in the host ppoll on Linux, with a read mask: its only caller, the equeue wait, gets non-socket interests only from these knotes. Windows can't wait on a pipe, so there an infinite or longer WSAPoll is cut to 1 ms, only when the queue has a non-socket interest.
  • On Windows the equeue waker calls WSAStartup itself, as socket and socketpair already do. A kqueue can now wait on a pipe before the process has opened any socket, and the waker's loopback socket failed then with equeue waker: loopback socket setup failed, guest errno 5.

Still throwing: EVFILT_WRITE on a pipe, EVFILT_READ on the write end (FreeBSD opens both ends for reading and writing, kern_pipe L422; host pipes are one-way), and EV_CLEAR and NOTE_LOWAT on a pipe. A new kevent on a pipe entry in the debt doc lists them, along with the 1 ms slices.

pr_overlap flags #5847 (EV_DISABLE, EV_DISPATCH, EV_FORCEONESHOT) and #5493 on the kqueue entry of docs/dev/debt/libkernel.md. This PR only changes two phrases on that line, the pointer to the new entry and EV_CLEAR on a pipe, so whichever goes second keeps both. Kqueue.cpp and GuestKqueue.cpp merge cleanly with #5847.

Tested

Windows 11 Pro (26300), WinLibs GCC 15.2.0 r7, Ninja Release, on main f3b5daaf8:

  • guest_kqueue passes 5/5. The new case registers the read end and checks that it's quiet. Then 3 bytes give data == 3 and no EV_EOF, and a kevent with a 5 s timeout returns the event after a write from another thread at 50 ms. Closing the write end gives EV_EOF with the 1 byte still buffered, and data == 0 once it's read.
  • On main e34c47921, with the source changes reverted and the test kept, it aborts with kevent: EVFILT_READ on a non-socket descriptor not implemented. With only the 1 ms cut removed, that kevent comes back at the 5 s timeout instead and the test fails after 5.35 s, so it can't hang CI. Without the waker's WSAStartup, it aborts with equeue waker: loopback socket setup failed, guest errno 5, since the pipe case runs before any socket is opened.
  • The socket, poll and equeue tests (guest_poll, guest_select, guest_equeue_events, guest_socket_*, guest_tcp, guest_udp, guest_sce_net, ...) pass, 31/31. The full suite passed on e34c47921, 1198/1198.

Linux, Ubuntu 26.04 under WSL 2, GCC 15.2.0, CMake 4.2.3, Ninja Release, BUILD_TESTING=ON, on the same commit, from an archive of this branch with its submodules. FFMPEG_PREBUILT_DIR points at the headers of the local Windows FFmpeg package instead of downloading the Linux one, and none of the built targets link FFmpeg. I built the targets of the tests below and ran them, not the whole suite:

  • guest_kqueue passes 5/5. With Kqueue.cpp, Udp.cpp and SocketRuntime.hpp from main, it aborts with the same kevent: EVFILT_READ on a non-socket descriptor not implemented.
  • guest_dup, guest_equeue_events, guest_kernel_fcntl, guest_poll, guest_rudp_*, guest_sce_net, guest_select, guest_select_wait, guest_socket_*, guest_tcp, guest_udp and guest_kqueue pass, 19/19.

Java 25 Networking (PPSA99868, 01.000.003, SHA-256 9765536f274f3b3e35922b29564ac67afc7cfe979df5d8f3cf5f2a3b3649ebc2, matching the release's SHA256SUMS), relinked with --trap-sdk-syscall-stubs, on main e34c47921:

  • on main, the third row stops right after STAGE: Network: NIO channels and selector with what(): kevent: EVFILT_READ on a non-socket descriptor not implemented.
  • with this PR, STAGE: Network25 returned normally, then STAGE: DestroyJavaVM complete, as on the console.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: PPSA99868 Java 25 Networking (PPSA99868) #5972
  • GPUs tested: n/a
  • AI-assisted: yes

@github-actions github-actions Bot added libkernel docs needs rebase Conflicts with main, rebase needed labels Oct 14, 2026
@pipe1os
pipe1os force-pushed the feat/libkernel-kevent-pipe-read branch from 23f5297 to 682e213 Compare October 14, 2026 19:28
@github-actions github-actions Bot added needs rebase Conflicts with main, rebase needed and removed needs rebase Conflicts with main, rebase needed labels Oct 14, 2026
@pipe1os
pipe1os force-pushed the feat/libkernel-kevent-pipe-read branch from 682e213 to 45bf549 Compare October 15, 2026 04:48
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 15, 2026
pipe1os pushed a commit to pipe1os/personal-portfolio that referenced this pull request Oct 15, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q9h7doEdokbmaQm6eLBFit
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 6343 core/libs/prx/libkernel/Equeue/src/Kqueue.cpp L137-139
docs/dev/debt/libkernel.md L112
PR 6542 docs/dev/debt/libkernel.md L111-112
PR 6962 docs/dev/debt/libkernel.md L173

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 core/libs/prx/libkernel/Socket/src/Udp.cpp
docs/dev/debt/libkernel.md
PR 6044 core/libs/tests/GuestKqueue.cpp
PR 6222 core/libs/prx/libkernel/Socket/src/Udp.cpp
docs/dev/debt/libkernel.md
PR 6228 docs/dev/debt/libkernel.md
PR 6487 docs/dev/debt/libkernel.md
PR 6510 core/libs/prx/libkernel/Equeue/src/Kqueue.cpp
core/libs/tests/GuestKqueue.cpp
docs/dev/debt/libkernel.md
PR 6710 docs/dev/debt/libkernel.md
PR 6828 core/libs/prx/libkernel/Socket/src/Udp.cpp
PR 6836 docs/dev/debt/libkernel.md

kevent threw for any descriptor that is not a guest socket. The read end
of a pipe now gets the descriptor knote, following FreeBSD 11's
filt_piperead: data is the number of buffered bytes and EV_EOF is set
once the write end is closed. GuestSockets::Ready falls back to the new
PipeReadState for a descriptor that is not a socket, and WaitAny waits on
the pipe with ppoll on Linux or in 1 ms WSAPoll slices on Windows, only
when the queue holds a pipe. On Windows the equeue waker now starts
Winsock itself, since a queue can wait on a pipe before any socket exists.
@pipe1os
pipe1os force-pushed the feat/libkernel-kevent-pipe-read branch from 45bf549 to 46d4913 Compare October 15, 2026 21:56
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 15, 2026

@karimibrahim07 karimibrahim07 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Windows check of head 46d49131e, merged onto main 5d9dcc42d (clean merge). Windows 11 Pro (26300), Ryzen 5 3600, RTX 3060, WinLibs GCC 15.2.0 r7, Ninja Release.

Title: Java 25 Networking (PPSA99868, 01.000.003, SHA-256 9765536f…bc2, matching the release's SHA256SUMS), relinked with --windows --to-intel --trap-sdk-syscall-stubs --registry. On main the third row stopped right after STAGE: Network: NIO channels and selector with kevent: EVFILT_READ on a non-socket descriptor not implemented. With this PR, all four rows pass:

STAGE: Network: security configuration
STAGE: Network: numeric IPv4 loopback
STAGE: Network: TCP client/server echo
STAGE: Network: UDP two-way datagrams
STAGE: Network: NIO channels and selector
STAGE: Network25 returned normally
STAGE: DestroyJavaVM complete

Tests: guest_kqueue, guest_poll, guest_select*, guest_equeue*, guest_socket*, guest_tcp, guest_udp, guest_sce_net*: 29 of 29 pass.

Code: I checked the Windows wait path for the case where an equeue waits only on pipes. entries can then hold no socket, but the thread's waker socket is always appended, so WSAPoll never gets an empty array, and the 1 ms slices only apply while a pipe interest is present. PipeReadState tells the read end from the write end by FILE_READ_DATA on Windows and O_ACCMODE on Linux, which matches the new throwing cases in Errors(). I found no issue.

AI-assisted: yes. I ran the build, the tests and the title above.

@screamm

screamm commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Windows run of head 46d49131e, merged onto main 5d9dcc42d together with #6064 (libSceSysmodule only, no shared file; both merges clean).

  • Host: Windows 11 Pro 26220, Ryzen 7 5700X, RTX 4070 (driver 596.21), WinLibs MinGW-w64 GCC 15.2.0 r7, Ninja, Release.
  • Build: everything builds except the known MinGW 64 KiB alignas AGC test (test(agc): align the indirect arguments block at run time #6452/fix(agc): allocate indirect arguments test block dynamically on Windows #6478). libs passes, with no warnings in the changed files.
  • --repeat until-fail:3, 102/102 runs: guest_kqueue together with guest_poll, guest_select, guest_select_wait, guest_equeue_events, guest_dup, guest_kernel_fcntl, guest_tcp, guest_udp, guest_sce_net, the 21 guest_socket_* tests and the 3 guest_rudp_* tests.
  • Negative controls:
    • With Kqueue.cpp, Udp.cpp and SocketRuntime.hpp from main and the test kept, guest_kqueue aborts with kevent: EVFILT_READ on a non-socket descriptor not implemented.
    • With only the 1 ms WSAPoll cut removed, the 5 s kevent wait times out and the test fails after 7.58 s.
    • With the source restored, it passes.
  • Non-GPU suite (ctest -j8 --timeout 120 -E '^agc_driver_|^video_out_flip'): 834/837, 2 skipped. Two load timeouts passed serially. guest_memory failed once at its host-wide commit-charge check, then passed serially and passed a second 10x run 10/10; it is not related to this change.
  • tools/check_conventions.py --base 5d9dcc42d and git diff --check pass. There are no new exports.

Titles: PPSA99868 is not available here. In the relinked titles I have, kevent is imported by PPSA99169, PPSA99360, PPSA99420 and PPSA99764, and kqueue by the first three. I ran each for 45 s with a local diagnostic libkernel that logs kqueue/kevent calls and the filter of each change. None of them called kqueue or kevent in that window (for PPSA99169 this covers app.exe only, not its picker child). So no local title reaches this path, and I ran no title before/after.

Notes on the diff, none blocking:

  • PipeReadState repeats the NtQueryInformationFile access and pipe-information query from PipeReadiness. A shared helper would remove the copy.
  • On Windows, WaitAny sets pipes for any interest that is not a guest socket, not only for pipes. That is equivalent today, but a later non-socket knote would get 1 ms polling without anyone noticing.
  • While a pipe knote is registered, a blocked kevent wakes about every 1 ms, which is the JDK selector's idle state. The debt entry documents this; I did not measure the CPU cost.

Not checked: Linux, PPSA99868.

Written with Claude Code assistance (validation by a subagent, reviewed in this session).

…vent

PipeReadiness and PipeReadState both opened the pipe handle and asked
NtQueryInformationFile for its access mask and local information. Both
now go through QueryPipe, which keeps each caller's error prefix.
@pipe1os

pipe1os commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the run.

PipeReadState repeats the NtQueryInformationFile access and pipe-information query from PipeReadiness. A shared helper would remove the copy.

Done in 647b5a1: both go through QueryPipe now. I'd kept them apart while #5013 was open, to stay out of its way, and it's merged now. On Windows, guest_kqueue, guest_poll and guest_select pass 5/5, and the 34 socket, poll and equeue tests pass.

On Windows, WaitAny sets pipes for any interest that is not a guest socket, not only for pipes.

Right. Today that only happens for pipes: AddDescriptorKnote throws for any other non-socket descriptor, and sceKernelAddReadEvent/sceKernelAddWriteEvent don't add non-socket knotes. Whoever lifts one of those throws for another descriptor type has to go through this path anyway, so I left it. I can rename the flag if you think that reads better.

While a pipe knote is registered, a blocked kevent wakes about every 1 ms

Yes, same as poll on a pipe since #3644. I didn't measure the CPU cost either.

@adriadam10 adriadam10 self-assigned this Oct 16, 2026
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked on Linux (build + guest_kqueue, guest_poll, guest_select, guest_select_wait, guest_equeue_events all pass; guest_kqueue fails with the source changes reverted), against FreeBSD's filt_piperead, and the Windows runs from @karimibrahim07 and @screamm; merging once the full Build is green.

@adriadam10

Copy link
Copy Markdown
Collaborator

Re-checked against current main: the only shared file that changed is Udp.cpp (#5141, a Windows-only check in connect), unrelated to the pipe path. Build + guest_kqueue, guest_poll, guest_select, guest_select_wait, guest_equeue_events, guest_tcp and guest_udp pass on Linux, and the Windows runs from @karimibrahim07 and @screamm cover the Windows branch; merging once the full Build is green.

@adriadam10
adriadam10 merged commit 8730dde into boykopovar:main Oct 17, 2026
8 checks passed
@pipe1os
pipe1os deleted the feat/libkernel-kevent-pipe-read branch October 17, 2026 05:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants