Use kernel event queues for socket monitoring - #23
Open
colemancda wants to merge 34 commits into
Open
Conversation
This reverts commit 1abddea.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the global
poll(2)set with per-platform kernel event queues, behind a seam that keepspollas the fallback for platforms without one.Event queue seam
EventQueueis an internal protocol with three implementations, selected at compile time:epoll(7)kqueue(2)poll(2)FileEventsstays the only vocabulary above the seam, so noEPOLLINorEVFILT_READleaks into the manager.waityields a borrowed buffer instead of returning an array, so a tick allocates nothing. Both kernel backends are level triggered to matchpollsemantics exactly, and each carries a wake channel (eventfdon Linux,EVFILT_USERon Darwin) for a future blocking driver.kqueuereports read and write as separate filters, so entries are coalesced per descriptor before being handed up, otherwise the manager would resume the same continuation twice.Write readiness is now registered on demand
A connected socket is almost always writable, so a standing
EPOLLOUT/EVFILT_WRITEregistration made every wait return every socket on every tick, which is most of what the oldpollloop was doing. Sockets now register for read, error and hangup only, and write readiness is added while a write is pending and dropped once nothing is waiting on it.Idle CPU over 5 seconds with 400 connected sockets:
Behavior change:
Socket.eventno longer emits spurious.writenotifications for idle sockets. A socket that only reads never sees.write, and a socket that writes sees it once for the pending write rather than repeatedly. Code that treated.writeas a periodic heartbeat will notice.Descriptor ownership
pollreportedPOLLNVALfor a descriptor closed behind the manager's back, so its entry was cleaned up promptly. Kernel queues drop closed descriptors silently, which let stale entries accumulate until a lateremoveclosed a descriptor number the kernel had already handed to a new socket. This reproducibly crashed the suite on Linux.The manager no longer closes descriptors it did not open. On hangup or error it deregisters and finishes the event stream but leaves the descriptor open, recording it so its owner can still close it exactly once. Because the descriptor stays open, the kernel cannot recycle the number, which removes the race. No public API change.
Syscall mocking
The
ENABLE_MOCKINGhooks were already scattered throughSyscalls.swift, but the driver types they referenced never existed, so that path could not compile. Adds a thread localMockingDriverwith a syscallTraceandForceErrno, modeled on swift-system, fixes four hooks that could not type check, and enables the flag for debug builds.Tests
EventQueueTestsruns one conformance suite against every backend, so thepollimplementation stays correct on platforms where it is no longer the default. Covers write then read readiness, level triggered persistence, interest mask changes, end of file, and deregistration.IdleTestsasserts idle CPU stays under 20µs per socket second. Restoring the standing write registration makes it fail at 87µs against 3µs, so it is a real guard rather than decoration. A companion test asserts idle sockets report no events at all, which is deterministic and cannot flake.MockingTestscovers tracing, forced errno, counted errno and scoping.CI now runs tests serially, since these share the process file descriptor table and the idle benchmark measures CPU for the whole process. The Linux job previously only built; it now runs the suite too, which is the first coverage the epoll path gets.
Verification
macOS is green across repeated full runs, apart from a pre-existing
No route to hostfailure in the UDP test that also fails onmain. On Linux the conformance and mocking suites pass underswift:6.0, but the full suite could not be verified locally: the socket tests are unreliable in local containers onmainas well, hanging or failing on loopback. The newly enabled Linux job is the real check.