-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
One EventLoop per process #30
Comments
Let's note that with many event loops, we could still do the same (enqueue the fd is all event loops), but then all event loops would be notified about the fd readyness, whether they care about it or not, and won't be just once (thanks to edge triggered) but everytime a fiber reached EAGAIN, which would repeatedly interrupt other event loops. We'd also have to iterate all event loops on each fd/socket open/close, keep a map of fd => node for each event loop, and so on. |
We identified different issues with a single evloop per process:
We tried to have a dedicated thread running the evloop to cirsumvent issue 1, but we then fall into issue 2 where all enqueues are always cross context, and performance degrades. @straight-shoota proposed to still have multiple evloops, and to keep the A downside is that if you have a server running in a context, creating client connections, doing some user/vhost authentication and so on, then pass the client socket to another context to handle the actual communication, then all enqueues from the evloop would be cross context, which would hinder performance. To allow this scenario, we proposed to transfer the ownership of an An advantage is that the A drawback is that the An example scenario that wouldn't fare well, is trying to print messages to a socket from different contexts; the |
I implemented this, codenamed the "lifetime" evloop against the current crystal (without execution contexts). The initial results were attractive, and the semi-final results are impressive: With the basic HTTP::Server bench, tested with
NOTE: |
I presume there'll be some EC impls with different event loops kept around to ensure that the design doesn't preclude per-EC event loops or global event loops? |
This is almost merged: crystal-lang/crystal#14996 |
Instead of one EL per thread (
preview_mt
) and instead of one EL per EC (#7), we could have one single EL for the whole process!We currently follow the libevent logic (even in #29) where we repeatedly add and remove a fd from the system epoll or kqueue instance, which means that whenever a fiber would block trying to read (or write) we add the fd to the polling instance, suspend, then on wakeup we remove it, read until we block again and we repeat (add -> wait -> del).
With a single event loop instance, we could add the fd when an
IO::FileDescriptor
orSocket
is created (registering for both read/write) then remove it when we close it. That's it.Advantages:
EventQueue::Node
right intoIO::FileDescriptor
&Socket
;EventQueue
to map fd => Node: the Node can directly link the IO object;Disadvantages:
fd
across contexts, prefer to move it);To limit the "one context processes events for another context", there could be a dedicated evloop thread always waiting on the eventloop. Ready fibers would then always be globally enqueued, which might defeat the stealing logic (we can grab more fibers out of the global queue instead of dividing).
The text was updated successfully, but these errors were encountered: