[Loom] Does junixsocket support virtual threads?

[cenodis]

Hi, I have been trying to figure out to what extent junixsocket supports virtual threads which were stabilized in Java 21 (JEP-444). Unfortunately neither the javadoc nor website make any mention of this feature or how, if at all, it behaves together with junixsocket.

Specifically I have two questions:

1. Will a call to a blocking I/O method on a junix socket pin the underlying carrier thread?
2. To better support task cancellation with virtual threads the blocking methods on the socket API (streams, connect, accept) have been amended to support interrupts:

This method is interruptible in the following circumstances:
[...]
2. The socket uses the system-default socket implementation and a virtual thread is accepting a connection. In that case, interrupting the virtual thread will cause it to wakeup and close the socket. This method will then throw SocketException with the interrupt status set.

https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/net/Socket.html

junisocket is, of course, not the default socket implementation. But it does offer "fake" sockets that behave like normal IP sockets except they pass their data through a unix socket instead. Does junixsocket offer similar interrupt functionality in those sockets?

[kohlschuetter]

Hi @cenodis!

Yes. Currently, junixsocket doesn't do anything special for virtual threads, so a blocking call with pin the underlying carrier thread.

Adding support for virtual threads would be a nice contribution.

Regarding interrupting at the Java level, and generally testing virtual threads, it would also be helpful to have some more unit tests as great start to dig into this topic.

[kohlschuetter]

@cenodis I have added some test code (MassiveParallelTest) and also improved the codebase for VirtualThreads with respect to ThreadLocal

Try the latest code (or junixsocket-selftest-2.10.0-20240415.192201-2-jar-with-dependencies.jar)

java -Dselftest.only=MassiveParallelTest -jar junixsocket-selftest/target/junixsocket-selftest-2.10.0-*-jar-with-dependencies.jar | grep MassiveParallelTest

and compare the output for Java 17 and Java 21 (for example)

In my testing on macOS, I was able to see a speed up of up to a factor of 6 when using VirtualThreads for the client connections, especially when using thousands/ten-thousands of client threads.

While we should definitely currently pin threads (due to JNI), I don't see any messages even with -Djdk.tracePinnedThreads=full specified. Maybe the pinning is too short to be noticed?

In any case, now that we have some benchmarks, I'll be looking into adding support for kqueue/epoll to see if that makes some further difference.

If you have other test cases, especially those where we get a "pinned thread" log message, please send them my way.

[cenodis]

I have done some experimentation with both SocketChannel and AFUnixSocket to try and provoke possible issues with pinning and have written it as a "test" class here:
Make sure to run this code with a single carrier thread (-Djdk.virtualThreadScheduler.maxPoolSize=1). Otherwise the pool size is set to the amount of cores on your machine, allowing both virtual threads to make progress by consuming a platform thread each.
https://gist.github.com/cenodis/57c5239a879821211ec0b9ca9aa4f863

It does little more than listen on a unix socket in one (virtual) thread and read from said socket in another. But switching between SocketChannel and Junixsocket shows a clear difference in behaviour:

• SocketChannel allows the virtual thread to be unmounted from the carrier once it blocks on accept(). The carrier can them make progress by running the client thread until it blocks on connect() at which point it jumps back into the server thread to accept the connection and so on. This way both threads can make progress despite only having a single platform thread available.
• Junixsocket only manages to create the server socket and call accept(). At this point the carrier appears to be pinned and the application locks up entirely. The client never runs until the main thread exits and destroys the process.

As for -Djdk.tracePinnedThreads, I have seen conflicting statements regarding this property with some saying it only works with pinning due to synchronized blocks. Despite the code above being obviously blocked for multiple seconds I have been unable to provoke a trace even with this option set to full. So either I'm doing something wrong or it doesn't work for JNI calls.

Unfortunately I have not yet found a way to make this class into a proper unit test. Outside of the jdk property there doesn't seem to be a way to ensure that both virtual threads are scheduled to the same carrier. All the ways to control either the scheduler or the virtual thread I have found are restricted as JDK interals. Another option might be to exhaust all other carriers on purpose to ensure deterministic assignment. But this also feels a bit too hacky and unstable to write a proper test against.

Regarding possible solutions on the side of junixsocket:

The "proper" fix is probably to do the same thing that Java did with its Socket: Implement the blocking semantics on top of non-blocking OS calls.
Socket has been reimplemented to use SocketChannel in Java 16 so there is now only one implementation. I have done some reading on how SocketChannel handles virtual threads and, on a high level, it works like this:

1. Test if the current thread is virtual and if so "force" the channel into non-blocking mode. (SocketChannel::configureSocketNonBlockingIfVirtualThread)
2. Perform the, now non-blocking, IO call. Handing the task to whatever async process the runtime uses. (probably polling based but I have seen io-uring being thrown around in some loom docs so this might change)
3. Use LockSupport::park to block the current Thread, freeing the underlying carrier.
4. Once the call is completed, unpark the Thread, allowing another carrier to pick it up again and continue execution.

Now I have no experience with JNI so I don't know how much effort this change would be in this project. As a simpler, but less ideal fix one could also use a pool of platform threads to do the blocking:

1. Create a pool of platform threads.
2. When performing a blocking call inside a virtual Thread, push the native call onto the pool and use LockSupport::park or Future::get to block and free the carrier.
3. Once the native call is complete have the worker Thread unpark the virtual Thread and return it to the pool.
4. The virtual Thread can now be mounted onto another carrier and continue as normal.

This is less ideal because it does not fix the resource issue (you are still consuming a platform Thread for each native call). But it should fix the excessive carrier consumption and avoid interfering with other virtual threads.

[kohlschuetter]

Yes, converting the blocking into non-blocking + LockSupport seems to be the way forward.

I will look into implementing support for kqueue (on macOS/BSD) and epoll (Linux), which are kind of required for this use case. On other platforms, I may leave the current, pinned/blocking strategy in place, at least for the time being (Solaris has its own /dev/poll device fwiw); maybe I will try a select() approach.

If I'm not mistaken, I'll only need 1 platform thread to manage the kqueue/epoll event loop.

[cenodis]

Having dug a bit further through the JVM code I am now almost certain that jdk.tracePinnedThreads does not work for JNI calls. The relevant methods are only invoked when the Thread is being parked (or otherwise attempts to yield) while it is marked as pinned. With JNI the carrier thread enters native code directly so a park is never attempted and therefore no trace will be printed.

[kohlschuetter]

@cenodis OK, I think I have managed to get it working correctly. Your test also appears to work as expected.
Please double-check with the latest 2.10.0-SNAPSHOT (besure to update snapshots with mvn -U or similar)

[kohlschuetter]

Continuing in #157