sh.py:1980: DeprecationWarning: This process ... is multi‐threaded, use of fork() may lead to deadlocks in the child. self.pid = os.fork()

Where is it really coming from?

• os.fork()
  • no hits in SD
• os.forkpty()
  • no hits in SD
• socketserver.ForkingMixIn
  • no hits in SD
• http.server.CGIHTTPRequestHandler
  • no hits in SD
• multiprocessing and concurrent.futures.ProcessPool
  • no hits in SD
• pty.fork()
  • no hits in SD

../../../../../opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/sh.py:1980 /opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/sh.py:1980: DeprecationWarning: This process (pid=2077) is multi-threaded, use of fork() may lead to deadlocks in the child. self.pid = os.fork()

See: https://discuss.python.org/t/concerns-regarding-deprecation-of-fork-with-alive-threads/33555/2

Developers should respond by adjusting their use of multiprocessing or concurrent.futures to explicitly specify either the "forkserver" or "spawn" start methods via a context.

I’m not entirely sure if this idea is both useful and doable, but here’s what I’m thinking off the bat:

Suggesting to use the "spawn" start method instead of "fork" for multi-threaded programs. Including the name of one of the background thread (e.g., "QueueFeederThread"), which is especially important if it originates from a third-party library. The user may not be directly using threading, which could make it challenging for them to pinpoint the source of the warning.

more:

I don’t think it is possible to surface such a warning at the ideal place all of the time. This warning serves as an indication of the fundamental low level problem. The why behind that and which code deserves “blame” for choosing something that happens to use os.fork and thus what workarounds or solutions are needed is rarely so simple and clear cut.

I don’t see that high bar of where it shows up criteria for enabling the warning ever being possible to meet.

The list of things in the stdlib that use os.fork() is very small:

os.fork() - blamed directly on use if threaded. os.forkpty() - blamed directly on use if threaded. socketserver.ForkingMixIn - socketserver.py ForkingMixIn.process_request’s os.fork call can be blamed. TODO: We can update the documentation to mention or link to the fork+threading problem. http.server.CGIHTTPRequestHandler - http/server.py CGIHTTPRequestHandler.run_cgi called from its do_POST method can be blamed. But as far as I can tell, nobody uses this ancient thing so I’m proposing we just deprecate CGIHTTPRequestHandler like we already did with the cgi and cgitb modules. multiprocessing and concurrent.futures.ProcessPool when using the multiprocessing “fork” start method (posix platform default start method until 3.14). pty.fork() - The pty.py pty.fork() function’s os.forkpty or os.fork call can be blamed. Those seem obvious enough already. TODO: We can update documentation to mention this.