Orphan process left over after SIGTERM under multiprocessing.Manager

[invisibleroads]

Description

IF
we start watchfiles.watch under a multiprocessing.Manager
AND
we send a SIGTERM
THEN
watchfiles.watch leaves behind an orphan process that remains even after the script exits

For now, the workaround is to send a SIGINT or to use the stop_event functionality.

Example Code

'''
python watch-orphan.py 0 -- without manager, interrupt -> no orphan
python watch-orphan.py 1 -- without manager, terminate -> no orphan
python watch-orphan.py 2 -- with manager, interrupt -> no orphan
python watch-orphan.py 3 -- with manager, terminate -> !!! ORPHAN !!!
'''
import multiprocessing as mp
import os
import signal
import subprocess
from sys import argv
from time import sleep
from watchfiles import watch


def watch_without_manager():
    print('without manager')
    try:
        for changes in watch('.'):
            print(changes)
    except KeyboardInterrupt:
        pass


def watch_with_manager():
    print('with manager')
    try:
        with mp.Manager():
            for changes in watch('.'):
                print(changes)
    except KeyboardInterrupt:
        pass


def get_process_count():
    return len(subprocess.check_output(
        'ps aux | grep watch-orphan | grep python | grep -v grep',
        shell=True).splitlines())


scenario_index = int(argv[1])
print('process_count = %s' % get_process_count())
if scenario_index == 0 or scenario_index == 1:
    f = watch_without_manager
else:
    f = watch_with_manager
p = mp.Process(target=f)
p.start()
print('process_count = %s' % get_process_count())
sleep(1)


if scenario_index == 0 or scenario_index == 2:
    print('interrupt')
    os.kill(p.pid, signal.SIGINT)
    sleep(2)
else:
    print('terminate')
    p.terminate()
    p.join()
    sleep(2)
print('process_count = %s' % get_process_count())

Watchfiles Output

$ python watch-orphan.py 0
process_count = 1
without manager
process_count = 2
interrupt
process_count = 1

$ python watch-orphan.py 1
process_count = 1
without manager
process_count = 2
terminate
process_count = 1

$ python watch-orphan.py 2
process_count = 1
with manager
process_count = 2
interrupt
process_count = 1

$ python watch-orphan.py 3
process_count = 1
with manager
process_count = 3
terminate
process_count = 2

Operating System & Architecture

Linux-6.0.12-100.fc35.x86_64-x86_64-with-glibc2.34
#1 SMP PREEMPT_DYNAMIC Thu Dec 8 16:53:55 UTC 2022

Environment

No response

Python & Watchfiles Version

python: 3.10.8 (main, Nov 14 2022, 00:00:00) [GCC 11.3.1 20220421 (Red Hat 11.3.1-3)], watchfiles: 0.18.1

Rust & Cargo Version

No response