Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save 20~30 minutes on every shared memory run. #15

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions hulc/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,35 @@
__author__ = "Oier Mees"
__license__ = "MIT"
__email__ = "[email protected]"



def remove_shm_from_resource_tracker():
"""
Monkey patch multiprocessing.resource_tracker so SharedMemory won't be tracked
More details at: https://bugs.python.org/issue38119
"""
# pylint: disable=protected-access, import-outside-toplevel
# Ignore linting errors in this bug workaround hack
from multiprocessing import resource_tracker

def fix_register(name, rtype):
if rtype == "shared_memory":
return None
return resource_tracker._resource_tracker.register(name, rtype)

resource_tracker.register = fix_register

def fix_unregister(name, rtype):
if rtype == "shared_memory":
return None
return resource_tracker._resource_tracker.unregister(name, rtype)

resource_tracker.unregister = fix_unregister
if "shared_memory" in resource_tracker._CLEANUP_FUNCS:
del resource_tracker._CLEANUP_FUNCS["shared_memory"]


# More details at: https://bugs.python.org/issue38119
remove_shm_from_resource_tracker()