Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sabctools and No GIL? "'sabctools.sabctools', which has not declared that it can run safely without the GIL. " #131

Open
sanderjo opened this issue Dec 18, 2024 · 3 comments

Comments

@sanderjo
Copy link
Collaborator

I'm playing with python 3.13 No GIL / free-threading, and I get this:

root@87c5b0d55fce:/sabnzbd# python3.13-nogil                     
Python 3.13.1 experimental free-threading build (main, Dec  4 2024, 08:54:14) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sabctools
<frozen importlib._bootstrap>:488: RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module 'sabctools.sabctools', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.
>>> 

So is sabctools safe with no GIL?

FWIW

Not an answer to my question, but indeed you can overwrite it

root@87c5b0d55fce:/sabnzbd# python3.13-nogil -Xgil=0             
Python 3.13.1 experimental free-threading build (main, Dec  4 2024, 08:54:14) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sabctools
>>> sabctools.__version__
'8.2.5'
>>> 

@sanderjo
Copy link
Collaborator Author

@mnightingale
Copy link
Contributor

I did have a brief look at it when 3.13 was released, but SAB needs to wait for compatability from python-cffi/cffi#126 and there may be others so I didn't pursue it further.

As you have found it's only 3 lines to mark the module as nogil compatibile, but as for the question of whether it's safe I'm not sure. There is some guidance in https://docs.python.org/3/howto/free-threading-extensions.html and as far as I can see most method are isolated with no global state modifications. Likely need to look at the unlocked ssl parts and see if that needs to merge any upstream changes, but it's also possible SAB won't need it in nogil.

#ifdef Py_GIL_DISABLED
    PyUnstable_Module_SetGIL(m, Py_MOD_GIL_NOT_USED);
#endif 

The real fun is in SAB itself, there could be all sorts of bugs were locking will be required but it currently relies on GIL behaviour.

@sanderjo
Copy link
Collaborator Author

sanderjo commented Dec 18, 2024

The real fun is in SAB itself, there could be all sorts of bugs were locking will be required but it currently relies on GIL behaviour.

I hacked together a working SABnzbd no-GIL : https://github.com/sanderjo/sabnzbd-nogil-docker . "Working" as in: it starts up, I can download, and no problems so far.
(Not working: SABnzbd listening to HTTPS)

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants