Fix runtime conflict between embedded and system Python #37
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`embedded_python-core` already provides the include dirs and non-core inherits it. There's no reason for non-core to add them again.
We didn't catch this on CI since we only test lowest and highest supported versions. I don't think I'd change this CI setup just for this, but maybe reconsider in the future if such a case happens again.
hellozee
reviewed
Sep 25, 2023
|
It's impossible to tell. It's any package from pypi.org that has a |
The issue was that our embedded Python could pick up extra Python packages installed in the user's home directory. Python adds user paths by default unless instructed otherwise. The `._pth` file puts it into isolated mode. See the docstrings of `_isolate()` for all the details.
dean0x7d
force-pushed
the
fix-double-include
branch
from
d50ffd2 to
bb0c9bb
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR primarily fixes an issue where the embedded
python(3)executable could still pick up some paths from the system Python which resulted in obscure, hard-to-debug issues. It's important to keep in mind that there are two ways to run the embedded Python:python(3)executable in the embedded package.Case 1 is how we mainly use it and it wasn't really an issue because we already enabled isolated mode at initialization time using the Python C API.
Case 2 is something we do at build time to run some utilities (e.g. convert Jupytext to
. ipynbfiles) or at runtime (launch Jupyter notebooks in a standalone process). When calling thepython(3)exe directly, isolated mode can be set either via the-Icommand line parameter or by adding a._pthfile next to the executable. It's easy to forget or not know to always add-Iso it's best for the embedded package itself to have a._pthfile to avoid issues. See the docstrings for more details.This PR also fixes #36 and a few other smaller bugs. See the individual commit messages. Thanks to @hrishikeshkelkar for finding both the bug and what causes it.