Editable install of screenpy in IDE fails to recognize code

[MarcelWilson]

When using screepy package via editable install (i.e. pip install -e ~/projects/screenpy) IDE code editors fail to recognize the package is installed (imports appear with red underlines). The package will work at runtime but the editor is rendered useless because the IDE can't figure out where the code is on the drive.

This appears to have started with the introduction of the pyproject.toml file in 352c97c .

Further explanation about what's happening can be found here:
python/mypy#13392 (comment)

I also experienced this problem and traced it down to setuptools v64 causing the issue. This version of setuptools seems to cause editable packages to no longer be installed with an egg-link file and now uses another mechanism. See release notes here: https://setuptools.pypa.io/en/latest/history.html#v64-0-0

If the editable package belongs to you, you could modify your pyproject.toml like this:

requires = [
  "setuptools==63.4.3",
  "wheel"
]

Apparently pip prioritizes using the pyproject.toml over the setup.py when installing the package. In its current state, our pyproject.toml doesn't define [build-system] so pip defaults to the Fallback Behavior by using the latest version of setuptools. As mentioned in the quote above, v64 setuptools changed how editable installations work. See python/mypy#13392 (comment)

Oh, so setuptools has switched to using some import hook based approach instead of static entries in pth files? Note that the super standard pth file static directory install that everyone has been using for decades is perfectly valid under PEP 660.

I don't think there's any way for IDEs and static analysis tools to support import hooks. See
https://mail.python.org/archives/list/[email protected]/thread/IIVBPYDZR5T5BGPAWFVYS5ZPYDXGVHQN/#OSWHT5VSRGKPSPYD7PQWR2M4OCSL5WO3 where maintainers of PyCharm, VSCode, Pyright, Pyre, etc are all on the same page about this.

The immediate solution would be to configure the toml to use setuptools 63 until pypa/setuptools#3518 is resolved. Or we could consider using a different backend to install the project (e.g. poetry, flit, etc).

Since python is moving away from using setup.py to pyproject.toml to configure the install of packages this might be the perfect time to look at what we want in the toml beyond the immediate issue of editable installs.

[MarcelWilson]

As mentioned above, there are alternatives to setuptool we could consider.

At the risk of sounding self-serving, I have found poetry-core configured as the backend in the pyproject.toml works when doing editable installs. poetry-core handles the installation of the packages whereas poetry is a full fledged project management tool. Users would not need install poetry in order for us to take advantage of their installer.

It also has the added benefit of allowing screenpy contributors to use poetry to fully manage their environments when branch switching without having to manually manipulate dependencies.