Fails to install on MinGW x64

[RoDuth]

Until 2 days ago the below was working. Not sure this is a msys2/mingw or pyproj issue. First noticed on an appveyor (Visual Studio 2109) build, since confirmed locally. Need mingw x64 due to GTK+3 dependency.

Installation method/steps

• Installation method (conda, pip wheel, from source, etc...)
  pip install pyproj
• How did you install PROJ? Where is it installed?
  $ pacman -S mingw-w64-x86_64-proj
C:\msys64\mingw64\bin\proj.exe
• Please provide all commands/steps you used to install pyproj and PROJ.
  am not sure these are all needed but..

$ pacman -Syuu  # twice if needed
$ pacman --needed -S base-devel mingw-w64-x86_64-toolchain mingw-w64-x86_64-cmake mingw-w64-x86_64-python-setuptools mingw-w64-x86_64-python-pip mingw-w64-x86_64-python-wheel mingw-w64-x86_64-proj
$ pip install pyproj

Environment Information

• pyproj version you are attempting to install
  3.3.0 and 3.2.1
• PROJ version (Execute proj command and give version here.)
  Rel. 8.1.1, September 1st, 2021
• Python version (python -c "import sys; print(sys.version.replace('\n', ' '))")
  3.9.7 (default, Nov 21 2021, 22:02:56) [GCC 11.2.0 64 bit (AMD64)]
• Operation System Information (python -c "import platform; print(platform.platform())")
  Windows-10-10.0.19042-SP0

My temporary solution was to use the last successful local installs wheel.

[RoDuth]

Looking at this again I'm at a bit of a loss as to why pip install pyproj (regardless of version of pyproj, pip, etc. used) stopped working. I can't find anything obvious in MinGW that seems responsible.

What I can add is that in a fresh windows 10 vm with a fresh install of msys2 using MinGW x64 terminal and installing as above (although, after noticing that pyproject.toml contains "cython>=0.28.4" in build-system requires, I have now added pacman -S mingw-w64-x86_64-cython - installs version 0.29.26-1).

with this pip install pyproj produces this error:

Building wheels for collected packages: pyproj
  Building wheel for pyproj (pyproject.toml) ... error
  ERROR: Command errored out with exit status 1:
   command: C:/msys64/mingw64/bin/python.exe C:/msys64/mingw64/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel C:/Users/test/AppData/Local/Temp/tmpfm6b4vcs
       cwd: C:/Users/test/AppData/Local/Temp/pip-install-c7b5s2k4/pyproj_502845de6695480b9a22b34f9466fbff
  Complete output (6 lines):
  running bdist_wheel
  running build
  running build_py
  running build_ext
  building 'pyproj._proj' extension
  error: --plat-name must be one of ('win32', 'win-amd64', 'win-arm32', 'win-arm64')
  ----------------------------------------
  ERROR: Failed building wheel for pyproj
Failed to build pyproj
ERROR: Could not build wheels for pyproj, which is required to install pyproject.toml-based projects

in appveyor it is different, seeming to use 'win-amd64'... Is this because Visual Studio is found? (Doubt this would work in mingw anyway) Fails because of a missing header file.

  Building wheel for pyproj (pyproject.toml): started
  Building wheel for pyproj (pyproject.toml): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: C:/msys64/mingw64/bin/python.exe C:/msys64/mingw64/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel C:/Users/appveyor/AppData/Local/Temp/1/tmpasacfwmt
       cwd: C:/Users/appveyor/AppData/Local/Temp/1/pip-install-5a93kwzg/pyproj_9a1786abda5e49c8a2e1aaf53692750b
  Complete output (45 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.win-amd64-3.9
  creating build/lib.win-amd64-3.9/pyproj
  copying pyproj/aoi.py -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/datadir.py -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/enums.py -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/exceptions.py -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/geod.py -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/network.py -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/proj.py -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/sync.py -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/transformer.py -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/utils.py -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/_show_versions.py -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/__init__.py -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/__main__.py -> build/lib.win-amd64-3.9/pyproj
  creating build/lib.win-amd64-3.9/pyproj/crs
  copying pyproj/crs/coordinate_operation.py -> build/lib.win-amd64-3.9/pyproj/crs
  copying pyproj/crs/coordinate_system.py -> build/lib.win-amd64-3.9/pyproj/crs
  copying pyproj/crs/crs.py -> build/lib.win-amd64-3.9/pyproj/crs
  copying pyproj/crs/datum.py -> build/lib.win-amd64-3.9/pyproj/crs
  copying pyproj/crs/enums.py -> build/lib.win-amd64-3.9/pyproj/crs
  copying pyproj/crs/_cf1x8.py -> build/lib.win-amd64-3.9/pyproj/crs
  copying pyproj/crs/__init__.py -> build/lib.win-amd64-3.9/pyproj/crs
  copying pyproj/database.pyi -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/list.pyi -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/_compat.pyi -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/_crs.pyi -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/_datadir.pyi -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/_geod.pyi -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/_network.pyi -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/_sync.pyi -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/_transformer.pyi -> build/lib.win-amd64-3.9/pyproj
  copying pyproj/py.typed -> build/lib.win-amd64-3.9/pyproj
  running build_ext
  creating build/temp.win-amd64-3.9
  creating build/temp.win-amd64-3.9/Release
  creating build/temp.win-amd64-3.9/Release/pyproj
  "C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/HostX86/x64/cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:/msys64/mingw64/include -IC:/msys64/mingw64/include -IC:/msys64/mingw64/include/python3.9 "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\cppwinrt" /Tcpyproj/_geod.c /Fobuild/temp.win-amd64-3.9/Release/pyproj/_geod.obj
  _geod.c
  C:/msys64/mingw64/include/python3.9\Python.h(36): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory
  error: command 'C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/HostX86/x64/cl.exe' failed with exit code 2
  ----------------------------------------
  ERROR: Failed building wheel for pyproj

I did have success producing and installing a wheel (that appears to be working OK) using a setuptools approach from source:

pacman -S git
git clone https://github.com/pyproj4/pyproj.git
cd pyproj
git checkout 3.3.0
python setup.py bdist_wheel
pip install dist/pyproj-3.3.0-cp39-cp39-mingw_x86_64.whl

Any ideas?

[snowman2]

I wonder if this is related to #1113.

What is the output of os.name and sys.platform?

[RoDuth]

I'd think maybe generally speaking it is?

Although I get:

>>> import os, sys
>>> os.name
'nt'
>>> sys.platform
'win32'

And still get the same --plat-name problem as before so obviously not the same issue.

All this is outside the limits of my knowledge but could this also be related?

pypa/distutils/issues/34

Maybe @lazka could shed some light?

[RoDuth]

Just thought I would try setting the version of setuptools in pyproject.toml. If I set it to "setuptools==59.8.0" (working back from the release around the time I first had the problem) I can get pip to install it:

$ pip install .
Processing c:/msys64/home/rodem/pyproj
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: certifi in c:/msys64/mingw64/lib/python3.10/site-packages (from pyproj==3.3.2.dev0) (2022.6.15)
Building wheels for collected packages: pyproj
  Building wheel for pyproj (pyproject.toml) ... done
  Created wheel for pyproj: filename=pyproj-3.3.2.dev0-cp310-cp310-mingw_x86_64.whl size=454559 sha256=96bf5afa1882e23ef08543203c65acae1b69de8935f1bdf64b511957d2373fe0
  Stored in directory: C:/Users/rodem/AppData/Local/Temp/pip-ephem-wheel-cache-k4easvg5/wheels/32/08/1b/6203a34dce79f449543dc73afcb3a4381e3d19384b45dc8f10
Successfully built pyproj
Installing collected packages: pyproj
  Attempting uninstall: pyproj
    Found existing installation: pyproj 3.3.2.dev0
    Can't uninstall 'pyproj'. No files were found to uninstall.
Successfully installed pyproj-3.3.2.dev0

But can not import it:

$ python
Python 3.10.6 (main, Aug 12 2022, 18:00:29)  [GCC 12.1.0 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyproj
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:/msys64/home/rodem/pyproj/pyproj/__init__.py", line 49, in <module>
    import pyproj.network
  File "C:/msys64/home/rodem/pyproj/pyproj/network.py", line 10, in <module>
    from pyproj._network import (  # noqa: F401 pylint: disable=unused-import
ModuleNotFoundError: No module named 'pyproj._network'

[RoDuth]

Thanks @snowman2 you pointed me in the right direction to finally find a solutions (for now)...

First of all scrap my "can not import" comment above, rookie mistake, didn't change out of the pyproj directory first...

From what I gather: Since setuptools>=60.0.0 setuptools "includes a local, vendored copy of distutils". Therefore the MSYS2 patched version of standard libary distutils is not used unless you make it e.g.:

SETUPTOOLS_USE_DISTUTILS=stdlib pip install pyproj

see: https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html

Ideally the MSYS2 patches find their way into setuptools' distutils and SETUPTOOLS_USE_DISTUTILS=stdlib becomes uneeded.

see:
pypa/distutils/issues/34
pypa/distutils/pull/78

[wsgalaxy]

Thanks @snowman2 you pointed me in the right direction to finally find a solutions (for now)...

First of all scrap my "can not import" comment above, rookie mistake, didn't change out of the pyproj directory first...

From what I gather: Since setuptools>=60.0.0 setuptools "includes a local, vendored copy of distutils". Therefore the MSYS2 patched version of standard libary distutils is not used unless you make it e.g.:

SETUPTOOLS_USE_DISTUTILS=stdlib pip install pyproj

see: https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html

Ideally the MSYS2 patches find their way into setuptools' distutils and SETUPTOOLS_USE_DISTUTILS=stdlib becomes uneeded.

see: pypa/distutils/issues/34 pypa/distutils/pull/78

This solves my problem, thanks!

[Gabogogi]

Great thread! I had issues with SQLAlchemy but now I am sorted

[MAjeRoK]

For Windows before installing the desired package set the variable in PowerShell by this command:

$env:SETUPTOOLS_USE_DISTUTILS="stdlib"

It helped me make pip work with MSYS2