Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cibuildwheel failed on pp38-manylinux_x86_64 and pp38-macosx_arm64 #2015

Closed
wind23 opened this issue Sep 18, 2024 · 3 comments
Closed

cibuildwheel failed on pp38-manylinux_x86_64 and pp38-macosx_arm64 #2015

wind23 opened this issue Sep 18, 2024 · 3 comments

Comments

@wind23
Copy link

wind23 commented Sep 18, 2024 •
edited
Loading

Description

My repo and branch is here: https://github.com/wind23/whole_history_rating/tree/dependabot/github_actions/actions-ddff5e9989

Here is my wheels.yml file, or you can find it here: https://github.com/wind23/whole_history_rating/blob/dependabot/github_actions/actions-ddff5e9989/.github/workflows/wheels.yml

name: Wheels

on:
  workflow_dispatch:
  pull_request:
  push:
    branches:
      - master
  release:
    types:
      - published

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  build_sdist:
    name: Build SDist
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build SDist
        run: pipx run build --sdist

      - name: Check metadata
        run: pipx run twine check dist/*

      - uses: actions/upload-artifact@v4
        with:
          name: cibw-sdist
          path: dist/*.tar.gz

  build_wheels:
    name: Wheels on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, windows-latest, macos-13, macos-14]

    steps:
      - uses: actions/checkout@v4

      - uses: pypa/[email protected]
        env:
          CIBW_ARCHS_MACOS: auto universal2
          SYSTEM_VERSION_COMPAT: 0

      - name: Verify clean directory
        run: git diff --exit-code
        shell: bash

      - name: Upload wheels
        uses: actions/upload-artifact@v4
        with:
          name: cibw-wheels-${{ matrix.os }}
          path: wheelhouse/*.whl

  upload_all:
    name: Upload if release
    needs: [build_wheels, build_sdist]
    runs-on: ubuntu-latest
    if: github.event_name == 'release' && github.event.action == 'published'

    steps:
      - uses: actions/setup-python@v5
        with:
          python-version: "3.x"

      - uses: actions/download-artifact@v4
        with:
          pattern: cibw-*
          path: dist
          merge-multiple: true
      - uses: pypa/gh-action-pypi-publish@release/v1
        with:
          user: __token__
          password: ${{ secrets.pypi_password }}

As you can see, it works on windows-latest, but failed on ubuntu-latest and macos-13. More precisely, it failed on pp38-manylinux_x86_64 and pp38-macosx_x86_64, but successed in the cp3* wheels before.

Also, my same repo used to work a month ago (https://github.com/wind23/whole_history_rating/actions/runs/10270004681), but it fails to work now (https://github.com/wind23/whole_history_rating/actions/runs/10925037181). What would be the problem mostly like?

Here is an example of the failing log:

Building pp38-manylinux_x86_64 wheel
PyPy 3.8 manylinux x86_64

Setting up build environment...
  
      + /opt/python/cp38-cp38/bin/python -c 'import sys, json, os; json.dump(os.environ.copy(), sys.stdout)'
      + which python
      + which pip
                                                              ✓ 0.11s
Building wheel...
  
      + rm -rf /tmp/cibuildwheel/built_wheel
      + mkdir -p /tmp/cibuildwheel/built_wheel
      + python -m pip wheel /project --wheel-dir=/tmp/cibuildwheel/built_wheel --no-deps
  Processing /project
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Building wheels for collected packages: whr
    Building wheel for whr (pyproject.toml): started
    Building wheel for whr (pyproject.toml): finished with status 'error'
    error: subprocess-exited-with-error
    
    × Building wheel for whr (pyproject.toml) did not run successfully.
    │ exit code: 1
    ╰─> [78 lines of output]
        running bdist_wheel
        running build
        running build_py
        creating build/lib.linux-x86_64-pypy38/whr
        copying whr/evaluate.py -> build/lib.linux-x86_64-pypy38/whr
        copying whr/__init__.py -> build/lib.linux-x86_64-pypy38/whr
        copying whr/base.py -> build/lib.linux-x86_64-pypy38/whr
        running build_ext
        g++ -pthread -DNDEBUG -O2 -fPIC -I/opt/python/pp38-pypy38_pp73/include/pypy3.8 -c flagcheck.cpp -o flagcheck.o -std=c++17
        building 'whr_core' extension
        creating build/temp.linux-x86_64-pypy38/src
        g++ -pthread -DNDEBUG -O2 -fPIC -DVERSION_INFO=2.0.3 -I/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/pybind11/include -I/opt/python/pp38-pypy38_pp73/include/pypy3.8 -c src/base.cc -o build/temp.linux-x86_64-pypy38/src/base.o -std=c++17 -fvisibility=hidden -g0
        g++ -pthread -DNDEBUG -O2 -fPIC -DVERSION_INFO=2.0.3 -I/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/pybind11/include -I/opt/python/pp38-pypy38_pp73/include/pypy3.8 -c src/evaluate.cc -o build/temp.linux-x86_64-pypy38/src/evaluate.o -std=c++17 -fvisibility=hidden -g0
        g++ -pthread -DNDEBUG -O2 -fPIC -DVERSION_INFO=2.0.3 -I/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/pybind11/include -I/opt/python/pp38-pypy38_pp73/include/pypy3.8 -c src/game.cc -o build/temp.linux-x86_64-pypy38/src/game.o -std=c++17 -fvisibility=hidden -g0
        g++ -pthread -DNDEBUG -O2 -fPIC -DVERSION_INFO=2.0.3 -I/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/pybind11/include -I/opt/python/pp38-pypy38_pp73/include/pypy3.8 -c src/player.cc -o build/temp.linux-x86_64-pypy38/src/player.o -std=c++17 -fvisibility=hidden -g0
        g++ -pthread -DNDEBUG -O2 -fPIC -DVERSION_INFO=2.0.3 -I/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/pybind11/include -I/opt/python/pp38-pypy38_pp73/include/pypy3.8 -c src/player_day.cc -o build/temp.linux-x86_64-pypy38/src/player_day.o -std=c++17 -fvisibility=hidden -g0
        g++ -pthread -DNDEBUG -O2 -fPIC -DVERSION_INFO=2.0.3 -I/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/pybind11/include -I/opt/python/pp38-pypy38_pp73/include/pypy3.8 -c src/pybind11.cc -o build/temp.linux-x86_64-pypy38/src/pybind11.o -std=c++17 -fvisibility=hidden -g0
        Traceback (most recent call last):
          File "/opt/python/pp38-pypy38_pp73/lib/pypy3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
            main()
          File "/opt/python/pp38-pypy38_pp73/lib/pypy3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
            json_out['return_val'] = hook(**hook_input['kwargs'])
          File "/opt/python/pp38-pypy38_pp73/lib/pypy3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
            return _build_backend().build_wheel(wheel_directory, config_settings,
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/build_meta.py", line 421, in build_wheel
            return self._build_with_temp_dir(
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/build_meta.py", line 403, in _build_with_temp_dir
            self.run_setup()
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/build_meta.py", line 318, in run_setup
            exec(code, locals())
          File "<string>", line 20, in <module>
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/__init__.py", line 117, in setup
            return distutils.core.setup(**attrs)
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/core.py", line 183, in setup
            return run_commands(dist)
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/core.py", line 199, in run_commands
            dist.run_commands()
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/dist.py", line 954, in run_commands
            self.run_command(cmd)
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/dist.py", line 950, in run_command
            super().run_command(command)
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
            cmd_obj.run()
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/command/bdist_wheel.py", line 398, in run
            self.run_command("build")
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
            self.distribution.run_command(command)
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/dist.py", line 950, in run_command
            super().run_command(command)
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
            cmd_obj.run()
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/command/build.py", line 135, in run
            self.run_command(cmd_name)
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
            self.distribution.run_command(command)
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/dist.py", line 950, in run_command
            super().run_command(command)
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
            cmd_obj.run()
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/command/build_ext.py", line 98, in run
            _build_ext.run(self)
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
            self.build_extensions()
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/pybind11/setup_helpers.py", line 287, in build_extensions
            super().build_extensions()
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 476, in build_extensions
            self._build_extensions_serial()
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 502, in _build_extensions_serial
            self.build_extension(ext)
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/command/build_ext.py", line 263, in build_extension
            _build_ext.build_extension(self, ext)
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 581, in build_extension
            self.compiler.link_shared_object(
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/ccompiler.py", line 758, in link_shared_object
            self.link(
          File "/tmp/pip-build-env-v81ucbg4/overlay/lib/pypy3.8/site-packages/setuptools/_distutils/unixccompiler.py", line 269, in link
            self.linker_exe
        TypeError: 'NoneType' object is not subscriptable (key slice(None, None, None))
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
    ERROR: Failed building wheel for whr
  Failed to build whr
  ERROR: Failed to build one or more wheels
                                                             ✕ 20.46s
Error: Command ['python', '-m', 'pip', 'wheel', PurePosixPath('/project'), '--wheel-dir=/tmp/cibuildwheel/built_wheel', '--no-deps'] failed with code 1. 

Error: Process completed with exit code 1.

My repo is written according to the python example of pybind: https://github.com/pybind/python_example/
As you can see, the wheels of their repo is also failing: https://github.com/pybind/python_example/actions/runs/10893406296/job/30228259053

Build log

https://github.com/wind23/whole_history_rating/actions/runs/10925037181/job/30325566939

CI config

https://github.com/wind23/whole_history_rating/blob/dependabot/github_actions/actions-ddff5e9989/.github/workflows/wheels.yml
@wind23 wind23 changed the title cibuildwheel failed on pp38-manylinux_x86_64 cibuildwheel failed on pp38-manylinux_x86_64 and pp38-macosx_arm64 Sep 18, 2024
@Czaki
Copy link
Contributor

Czaki commented Sep 18, 2024

It is the same bug as #1983. It is pypy and recent setuptools incompatibility.

You need either to skip pyp38 or pin lower setuptools version for this not maintained pypy version.

@wind23
Copy link
Author

wind23 commented Sep 21, 2024

It is the same bug as #1983. It is pypy and recent setuptools incompatibility.

You need either to skip pyp38 or pin lower setuptools version for this not maintained pypy version.

Thank you. I skipped pypy in the pipeline and it works now.

BTW, in #1983, it mentioned that pypy310 should work. However, in my project, pypy310 failed to work on Mac OS, although it is working on Windows and Ubuntu:
https://github.com/wind23/whole_history_rating/actions/runs/10925866795/job/30328408987

Building pp310-macosx_arm64 wheel
PyPy 3.10 macOS arm64 - Apple Silicon

Installing Python pp310...
                                                              ✓ 2.30s
Setting up build environment...
                                                              ✓ 2.52s
Installing build tools...
                                                              ✓ 1.36s
Building wheel...
  
  + python -m pip wheel /Users/runner/work/whole_history_rating/whole_history_rating --wheel-dir=/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/cibw-run-4dghxnj3/pp310-macosx_arm64/built_wheel --no-deps
  Processing /Users/runner/work/whole_history_rating/whole_history_rating
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Building wheels for collected packages: whr
    Building wheel for whr (pyproject.toml): started
    Building wheel for whr (pyproject.toml): finished with status 'error'
    error: subprocess-exited-with-error
    
    × Building wheel for whr (pyproject.toml) did not run successfully.
    │ exit code: 1
    ╰─> [21 lines of output]
        running bdist_wheel
        running build
        running build_py
        creating build/lib.macosx-11.0-arm64-pypy310/whr
        copying whr/__init__.py -> build/lib.macosx-11.0-arm64-pypy310/whr
        copying whr/evaluate.py -> build/lib.macosx-11.0-arm64-pypy310/whr
        copying whr/base.py -> build/lib.macosx-11.0-arm64-pypy310/whr
        running build_ext
        g++ -pthread -DNDEBUG -O2 -fPIC -arch arm64 -I/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/cibw-run-4dghxnj3/pp310-macosx_arm64/build/venv/include -I/Users/runner/Library/Caches/cibuildwheel/pypy3.10-v7.3.17-macos_arm64/include/pypy3.10 -c flagcheck.cpp -o flagcheck.o -std=c++17
        building 'whr_core' extension
        creating build/temp.macosx-11.0-arm64-pypy310/src
        g++ -pthread -DNDEBUG -O2 -fPIC -arch arm64 -DVERSION_INFO=2.0.4 -I/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/pip-build-env-ev9sgrmb/overlay/lib/pypy3.10/site-packages/pybind11/include -I/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/cibw-run-4dghxnj3/pp310-macosx_arm64/build/venv/include -I/Users/runner/Library/Caches/cibuildwheel/pypy3.10-v7.3.17-macos_arm64/include/pypy3.10 -c src/base.cc -o build/temp.macosx-11.0-arm64-pypy310/src/base.o -std=c++17 -fvisibility=hidden -g0
        g++ -pthread -DNDEBUG -O2 -fPIC -arch arm64 -DVERSION_INFO=2.0.4 -I/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/pip-build-env-ev9sgrmb/overlay/lib/pypy3.10/site-packages/pybind11/include -I/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/cibw-run-4dghxnj3/pp310-macosx_arm64/build/venv/include -I/Users/runner/Library/Caches/cibuildwheel/pypy3.10-v7.3.17-macos_arm64/include/pypy3.10 -c src/evaluate.cc -o build/temp.macosx-11.0-arm64-pypy310/src/evaluate.o -std=c++17 -fvisibility=hidden -g0
        g++ -pthread -DNDEBUG -O2 -fPIC -arch arm64 -DVERSION_INFO=2.0.4 -I/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/pip-build-env-ev9sgrmb/overlay/lib/pypy3.10/site-packages/pybind11/include -I/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/cibw-run-4dghxnj3/pp310-macosx_arm64/build/venv/include -I/Users/runner/Library/Caches/cibuildwheel/pypy3.10-v7.3.17-macos_arm64/include/pypy3.10 -c src/game.cc -o build/temp.macosx-11.0-arm64-pypy310/src/game.o -std=c++17 -fvisibility=hidden -g0
        g++ -pthread -DNDEBUG -O2 -fPIC -arch arm64 -DVERSION_INFO=2.0.4 -I/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/pip-build-env-ev9sgrmb/overlay/lib/pypy3.10/site-packages/pybind11/include -I/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/cibw-run-4dghxnj3/pp310-macosx_arm64/build/venv/include -I/Users/runner/Library/Caches/cibuildwheel/pypy3.10-v7.3.17-macos_arm64/include/pypy3.10 -c src/player.cc -o build/temp.macosx-11.0-arm64-pypy310/src/player.o -std=c++17 -fvisibility=hidden -g0
        g++ -pthread -DNDEBUG -O2 -fPIC -arch arm64 -DVERSION_INFO=2.0.4 -I/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/pip-build-env-ev9sgrmb/overlay/lib/pypy3.10/site-packages/pybind11/include -I/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/cibw-run-4dghxnj3/pp310-macosx_arm64/build/venv/include -I/Users/runner/Library/Caches/cibuildwheel/pypy3.10-v7.3.17-macos_arm64/include/pypy3.10 -c src/player_day.cc -o build/temp.macosx-11.0-arm64-pypy310/src/player_day.o -std=c++17 -fvisibility=hidden -g0
        g++ -pthread -DNDEBUG -O2 -fPIC -arch arm64 -DVERSION_INFO=2.0.4 -I/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/pip-build-env-ev9sgrmb/overlay/lib/pypy3.10/site-packages/pybind11/include -I/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/cibw-run-4dghxnj3/pp310-macosx_arm64/build/venv/include -I/Users/runner/Library/Caches/cibuildwheel/pypy3.10-v7.3.17-macos_arm64/include/pypy3.10 -c src/pybind11.cc -o build/temp.macosx-11.0-arm64-pypy310/src/pybind11.o -std=c++17 -fvisibility=hidden -g0
        g++ -pthread -DNDEBUG -O2 -shared -Wl,-O1 -Wl,-Bsymbolic-functions -arch arm64 build/temp.macosx-11.0-arm64-pypy310/src/base.o build/temp.macosx-11.0-arm64-pypy310/src/evaluate.o build/temp.macosx-11.0-arm64-pypy310/src/game.o build/temp.macosx-11.0-arm64-pypy310/src/player.o build/temp.macosx-11.0-arm64-pypy310/src/player_day.o build/temp.macosx-11.0-arm64-pypy310/src/pybind11.o -o build/lib.macosx-11.0-arm64-pypy310/whr_core.pypy310-pp73-darwin.so
        ld: unknown options: -Bsymbolic-functions
        clang: error: linker command failed with exit code 1 (use -v to see invocation)
        error: command '/usr/bin/g++' failed with exit code 1
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
    ERROR: Failed building wheel for whr
  Failed to build whr
  ERROR: Failed to build one or more wheels
                                                             ✕ 11.37s
Error: Command ['python', '-m', 'pip', 'wheel', '/Users/runner/work/whole_history_rating/whole_history_rating', '--wheel-dir=/private/var/folders/n3/x0zngmn15fzg2jfkglj4ys4w0000gn/T/cibw-run-4dghxnj3/pp310-macosx_arm64/built_wheel', '--no-deps'] failed with code 1. 

Error: Process completed with exit code 1.

@QuLogic
Copy link

QuLogic commented Sep 26, 2024

The setuptools bug is pypa/distutils#283; you can pin setuptools<72.2.0 on PyPy builds to get it working.

@lexdene lexdene mentioned this issue Oct 18, 2024
Merged
@joerick joerick closed this as not planned Won't fix, can't repro, duplicate, stale Oct 27, 2024
mxmlnkn added a commit to mxmlnkn/cppbktree that referenced this issue Nov 23, 2024
@mxmlnkn
[build] Require Python 3.9 because pp38 fails with cibuildwheel
7cb59fb 
pypa/cibuildwheel#2015
mxmlnkn added a commit to mxmlnkn/cppbktree that referenced this issue Nov 23, 2024
@mxmlnkn
[build] Require Python 3.9 because pp38 fails with cibuildwheel
d1e8d3d 
pypa/cibuildwheel#2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@QuLogic @joerick @Czaki @wind23