CPython < 3.11 x64 failing on m1 runners?

[masklinn]

As Github is shutting down the macos-12 workers, I figured I'd look at running x86 python on m1.

However, while it seems to work fine with the >= 3.11 universal binary (using arch to force running a specific architecture) as well as passing an explicit architecture to select the correct pypy or graal, the non-universal cpython binaries seem problematic e.g. 3.10 x64 fails with

  Error: dyld[2227]: Library not loaded: /usr/local/opt/gettext/lib/libintl.8.dylib
    Referenced from: <B5E916E8-01CA-35D0-8CA1-DB48CC803EB7> /Users/runner/hostedtoolcache/Python/3.10.15/x64/bin/python3.10
    Reason: tried: '/usr/local/opt/gettext/lib/libintl.8.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/gettext/lib/libintl.8.dylib' (no such file), '/usr/local/opt/gettext/lib/libintl.8.dylib' (no such file), '/usr/local/lib/libintl.8.dylib' (no such file), '/usr/lib/libintl.8.dylib' (no such file,
  Error: not in dyld cache)

Here's the workflow file I'm using: https://github.com/masklinn/ci-test/blob/main/.github/workflows/crossplatform.yml

name: crossplatform

# Controls when the workflow will run
on:
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  check-cross:
    # The type of runner that the job will run on
    runs-on: macos-latest
    strategy:
      fail-fast: false
      matrix:
        python-version:
          - "3.x" # should be universal
          - "3.10" # should be *not* universal
          - "pypy-3.10"
          - "graalpy-24"
        archname: [x86_64, aarch64]
        include:
          - archname: x86_64
            arch: x86_64
            architecture: x64
            processor: i386
          - archname: aarch64
            arch: arm64
            architecture: arm64
            processor: arm

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          architecture: ${{ matrix.architecture }}
      - name: arch(1)
        env:
          ARCHPREFERENCE: ${{ matrix.arch }}
        run: |
          proc=$(arch python -c "import platform; print(platform.processor())")
          echo $proc
          test $proc = ${{ matrix.processor }}

here's the nearly successful run: https://github.com/masklinn/ci-test/actions/runs/11296863554/job/31422564472

[aparnajyothi-y]

Hello @masklinn, Thank you for creating this issue and we will look into it :)

[masklinn]

Hey @aparnajyothi-y thanks.

Also to be clear as re-reading I think I’ve been confusing, by “3.10 specifically” I meant that as the example I tested out but would assume the issue can occur with 3.9 (and 3.8 but that’s going away) as well. Basically with the non-universal cpython binaries. I updated the post.

[haampie]

Very similar issue seen with Python 3.7 on macos-13 (x86).

bash-3.2$ wget -q https://github.com/actions/python-versions/releases/download/3.7.17-5356448435/python-3.7.17-darwin-x64.tar.gz                                                                                                                              
bash-3.2$ tar xf python-3.7.17-darwin-x64.tar.gz 
bash-3.2$ otool -L ./lib/python3.7/lib-dynload/binascii.cpython-37m-darwin.so
./lib/python3.7/lib-dynload/binascii.cpython-37m-darwin.so:
        /usr/local/opt/zlib/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.13)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)
bash-3.2$ file /usr/local/opt/zlib/lib/libz.1.dylib
/usr/local/opt/zlib/lib/libz.1.dylib: cannot open `/usr/local/opt/zlib/lib/libz.1.dylib' (No such file or directory)

So, the binaries refer to dependencies of Python by absolute paths, but the dependencies are not installed.

Potential workaround is brew install zlib.

[haampie]

My issue does not happen on Python 3.6 on macos-13.

Indeed:

$ wget -q https://github.com/actions/python-versions/releases/download/3.6.15-116025/python-3.6.15-darwin-x64.tar.gz
$ tar xf python-3.6.15-darwin-x64.tar.gz 
$ otool -L ./lib/python3.6/lib-dynload/binascii.cpython-36m-darwin.so
./lib/python3.6/lib-dynload/binascii.cpython-36m-darwin.so:
        /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

it records system zlib instead of homebrew zlib.

Sounds like Python 3.7 was linked against homebrew zlib by accident. Or in general: looks like there wasn't enough build isolation, that's causing both @masklinn's and my issue.

If it's difficult to do a rebuild of Python 3.7 I guess you could patch the binaries and replace homebrew dependencies with system dependencies... but I did not check if they are ABI compatible.

[gowridurgad]

Hi @masklinn, The actions/setup-python repository does not support x64 packages for Python <3.11 version on macOS arm64 runners due to compatibility and build issues. The build process for Python versions less than 3.11 involves downloading the official Python sources from python.org and building them using the make tool on the oldest available version of macOS that was available when that Python version was released. This is done to ensure backward compatibility for those Python versions as mentioned in this python-versions documentation.
However, this build process can run into compatibility issues on macOS arm64 runners. The arm64 architecture is used in Apple's newer Mac models that use Apple Silicon processors, which are based on a different instruction set than the x86-64 processors used in older models.
Therefore, the built Python packages for versions less than 3.11 may trigger the issues like few modules missing on macOS arm64 runners. This is why the actions/setup-python repository does not support x64 packages for <Python 3.11 versions on macOS arm64 runners.
For Python versions 3.11 and onwards, the actions/setup-python repository uses the official macOS universal2 Python binaries from python.org, which are compatible with both x86-64 and arm64 architectures. These can be used on macOS arm64 runners without any issues.
For Arm64 platform, we use official macOS universal2 Python binaries and provide only versions which are provided by Python org.
Hence we are closing this issue as x64 packages for < python 3.11 from source will not be supported by macos arm64 runners.
Please feel free to reach us needed to reopen this issue for more clarifications.

[haampie]

Please read my comment. Python 3.7 is broken on macos-13 runners that are x86_64.

[haampie]

I've opened #964