Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix GitHub actions to publish to pypi #24

Closed
wants to merge 21 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 100 additions & 8 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,15 @@ concurrency:
cancel-in-progress: true

jobs:
#
# 1) Fast builds for ubuntu + macOS
#
build-wheels:
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
name: Build and publish Python package (${{ matrix.os }})
timeout-minutes: 15
timeout-minutes: 60
runs-on: ${{ matrix.os }}

steps:
Expand All @@ -36,17 +39,15 @@ jobs:
with:
toolchain: stable

# We do NOT set up QEMU here,
# so macOS won't fail on QEMU steps.

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install cibuildwheel
shell: bash

- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: all

- name: Build package
env:
CIBW_SKIP: "pp* *-musllinux*" # Skip PyPy and musllinux builds
Expand All @@ -57,8 +58,6 @@ jobs:
mv protoc/include/google /usr/local/include/google
CIBW_BEFORE_ALL_MACOS: |
brew install protobuf
CIBW_ARCHS_LINUX: auto aarch64 ppc64le s390x
CIBW_QEMU: "true"
CIBW_BEFORE_BUILD: |
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
source $HOME/.cargo/env
Expand All @@ -76,6 +75,9 @@ jobs:
path: |
dist/*.whl

#
# 2) Build source distribution
#
build-source-dist:
name: Build and publish Python package (source distribution)
timeout-minutes: 10
Expand Down Expand Up @@ -112,6 +114,9 @@ jobs:
path: |
dist/*.tar.gz

#
# 3) Publish the main wheels first
#
publish-wheels:
needs: [build-wheels, build-source-dist]
name: Publish Python wheels
Expand All @@ -135,6 +140,9 @@ jobs:
with:
packages-dir: final_dist/

#
# 4) Publish Rust package
#
publish-rust:
name: Build and publish Rust package
timeout-minutes: 10
Expand Down Expand Up @@ -178,3 +186,87 @@ jobs:
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
run: |
cargo publish -p krec

#
# 5) Build QEMU-based wheels for aarch64 only
#
build-qemu-wheels:
name: Build QEMU-based wheels (aarch64 only)
runs-on: ubuntu-latest
timeout-minutes: 120

steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.11"

- name: Set up Rust
uses: actions-rs/toolchain@v1
with:
toolchain: stable

# We do QEMU setup only here, for aarch64
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
# "platforms: aarch64" ensures we only enable QEMU for ARM64
# so we don't attempt ppc64le or s390x, etc.
with:
platforms: aarch64

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install cibuildwheel
shell: bash

- name: Build package with QEMU
env:
CIBW_ARCHS_LINUX: "aarch64"
CIBW_QEMU: "true"
# Skip PyPy, musllinux, i686, ppc64le, s390x, etc.
CIBW_SKIP: "pp* *-musllinux* *-i686* *-ppc64le* *-s390x*"
CIBW_BEFORE_ALL_LINUX: |
curl -L https://github.com/protocolbuffers/protobuf/releases/download/v28.3/protoc-28.3-linux-x86_64.zip -o protoc.zip
unzip protoc.zip -d protoc
mv protoc/bin/protoc /usr/local/bin/protoc
mv protoc/include/google /usr/local/include/google
CIBW_BEFORE_BUILD: |
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
source $HOME/.cargo/env
pip install setuptools-rust
CIBW_ENVIRONMENT: |
PATH="/usr/local/bin:$HOME/.cargo/bin:$PATH"
CARGO_NET_GIT_FETCH_WITH_CLI=true
run: |
cibuildwheel --output-dir dist

- name: Upload QEMU wheel artifacts
uses: actions/upload-artifact@v3
with:
name: wheels-qemu
path: dist/*.whl

#
# 6) Publish the QEMU-based wheels
#
publish-qemu-wheels:
name: Publish QEMU-based wheels
needs: [build-qemu-wheels]
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- name: Download QEMU wheels
uses: actions/download-artifact@v3
with:
name: wheels-qemu
path: dist

- name: Publish QEMU wheels
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: dist
Loading