Fix version_suffix for nighly shark-ai packages in build_packages.yml. #666
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This fixes the nightly install issues I noticed on iree-org/iree#19391 (see the last section of the PR description) |
marbre
approved these changes
Dec 10, 2024
ScottTodd
added a commit
that referenced
this pull request
Dec 10, 2024
Progress on #584. ~~Depends on #666 (the first commit).~~ This is refactors the `build_packages.yml` workflow so it can be used via `workflow_call` as part of a "pkgci" setup, as an alternative to creating a new `pkgci_build_packages.yml` workflow as originally proposed in #589. This lets us reuse the same workflow for building stable, nightly, and dev packages, all across the same matrix of Python versions and operating systems. Package builds take about 2 minutes (wall time) across the full matrix, so we might as well build them all, instead of artificially constraining ourselves to a subset like only Linux on Python 3.11. Triggers for the workflow are now this: Trigger | Scenario | Build type(s) -- | -- | -- `schedule` | Nightly pre-release build | `rc` `workflow_dispatch` | Workflow testing, manual releasing | `rc` default, `stable` and `dev` possible `workflow_call` | Pull request or push "pkgci" dev builds | `dev` default, `stable` and `rc` possible With this workflow behavior: Build type | Version suffix | Cache enabled? | Tracing enabled? | Pushes to release? -- | -- | -- | -- | -- `stable` | None | No | Yes | No `rc` | `rcYYYYMMDD` | No | Yes | Yes `dev` | `.dev0+${{ github.sha }}` | Yes | No | No Tested over at https://github.com/ScottTodd/shark-ai/actions/workflows/build_packages.yml. Example run: https://github.com/ScottTodd/shark-ai/actions/runs/12245900071 (warm cache)
IanNod
pushed a commit
to IanNod/SHARK-Platform
that referenced
this pull request
Dec 17, 2024
nod-ai#666) Tested here: https://github.com/ScottTodd/shark-ai/actions/runs/12244417956 and confirmed that the `METADATA` embedded within `shark_ai.*.whl` has `Requires-Dist: shortfin==3.1.0rc20241209` as expected, compared to the baseline of `Requires-Dist: shortfin==3.1.0`. I then also ran ```bash pip install --pre --find-links https://github.com/ScottTodd/shark-ai/releases/expanded_assets/dev-wheels shark-ai # Looking in links: https://github.com/ScottTodd/shark-ai/releases/expanded_assets/dev-wheels # Collecting shark-ai # Downloading https://github.com/ScottTodd/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241209-py3-none-any.whl (1.4 kB) # Collecting iree-base-compiler (from shark-ai) # Using cached iree_base_compiler-3.0.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (1.0 kB) # Collecting iree-base-runtime (from shark-ai) # Using cached iree_base_runtime-3.0.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (1.0 kB) # Collecting iree-turbine (from shark-ai) # Using cached iree_turbine-3.0.0-py3-none-any.whl.metadata (5.8 kB) # Collecting shortfin==3.1.0rc20241209 (from shark-ai) # Downloading https://github.com/ScottTodd/shark-ai/releases/download/dev-wheels/shortfin-3.1.0rc20241209-cp311-cp311-manylinux_2_28_x86_64.whl (2.7 MB) pip freeze | grep sh # shark-ai==3.1.0rc20241209 # shortfin==3.1.0rc20241209 ``` Compared to the baseline of ```bash pip install --pre --find-links https://github.com/nod-ai/shark-ai/releases/expanded_assets/dev-wheels shark-ai # Looking in links: https://github.com/nod-ai/shark-ai/releases/expanded_assets/dev-wheels # Collecting shark-ai # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241209-py3-none-any.whl (1.4 kB) # Collecting iree-base-compiler (from shark-ai) # Using cached iree_base_compiler-3.0.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (1.0 kB) # Collecting iree-base-runtime (from shark-ai) # Using cached iree_base_runtime-3.0.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (1.0 kB) # Collecting iree-turbine (from shark-ai) # Using cached iree_turbine-3.0.0-py3-none-any.whl.metadata (5.8 kB) # INFO: pip is looking at multiple versions of shark-ai to determine which version is compatible with other requirements. This could take a while. # Collecting shark-ai # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241208-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241207-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241206-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241205-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241204-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241203-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241202-py3-none-any.whl (1.4 kB) # INFO: pip is still looking at multiple versions of shark-ai to determine which version is compatible with other requirements. This could take a while. # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241201-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241130-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241129-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241128-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241127-py3-none-any.whl (1.4 kB) # INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C. # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241126-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241125-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241124-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241123-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241122-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241121-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241120-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241119-py3-none-any.whl (1.4 kB) # Using cached shark_ai-3.0.0-py3-none-any.whl.metadata (764 bytes) # Collecting shortfin==3.0.0 (from shark-ai) # Using cached shortfin-3.0.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (8.2 kB) pip freeze | grep sh # shark-ai==3.0.0 # shortfin==3.0.0 ```
IanNod
pushed a commit
to IanNod/SHARK-Platform
that referenced
this pull request
Dec 17, 2024
Progress on nod-ai#584. ~~Depends on nod-ai#666 (the first commit).~~ This is refactors the `build_packages.yml` workflow so it can be used via `workflow_call` as part of a "pkgci" setup, as an alternative to creating a new `pkgci_build_packages.yml` workflow as originally proposed in nod-ai#589. This lets us reuse the same workflow for building stable, nightly, and dev packages, all across the same matrix of Python versions and operating systems. Package builds take about 2 minutes (wall time) across the full matrix, so we might as well build them all, instead of artificially constraining ourselves to a subset like only Linux on Python 3.11. Triggers for the workflow are now this: Trigger | Scenario | Build type(s) -- | -- | -- `schedule` | Nightly pre-release build | `rc` `workflow_dispatch` | Workflow testing, manual releasing | `rc` default, `stable` and `dev` possible `workflow_call` | Pull request or push "pkgci" dev builds | `dev` default, `stable` and `rc` possible With this workflow behavior: Build type | Version suffix | Cache enabled? | Tracing enabled? | Pushes to release? -- | -- | -- | -- | -- `stable` | None | No | Yes | No `rc` | `rcYYYYMMDD` | No | Yes | Yes `dev` | `.dev0+${{ github.sha }}` | Yes | No | No Tested over at https://github.com/ScottTodd/shark-ai/actions/workflows/build_packages.yml. Example run: https://github.com/ScottTodd/shark-ai/actions/runs/12245900071 (warm cache)
monorimet
pushed a commit
that referenced
this pull request
Jan 8, 2025
#666) Tested here: https://github.com/ScottTodd/shark-ai/actions/runs/12244417956 and confirmed that the `METADATA` embedded within `shark_ai.*.whl` has `Requires-Dist: shortfin==3.1.0rc20241209` as expected, compared to the baseline of `Requires-Dist: shortfin==3.1.0`. I then also ran ```bash pip install --pre --find-links https://github.com/ScottTodd/shark-ai/releases/expanded_assets/dev-wheels shark-ai # Looking in links: https://github.com/ScottTodd/shark-ai/releases/expanded_assets/dev-wheels # Collecting shark-ai # Downloading https://github.com/ScottTodd/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241209-py3-none-any.whl (1.4 kB) # Collecting iree-base-compiler (from shark-ai) # Using cached iree_base_compiler-3.0.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (1.0 kB) # Collecting iree-base-runtime (from shark-ai) # Using cached iree_base_runtime-3.0.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (1.0 kB) # Collecting iree-turbine (from shark-ai) # Using cached iree_turbine-3.0.0-py3-none-any.whl.metadata (5.8 kB) # Collecting shortfin==3.1.0rc20241209 (from shark-ai) # Downloading https://github.com/ScottTodd/shark-ai/releases/download/dev-wheels/shortfin-3.1.0rc20241209-cp311-cp311-manylinux_2_28_x86_64.whl (2.7 MB) pip freeze | grep sh # shark-ai==3.1.0rc20241209 # shortfin==3.1.0rc20241209 ``` Compared to the baseline of ```bash pip install --pre --find-links https://github.com/nod-ai/shark-ai/releases/expanded_assets/dev-wheels shark-ai # Looking in links: https://github.com/nod-ai/shark-ai/releases/expanded_assets/dev-wheels # Collecting shark-ai # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241209-py3-none-any.whl (1.4 kB) # Collecting iree-base-compiler (from shark-ai) # Using cached iree_base_compiler-3.0.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (1.0 kB) # Collecting iree-base-runtime (from shark-ai) # Using cached iree_base_runtime-3.0.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (1.0 kB) # Collecting iree-turbine (from shark-ai) # Using cached iree_turbine-3.0.0-py3-none-any.whl.metadata (5.8 kB) # INFO: pip is looking at multiple versions of shark-ai to determine which version is compatible with other requirements. This could take a while. # Collecting shark-ai # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241208-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241207-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241206-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241205-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241204-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241203-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241202-py3-none-any.whl (1.4 kB) # INFO: pip is still looking at multiple versions of shark-ai to determine which version is compatible with other requirements. This could take a while. # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241201-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241130-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241129-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241128-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241127-py3-none-any.whl (1.4 kB) # INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C. # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241126-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241125-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241124-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241123-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241122-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241121-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241120-py3-none-any.whl (1.4 kB) # Downloading https://github.com/nod-ai/shark-ai/releases/download/dev-wheels/shark_ai-3.1.0rc20241119-py3-none-any.whl (1.4 kB) # Using cached shark_ai-3.0.0-py3-none-any.whl.metadata (764 bytes) # Collecting shortfin==3.0.0 (from shark-ai) # Using cached shortfin-3.0.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (8.2 kB) pip freeze | grep sh # shark-ai==3.0.0 # shortfin==3.0.0 ```
monorimet
pushed a commit
that referenced
this pull request
Jan 8, 2025
Progress on #584. ~~Depends on #666 (the first commit).~~ This is refactors the `build_packages.yml` workflow so it can be used via `workflow_call` as part of a "pkgci" setup, as an alternative to creating a new `pkgci_build_packages.yml` workflow as originally proposed in #589. This lets us reuse the same workflow for building stable, nightly, and dev packages, all across the same matrix of Python versions and operating systems. Package builds take about 2 minutes (wall time) across the full matrix, so we might as well build them all, instead of artificially constraining ourselves to a subset like only Linux on Python 3.11. Triggers for the workflow are now this: Trigger | Scenario | Build type(s) -- | -- | -- `schedule` | Nightly pre-release build | `rc` `workflow_dispatch` | Workflow testing, manual releasing | `rc` default, `stable` and `dev` possible `workflow_call` | Pull request or push "pkgci" dev builds | `dev` default, `stable` and `rc` possible With this workflow behavior: Build type | Version suffix | Cache enabled? | Tracing enabled? | Pushes to release? -- | -- | -- | -- | -- `stable` | None | No | Yes | No `rc` | `rcYYYYMMDD` | No | Yes | Yes `dev` | `.dev0+${{ github.sha }}` | Yes | No | No Tested over at https://github.com/ScottTodd/shark-ai/actions/workflows/build_packages.yml. Example run: https://github.com/ScottTodd/shark-ai/actions/runs/12245900071 (warm cache)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tested here: https://github.com/ScottTodd/shark-ai/actions/runs/12244417956 and confirmed that the
METADATAembedded withinshark_ai.*.whlhasRequires-Dist: shortfin==3.1.0rc20241209as expected, compared to the baseline ofRequires-Dist: shortfin==3.1.0.I then also ran
Compared to the baseline of