Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add workflow for building and publishing wheels #2260

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions .github/reusable-build/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
name: Reusable action which builds serving

inputs:
python-version:
description: 'Python version'
required: true
env:
USE_BAZEL_VERSION: '6.5.0'
runs:
using: 'composite'
steps:
- name: Build the package for Python ${{ inputs.python-version }}
shell: bash
run: |
docker build -f tensorflow_serving/tools/docker/Dockerfile.devel

# - name: Install runner dependencies
# shell: bash
# run: |
# sudo apt update
# sudo apt install -y automake build-essential ca-certificates curl git gcc-10 g++-10 libcurl3-dev libfreetype6-dev libpng-dev libtool libzmq3-dev mlocate openjdk-8-jdk openjdk-8-jre-headless pkg-config software-properties-common swig unzip wget zip zlib1g-dev
#
# - name: Install python dependencies
# shell: bash
# run: |
# # This uses the pre-PEP517 setuptools approach, which means
# # wheels aren't built in isolated environments. We therefore need to install
# # a bunch of build-time stuff here.
# pip install future>=0.17.1 grpcio h5py keras_applications>=1.0.8 keras_preprocessing>=1.1.0 mock numpy portpicker requests
#
# - uses: bazel-contrib/[email protected]
# name: Set up Bazel
# with:
# # Avoid downloading Bazel every time.
# bazelisk-cache: true
# # Store build cache per workflow.
# disk-cache: ${{ github.workflow }}-${{ hashFiles('.github/workflows/wheels.yml') }}
# # Share repository cache between workflows.
# repository-cache: true
#
# - name: Verify bazel installation
# shell: bash
# run: |
# which bazel
# bazel info
# bazel version
#
# - name: Build the package with bazel
# shell: bash
# run: |
# bazel build --output_filter=DONT_MATCH_ANYTHING --config=kokoro tensorflow_serving/model_servers:tensorflow_model_server
# # cp bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server /usr/local/bin
#
# # Build and install TensorFlow Serving API
# bazel build --output_filter=DONT_MATCH_ANYTHING --config=kokoro tensorflow_serving/tools/pip_package:build_pip_package
# bazel-bin/tensorflow_serving/tools/pip_package/build_pip_package /tmp/pip
# cp /tmp/pip/tensorflow_serving_api-*.whl ./dist/
# # pip --no-cache-dir install --upgrade /tmp/pip/tensorflow_serving_api-*.whl

- name: Upload wheel artifact for Python ${{ inputs.python-version }}
uses: actions/upload-artifact@v4
with:
name: ml-metadata-wheel-py${{ inputs.python-version }}
path: dist/tensorflow_serving_api-*.whl
65 changes: 65 additions & 0 deletions .github/workflows/wheels.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: Build serving wheels

on:
pull_request:
workflow_dispatch:
release:
types: [published]

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11"]

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Build package
id: build-package
uses: ./.github/reusable-build
with:
python-version: ${{ matrix.python-version }}

- name: List and check wheels
shell: bash
run: |
# Pin transitive dependency on pkginfo until https://github.com/pypa/twine/issues/1070 is
# fixed
pip install twine pkginfo>=1.10.0
${{ matrix.ls || 'ls -lh' }} wheels/
twine check wheels/*


upload_to_pypi:
name: Upload to PyPI
runs-on: ubuntu-latest
if: (github.event_name == 'release' && startsWith(github.ref, 'refs/tags')) || (github.event_name == 'workflow_dispatch')
needs: [build]
environment:
name: pypi
url: https://pypi.org/p/tensorflow_serving/
permissions:
id-token: write
steps:
- name: Retrieve wheels
uses: actions/download-artifact@v4
with:
merge-multiple: true
path: wheels

- name: List the build artifacts
run: |
ls -lAs wheels/

- name: Upload to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages_dir: wheels/
176 changes: 162 additions & 14 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,14 +1,162 @@
node_modules
/bazel-bin
/bazel-ci_build-cache
/bazel-genfiles
/bazel-out
/bazel-serving
/bazel-tensorflow
/bazel-tensorflow_serving
/bazel-testlogs
/bazel-tf
/bazel-workspace
/third_party/py/numpy/numpy_include
/util/python/python_include
/util/python/python_lib
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock

# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/latest/usage/project/#working-with-version-control
.pdm.toml
.pdm-python
.pdm-build/

# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
10 changes: 10 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
services:
build-wheels:
build:
context: .
dockerfile: tensorflow_serving/tools/docker/Dockerfile.devel
args:
- PYTHON_VERSION=${PYTHON_VERSION}
- USE_BAZEL_VERSION=${USE_BAZEL_VERSION}
volumes:
- .:/build:delegated
Loading
Loading