Skip to content

Commit

Permalink
use rapids infra to run testing
Browse files Browse the repository at this point in the history
  • Loading branch information
jperez999 committed Oct 12, 2023
1 parent 96fccce commit 07b3d3e
Showing 1 changed file with 69 additions and 45 deletions.
114 changes: 69 additions & 45 deletions .github/workflows/gpu.yml
Original file line number Diff line number Diff line change
@@ -1,58 +1,82 @@
name: gpu-ci
name: GPU CI

on:
workflow_dispatch:
push:
branches: [main]
branches:
- main
- "pull-request/[0-9]+"
tags:
- "v[0-9]+.[0-9]+.[0-9]+"
pull_request:
branches: [main]
types: [opened, synchronize, reopened]

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
gpu-ci-tensorflow:
runs-on: linux-amd64-gpu-p100-latest-1

Check failure on line 14 in .github/workflows/gpu.yml

View workflow job for this annotation

GitHub Actions / actionlint

label "linux-amd64-gpu-p100-latest-1" is unknown. available labels are "windows-latest", "windows-2022", "windows-2019", "windows-2016", "ubuntu-latest", "ubuntu-22.04", "ubuntu-20.04", "ubuntu-18.04", "macos-latest", "macos-12", "macos-12.0", "macos-11", "macos-11.0", "macos-10.15", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows", "1GPU", "2GPU". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file

Check failure on line 14 in .github/workflows/gpu.yml

View workflow job for this annotation

GitHub Actions / actionlint

label "linux-amd64-gpu-p100-latest-1" is unknown. available labels are "windows-latest", "windows-2022", "windows-2019", "windows-2016", "ubuntu-latest", "ubuntu-22.04", "ubuntu-20.04", "ubuntu-18.04", "macos-latest", "macos-12", "macos-12.0", "macos-11", "macos-11.0", "macos-10.15", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows", "1GPU", "2GPU". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
container:
image: nvcr.io/nvidia/merlin/merlin-tensorflow:nightly
env:
NVIDIA_VISIBLE_DEVICES: ${{ env.NVIDIA_VISIBLE_DEVICES }}
options: --shm-size=1G
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Run tests
run: |
pip install tox
PYTEST_MARKERS="unit and not (examples or integration or notebook) and (singlegpu or not multigpu) $extra_pytest_markers" MERLIN_BRANCH=$branch COMPARE_BRANCH=${{ github.base_ref }} tox -e gpu
jobs:

Check failure on line 29 in .github/workflows/gpu.yml

View workflow job for this annotation

GitHub Actions / actionlint

key "jobs" is duplicated in workflow. previously defined at line:12,col:1. note that key names are case insensitive

Check failure on line 29 in .github/workflows/gpu.yml

View workflow job for this annotation

GitHub Actions / actionlint

key "jobs" is duplicated in workflow. previously defined at line:12,col:1. note that key names are case insensitive
gpu-ci:
runs-on: 1GPU
gpu-ci-pytorch:
runs-on: linux-amd64-gpu-p100-latest-1
container:
image: nvcr.io/nvidia/merlin/merlin-pytorch:nightly
env:
NVIDIA_VISIBLE_DEVICES: ${{ env.NVIDIA_VISIBLE_DEVICES }}
options: --shm-size=1G
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Run tests
run: |
pip install tox
PYTEST_MARKERS="unit and not (examples or integration or notebook) and (singlegpu or not multigpu) $extra_pytest_markers" MERLIN_BRANCH=$branch COMPARE_BRANCH=${{ github.base_ref }} tox -e gpu
jobs:

Check failure on line 47 in .github/workflows/gpu.yml

View workflow job for this annotation

GitHub Actions / actionlint

key "jobs" is duplicated in workflow. previously defined at line:12,col:1. note that key names are case insensitive

Check failure on line 47 in .github/workflows/gpu.yml

View workflow job for this annotation

GitHub Actions / actionlint

key "jobs" is duplicated in workflow. previously defined at line:12,col:1. note that key names are case insensitive
gpu-ci-tensorflow-examples:
runs-on: linux-amd64-gpu-p100-latest-1
container:
image: nvcr.io/nvidia/merlin/merlin-tensorflow:nightly
env:
NVIDIA_VISIBLE_DEVICES: ${{ env.NVIDIA_VISIBLE_DEVICES }}
options: --shm-size=1G
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Run tests
run: |
ref_type=${{ github.ref_type }}
branch=main
if [[ $ref_type == "tag"* ]]
then
git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +refs/heads/release*:refs/remotes/origin/release*
branch=$(git branch -r --contains ${{ github.ref_name }} --list '*release*' --format "%(refname:short)" | sed -e 's/^origin\///')
fi
if [[ "${{ github.ref }}" != 'refs/heads/main' ]]; then
extra_pytest_markers="and changed"
fi
cd ${{ github.workspace }}; PYTEST_MARKERS="unit and not (examples or integration or notebook) and (singlegpu or not multigpu) $extra_pytest_markers" MERLIN_BRANCH=$branch COMPARE_BRANCH=${{ github.base_ref }} tox -e gpu
tests-examples:
runs-on: 1GPU
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Run tests
run: |
pip install tox
PYTEST_MARKERS="(examples or notebook) $extra_pytest_markers" tox -e gpu
jobs:

Check failure on line 65 in .github/workflows/gpu.yml

View workflow job for this annotation

GitHub Actions / actionlint

key "jobs" is duplicated in workflow. previously defined at line:12,col:1. note that key names are case insensitive

Check failure on line 65 in .github/workflows/gpu.yml

View workflow job for this annotation

GitHub Actions / actionlint

key "jobs" is duplicated in workflow. previously defined at line:12,col:1. note that key names are case insensitive
gpu-ci-pytorch-examples:
runs-on: linux-amd64-gpu-p100-latest-1
container:
image: nvcr.io/nvidia/merlin/merlin-pytorch:nightly
env:
NVIDIA_VISIBLE_DEVICES: ${{ env.NVIDIA_VISIBLE_DEVICES }}
options: --shm-size=1G
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Run tests
run: |
ref_type=${{ github.ref_type }}
branch=main
if [[ $ref_type == "tag"* ]]
then
git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +refs/heads/release*:refs/remotes/origin/release*
branch=$(git branch -r --contains ${{ github.ref_name }} --list '*release*' --format "%(refname:short)" | sed -e 's/^origin\///')
fi
if [[ "${{ github.ref }}" != 'refs/heads/main' ]]; then
extra_pytest_markers="and changed"
fi
cd ${{ github.workspace }}; PYTEST_MARKERS="(examples or notebook) $extra_pytest_markers" MERLIN_BRANCH=$branch COMPARE_BRANCH=${{ github.base_ref }} tox -e gpu
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Run tests
run: |
pip install tox
PYTEST_MARKERS="(examples or notebook) $extra_pytest_markers" tox -e gpu

0 comments on commit 07b3d3e

Please sign in to comment.