Skip to content

Commit

Permalink
Fix pretty broken CI
Browse files Browse the repository at this point in the history
This was prompted by a bug in setup-python@v5, which means that the
post-clean-up job will fail if you do not actually install anything in
pip, as the cache directory will not be created. This has hit a few of
our GHA workflows.

actions/setup-python#815

The reason this was triggering for job-runner was that the current CI
setup was pretty much broken.

Firstly, the PYTHON_VERSION env var was not being set, which meant that
every test run was always testing 3.8, not 3.9 or 3.10 as per the
matrix.  Fixing this meant a bit of windows/powershell finesse.  It
always explains the fact that for some reason, even though we have 22.04
in the matrix, we did not run the tests on it (this was what triggered
the above failure).

Additionally, we were going to the trouble of installing/starting docker
on macos, and then not even running the docker tests. This feels like we
had the abandonded attempts of getting docker tests running on macos.

To fix this, I just removed those steps and fully embraced not running
docker tests (which required excluding a few more tests).  Additionally,
macos-12 is old, and has very slow runners, so I took the opportunity to
update to using macos-13.

I did attempt to use our opensafely-core/setup-action, but it still
seems to have problems with just on windows, and my yak stack was full.
  • Loading branch information
bloodearnest committed Sep 30, 2024
1 parent 24d49c1 commit 1df7f2b
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 24 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/build_and_publish.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,9 @@ jobs:
run: docker login https://ghcr.io -u ${{ github.actor }} --password ${{ secrets.GITHUB_TOKEN }}
- name: Push image to GitHub Container Registry
run: |
IMAGE_ID=ghcr.io/${{ github.repository_owner }}/$IMAGE_NAME
docker tag $IMAGE_NAME $IMAGE_ID:latest
docker push $IMAGE_ID:latest
IMAGE_ID="ghcr.io/${{ github.repository_owner }}/$IMAGE_NAME"
docker tag "$IMAGE_NAME" "$IMAGE_ID:latest"
docker push "$IMAGE_ID:latest"
create-release-from-tag:
name: Create release from tag
Expand Down
38 changes: 17 additions & 21 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,17 @@ jobs:

test-job:
strategy:
fail-fast: false
matrix:
os: [ubuntu-20.04, ubuntu-22.04, windows-2019, macos-12]
os: [ubuntu-20.04, ubuntu-22.04, windows-2019, macos-13]
# Python 3.8 is what we currently support for running cohortextractor
# locally, and 3.9 is what we required for databuilder so we need to make
# sure we can run with those
python: ["3.8", "3.9", "3.10"]
runs-on: ${{ matrix.os }}
name: Run test suite
env:
PYTHON_VERSION: "python${{ matrix.python }}"
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -46,30 +49,23 @@ jobs:
cache: "pip"
cache-dependency-path: requirements.*.txt

- name: Set up default Python 3.8 for MacOS
if: ${{ matrix.os == 'macos-12' && matrix.python != '3.8' }}
uses: actions/setup-python@v5
with:
python-version: 3.8
cache: "pip"
cache-dependency-path: requirements.*.txt

- name: install Docker/Colima on MacOS
if: ${{ matrix.os == 'macos-12' }}
run: brew install docker

- name: start Colima on MacOS
if: ${{ matrix.os == 'macos-12' }}
run: colima start

- uses: extractions/setup-just@69d82fb0233557aec017ef13706851d0694e0f1d

- name: Run actual tests on ${{ matrix.os }}
if: ${{ matrix.os == 'ubuntu-20.04' }}
run: just test -vvv
if: ${{ matrix.os == 'ubuntu-20.04' || matrix.os == 'ubuntu-22.04' }}
run: |
echo "$PYTHON_VERSION"
just test -vvv
- name: Run actual tests on ${{ matrix.os }}
if: ${{ matrix.os == 'windows-2019' || matrix.os == 'macos-12' }}
- name: Run actual tests on windows
if: ${{ matrix.os == 'windows-2019' }}
run: |
# windows powershell: set to executable w/o version, as it doesn't have one on windows
$Env:PYTHON_VERSION = "python"
just test-no-docker -vvv
- name: Run actual tests on macos
if: ${{ matrix.os == 'macos-13' }}
run: just test-no-docker -vvv

test-package-build:
Expand Down

0 comments on commit 1df7f2b

Please sign in to comment.