Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate image and server setup in multiple checks #14422

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .github/actions/awx_devel_image/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Setup images for AWX
description: Builds new awx_devel image
inputs:
github-token:
description: GitHub Token for registry access
required: true
runs:
using: composite
steps:
- name: Get python version from Makefile
shell: bash
run: echo py_version=`make PYTHON_VERSION` >> $GITHUB_ENV

- name: Log in to registry
shell: bash
run: |
echo "${{ inputs.github-token }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin

- name: Pre-pull latest devel image to warm cache
shell: bash
run: docker pull ghcr.io/${{ github.repository_owner }}/awx_devel:${{ github.base_ref }}

- name: Build image for current source checkout
shell: bash
run: |
DEV_DOCKER_TAG_BASE=ghcr.io/${{ github.repository_owner }} \
COMPOSE_TAG=${{ github.base_ref }} \
make docker-compose-build
28 changes: 6 additions & 22 deletions .github/actions/run_awx_devel/action.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,8 @@
# This currently *always* uses the "warm build cache" image
# We should do something to allow forcing a rebuild, probably by looking for
# some string in the commit message or something.

name: Run AWX (devel environment)
name: Run AWX docker-compose
description: Runs AWX with `make docker-compose`
inputs:
github-token:
description: GitHub Token for registry access
description: GitHub Token to pass to awx_devel_image
required: true
build-ui:
description: Should the UI be built?
Expand All @@ -23,9 +19,10 @@ outputs:
runs:
using: composite
steps:
- name: Get python version from Makefile
shell: bash
run: echo py_version=`make PYTHON_VERSION` >> $GITHUB_ENV
- name: Build awx_devel image for running checks
uses: ./.github/actions/awx_devel_image
with:
github-token: ${{ inputs.github-token }}

- name: Upgrade ansible-core
shell: bash
Expand All @@ -35,19 +32,6 @@ runs:
shell: bash
run: sudo apt-get install -y gettext

- name: Log in to registry
shell: bash
run: |
echo "${{ inputs.github-token }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin

- name: Pre-pull latest available devel image and build HEAD on top of it
shell: bash
run: |
docker pull ghcr.io/${{ github.repository_owner }}/awx_devel:${{ github.base_ref }}
DEV_DOCKER_TAG_BASE=ghcr.io/${{ github.repository_owner }} \
COMPOSE_TAG=${{ github.base_ref }} \
make docker-compose-build

- name: Start AWX
shell: bash
run: |
Expand Down
15 changes: 13 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,16 +37,27 @@ jobs:
steps:
- uses: actions/checkout@v3

- name: Build awx_devel image for running checks
uses: ./.github/actions/awx_devel_image
with:
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Run check ${{ matrix.tests.name }}
run: AWX_DOCKER_CMD='${{ matrix.tests.command }}' make github_ci_runner
run: AWX_DOCKER_CMD='${{ matrix.tests.command }}' make docker-runner

dev-env:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- uses: ./.github/actions/run_awx_devel
id: awx
with:
build-ui: false
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Run smoke test
run: make github_ci_setup && ansible-playbook tools/docker-compose/ansible/smoke-test.yml -v
run: ansible-playbook tools/docker-compose/ansible/smoke-test.yml -v

awx-operator:
runs-on: ubuntu-latest
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/e2e_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ jobs:
with:
build-ui: true
github-token: ${{ secrets.GITHUB_TOKEN }}
log-filename: e2e-${{ matrix.job }}.log
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@relrod This log-filename is an input to the other action (upload_awx_devel_logs), not this action. It seems to have not failed anything, but this seemed to be a bug.

I could, however, see the argument for having run_awx_devel call upload_awx_devel_logs in the same way that I'm now having it call the image build action. In that case, it would be an input, and we also would be more compact by never calling upload_awx_devel_logs

But that would require adding it as something like a "finalizer" step, which needs research.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like it's not a thing actions/runner#1478

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that is what I really wanted to do originally, but there's no way when using a composite action. :(


- name: Pull awx_cypress_base image
run: |
Expand Down
13 changes: 1 addition & 12 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ I18N_FLAG_FILE = .i18n_built
sdist \
ui-release ui-devel \
VERSION PYTHON_VERSION docker-compose-sources \
.git/hooks/pre-commit github_ci_setup github_ci_runner
.git/hooks/pre-commit

clean-tmp:
rm -rf tmp/
Expand Down Expand Up @@ -324,21 +324,10 @@ test:
cd awxkit && $(VENV_BASE)/awx/bin/tox -re py3
awx-manage check_migrations --dry-run --check -n 'missing_migration_file'

## Login to Github container image registry, pull image, then build image.
github_ci_setup:
# GITHUB_ACTOR is automatic github actions env var
# CI_GITHUB_TOKEN is defined in .github files
echo $(CI_GITHUB_TOKEN) | docker login ghcr.io -u $(GITHUB_ACTOR) --password-stdin
docker pull $(DEVEL_IMAGE_NAME) || : # Pre-pull image to warm build cache
$(MAKE) docker-compose-build

## Runs AWX_DOCKER_CMD inside a new docker container.
docker-runner:
docker run -u $(shell id -u) --rm -v $(shell pwd):/awx_devel/:Z --workdir=/awx_devel $(DEVEL_IMAGE_NAME) $(AWX_DOCKER_CMD)

## Builds image and runs AWX_DOCKER_CMD in it, mainly for .github checks.
github_ci_runner: github_ci_setup docker-runner

test_collection:
rm -f $(shell ls -d $(VENV_BASE)/awx/lib/python* | head -n 1)/no-global-site-packages.txt
if [ "$(VENV_BASE)" ]; then \
Expand Down
17 changes: 0 additions & 17 deletions tools/docker-compose/ansible/smoke-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,23 +11,6 @@
- name: Test that the development environment is able to launch a job
hosts: localhost
tasks:
- name: Boot the development environment
command: |
make docker-compose
environment:
COMPOSE_UP_OPTS: -d
args:
chdir: "{{ playbook_dir }}/../../../"

# Takes a while for migrations to finish
- name: Wait for the dev environment to be ready
uri:
url: "http://localhost:8013/api/v2/ping/"
register: _result
until: _result.status == 200
retries: 120
delay: 5

- name: Reset admin password
shell: |
docker exec -i tools_awx_1 bash <<EOSH
Expand Down
Loading