Merge pull request #495 from una-auxme/465-update-drive-action-docume…

…ntation-as-soon-as-our-cicd-pipeline-runs-reliably

Updated docs of GitHub actions

.github/workflows/build.yml

@@ -123,5 +123,5 @@ jobs:
           rm -rf /tmp/.buildx-cache/cache/latest
           mv /tmp/.buildx-cache/cache-new/latest /tmp/.buildx-cache/cache/latest
       
-      - name: Prune all images older than 1 days from self-hosted runner
+      - name: Prune all images older than 1 day from self-hosted runner
         run: docker image prune -a -f --filter "until=24h"

doc/development/build_action.md

@@ -8,9 +8,16 @@
 - [The `build-and-push-image` job](#the-build-and-push-image-job)
   - [1. Checkout repository (`actions/checkout@v3`)](#1-checkout-repository-actionscheckoutv3)
   - [2. Set up Docker Buildx (`docker/setup-buildx-action@v2`)](#2-set-up-docker-buildx-dockersetup-buildx-actionv2)
-  - [3. Log in to the Container registry (`docker/login-action@v2`)](#3-log-in-to-the-container-registry-dockerlogin-actionv2)
-  - [4. Test building the image (`docker/build-push-action@v3`)](#4-test-building-the-image-dockerbuild-push-actionv3)
-  - [5. Build and push the image (`docker/build-push-action@v3`)](#5-build-and-push-the-image-dockerbuild-push-actionv3)
+  - [3. Cache Docker layers](#3-cache-docker-layers)
+  - [4. Log in to the Container registry (`docker/login-action@v2`)](#4-log-in-to-the-container-registry-dockerlogin-actionv2)
+  - [5. Test building the image (`docker/build-push-action@v3`)](#5-test-building-the-image-dockerbuild-push-actionv3)
+  - [6. Build and push the image (`docker/build-push-action@v3`)](#6-build-and-push-the-image-dockerbuild-push-actionv3)
+  - [7. Save pull request artifact](#7-save-pull-request-artifact)
+  - [8. Save merge artifact](#8-save-merge-artifact)
+  - [9. Upload artifact](#9-upload-artifact)
+  - [10. Clean up PR Cache](#10-clean-up-pr-cache)
+  - [11. Clean up merge Cache](#11-clean-up-merge-cache)
+  - [12. Prune all images older than one day](#12-prune-all-images-older-than-one-day)
 
 ## General
 
@@ -21,8 +28,7 @@ to [GitHub Packages](ghcr.io).
 The image can then be pulled with `docker pull ghcr.io/una-auxme/paf:latest` to get the latest version
 or `docker pull ghcr.io/una-auxme/paf:<version>` to get a specific version.
 
-If action is triggered by a pull request the created image is then used to execute a test run in the leaderboard, using
-the devtest routes. The results of this simulation are then added as a comment to the pull request.
+After the action is finished the `drive` action is triggered.
 
 ## The `build-and-push-image` job
 
@@ -37,19 +43,47 @@ Set's up Buildx. This is needed to set up the correct driver to allow caching in
 Detailed description why this is needed can be
 found [here](https://github.com/docker/build-push-action/issues/163#issuecomment-1053657228).
 
-### 3. Log in to the Container registry ([`docker/login-action@v2`](https://github.com/docker/login-action))
+### 3. Cache Docker layers
+
+Creates (if not done previously) and sets up the cache for the Docker layers.
+
+### 4. Log in to the Container registry ([`docker/login-action@v2`](https://github.com/docker/login-action))
 
 Logs in with `GITHUB_TOKEN` into the registry (ghcr.io).
 
 Example taken from [here](https://docs.github.com/en/actions/publishing-packages/publishing-docker-images)
 
-### 4. Test building the image ([`docker/build-push-action@v3`](https://github.com/docker/build-push-action/))
+### 5. Test building the image ([`docker/build-push-action@v3`](https://github.com/docker/build-push-action/))
 
 Tries to build the image without pushing it to the repository packages if the workflow was triggered with a pull request.
 
-### 5. Build and push the image ([`docker/build-push-action@v3`](https://github.com/docker/build-push-action/))
+### 6. Build and push the image ([`docker/build-push-action@v3`](https://github.com/docker/build-push-action/))
 
 This action builds the image and pushes it to repository under the `latest` tag if the workflow was triggered with a merge to the `main` branch.
 To avoid large downloads of the base image
 the [GitHub Actions cache](https://docs.docker.com/build/building/cache/backends/gha/)
 is used to cache the image after build.
+
+### 7. Save pull request artifact
+
+If the action was triggered by a pull request an artifact is created with the corresponding PR ID (used for commenting on the PR in the `drive` action).
+
+### 8. Save merge artifact
+
+If the action was triggered by a merge an artifact is created with an invalid PR ID to signalise the `drive` action that it was not triggered by a PR.
+
+### 9. Upload artifact
+
+Uploads the artifact to the given path with a retention of the given number of days.
+
+### 10. Clean up PR Cache
+
+Removes the previous obsolete PR cache.
+
+### 11. Clean up merge Cache
+
+Removes the previous obsolete test cache.
+
+### 12. Prune all images older than one day
+
+Removes all images from the runner that are older than one day to free disk space.

doc/development/drive_action.md

@@ -4,11 +4,17 @@
 
 - [The drive job](#the-drive-job)
   - [1. Checkout repository (`actions/checkout@v3`)](#1-checkout-repository-actionscheckoutv3)
-  - [2. Run agent with docker-compose](#2-run-agent-with-docker-compose)
-  - [3. Copy simulation results file out of container](#3-copy-simulation-results-file-out-of-container)
-  - [4. Stop docker-compose stack](#4-stop-docker-compose-stack)
-  - [5. Comment result in pull request `actions/github-script@v6`](#5-comment-result-in-pull-request-actionsgithub-scriptv6)
+  - [2. Download artifact](#2-download-artifact)
+  - [3. Unzip artifact](#3-unzip-artifact)
+  - [4. Return artifact JSON](#4-return-artifact-json)
+  - [5. Run agent with docker-compose](#5-run-agent-with-docker-compose)
+  - [6. Copy simulation results file out of container](#6-copy-simulation-results-file-out-of-container)
+  - [7. Stop docker-compose stack](#7-stop-docker-compose-stack)
+  - [8. Create simulation results table](#8-create-simulation-results-table)
+  - [9. Print simulation results](#9-print-simulation-results)
+  - [10. Comment result in pull request `actions/github-script@v6`](#10-comment-result-in-pull-request-actionsgithub-scriptv6)
 - [Simulation results](#simulation-results)
+  - [11. Prune all images older than one day](#11-prune-all-images-older-than-one-day)
 
 ## The drive job
 
@@ -20,7 +26,19 @@ The `drive` job is executed conditionally on `pull_request`, after the build suc
 
 Same step as in the [build job](#1-checkout-repository--actionscheckoutv3-)
 
-### 2. Run agent with docker-compose
+### 2. Download artifact
+
+Downloads the artifact that was uploaded during the preceeding `build` action.
+
+### 3. Unzip artifact
+
+Extracts the files of the downloaded artifact.
+
+### 4. Return artifact JSON
+
+Parses the extracted file in the JSON format to read the information inside the file.
+
+### 5. Run agent with docker-compose
 
 Runs the agent with the [`build/docker-compose.cicd.yaml`](../../build/docker-compose.cicd.yaml) that only contains the
 bare minimum components for test execution:
@@ -30,19 +48,27 @@ bare minimum components for test execution:
 - Agent container, run through the
   Carla [`leaderboard_evaluator`](https://github.com/carla-simulator/leaderboard/blob/leaderboard-2.0/leaderboard/leaderboard_evaluator.py).
 
-### 3. Copy simulation results file out of container
+### 6. Copy simulation results file out of container
 
 Copies the created `simulation_results.json` file out of the agent container into the current container
 
-### 4. Stop docker-compose stack
+### 7. Stop docker-compose stack
 
 Stops the remaining containers (Carla, roscore) and removes the volumes with:
 `$ docker-compose down -v`.
 
 This step is important to clean up the remaining containers to have a clean run everytime. This is also the reason for
 the `if: always()`, that ensures step execution.
 
-### 5. Comment result in pull request [`actions/github-script@v6`](https://github.com/marketplace/actions/github-script)
+### 8. Create simulation results table
+
+Reads the simulation results an creates a table for better readability.
+
+### 9. Print simulation results
+
+Prints the simulation results table to the action.
+
+### 10. Comment result in pull request [`actions/github-script@v6`](https://github.com/marketplace/actions/github-script)
 
 This steps uses a JS script to parse the simulation results and add a comment with a results table to the corresponding
 pull request.
@@ -68,3 +94,7 @@ An example comment for this would be:
 | Yield emergency vehicles infractions | 0.0     |
 | Scenario timeouts                    | 62.046  |
 | Min speed infractions                | 0.0     |
+
+### 11. Prune all images older than one day
+
+Removes all images from the runner that are older than one day to free disk space.