Skip to content

Commit

Permalink
Build & push instructions and test
Browse files Browse the repository at this point in the history
  • Loading branch information
@NicholasCote
NicholasCote authored Aug 5, 2024
1 parent 42c1724 commit 0e3d587
Show file tree
Hide file tree
Showing 2 changed files with 117 additions and 35 deletions.
43 changes: 43 additions & 0 deletions .github/workflows/webapp-build-push.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Name the Action
name: Automation to build a container image
# Define when the Action is run. This example is run when there is a push to the flask-app/ directory on the app branch.
on:
push:
paths:
- August-08-2024/**
branches:
- main
# Define the jobs to run. A job can have multiple steps and an Action can contain multiple jobs.
jobs:
build-image:
# Use the latest ubuntu image to run the jobs
runs-on: ubuntu-latest
steps:
# Step 1 is to checkout the github repo used to build the Dockerfile
- name: Check out the repo
uses: actions/checkout@v4
# Step 2 is to login to docker hub so the image can be pushed
- name: Login to Docker Hub
uses: docker/login-action@v2
# GitHub repository secrets are used as variables to provide login information to Docker Hub
# DOCKERHUB_USERNAME and DOCKERHUB_TOKEN need to be added to the Actions secrets
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# Step 3 gets the date and sends it to GITHUB_OUTPUT to apply as an image tag
- name: Get current date
id: date
run: echo "date=$(date +'%Y-%m-%d.%H')" >> $GITHUB_OUTPUT
# Step 4 builds and pushes the docker image
- name: Build Docker image
uses: docker/build-push-action@v4
with:
# Provide the August-08-2024 directory as build context
context: August-08-2024/.
# Specify where the Containerfile is located in relation to the repo base path
file: August-08-2024/Containerfile
# Enable the push to docker hub
push: true
# Provide the tags to apply to the image, this example uses the latest image tag
tags: |
ncote/workshop-webapp:${{ steps.date.outputs.date }}
109 changes: 74 additions & 35 deletions August-08-2024/walkthrough.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -170,21 +170,21 @@
" # Step 1 is to checkout the github repo used to build the Dockerfile\n",
" - name: Check out the repo\n",
" uses: actions/checkout@v4\n",
" # Get the date and send to GITHUB_OUTPUT to apply as image tag\n",
" # Step 2 gets the date and sends it to GITHUB_OUTPUT to apply as an image tag\n",
" - name: Get current date\n",
" id: date\n",
" run: echo \"date=$(date +'%Y-%m-%d.%H')\" >> $GITHUB_OUTPUT\n",
" # Build the docker image\n",
" # Step 3 builds the docker image\n",
" - name: Build Docker image\n",
" uses: docker/build-push-action@v4\n",
" with:\n",
" # Provide the current directory as build context \n",
" # Provide the August-08-2024 directory as build context \n",
" context: August-08-2024/.\n",
" # Specify where the Dockerfile is located in relation to the repo base path\n",
" # Specify where the Containerfile is located in relation to the repo base path\n",
" file: August-08-2024/Containerfile\n",
" # Enable the push to docker hub\n",
" # Disable the push to docker hub\n",
" push: false\n",
" # Provide the tags to apply to the image, this example uses the latest image tag \n",
" # Provide the tags to apply to the image, this example uses the date from step 2\n",
" tags: |\n",
" ncote/workshop-webapp:${{ steps.date.outputs.date }}\n",
"```"
Expand All @@ -208,7 +208,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The GitHub action defined only triggers when changes are made to the August-08-2024 directory on the main branch. Let's open the Containerfile and add a new comment so it changes the Containerfile, but not actually how it's built and run. Once the comment has been added you will see that the GitHub extension in the left bar shows we have another change that can be pushed. Let's go ahead and add another commit message and push the new changes to kick off the GitHub Action. Once the push has been made we can watch the status of our GitHub action by going back to teh repository URL and selecting Actions in the middle of the top navigation bar on the site. "
"The GitHub action defined only triggers when changes are made to the August-08-2024 directory on the main branch. Let's open the Containerfile and add a new comment so it changes the Containerfile, but not actually how it's built and run. Once the comment has been added you will see that the GitHub extension in the left bar shows we have another change that can be pushed. Let's go ahead and add another commit message and push the new changes to kick off the GitHub Action. Once the push has been made we can watch the status of our GitHub action by going back to the repository URL and selecting Actions in the middle of the top navigation bar on the site. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Once the job has completed successfully let's disable it from running. From the All workflows Actions page, on the left side select the individual job name, \"Automation to build a container image\". Once on this page, in the upper right there's a button with 3 dots. Click this to bring up a menu with 3 options, the bottom one being Disable workflow. Click the Disable workflow button to turn off the Action for now. "
]
},
{
Expand All @@ -222,7 +229,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Once we know the image is building correctly we can go ahead and push that image to a container registry so it can be shared with others or reused. The image created in the example is pushed to [Docker Hub](https://hub.docker.com/) and uses credentials stored as repository [Actions secrets](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions) to log in. If we go back to the code repository in our browser we can open up the repository settings by using the far right option on the top navigation bar. Once in the repo security settings we can look at the left navigation menu that is opened and under Security there is a dropdown for Secrets and variables that contains a link to Actions. Once we have the Actions repository secrets open we can add our Docker Hub information based on the variables used in the workflow file, DOCKERHUB_USERNAME & DOCKERHUB_TOKEN. Copy and paste the contents below into our existing GitHub Actions workflow file and overwrite everything that was in there before. "
"Once we know the image is building correctly we can go ahead and push that image to a container registry so it can be shared with others or reused. The image created in the example is pushed to [Docker Hub](https://hub.docker.com/) and uses credentials stored as repository [Actions secrets](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions) to log in. If we go back to the code repository in our browser we can open up the repository settings by using the far right option on the top navigation bar. Once in the repo security settings we can look at the left navigation menu that is opened and under Security there is a dropdown for Secrets and variables that contains a link to Actions. Once we have the Actions repository secrets open we can add our Docker Hub information based on the variables used in the workflow file, DOCKERHUB_USERNAME & DOCKERHUB_TOKEN. Inside the `.github/workflows/` directory create another file that will build and push the image using a descriptive name like `webapp-build-push.yaml`. The contents to place inside the file can be found below with explanations of each line, or group of lines, included as comments:"
]
},
{
Expand Down Expand Up @@ -255,27 +262,66 @@
" # DOCKERHUB_USERNAME and DOCKERHUB_TOKEN need to be added to the Actions secrets\n",
" with:\n",
" username: ${{ secrets.DOCKERHUB_USERNAME }}\n",
" password: ${{ secrets.DOCKERHUB_TOKEN }}\n",
" # Get the date and send to GITHUB_OUTPUT to apply as image tag\n",
" password: ${{ secrets.DOCKERHUB_TOKEN }}\n",
" # Step 3 gets the date and sends it to GITHUB_OUTPUT to apply as an image tag\n",
" - name: Get current date\n",
" id: date\n",
" run: echo \"date=$(date +'%Y-%m-%d.%H')\" >> $GITHUB_OUTPUT\n",
" # Build and push the docker image\n",
" # Step 4 builds and pushes the docker image\n",
" - name: Build Docker image\n",
" uses: docker/build-push-action@v4\n",
" with:\n",
" # Provide the current directory as build context \n",
" context: .\n",
" # Specify where the Dockerfile is located in relation to the repo base path\n",
" file: Containerfile\n",
" # Provide the August-08-2024 directory as build context \n",
" context: August-08-2024/.\n",
" # Specify where the Containerfile is located in relation to the repo base path\n",
" file: August-08-2024/Containerfile\n",
" # Enable the push to docker hub\n",
" push: true\n",
" # Provide the tags to apply to the image, this example uses the latest image tag \n",
" tags: |\n",
" ncote/nbviz2cntnr:${{ steps.date.outputs.date }}\n",
" ncote/workshop-webapp:${{ steps.date.outputs.date }}\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Push the changes, the 2nd workflow action file, by using the branch on the left side of your browser window like we did before. Enter a descriptive message like added build and push workflow file and click the green Commit & Push button. Our Action is now live in our repository and we can use it to build and push images. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Test the new workflow"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Like before, the new workflow will only trigger when a change is made to the August-08-2024 directory on the main branch. Let's add another comment, or delete the one we added before, to the Containerfile and push the changes to trigger the new Action."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can watch the job progress from the Actions tab in our repository. Once it's successfully completed let's go to Docker Hub to see our image stored in our public repository with the date tag assigned by the workflow. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If we want to test the image locally we could run a command like\n",
"\n",
"`docker run -p 5006:5006 docker.io/ncote/workshop-webapp:2024-08-20`\n",
"\n",
"We would want to make sure the repository name, image name, and image tag matched up with what was assigned during our workflow. "
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand All @@ -287,23 +333,21 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"```{note}\n",
"The self-hosted runner example depends on resources that currently are only available on the UCAR internal network. A VPN connection or access onsite at a UCAR facility is required to successfully run the commands and Action example. \n",
"```"
"GitHub self-hosted runners allow you to run GitHub Actions workflows on your own infrastructure, providing more control over the environment and resources used for continuous integration and deployment."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"I have built a custom container image to do rootless podman builds and the configuration information can be found at this [link to my github-runner repository](https://github.com/NicholasCote/github-runner). The up to date container image is stored on the NSF NCAR CISL On-premise Cloud private container registry running on Harbor. In order to run the self-hosted runner on your local machine a GitHub API token is required when the container is launched. If you do not have an existing token use this [link to create an API token](https://github.com/settings/tokens). The example below uses a local environment variable, ${GITHUB_RUNNER_TOKEN}, instead of supplying the token as plain text in the command:"
"I have a custom container image that uses podman to do builds. I have included the container image file in this repository as a reference in the github-runner directory. The up to date container image is also stored in my Docker Hub registry. In order to start the self-hosted runner, a GitHub API token is required when the container is launched along with the repository location. If you do not have an existing token, see [Personal access tokens](https://github.com/settings/tokens). The example below uses a local environment variable, ${GITHUB_RUNNER_TOKEN}, instead of supplying the token as plain text in the command:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`podman run -e REPO={user/org}/{repo_name} -e TOKEN=${GITHUB_RUNNER_REGISTRATION} hub.k8s.ucar.edu/ncote/github-runner:2024-06-04.00.56`"
"`docker run -e REPO={user/org}/{repo_name} -e TOKEN=${GITHUB_RUNNER_TOKEN} docker.io/ncote/github-runner:2024-08-08`"
]
},
{
Expand All @@ -325,7 +369,11 @@
"# Define the trigger that starts the action\n",
"# For this workflow the trigger is on a push that changes anything in the main branch\n",
"on:\n",
" - push\n",
" push:\n",
" paths:\n",
" - August-08-2024/**\n",
" branches:\n",
" - main\n",
"\n",
"# Define the actions that are going to take place as part of this workflow \n",
"jobs:\n",
Expand All @@ -344,25 +392,16 @@
" run: echo \"date=$(date +'%Y-%m-%d.%H.%M')\" >> $GITHUB_OUTPUT\n",
" # Use podman to build the image and tag it with the current date\n",
" - name: Build and push image\n",
" run: podman build -t hub.k8s.ucar.edu/ncote/nbviz2cntnr:${{ steps.date.outputs.date }} .\n",
" run: podman build -t docker.io/ncote/workshop-webapp:${{ steps.date.outputs.date }} .\n",
" # Login to Harbor with a robot account for the ncote project in Harbor\n",
" - name: Login to Harbor\n",
" run: podman login hub.k8s.ucar.edu -u robot-ncote+ncote -p ${{ secrets.HARBOR_ROBOT_PW }}\n",
" - name: Login to Docker Hub\n",
" run: podman login docker.io -u ${{ secrets.DOCKERHUB_USERNAME }} -p ${{ secrets.DOCKERHUB_TOKEN }}\n",
" # Push the image to Harbor\n",
" - name: Push image to Harbor\n",
" run: podman push hub.k8s.ucar.edu/ncote/nbviz2cntnr:${{ steps.date.outputs.date }}\n",
" run: podman push docker.io/ncote/workshop-webapp:${{ steps.date.outputs.date }}\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```{note}\n",
"In this example, Workflow permissions need to be changed in the GitHub repository Settings under Actions -> General. Update the Workflow permissions to be Read and write to allow the Action to make changes to the repository.\n",
"``` "
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down

0 comments on commit 0e3d587

Please sign in to comment.