cml_automation

The intention of this repository is to support the testing of nightly built software. This pipeline will do the following:

• Query a repository for the most recent software build, defined by the user.
• Download, convert to CML supported image, and upload the image to CML.
• Generates API token for CML.
• Creates or updates a node defintion defined by the user.
• Adds a new image definition for the new software build.
• Finds user defined topology, identifies correct nodes, and assigns the new image definition.
• Restarts the nodes.

Gotchas

• CML seems to only accept image definitions with images that are over 1 GB in size. I first attempted this using Tiny Core Linux and ran into an error when attempting to update the image definition. The qcow file was 32 MB, but CML read it as 32 GB.

Running Workflow in Jenkins

Running Workflow in Gitlab

Setting your Environment Variables

The scripts in this repository can be executed manually or as an automated pipeline. Included in this repository is a pre-built gitlab-ci.yml file that will work by setting your appropirate environment variables.

CML Environment Variables

• CML_HOST: This is the IP address of your CML server. This is needed to SCP the updated qcow file to your CML server to use with the new node definition.
• CML_URL: This will be your https://cml.server.com URL used to access the API once the image is uploaded.
• CML_LAB: This will be the name of the lab you are targeting for the image definition update.
• CML_PASSWORD: Used to generate your API token.
• CML_USERNAME: Used to generate your API token.
• NODE_DEFINITION: IMPORTANT - This will be used to target the correct nodes in your topology that will be updated with the newest image defintion.

GitLab Environment Variables

The current working scripts use the GitLab API to dynamically store certain variables for access in different stages of the pipeline. There are easier ways to do this, but because this was meant to be used for other runners I set the scripts to use the API. If you need to re-factor these to use something like Jenkins or GitHub, you can replace the API call with the one for your system.

• GITLAB_URL: Your https://my.gitlab.server web address.
• GITLAB_TOKEN: Your API token that will be used for access.
• CI_PROJECT_ID: This is used to idenfity the right project to set additional environment variables as the pipeline runs.

Image Environment Variables

The current working script allows the user to set a URL where image updates are posted. The example in this pipeline is using basic ubuntu images on their publically available website. This will change based on your use case.

• IMAGE_URL: This is the URL you are scraping with downloadable images.
• IMAGE_KEYWORD: This is not mandatory, but in scenarios where different platform images are posted, this helps you target the correct image that can be supported in CML. For example, the ubuntu repository posts the same images for amd64, arm64, i386, etc. This keyword lets you target a specefic platform.

Dynamic Environment Variables

The following environment variables will be created as the pipeline executes. This allows them to be passed on to other stages of the pipeline. These DO NOT need to be set by the user.

DO NOT SET THESE

• API_TOKEN: This will be auto-generated by a script once the CML token is recieved. This token is needed for API calls to CML.
• IMAGE_NAME: This will be auto-generated based on the software name and version of your new image. When the initial script is executed, it sorts the image repository by date and stores the name of the new image and version as a variable. This is important as it will be used to dynamically generate your image definition.