Skip to content

Latest commit

 

History

History
133 lines (96 loc) · 6.76 KB

3-forecast.md

File metadata and controls

133 lines (96 loc) · 6.76 KB

Forecast potential build runner usage

In this lab you will use the forecast command to forecast potential GitHub Actions usage by computing metrics from completed pipeline runs in your Azure DevOps project.

Prerequisites

  1. Followed the steps here to set up your GitHub Codespaces environment and bootstrap an Azure DevOps project.
  2. Completed the configure lab.

Perform a forecast

Answer the following questions before running the forecast command:

  1. What is the Azure DevOps organization name that you want to audit?

    • :organization. This should be the same organization used in the setup steps here

  2. What is the Azure DevOps project name that you want to audit?

    • :project. This should be the same project name used in the setup steps here

  3. Where do you want to store the results?

    • tmp/forecast

Steps

  1. Navigate to the codespace terminal.

  2. Run the following command from the root directory:

    gh actions-importer forecast azure-devops --output-dir tmp/forecast

    Note: The Azure DevOps organization and project name can be omitted from the forecast command because they were persisted in the .env.local file in the configure lab. You can optionally provide these arguments on the command line with the --azure-devops-organization and --azure-devops-project CLI options.

  3. The command will output a message that says "No jobs found" because no jobs have been executed in your bootstrapped project.

    $ gh actions-importer forecast azure-devops --output-dir tmp/forecast
[2022-08-20 22:08:20] Logs: 'tmp/forecast/log/actions-importer-20220916-021004.log'
[2022-08-20 22:08:20] Forecasting 'http://dev.azure.com/mona/actions-bootstrap/_build'
[2022-08-20 22:08:20] No jobs found

  4. If you inspect the help menu using the gh actions-importer azure-devops forecast --help command, you will see a --source-file-path option. You can use this option to perform a forecast using json files that are already present on the filesystem. These labs come bundled with sample json files located here.

    $ gh actions-importer forecast azure-devops -h
Options:
  -g, --azure-devops-organization <azure-devops-organization>  The Azure DevOps organization name.
  -p, --azure-devops-project <azure-devops-project>            The Azure DevOps project name.
  -u, --azure-devops-instance-url <azure-devops-instance-url>  The URL of the Azure DevOps instance.
  -t, --azure-devops-access-token <azure-devops-access-token>  Access token for the Azure DevOps instance.
  --source-file-path <source-file-path>                        The file path(s) to existing jobs data.
  -o, --output-dir <output-dir> (REQUIRED)                     The location for any output files.
  --start-date <start-date>                                    The start date of the forecast analysis in YYYY-MM-DD format. [default:
                                                               10/31/2022 8:35:17 AM]
  --time-slice <time-slice>                                    The time slice in seconds to use for computing concurrency metrics.
                                                               [default: 60]
  --credentials-file <credentials-file>                        The file containing the credentials to use.
  --no-telemetry                                               Boolean value to disallow telemetry.
  --no-ssl-verify                                              Disable ssl certificate verification.
  --no-http-cache                                              Disable caching of http responses.
  -?, -h, --help                                               Show help and usage information

  5. Run the following forecast command while specifying the path to the sample json files:

    gh actions-importer forecast azure-devops --output-dir tmp/forecast --source-file-path azure_devops/bootstrap/jobs.json

  6. The command will list all the files written to disk when the command succeeds.

    $ gh actions-importer forecast azure-devops --output-dir tmp/forecast --source-file-path azure_devops/bootstrap/jobs.json
[2022-08-20 22:08:20] Logs: 'tmp/forecast/log/actions-importer-20220916-021004.log'
[2022-08-20 22:08:20] Forecasting 'http://dev.azure.com/mona/actions-bootstrap/_build'
[2022-08-20 22:08:20] Outfile file(s):
[2022-08-20 22:08:20]   ./tmp/forecast/forecast_report.md

Review the forecast report

The forecast report, logs, and completed job data will be located within the tmp/forecast folder.

  1. Find the forecast_report.md file in the file explorer.
  2. Right-click the forecast_report.md file and select Open Preview.
  3. This file contains metrics used to forecast potential GitHub Actions usage.

Total

The Total section of the forecast report contains high level statistics related to all the jobs completed after the --start-date CLI option:

- Job count: **84**
- Pipeline count: **32**

- Execution time

  - Total: **82 minutes**
  - Median: **0 minutes**
  - P90: **2 minutes**
  - Min: **0 minutes**
  - Max: **4 minutes**

- Queue time

  - Median: **0 minutes**
  - P90: **1 minutes**
  - Min: **0 minutes**
  - Max: **5 minutes**

- Concurrent jobs

  - Median: **0**
  - P90: **0**
  - Min: **0**
  - Max: **5**

Here are some key terms of items defined in the forecast report:

  • The Job count is the total number of completed jobs.
  • The Pipeline count is the number of unique pipelines used.
  • Execution time describes the amount of time a runner spent on a job. This metric can be used to help plan for the cost of GitHub hosted runners.
    • This metric is correlated to how much you should expect to spend in GitHub Actions. This will vary depending on the hardware used for these minutes. You can use the Actions pricing calculator to estimate a dollar amount.
  • Queue time metrics describe the amount of time a job spent waiting for a runner to be available to execute it.
  • Concurrent jobs metrics describe the amount of jobs running at any given time. This metric can be used to define the number of runners a customer should configure.

Additionally, these metrics are defined for each queue of runners defined in Azure DevOps. This is especially useful if there are a mix of hosted/self-hosted runners or high/low spec machines to see metrics specific to different types of runners.

Next steps

Perform a dry-run migration of an Azure DevOps pipeline