-
Notifications
You must be signed in to change notification settings - Fork 13
Continuous Performance Benchmarking
Siddhartha Kasivajhula edited this page Jul 5, 2022
·
6 revisions
Qi uses github-action-benchmark to track performance changes across commits. Here's how it's set up:
- There is a workflow file in
.github/workflows/benchmarks.yml
- Like most development workflows, the CI workflow is configured to run a Makefile target, in this case
make report-benchmarks
, to generate the performance data. This can be run locally as well, and would produce the data in JSON format as standard output (STDOUT
). - In the
benchmarks.yml
workflow, the output from the Make target is dumped into a file calledbenchmarks.txt
. -
github-action-benchmark
is then invoked, and it expects the input as a JSON-formatted text file -- we simply tell it, via workflow config, that it can use the filebenchmarks.txt
that was produced by the previous step. - This GitHub Action works by pushing the data and an HTML facade to the
gh-pages
branch of the Qi repo, at the specified folder (benchmarks
). - Note that the
gh-pages
branch is also being used to host backup docs via the GitHub Pages Deploy action, so we need to tell that workflow (docs.yml
) to exclude thebenchmarks
folder from its "clean" step, so that it doesn't delete the benchmarks whenever there is an update to the documentation. - The deployed HTML page, including the benchmark data in charts indexed by commit, are available at https://countvajhula.github.io/qi/benchmarks/
Home | Developer's Guide | Calendar | Events | Projects | Meeting Notes