From a7b3d70c3a2d4fb4b1d36546fe25903dfe2895a6 Mon Sep 17 00:00:00 2001
From: Erik Schierboom <erik_schierboom@hotmail.com>
Date: Fri, 16 Aug 2024 12:27:32 +0200
Subject: [PATCH] Add measuring section to performance

---
 building/tooling/best-practices.md | 23 +++++++++++++++++++++++
 building/tooling/docker.md         |  3 ++-
 2 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/building/tooling/best-practices.md b/building/tooling/best-practices.md
index bf148dfc..84feb825 100644
--- a/building/tooling/best-practices.md
+++ b/building/tooling/best-practices.md
@@ -9,6 +9,29 @@ The official [Dockerfile best practices](https://docs.docker.com/develop/develop
 You should primarily optimize for performance (especially for test runners).
 This will ensure your tooling runs as fast as possible and does not time-out.
 
+### Measure
+
+Measuring execution time often is a great way to get a feel for the performance of tooling.
+Make it a habit to measure execution time both after _and_ before a change.
+Even when you feel "certain" that a change will improve performance, you should still measure execution time.
+
+#### Scripts
+
+When possible, create scripts to automatically measure performance (also known as _benchmarking_).
+A very helpful command-line tool is [hyperfine](https://github.com/sharkdp/hyperfine), but feel free to use whatever makes the most sense for your tooling.
+
+Newer track tooling repos will have access to the following two scripts:
+
+1. `./bin/benchmark.sh`: benchmark the track tooling code ([source code](https://github.com/exercism/generic-test-runner/blob/a6886f4d84d2a2030f766a658c334bbfbe97b79c/bin/benchmark.sh))
+2. `./bin/benchmark-in-docker.sh`: benchmark the track tooling Docker image ([source code](https://github.com/exercism/generic-test-runner/blob/a6886f4d84d2a2030f766a658c334bbfbe97b79c/bin/benchmark-in-docker.sh))
+
+If you're working on a track tooling repo without these files, feel free to copy them into your repo.
+
+```exercism/caution
+Benchmarking scripts can help estimate the tooling's performance.
+Bear in mind though that the performance on Exercism's production servers is often lower.
+```
+
 ### Experiment with different Base images
 
 Try experimenting with different base images (e.g. Alpine instead of Ubuntu), to see if one (significantly) outperforms the other.
diff --git a/building/tooling/docker.md b/building/tooling/docker.md
index 5cadccc2..bc4060cd 100644
--- a/building/tooling/docker.md
+++ b/building/tooling/docker.md
@@ -15,6 +15,8 @@ Our [Best Practices page](/docs/building/tooling/best-practices) has lots of tip
 The test runner gets 100% CPU with 3GB of memory for a 20 second window per solution.
 After 20 seconds, the process is halted and reports a time-out with a 408 error code.
 
+We highly recommend following our [Performance Best Practices document](/docs/building/tooling/best-practices#h-performance) to reduce the chance of timeout occuring.
+
 ### Stdout/stderr
 
 A tooling run may produce up to a maximum of one-megabyte of stdout and stderr.
@@ -35,7 +37,6 @@ If the file is larger than this, the tooling run will be killed with a 460 error
 ## Configuration
 
 Each solution gets 100% machine resources for a twenty second window.
-
 After 20 seconds, the process is halted and reports as a time-out.
 
 Some tools require (slight) deviations from the default configuration.