From f7d210bf93366a7cf79c021d3eab4a10cc408a59 Mon Sep 17 00:00:00 2001
From: Giovanni Bussi <giovanni.bussi@gmail.com>
Date: Fri, 1 Mar 2024 15:05:19 +0100
Subject: [PATCH] documentation of benchmark tool

---
 src/cltools/Benchmark.cpp | 51 +++++++++++++++++++++++++++++++++++++--
 1 file changed, 49 insertions(+), 2 deletions(-)

diff --git a/src/cltools/Benchmark.cpp b/src/cltools/Benchmark.cpp
index d96e617b91..2c12fc73c9 100644
--- a/src/cltools/Benchmark.cpp
+++ b/src/cltools/Benchmark.cpp
@@ -58,14 +58,61 @@ benchmarks agaist previous plumed versions
 
 \par Examples
 
+First, you should create a sample `plumed.dat` file for testing. For instance:
+```
+WHOLEMOLECULES ENTITY0=1-10000
+p: POSITION ATOM=1
+RESTRAINT ARG=p.x KAPPA=1 AT=0
+
+```
+
+Then you can test the performance of this input with the following command:
+\verbatim
+plumed benchmark
+\endverbatim
+
+You can also test a different (older) version of PLUMED with the same input. To do so,
+you should run
 \verbatim
-plumed benchmark --plumed plumed.dat
+plumed-runtime benchmark --kernel /path/to/lib/libplumedKernel.so
 \endverbatim
 
+\warning It is necessary to use the `plumed-runtime` executable here to avoid conflicts between different
+plumed versions. You will find it in `src/lib` if you are using the non installed version of plumed,
+and in `$prefix/lib/plumed` if you installed plumed in $prefix,.
+
+The best way to compare two versions of plumed on the same input is to pass multiple colon-separated kernels:
+
 \verbatim
-plumed benchmark --plumed plumed.dat --kernel /path/to/libplumedKernel.so
+plumed-runtime benchmark --kernel /path/to/lib/libplumedKernel.so:/path2/to/lib/libplumedKernel.so:this
 \endverbatim
 
+Here `this` means the kernel of the version with which you are running the benchmark. This comparison runs the three
+instances simultaneously (alternating them) so that systematic differences in the load of your machine will affect them
+to the same extent.
+
+In case the different versions require modified plumed.dat files, or if you simply want to compare 
+two different plumed input files that compute the same thing, you can also use multiple plumed input files:
+
+\verbatim
+plumed-runtime benchmark --kernel /path/to/lib/libplumedKernel.so:this --plumed plumed1.dat:plumed2.dat
+\endverbatim
+
+\par Output
+
+In the output you will see the usual reports about timing produced by the internal
+timers of the tested plumed instances.
+In addition, this tool will monitor the timing externally, with some slightly different criterion:
+- First, the initialization (construction of the input) will be shown with a separate timer
+- Second, the timer corresponding to the calculation will be split in two parts, reporting
+  execution of the first half of the calculation and the second half.
+- Finally, you might notice some discrepancy because some of the actions that are usually
+  not expensive are not included in the internal timers. The external timer will
+  thus provide a better estimate of the total elapsed time, including everything.
+
+The internal timers are still useful to monitor what happens at the different stages
+and, with \ref DEBUG `DETAILED_TIMERS`, what happens in each action.
+
 */
 //+ENDPLUMEDOC