Benchmarking

Compilation

In order to compile the benchmarks JAR run:

./gradlew benchmarks

Running Codec Microbenchmarks

artio-system-tests/build/libs/artio-tests-0.1-SNAPSHOT-benchmarks.jar is just a standard JMH benchmark jar, so for example to run the stub decoder benchmark run:

java -jar ./artio-system-tests/build/libs/artio-tests-0.1-SNAPSHOT-benchmarks.jar StubDecoderBenchmark

Benchmark Images on AWS

In all the code snippets below we've set up environment variables to point to your ec2 host and key, ie:

export PEM=/path/to/pem-file.pem
export EC2=ec2-<XX-XX-XX-XX>.us-west-2.compute.amazonaws.com

From Scratch

If you want to build an EC2 image from scratch,

1. Pick an Ubuntu HVM image
2. At least a c3.2xlarge
3. Install requirements/utilities

If you're happy to benchmark without JMC:

sudo apt-get update
sudo apt-get install htop screen mosh sysbench default-jdk

If you want to use JMC:

sudo apt-get install htop screen mosh sysbench
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer

4. Upload the current benchmark directory

scp -i $PEM -r bench/ ubuntu@$EC2:fix-gateway

You can optionally upload your .screenrc

scp -i $PEM ~/.screenrc ubuntu@$EC2:

From an AMI

Having created the benchmark server, you might want to save it as an AMI in order to make re-using it easier.

Running benchmarks

1. Re-compile and upload the benchmarks jar:

./bench/upload.sh

2. Run the benchmark on the EC2 image using the ./server and ./client scripts in screen. You can mosh into the server with ./bench/mosh.sh.

3. Pull down the flight recording to analyse locally.

./bench/get_jfr.sh

Low Latency Tuning

Lowest Latency performance can be achieved by getting the benchmark to run with a noop idle strategy (NB: this is a benchmark specific configuration).

-Dfix.benchmark.engine_idle=noop

Both throughput and sometimes latency can be improved by increasing TCP buffer sizes, these can be configured through the following properties.

-Dfix.core.receiver_buffer_size=1048576 \
-Dfix.core.sender_socket_buffer_size=16777216 \
-Dfix.core.receiver_socket_buffer_size=16777216