Benchmarking

Compilation

In order to compile the benchmarks JAR run: ./gradlew benchmarks

Running Codec Microbenchmarks

fix-gateway-system-tests/build/libs/fix-gateway-system-tests-0.1-SNAPSHOT-benchmarks.jar is just a standard JMH benchmark jar, so for example to run the stub decoder benchmark run:

java -jar ./fix-gateway-system-tests/build/libs/fix-gateway-system-tests-0.1-SNAPSHOT-benchmarks.jar StubDecoderBenchmark

Benchmark Images on AWS

In all the code snippets below we've set up environment variables to point to your ec2 host and key, ie:

export PEM=/path/to/pem-file.pem
export EC2=ec2-<XX-XX-XX-XX>.us-west-2.compute.amazonaws.com

From Scratch

If you want to build an EC2 image from scratch,

1. Pick an Ubuntu HVM image
2. At least a c3.2xlarge
3. Install requirements/utilities

sudo apt-get update
sudo apt-get install htop screen mosh sysbench default-jdk

4. Upload the current benchmark directory

scp -i $PEM -r bench/ ubuntu@$EC2:

You can optionally upload your .screenrc

scp -i $PEM ~/.screenrc ubuntu@$EC2:

From an AMI

If you've got access to the AMI with a pre-built benchmarking instance, you can just boot the "FIX Benchmark 2" AMI. Remember to boot it on at least a C3.X2Large compute instance.

Running benchmarks

1. Re-compile and upload the benchmarks jar:

./bench/upload.sh

2. Run the benchmark on the EC2 image using the ./server and ./client scripts in screen. You can mosh into the server with ./bench/mosh.sh.

3. Pull down the flight recording to analyse locally.

./bench/get_jfr.sh

Low Latency Tuning

Lowest Latency performance can be achieved by getting the benchmark to run with a noop idle strategy (NB: this is a benchmark specific configuration).

-Dfix.benchmark.engine_idle=noop

Both throughput and sometimes latency can be improved by increasing TCP buffer sizes, these can be configured through the following properties.

-Dfix.core.receiver_buffer_size=1048576 \
-Dfix.core.sender_socket_buffer_size=16777216 \
-Dfix.core.receiver_socket_buffer_size=16777216