Profile-Guided Optimization (PGO) benchmark report

[zamazan4ik]

Hi!

As I have done many times before, I decided to test the Profile-Guided Optimization (PGO) technique to optimize the library performance. For reference, results for other projects are available at https://github.com/zamazan4ik/awesome-pgo . Since PGO helped a lot for many other libraries, I decided to apply it on rust_radix_trie to see if the performance win (or lose) can be achieved. Here are my benchmark results. For benchmarks, I used these benchmarks since this benchmark covers multiple trie libraries: this one and trie-hard from Cloudflare.

This information can be interesting for anyone who wants to achieve more performance with the library in their use cases.

Test environment

• Fedora 40
• Linux kernel 6.10.7
• AMD Ryzen 9 5900x
• 48 Gib RAM
• SSD Samsung 980 Pro 2 Tib
• Compiler - Rustc 1.79.0
• trie-hard version: main branch on commit 3d8163ca2d9208c663d6dbac48105a96ac540306
• Disabled Turbo boost

Benchmark

For benchmark purposes, I use built-in into the trie-hard project benchmarks. For PGO optimization I use cargo-pgo tool. Release bench results I got with taskset -c 0 cargo bench command. The PGO training phase is done with taskset -c 0 cargo pgo bench, PGO optimization phase - with taskset -c 0 cargo pgo optimize bench.

taskset -c 0 is used for reducing the OS scheduler's influence on the results. All measurements are done on the same machine, with the same background "noise" (as much as I can guarantee).

Results

I got the following results:

• Release: https://gist.github.com/zamazan4ik/1cec56aa3f4223bb70b613f468d5cf9a
• PGO optimized compared to Release: https://gist.github.com/zamazan4ik/62021e0b7e13a97a318c0a4463ce70e5
• (just for reference) PGO instrumented compared to Release: https://gist.github.com/zamazan4ik/2e80724b2d728d20c269da78e37e3646

According to the results, PGO measurably improves the libraries' performance in most cases.

Further steps

I understand that the steps above can be time-consuming and hard to implement in practice. At the very least, the library's users can find this performance report and decide to enable PGO for their applications if they care about wildcard performance in their workloads. Maybe a small note somewhere in the documentation (the README file?) will be enough to raise awareness about this work.

Please don't treat the issue like an actual issue - it's just a benchmark report (since Discussions are disabled for the repo). I posted the trie-hard results here too since in the trie-hard repository Issues and Discussions are disabled.

Thank you.