Make benchmark of parsing

[urv]

Make a benchmark of our FIX parser (maybe using Ciretorion) and compare to other FIX engine implementations.

[urv]

A first benchmark

warming up
estimating clock resolution...
mean is 10.21753 us (80001 iterations)
found 123868 outliers among 79999 samples (154.8%)
62343 (77.9%) low severe
61525 (76.9%) high severe
estimating cost of a clock call...
mean is 734.3272 ns (65 iterations)
found 3 outliers among 65 samples (4.6%)
1 (1.5%) high mild
2 (3.1%) high severe

benchmarking Parsing FIX
mean: 86.27621 us, lb 86.05795 us, ub 86.69280 us, ci 0.950
std dev: 1.495563 us, lb 947.5986 ns, ub 2.687498 us, ci 0.950

[urv]

With the -O3 we have

warming up
estimating clock resolution...
mean is 10.14286 us (80001 iterations)
found 2595 outliers among 79999 samples (3.2%)
1066 (1.3%) high mild
1529 (1.9%) high severe
estimating cost of a clock call...
mean is 725.4460 ns (66 iterations)
found 3 outliers among 66 samples (4.5%)
3 (4.5%) high severe

benchmarking Parsing FIX
mean: 20.83710 us, lb 20.79674 us, ub 20.93774 us, ci 0.950
std dev: 304.9626 ns, lb 138.0501 ns, ub 621.8556 ns, ci 0.950

[urv]

I now enforce, using DeepSeq, the evaluation of FIXMessage values.

warming up
estimating clock resolution...
mean is 10.31709 us (80001 iterations)
found 2309 outliers among 79999 samples (2.9%)
1742 (2.2%) high severe
estimating cost of a clock call...
mean is 732.6677 ns (66 iterations)
found 3 outliers among 66 samples (4.5%)
1 (1.5%) high mild
2 (3.0%) high severe

benchmarking Parsing FIX
mean: 26.77558 us, lb 26.69720 us, ub 26.98418 us, ci 0.950
std dev: 608.2751 ns, lb 288.5777 ns, ub 1.281486 us, ci 0.950

[urv]

warming up
estimating clock resolution...
mean is 10.45167 us (80001 iterations)
found 1803 outliers among 79999 samples (2.3%)
1358 (1.7%) high severe
estimating cost of a clock call...
mean is 742.4760 ns (65 iterations)
found 4 outliers among 65 samples (6.2%)
1 (1.5%) high mild
3 (4.6%) high severe

benchmarking FIX42/Heartbeat
mean: 25.62310 us, lb 25.57543 us, ub 25.70199 us, ci 0.950
std dev: 307.5366 ns, lb 210.3357 ns, ub 468.9663 ns, ci 0.950

benchmarking FIX42/TestRequest
mean: 25.75921 us, lb 25.71488 us, ub 25.84296 us, ci 0.950
std dev: 302.6919 ns, lb 191.8108 ns, ub 520.2413 ns, ci 0.950

benchmarking FIX42/ResendRequest
mean: 27.11603 us, lb 27.06496 us, ub 27.20892 us, ci 0.950
std dev: 343.8596 ns, lb 213.7963 ns, ub 539.9965 ns, ci 0.950

benchmarking FIX42/Reject
mean: 30.61395 us, lb 30.56567 us, ub 30.70327 us, ci 0.950
std dev: 326.4716 ns, lb 203.7285 ns, ub 538.6497 ns, ci 0.950

benchmarking FIX42/SequenceReset
mean: 26.81341 us, lb 26.76572 us, ub 26.90176 us, ci 0.950
std dev: 322.9629 ns, lb 191.5732 ns, ub 514.2450 ns, ci 0.950

benchmarking FIX42/Logout
mean: 27.58251 us, lb 27.47954 us, ub 27.70689 us, ci 0.950
std dev: 576.3262 ns, lb 472.6970 ns, ub 772.9875 ns, ci 0.950

benchmarking FIX42/IndicationofInterest
mean: 42.62343 us, lb 42.53541 us, ub 42.77250 us, ci 0.950
std dev: 575.8318 ns, lb 362.4185 ns, ub 904.8730 ns, ci 0.950

benchmarking FIX42/Advertisement
mean: 37.97709 us, lb 37.91716 us, ub 38.08886 us, ci 0.950
std dev: 403.4331 ns, lb 248.6157 ns, ub 691.5797 ns, ci 0.950

benchmarking FIX42/ExecutionReport
mean: 52.28659 us, lb 52.18149 us, ub 52.52528 us, ci 0.950
std dev: 768.8039 ns, lb 284.5223 ns, ub 1.336802 us, ci 0.950

benchmarking FIX42/OrderCancelReject
mean: 37.71394 us, lb 37.60606 us, ub 37.93442 us, ci 0.950
std dev: 756.8895 ns, lb 424.8940 ns, ub 1.280046 us, ci 0.950

benchmarking FIX42/Logon
mean: 29.10476 us, lb 29.02372 us, ub 29.23585 us, ci 0.950
std dev: 521.5266 ns, lb 354.3695 ns, ub 731.5535 ns, ci 0.950

benchmarking FIX42/News
mean: 25.07434 us, lb 25.03142 us, ub 25.15785 us, ci 0.950
std dev: 296.9808 ns, lb 150.5596 ns, ub 492.2937 ns, ci 0.950

benchmarking FIX42/Email
mean: 25.37018 us, lb 25.32134 us, ub 25.47484 us, ci 0.950
std dev: 348.3424 ns, lb 156.8204 ns, ub 595.5600 ns, ci 0.950

benchmarking FIX42/NewOrderSingle
mean: 43.09993 us, lb 43.02372 us, ub 43.26264 us, ci 0.950
std dev: 544.3960 ns, lb 230.4815 ns, ub 932.7006 ns, ci 0.950

benchmarking FIX42/NewOrderList
mean: 27.57427 us, lb 27.52941 us, ub 27.66448 us, ci 0.950
std dev: 312.5692 ns, lb 153.0878 ns, ub 522.4613 ns, ci 0.950

benchmarking FIX42/OrderCancelRequest
mean: 55.62130 us, lb 55.52528 us, ub 55.80689 us, ci 0.950
std dev: 658.3418 ns, lb 410.1085 ns, ub 1.209660 us, ci 0.950

benchmarking FIX42/OrderCancelReplaceRequest
mean: 43.68173 us, lb 43.61359 us, ub 43.81103 us, ci 0.950
std dev: 465.8716 ns, lb 286.1919 ns, ub 812.0760 ns, ci 0.950

benchmarking FIX42/OrderStatusRequest
mean: 46.67166 us, lb 46.59597 us, ub 46.81840 us, ci 0.950
std dev: 522.1631 ns, lb 271.2248 ns, ub 864.1614 ns, ci 0.950

benchmarking FIX42/Allocation
mean: 36.45555 us, lb 36.39401 us, ub 36.58093 us, ci 0.950
std dev: 434.5989 ns, lb 254.9190 ns, ub 817.3188 ns, ci 0.950

benchmarking FIX42/ListCancelRequest
mean: 29.90737 us, lb 29.84977 us, ub 30.03667 us, ci 0.950
std dev: 417.5856 ns, lb 163.5569 ns, ub 720.4645 ns, ci 0.950

benchmarking FIX42/ListExecute
mean: 31.44535 us, lb 31.38905 us, ub 31.57044 us, ci 0.950
std dev: 407.5955 ns, lb 176.1113 ns, ub 705.7748 ns, ci 0.950

benchmarking FIX42/ListStatusRequest
mean: 28.12923 us, lb 28.08388 us, ub 28.22507 us, ci 0.950
std dev: 322.7024 ns, lb 152.4204 ns, ub 543.8100 ns, ci 0.950

benchmarking FIX42/ListStatus
mean: 27.82942 us, lb 27.78284 us, ub 27.91186 us, ci 0.950
std dev: 309.3322 ns, lb 203.2743 ns, ub 497.4211 ns, ci 0.950

benchmarking FIX42/AllocationACK
mean: 27.57724 us, lb 27.53727 us, ub 27.63981 us, ci 0.950
std dev: 250.7287 ns, lb 179.3696 ns, ub 367.4196 ns, ci 0.950

benchmarking FIX42/DontKnowTrade
mean: 51.68879 us, lb 51.58808 us, ub 51.90712 us, ci 0.950
std dev: 724.8570 ns, lb 315.7626 ns, ub 1.248278 us, ci 0.950

benchmarking FIX42/QuoteRequest
mean: 25.34721 us, lb 25.31232 us, ub 25.42471 us, ci 0.950
std dev: 254.3756 ns, lb 128.9554 ns, ub 482.6656 ns, ci 0.950

benchmarking FIX42/Quote
mean: 31.87809 us, lb 31.82146 us, ub 31.99959 us, ci 0.950
std dev: 403.7725 ns, lb 175.1282 ns, ub 688.3723 ns, ci 0.950

benchmarking FIX42/SettlementInstructions
mean: 28.84651 us, lb 28.79562 us, ub 28.93739 us, ci 0.950
std dev: 339.8886 ns, lb 218.8846 ns, ub 525.5738 ns, ci 0.950

benchmarking FIX42/MarketDataRequest
mean: 24.50915 us, lb 24.46695 us, ub 24.59412 us, ci 0.950
std dev: 295.5547 ns, lb 144.8799 ns, ub 495.5370 ns, ci 0.950

benchmarking FIX42/MarketDataSnapshotFullRefresh
mean: 39.39159 us, lb 39.32408 us, ub 39.54911 us, ci 0.950
std dev: 497.1919 ns, lb 220.8823 ns, ub 945.4233 ns, ci 0.950

benchmarking FIX42/MarketDataIncrementalRefresh
mean: 25.49994 us, lb 25.45556 us, ub 25.59554 us, ci 0.950
std dev: 318.4819 ns, lb 142.3403 ns, ub 548.8173 ns, ci 0.950

benchmarking FIX42/MarketDataRequestReject
mean: 29.14978 us, lb 29.09495 us, ub 29.27193 us, ci 0.950
std dev: 397.5558 ns, lb 162.3003 ns, ub 723.8110 ns, ci 0.950

benchmarking FIX42/QuoteCancel
mean: 25.98389 us, lb 25.92940 us, ub 26.10163 us, ci 0.950
std dev: 392.4574 ns, lb 166.4445 ns, ub 673.4282 ns, ci 0.950

benchmarking FIX42/QuoteStatusRequest
mean: 44.59158 us, lb 44.51930 us, ub 44.73464 us, ci 0.950
std dev: 499.7010 ns, lb 256.8044 ns, ub 827.4158 ns, ci 0.950

benchmarking FIX42/QuoteAcknowledgement
mean: 27.37645 us, lb 27.28962 us, ub 27.50213 us, ci 0.950
std dev: 527.0144 ns, lb 409.2130 ns, ub 720.1722 ns, ci 0.950

benchmarking FIX42/SecurityDefinitionRequest
mean: 30.45666 us, lb 30.40263 us, ub 30.57096 us, ci 0.950
std dev: 386.8639 ns, lb 170.4657 ns, ub 655.2819 ns, ci 0.950

benchmarking FIX42/SecurityDefinition
mean: 30.21937 us, lb 30.16985 us, ub 30.32528 us, ci 0.950
std dev: 354.3173 ns, lb 156.6033 ns, ub 603.1271 ns, ci 0.950

benchmarking FIX42/SecurityStatusRequest
mean: 45.34873 us, lb 45.25502 us, ub 45.51058 us, ci 0.950
std dev: 614.3409 ns, lb 365.2996 ns, ub 980.2201 ns, ci 0.950

benchmarking FIX42/SecurityStatus
mean: 57.18824 us, lb 57.09567 us, ub 57.36581 us, ci 0.950
std dev: 633.9295 ns, lb 390.6769 ns, ub 1.101295 us, ci 0.950

benchmarking FIX42/TradingSessionStatusRequest
mean: 29.26424 us, lb 29.21614 us, ub 29.36523 us, ci 0.950
std dev: 342.4366 ns, lb 156.3864 ns, ub 577.6092 ns, ci 0.950

benchmarking FIX42/TradingSessionStatus
mean: 43.69483 us, lb 43.61978 us, ub 43.84701 us, ci 0.950
std dev: 525.1173 ns, lb 301.1078 ns, ub 917.9209 ns, ci 0.950

benchmarking FIX42/MassQuote
mean: 30.03651 us, lb 29.96399 us, ub 30.18009 us, ci 0.950
std dev: 505.2949 ns, lb 300.1334 ns, ub 853.7636 ns, ci 0.950

benchmarking FIX42/BusinessMessageReject
mean: 30.65060 us, lb 30.59722 us, ub 30.76924 us, ci 0.950
std dev: 387.7838 ns, lb 213.4831 ns, ub 761.3290 ns, ci 0.950

benchmarking FIX42/BidRequest
mean: 26.25447 us, lb 26.20992 us, ub 26.34558 us, ci 0.950
std dev: 312.6760 ns, lb 146.1855 ns, ub 526.6778 ns, ci 0.950

benchmarking FIX42/BidResponse
mean: 25.76480 us, lb 25.71625 us, ub 25.85793 us, ci 0.950
std dev: 331.9441 ns, lb 181.2820 ns, ub 552.2631 ns, ci 0.950

benchmarking FIX42/ListStrikePrice
mean: 26.27455 us, lb 26.22720 us, ub 26.37057 us, ci 0.950
std dev: 332.3810 ns, lb 160.9854 ns, ub 559.1898 ns, ci 0.950

[urv]

Compare the results with QuickFIX