Introduce client side caching benchmark capabilities. Enable rueidis/radix underlying vanilla client selection

[filipecosta90]

Client Side Caching and rueidis

The first CSC benchmark utility working as expected using rueidis vanilla client underneath.

• integration tests of the tool pass
• I’m not changing any behaviour when csc is disabled — meaning we don’t use ruedis by default and only use it when -csc and -rueidis flag are enabled.
• we can track CSC hits and evictions
• we've compared the ruedis vs radix benchmark client variations and they seem to obtain similar outcomes

Flags that were added concerning CSC:

  -csc
        Enable client side caching
  -csc-per-client-bytes int
        client side cache size that bind to each TCP connection to a single redis instance (default 134217728)
  -csc-ttl duration
        Client side cache ttl for cached entries (default 1m0s)
  -rueidis
        Use rueidis as the vanilla underlying client.

Sample using rueidis clients without csc

$ ./redis-benchmark-go -p 6379  -rueidis -r 10000  -cmd "SET __key__ __data__" -cmd-ratio 0.25 -cmd "GET __key__" -cmd-ratio 0.75 --json-out-file results.json
warning: GOCOVERDIR not set, no coverage data emitted
IPs [127.0.0.1]
Total clients: 50. Commands per client: 200000 Total commands: 10000000
Using random seed: 12345
                 Test time                    Total Commands              Total Errors                      Command Rate              CSC Hits/sec     CSC Invalidations/sec           p50 lat. (msec)
^C                      9s [22.6%]                   2261351                         0 [0.0%]                    125152                         0                         0                     0.200   
received Ctrl-c - shutting down

#################################################
Total Duration 9.579 Seconds
Total Errors 0
Throughput summary: 236067 requests per second
                    0 CSC Hits per second
                    0 CSC Evicts per second
Latency summary (msec):
          avg       p50       p95       p99
        0.225     0.202     0.460     0.661
2023/09/28 15:00:43 Saving JSON results file to results.json

Sample using rueidis clients with CSC enabled

$ ./redis-benchmark-go -p 6379  -rueidis -r 10000 -csc -cmd "SET __key__ __data__" -cmd-ratio 0.25 -cmd "GET __key__" -cmd-ratio 0.75 --json-out-file results.json
IPs [127.0.0.1]
Total clients: 50. Commands per client: 200000 Total commands: 10000000
Using random seed: 12345
                 Test time                    Total Commands              Total Errors                      Command Rate              CSC Hits/sec     CSC Invalidations/sec           p50 lat. (msec)
^C                     10s [16.3%]                   1630232                         0 [0.0%]                    146473                      6172                    104086                     0.333   
received Ctrl-c - shutting down

#################################################
Total Duration 10.715 Seconds
Total Errors 0
Throughput summary: 152139 requests per second
                    6325 CSC Hits per second
                    111911 CSC Evicts per second
Latency summary (msec):
          avg       p50       p95       p99
        0.337     0.333     0.595     0.694
2023/09/28 15:03:18 Saving JSON results file to results.json

confirmed we don’t burst the shard when csc is enabled and that the commandstats are as expected (1 command per client).

JSON output to store all CSC data

Apart from that, this PR also includes the --json-out-file flag that enables storing the results into a json.
The json file contains both the command rate, CSC hits rate, CSC evicts rate, and percentiles on the client side.

Here's a sample JSON file:

Confirmation of the same outcomes of using radix (default underlying client) and rueidis.

doing a quick comparison with radix (the previous client) we can see the metrics look similar. -- requires further metrics like resource consumption,etc... but it's a good start.
on the client side the metrics seem similar without CSC for radix and rueidis

RUN with rueidis

$ ./redis-benchmark-go -rueidis -n 10000000 -r 100  -p 6379 SET __key__ value
IPs [127.0.0.1]
Total clients: 50. Commands per client: 200000 Total commands: 10000000
Using random seed: 12345
                 Test time                    Total Commands              Total Errors                      Command Rate              CSC Hits/sec     CSC Invalidations/sec           p50 lat. (msec)
                       59s [100.0%]                  10000000                         0 [0.0%]                     92915                         0                         0                     0.229  
#################################################
Total Duration 59.001 Seconds
Total Errors 0
Throughput summary: 169489 requests per second
                    0 CSC Hits per second
                    0 CSC Evicts per second
Latency summary (msec):
          avg       p50       p95       p99
        0.296     0.229     0.693     0.971

RUN with radix v4

$ ./redis-benchmark-go -n 10000000 -r 100  -p 6379 SET __key__ value
IPs [127.0.0.1]
Total clients: 50. Commands per client: 200000 Total commands: 10000000
Using random seed: 12345
                 Test time                    Total Commands              Total Errors                      Command Rate              CSC Hits/sec     CSC Invalidations/sec           p50 lat. (msec)
                       62s [100.0%]                  10000000                         0 [0.0%]                     29064                         0                         0                     0.340  
#################################################
Total Duration 62.001 Seconds
Total Errors 0
Throughput summary: 161288 requests per second
                    0 CSC Hits per second
                    0 CSC Evicts per second
Latency summary (msec):
          avg       p50       p95       p99
        0.308     0.340     0.489     0.747

CPU and Memory consumption for both variations

CPU

Memory

[codecov]

Codecov Report

Attention: 32 lines in your changes are missing coverage. Please review.

Files	Coverage Δ
common.go	92.30% <ø> (ø)
standalone_conn.go	83.33% <100.00%> (ø)
redis-bechmark-go.go	69.30% <70.90%> (+1.74%)	⬆️

📢 Thoughts on this report? Let us know!.