Lettuce Redis Cluster client cluster topology refresh design

[ExploreNcrack]

I am curious why Lettuce Redis Cluster client opens up new connection for every cluster topology refresh? Opening up new connection can be very expensive and introduce additional burden to the server especially when TLS is enabled and there are lots of clients connected to the server. As a result, it can lead to connection churn issues. I am thinking can we maintain a set of connections just for topology refresh instead of opening up new connections every time?

I do know that there is the option of turning dynamicRefreshSources to false, but that will lose the advantage of gathering consensus on cluster nodes (info).

[mp911de]

We have a chicken and egg problem to some extent. We need to open up connections for the initial discovery to obtain the topology. Then, we perform the actual connection after the topology discovery connections have been closed.

RedisClusterClient can open up multiple cluster sessions (StatefulRedisClusterConnection) comprised of multiple TCP connections to the individual nodes. We do not maintain a shared connection pool. We also do not know in what state the connections are and whether they are in use. Just think of promises for asynchronous/reactive usage patterns where a call has been prepared but not yet sent to the Redis server, and we need to consider such states for connection allocation, too.

I think it could be possible to have a connection pool for cluster node connections that could be reused for topology refreshes. Such a pool would require a large-scale rewrite and introduce a lot of additional complexity.

I'm not keen on investing so much time given that I'm the sole maintainer and I do not draw anything beneficial from such an exercise.