Issue with asynchronous topology refresh

[jdeppe-pivotal]

Hi,

Developing a clustered redis-compatible backend, and testing with lettuce, I'm experiencing the following issue when a new cluster member is added:

1. A new member is added to the cluster and slots are automatically rebalanced across all members.
2. The RedisClusterClient sends a command to one of the initial members and receives a MOVED response.
3. An asynchronous topology refresh is triggered because of the MOVED response.
4. For the initial command, lettuce attempts to establish a connection to the host/port indicated in the MOVED response
5. The below exception is thrown because lettuce has not yet received a topology update with the correct node information:

io.lettuce.core.cluster.UnknownPartitionException: Connection to 127.0.0.1:62789 not allowed. This partition is not known in the cluster view.

	at io.lettuce.core.cluster.PooledClusterConnectionProvider.connectionAttemptRejected(PooledClusterConnectionProvider.java:621)
	at io.lettuce.core.cluster.PooledClusterConnectionProvider.beforeGetConnection(PooledClusterConnectionProvider.java:469)
	at io.lettuce.core.cluster.PooledClusterConnectionProvider.getConnectionAsync(PooledClusterConnectionProvider.java:446)
	at io.lettuce.core.cluster.ClusterDistributionChannelWriter.doWrite(ClusterDistributionChannelWriter.java:143)
	at io.lettuce.core.cluster.ClusterDistributionChannelWriter.write(ClusterDistributionChannelWriter.java:101)
	at io.lettuce.core.cluster.ClusterCommand.complete(ClusterCommand.java:58)
    
        <snip>

My ClusterTopologyRefreshOptions are simply:

    ClusterTopologyRefreshOptions refreshOptions =
        ClusterTopologyRefreshOptions.builder()
            .enableAllAdaptiveRefreshTriggers()
            .build();

Should lettuce be able to deal with this situation? Is there a configuration option I'm missing?

Thanks

[mp911de]

The client behaves as designed. MOVED triggers a subsequent topology refresh and UnknownPartitionException is a consequence for the current command. The topology refresh isn't performed within the receiving process as with sufficient traffic (consider multiple commands being completed with MOVED) topology refresh would result in a DDoS-like scenario.

You can disable cluster membership validation (ClusterClientOptions.validateClusterNodeMembership) which is more a security setting to avoid connection creation to arbitrary nodes.

So as a result of asynchronous topology refresh and by-default enabled cluster membership validation you see this failure.

[jdeppe-pivotal]

Thanks, that helps!