Quorum Queue Replication parameter for user policies

[ahi234]

Community Support Policy

• I have read RabbitMQ's Community Support Policy
• I run RabbitMQ 4.x, the only series currently covered by community support
• I promise to provide all relevant information (versions, logs from all nodes, rabbitmq-diagnostics output, detailed reproduction steps)

RabbitMQ version used

4.0.4

Erlang version used

27.1.x

Operating system (distribution) used

Linux (Kubernetes)

How is RabbitMQ deployed?

Community Docker image

rabbitmq-diagnostics status output

See https://www.rabbitmq.com/docs/cli to learn how to use rabbitmq-diagnostics

# PASTE OUTPUT HERE, BETWEEN BACKTICKS

Logs from node 1 (with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

# PASTE LOG HERE, BETWEEN BACKTICKS

Logs from node 2 (if applicable, with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

# PASTE LOG HERE, BETWEEN BACKTICKS

Logs from node 3 (if applicable, with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

# PASTE LOG HERE, BETWEEN BACKTICKS

rabbitmq.conf

See https://www.rabbitmq.com/docs/configure#config-location to learn how to find rabbitmq.conf file location

# PASTE rabbitmq.conf HERE, BETWEEN BACKTICKS

Steps to deploy RabbitMQ cluster

.

Steps to reproduce the behavior in question

Creating user policy with the parameter initial-group-size

advanced.config

See https://www.rabbitmq.com/docs/configure#config-location to learn how to find advanced.config file location

# PASTE advanced.config HERE, BETWEEN BACKTICKS

Application code

# PASTE CODE HERE, BETWEEN BACKTICKS

Kubernetes deployment file

# Relevant parts of K8S deployment that demonstrate how RabbitMQ is deployed
# PASTE YAML HERE, BETWEEN BACKTICKS

What problem are you trying to solve?

I have a RabbitMQ cluster with 5 nodes, but my Quorum Queues are only being replicated across three nodes, as this is the default setting for quorum queues. I noticed that the parameter x-quorum-initial-group-size can be specified according to the quorum queue documentation, but I prefer to define this parameter (or a similar one) as a user policy instead of during the queue creation process.

I discovered that the initial-group-size parameter can be set as a user policy, but it only affects streams by applying to all queues. Is there a comparable parameter specifically for quorum queues?

[mkuratczyk]

initial-group-size cannot be set through a policy, because policies are applied asynchronously, after a queue is declared. Therefore, they can't set the initial state of the queue. You can however change the default for the cluster: https://www.rabbitmq.com/docs/quorum-queues#configuration

You can also be interested in the related topic of membership reconciliation: https://www.rabbitmq.com/docs/quorum-queues#replica-reconciliation

Streams are not affected by initial-group-size setting - they just start a member on all cluster nodes by default.

[michaelklishin]

Three replicas in a five node cluster is a perfectly reasonable default. Replicating to all nodes is not necessary in a majority of cases, and when it is, it is almost certainly not necessary for every quorum queue in the cluster. Not all data is so important that it must survive a failure of two nodes.

In addition to the above response:

• Quorum Queue Replica Management
• Stream Replica Management

[ahi234]

Thank you very much for the explanation ! @michaelklishin @mkuratczyk