Unidirectional flow of messages on a topic in two Cluster

[Hitesh-Agrawal]

Observed behavior

I have two NATS Clusters each with 4 Node.
ClusterA
node1a.cluster.svc.local
node2a.cluster.svc.local
node3a.cluster.svc.local
node4a.cluster.svc.local
Load Balancer :- clustera.lb.internal.com

ClusterB
node1b.cluster.svc.local
node2b.cluster.svc.local
node3b.cluster.svc.local
node4b.cluster.svc.local
Load Balancer :- clusterb.lb.internal.com

I want messages generated in Cluster A on any topic to flow to ClusterB ( if there is interest for the topic), but not vice-versa.

I have configured 1 nodes in ClusterA (node1a.cluster.svc.local) with
leafnodes {
listen: 0.0.0.0:7222
}

=================
In ClusterB only 1 node (node1b.cluster.svc.local) is configured with leaf remote as follow
listen: 0.0.0.0:4222
http: 0.0.0.0:8222

cluster {
listen: 0.0.0.0:5222
name: ClusterB

routes = [
nats-route://node2b.cluster.svc.local:5222
nats-route://node3b.cluster.svc.local:5222
nats-route://node4b.cluster.svc.local:5222
]

no_advertise: true
connect_retries: 0
cluster_advertise: node1b.cluster.svc.local:5222
}
leafnodes {
remotes = [
{
urls: [
"nats-leaf://node1a.cluster.svc.local:7422",
]
deny_exports = [">"]
},
]
}

logtime: true
max_connections: 65535
max_control_line: 512
max_payload: 1048576
max_pending: 67108864
write_deadline: "2s"

=====================================================

Expected behavior

When a message is pub on "testtopic" in cluster "A",
nats pub testtopic "Test from cluster A" --server nats://clustera.lb.internal.com
07:27:03 Published 23 bytes to "testtopic"

It should be received on all the 4 nodes in cluster "B"

but its received only on node1b.cluster.svc.local
nats sub testtopic --server nats://node1b.cluster.svc.local
07:26:38 Subscribing on testtopic
[#1] Received on "testtopic"
Test from cluster A

When I do
nats sub testtopic --server nats://node2b.cluster.svc.local

I dont receive anything here

======================================
Expected Behaviour
Messages published by any node in "Cluster A" , should be receivable on all the nodes in "Cluster B"

The message received on node1b.cluster.svc.local from remote leaf "node1a.cluster.svc.local" should be forwarded to all nodes (node{2-4}b) in "Cluster B".

Question:
Do I need to configure all the nodes in cluster B as a leafnode to "nats-leaf://node1a.cluster.svc.local:7422" ?
I think node1b.cluster.svc.local should have acted as a forwarded for any interest generated in local clusterB for topic "testtopic"
I cannot remove
deny_exports = [">"] as I do not want any messages generated in ClusterB to flow to ClusterA.

Server and client version

[60] 2024/10/24 06:00:12.354477 [INF] Starting nats-server
[60] 2024/10/24 06:00:12.354522 [INF] Version: 2.3.4

Host environment

EKS Kubernetes v1.29

Steps to reproduce

Configure your clusters as above and test the same.

[Hitesh-Agrawal]

Just to add,
I want node1b.cluster.svc.local to act as an 1 way bridge between "Cluster A" and "Cluster B". Only node1b.cluster.svc.local can reach node1a.cluster.svc.local:7222 other nodes in "Cluster B" cannot reach Cluster A.

Message flow should be "Cluster A" -> node1a.cluster.svc.local:7222 -> node1b.cluster.svc.local -> All nodes in "Cluster B"
If there is any other solution in place , I would be happy to explore that.

[Hitesh-Agrawal]

Any update on this....

[derekcollison]

I would imagine you do not have a properly formed system in terms of the servers being clustered together.

[Hitesh-Agrawal]

@derekcollison I have provided all the relevant details of the NATS configuration at my end. Cluster A being "production" cluster and Cluster B being "development" Cluster. The clients connected to development NATS clusterB are interested in whatever NATS traffic is generated in "production" clusterA, but anything which is published on the same topic in ClusterB ( aka Development cluster) should never reach ClusterA (aka production cluster).

Any NATS topology which helps me to achieve the above with least amount of configuration is ok for me.

[derekcollison]

You can leaf node the two clusters today and put in a deny permission for the leafnode to prevent certain traffic or use the JWT permissions for the leafnode auth to control data flow.

[Hitesh-Agrawal]

You can leaf node the two clusters today and put in a deny permission for the leafnode to prevent certain traffic or use the JWT permissions for the leafnode auth to control data flow.

@derekcollison You mean all the nodes in ClusterB as a leaf node for ClusterA. Am I correct ?.
I cannot have
ClusterA -> node1a.cluster.svc.local ( leaf server & Cluster A node) -> node1b.cluster.svc.local ( leaf client & Cluster B node )-> node2b.cluster.svc.local ( ClusterB node)

I need to have
ClusterA -> node1a.cluster.svc.local ( leaf server) -> node1b.cluster.svc.local ( leaf client )
ClusterA -> node1a.cluster.svc.local ( leaf server) -> node2b.cluster.svc.local ( leaf client )