FRR EVPN Multihoming MAC-IP sync issue : Neighbor entries never become reachable.

[mittalsomya18]

I am currently testing EVPN Multihoming with FRR version 8.5.1. My setup is based on leaf-spine topology with leaf switches acting as VTEPS and spines acting as route reflectors. Two DELL servers are multihomed to two leaf switches.

I have created a VNI (10112) and corresponding SVI interface on both the Leaf switches. The SVI interfaces have same IP on both leaf switches. I have configured two Ethernet segments , one for each server on both the leaf switches.

When I ping from Leaf1 to a server1 IP , ARP request reaches from Leaf 1 to Server1 but ARP reply is sent by server1 to Leaf2 (as it’s a bond, reply can go out of any interface). Leaf2 learns the ARP entry and programs it as reachable. FRR syncs the MAC-IP with Leaf1 which programs it as peer active, local-inactive neighbor entry in FRR and stale in linux. Neighbor entry on Leaf1 is transitioning from stale->failed.
When it goes into failed state, neighbor entry is deleted from kernel before FRR reprograms it. Thus, packet drops happen in between. Also since neighor state is constantly changing , with large number of neighbors, this situation also keeps our systems busy by deleting and reprograming neighbor entries.