EVPN type-2 MACIP not imported into other VRFs by originating node, only at receivers

[toreanderson]

Description

I am not 100% certain whether or not this is a bug, or expected behaviour. In any case, it leads to suboptimal routing, so if it is not a bug, it could be considered a feature request at least.

When importing routes from an EVPN VRF into another VRF (such as the default VRF), type-2 MACIP routes (containing an IP address) do not get imported into the target VRF on the node that originated the type-2 MACIP route (i.e., the one that has the MAC/IP in the type-2 route locally attached). It is however imported on all other nodes.

This leads to suboptimal routing as traffic the traffic from the external network is always directed to a node where the MAC/IP destination is not present. From there it will be encapsulated in VXLAN and sent to the target node.

It would be better if also the node where the MAC/IP is present could also import the route into the target VRF and re-advertise it there, as that path would be preferred by the external network (due to a shorter AS path)

Version

FRRouting 10.1-dev (frrtest) on Linux(6.1.0-21-amd64).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--sbindir=/usr/lib/frr' '--with-vtysh-pager=/usr/bin/pager' '--libdir=/usr/lib/x86_64-linux-gnu/frr' '--with-moduledir=/usr/lib/x86_64-linux-gnu/frr/modules' '--disable-dependency-tracking' '--enable-rpki' '--enable-scripting' '--enable-pim6d' '--with-libpam' '--enable-doc' '--enable-doc-html' '--enable-snmp' '--enable-fpm' '--disable-protobuf' '--disable-zeromq' '--enable-ospfapi' '--enable-bgp-vnc' '--enable-multipath=256' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-configfile-mask=0640' '--enable-logfile-mask=0640' '--enable-sharpd' 'build_alias=x86_64-linux-gnu' 'PYTHON=python3'

(from frr_10.1-dev-master-ga24c805-20240604.084942-1~deb12u1_amd64.deb)

How to reproduce

To illustrate, I've set up a lab with three Debian 12 nodes connected in a triangle (full mesh between interfaces ens7 and ens8 on each node).

One of the nodes, frrtest, does not participate in EVPN - it represents the external network.

The other two nodes, frrtest2 and frrtest3, represents two EVPN routers with ASN 2 and 3, with a single L3VNI (10) bound to VRF 10, and an IRB on L2VNI 100. Static MAC/ARP entries are used to generate type-2 routes for (192.168.0.2 and 192.168.0.3) are used to represent downstream hosts on the L2VNI. The default VRF imports routes from VRF 10.

These are the scripts I use to configure the three nodes from scratch:

frrtest1

vtysh <<EOF
configure

interface lo
 ip address 10.0.0.1/32

interface ens7
 no shutdown

interface ens8
 no shutdown

router bgp 1
 no bgp ebgp-requires-policy
 neighbor ens7 interface remote-as external
 neighbor ens8 interface remote-as external

 address-family ipv4 unicast
  network 10.0.0.1/32
  neighbor ens7 activate
  neighbor ens8 activate
  exit-address-family
EOF

frrtest2 and frrtest3

# ID=2 on frrtest2
# ID=3 on frrtest3
ID=${HOSTNAME#frrtest}

vtysh <<EOF
configure

interface lo
 ip address 10.0.0.$ID/32

interface ens7
 no shutdown

interface ens8
 no shutdown

vrf vrf10
 vni 10

router bgp $ID
 no bgp ebgp-requires-policy
 neighbor ens7 interface remote-as external
 neighbor ens8 interface remote-as external

 address-family ipv4 unicast
  network 10.0.0.$ID/32
  neighbor ens7 activate
  neighbor ens8 activate
  import vrf vrf10
 exit-address-family

 address-family l2vpn evpn
  advertise-all-vni
  neighbor ens8 activate
 exit-address-family

router bgp $ID vrf vrf10
 address-family ipv4 unicast
  redistribute connected
 exit-address-family

 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
EOF

# L3VNI setup
ip link add up vrf10 type vrf table 10
ip link add up br10 type bridge
ip link set br10 master vrf10
ip link add up vni10 type vxlan id 10 local 10.0.0.$ID nolearning dstport 4789
ip link set vni10 master br10
bridge link set dev vni10 learning off

# L2VNI and IRB setup
ip link add up br100 type bridge
ip link set br100 master vrf10
ip link add up vni100 type vxlan id 100 local 10.0.0.$ID nolearning dstport 4789
ip link set vni100 master br100
bridge link set dev vni100 learning off
ip address add 192.168.0.1/24 dev br100

# Mock host setup (to cause type-2 MACIP advertisement)
ip link add up dummy0 type dummy
ip link set dummy0 master br100
bridge fdb add 02:00:00:00:00:$ID$ID dev dummy0 master static sticky
ip neigh add 192.168.0.$ID lladdr 02:00:00:00:00:$ID$ID dev br100

Expected behavior

frrtest should see a direct route to 192.168.0.2 via frrtest2, and a direct route to 192.168.0.3 via frrtest3.

Both frrtest2 and frrtest3 should see routes to 192.168.0.2 and 192.168.0.3 in the default VRF with an AS-path length of null for the locally generated route, and one for the route received from the other EVPN node.

Actual behavior

frrtest only sees a route to 192.168.0.2 via frrtest3 (as-path 3 2) and t 192.168.0.3 via frtest2 (as-path 2 3):

frrtest# show ip bgp 192.168.0.2
BGP routing table entry for 192.168.0.2/32, version 5
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  ens7 ens8
  3 2
    ::ffff:a00:3 from ens8 (10.0.0.3)
    (fe80::f816:3eff:fe87:37c2) (used)
      Origin IGP, valid, external, best (First path received)
      Extended Community: ET:8 Rmac:0e:cd:2f:40:19:54
      Last update: Tue Jun  4 14:27:28 2024
frrtest# show ip bgp 192.168.0.3
BGP routing table entry for 192.168.0.3/32, version 6
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  ens7 ens8
  2 3
    ::ffff:a00:2 from ens7 (10.0.0.2)
    (fe80::f816:3eff:fec0:6702) (used)
      Origin IGP, valid, external, best (First path received)
      Extended Community: ET:8 Rmac:86:42:e3:bc:7c:91
      Last update: Tue Jun  4 14:27:28 2024

This is also visible on frrtest2 and frrtest3, which only has routes in the default VRF to the remote MACIP route, not to its own:

frrtest2# show ip route vrf default 192.168.0.0/24 longer-prefixes 
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

B>* 192.168.0.0/24 [20/0] is directly connected, vrf10 (vrf vrf10), weight 1, 01:19:45
B>* 192.168.0.3/32 [20/0] via 10.0.0.3, br10 (vrf vrf10) onlink, weight 1, 01:19:36

frrtest3# show ip route vrf default 192.168.0.0/24 longer-prefixes
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

B>* 192.168.0.0/24 [20/0] is directly connected, vrf10 (vrf vrf10), weight 1, 01:19:47
B>* 192.168.0.2/32 [20/0] via 10.0.0.2, br10 (vrf vrf10) onlink, weight 1, 01:19:45

Additional context

Both frrtest2 and frrtest3 see their local and the remote MACIP route with the expected as-path lengths (null for the locally generated type-2, one AS for the remotely generated one):

frrtest2

frrtest2# show bgp l2vpn evpn route type 2 
BGP table version is 3, local router ID is 10.0.0.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]:[Frag-id]
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
                    Extended Community
Route Distinguisher: 10.0.0.2:3
 *>  [2]:[0]:[48]:[02:00:00:00:00:22]
                    10.0.0.2                           32768 i
                    ET:8 RT:2:100 MM:0, sticky MAC
 *>  [2]:[0]:[48]:[02:00:00:00:00:22]:[32]:[192.168.0.2]
                    10.0.0.2                           32768 i
                    ET:8 RT:2:100 RT:2:10 Rmac:0e:cd:2f:40:19:54
Route Distinguisher: 10.0.0.3:3
 *>  [2]:[0]:[48]:[02:00:00:00:00:33]
                    10.0.0.3                               0 3 i
                    RT:3:100 ET:8 MM:0, sticky MAC
 *>  [2]:[0]:[48]:[02:00:00:00:00:33]:[32]:[192.168.0.3]
                    10.0.0.3                               0 3 i
                    RT:3:10 RT:3:100 ET:8 Rmac:86:42:e3:bc:7c:91

Displayed 4 prefixes (4 paths) (of requested type)

frrtest3

frrtest3# show bgp l2vpn evpn route type 2 
BGP table version is 3, local router ID is 10.0.0.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]:[Frag-id]
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
                    Extended Community
Route Distinguisher: 10.0.0.2:3
 *>  [2]:[0]:[48]:[02:00:00:00:00:22]
                    10.0.0.2                               0 2 i
                    RT:2:100 ET:8 MM:0, sticky MAC
 *>  [2]:[0]:[48]:[02:00:00:00:00:22]:[32]:[192.168.0.2]
                    10.0.0.2                               0 2 i
                    RT:2:10 RT:2:100 ET:8 Rmac:0e:cd:2f:40:19:54
Route Distinguisher: 10.0.0.3:3
 *>  [2]:[0]:[48]:[02:00:00:00:00:33]
                    10.0.0.3                           32768 i
                    ET:8 RT:3:100 MM:0, sticky MAC
 *>  [2]:[0]:[48]:[02:00:00:00:00:33]:[32]:[192.168.0.3]
                    10.0.0.3                           32768 i
                    ET:8 RT:3:100 RT:3:10 Rmac:86:42:e3:bc:7c:91

Displayed 4 prefixes (4 paths) (of requested type)

I would be happy to give any interested developer access to this lab (including full sudo access) in case that is of interest.

Checklist

• I have searched the open issues for this bug.
• I have not included sensitive information in this report.

[github-actions]

This issue is stale because it has been open 180 days with no activity. Comment or remove the autoclose label in order to avoid having this issue closed.

[frrbot]

This issue will be automatically closed in the specified period unless there is further activity.

[MatthieuCoder]

Can this be re-opened ?