Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[9.2-dev] staticd/mgmtd doesn't load static routes on OS booting in a special case with more than 11 VRFs in system and wrong networking config #15267

Closed
2 tasks done
EasyNetDev opened this issue Jan 31, 2024 · 6 comments · Fixed by #15286
Closed
2 tasks done

[9.2-dev] staticd/mgmtd doesn't load static routes on OS booting in a special case with more than 11 VRFs in system and wrong networking config #15267

EasyNetDev opened this issue Jan 31, 2024 · 6 comments · Fixed by #15286
Labels
triage Needs further investigation

Comments

@EasyNetDev
Copy link
Contributor

EasyNetDev commented Jan 31, 2024 •
edited
Loading

Describe the bug

When the system is booting up and you have 12 or more VRFs defined in system and in system networking you have a wrong configuration, FRRouting doesn't load static routes and are missing from the running configuration!
I've notice this behavior from December on 4 PC and 1 VM running Debian 12 or 13 with latest development FRR. All of them have the same behavior.

  • Did you check if this is a duplicate issue?
  • Did you test it on the latest FRRouting/frr master branch?

To Reproduce

  1. Install latest git FRR.
  2. Create 11 or 12 VRFs in the system (example provided below).
  3. Create a wrong configuration interface in system configrau (example provided below).
  4. Under FRR add in each VRF and/or in GRT some routes (example provided below).
  5. Save the config.
  6. Reboot the machine.
  7. If you have 11 VRFs in system, routes are loaded in running config.
  8. If you have 12 or more VRFs in system, static routes are missing from running config.

Networking configuration:

  1. /etc/network/interfaces:
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source-directory /etc/network/interfaces.d/

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto ens33
iface ens33
    use dhcp
  1. /etc/network/interfaces.d/01-vrf:
auto red
iface red
    vrf-table 3000

auto red1
iface red1
    vrf-table 3001

auto red2
iface red2
    vrf-table 3002

auto red3
iface red3
    vrf-table 3003

auto red4
iface red4
    vrf-table 3004

auto red5
iface red5
    vrf-table 3005

auto blue
iface blue
    vrf-table 4000

auto blue1
iface blue1
    vrf-table 4001

auto blue2
iface blue2
    vrf-table 4002

auto blue3
iface blue3
    vrf-table 4003

auto blue4
iface blue4
    vrf-table 4004

#auto blue5
#iface blue5
#    vrf-table 4005

auto green
iface green
    vrf-table 5000

#auto green1
#iface green1
#    vrf-table 5001

#auto green2
#iface green2
#    vrf-table 5002

#auto green3
#iface green3
#    vrf-table 5003

#auto green4
#iface green4
#    vrf-table 5004

#auto green5
#iface green5
#    vrf-table 5005
  1. /etc/network/interfaces.d/05-loopback (with wrong configuration on purpose):
auto lo0
iface lo0
    link-type dummy

auto lo1
iface lo1
    link-type dummy
    vrf red10

auto lo2
iface lo2
    link-type dummy
    vrf blue10

red10 and blue10 VRFs are not existing in system.

Test with 12 VRFs:

# grep ^auto /etc/network/interfaces.d/01-vrf | wc -l
12

Output of networking.service and frr.service:

# systemctl status networking
● networking.service - ifupdown-ng networking initialization
     Loaded: loaded (/usr/lib/systemd/system/networking.service; enabled; preset: enabled)
     Active: active (exited) since Wed 2024-01-31 23:05:03 CET; 9min ago
       Docs: man:interfaces(5)
             man:ifup(8)
             man:ifdown(8)
    Process: 673 ExecStart=/usr/share/ifupdown-ng/sbin/networking start (code=exited, status=0/SUCCESS)
   Main PID: 673 (code=exited, status=0/SUCCESS)
      Tasks: 1 (limit: 9435)
     Memory: 5.9M (peak: 9.2M)
        CPU: 799ms
     CGroup: /system.slice/networking.service
             └─1683 /usr/sbin/dhclient -pf /var/run/dhclient.ens33.pid ens33

Jan 31 23:05:03 FRR01 networking[691]: /bin/run-parts /etc/network/if-up.d
Jan 31 23:05:03 FRR01 networking[691]: /usr/libexec/ifupdown-ng/link
Jan 31 23:05:03 FRR01 networking[691]: /usr/libexec/ifupdown-ng/dhcp
Jan 31 23:05:03 FRR01 systemd[1]: Finished networking.service - ifupdown-ng networking initialization.
Jan 31 23:09:06 FRR01 dhclient[1683]: DHCPREQUEST for 192.168.55.187 on ens33 to 192.168.55.2 port 67
Jan 31 23:09:06 FRR01 dhclient[1683]: DHCPACK of 192.168.55.187 from 192.168.55.2
Jan 31 23:09:06 FRR01 dhclient[1683]: bound to 192.168.55.187 -- renewal in 287 seconds.
Jan 31 23:13:53 FRR01 dhclient[1683]: DHCPREQUEST for 192.168.55.187 on ens33 to 192.168.55.2 port 67
Jan 31 23:13:53 FRR01 dhclient[1683]: DHCPACK of 192.168.55.187 from 192.168.55.2
Jan 31 23:13:53 FRR01 dhclient[1683]: bound to 192.168.55.187 -- renewal in 273 seconds.

# systemctl status frr.service
● frr.service - FRRouting
     Loaded: loaded (/usr/lib/systemd/system/frr.service; enabled; preset: enabled)
     Active: active (running) since Wed 2024-01-31 23:05:03 CET; 10min ago
       Docs: https://frrouting.readthedocs.io/en/latest/setup.html
    Process: 657 ExecStart=/usr/lib/frr/frrinit.sh start (code=exited, status=0/SUCCESS)
   Main PID: 791 (watchfrr)
     Status: "FRR Operational"
      Tasks: 45 (limit: 9435)
     Memory: 124.3M (peak: 159.1M)
        CPU: 2.410s
     CGroup: /system.slice/frr.service
             ├─ 791 /usr/lib/frr/watchfrr -d -F traditional zebra mgmtd bgpd ospfd ospf6d isisd pimd pim6d ldpd nhrpd eigrpd pbrd stat>
             ├─ 879 /usr/lib/frr/zebra -d -F traditional -A 127.0.0.1 -s 90000000 -M snmp
             ├─ 931 /usr/lib/frr/mgmtd -d -F traditional -A 127.0.0.1
             ├─1012 /usr/lib/frr/bgpd -d -F traditional -A 127.0.0.1 -M snmp
             ├─1060 /usr/lib/frr/ospfd -d -F traditional -A 127.0.0.1 -M snmp
             ├─1108 /usr/lib/frr/ospf6d -d -F traditional -A ::1 -M snmp
             ├─1159 /usr/lib/frr/isisd -d -F traditional -A 127.0.0.1 -M snmp
             ├─1204 /usr/lib/frr/pimd -d -F traditional -A 127.0.0.1
             ├─1235 /usr/lib/frr/pim6d -d -F traditional -A ::1
             ├─1254 /usr/lib/frr/ldpd -L -u frr -g frr
             ├─1255 /usr/lib/frr/ldpd -E -u frr -g frr
             ├─1275 /usr/lib/frr/ldpd -d -F traditional -A 127.0.0.1 -M snmp
             ├─1305 /usr/lib/frr/nhrpd -d -F traditional -A 127.0.0.1
             ├─1331 /usr/lib/frr/eigrpd -d -F traditional -A 127.0.0.1
             ├─1357 /usr/lib/frr/pbrd -d -F traditional -A 127.0.0.1
             ├─1383 /usr/lib/frr/staticd -d -F traditional -A 127.0.0.1
             ├─1452 /usr/lib/frr/bfdd -d -F traditional -A 127.0.0.1
             ├─1474 /usr/lib/frr/vrrpd -d -F traditional -A 127.0.0.1
             └─1502 /usr/lib/frr/pathd -d -F traditional -A 127.0.0.1

Jan 31 23:05:03 FRR01 watchfrr[791]: [QDG3Y-BY5TN] staticd state -> up : connect succeeded
Jan 31 23:05:03 FRR01 watchfrr[791]: [QDG3Y-BY5TN] bfdd state -> up : connect succeeded
Jan 31 23:05:03 FRR01 watchfrr[791]: [QDG3Y-BY5TN] vrrpd state -> up : connect succeeded
Jan 31 23:05:03 FRR01 watchfrr[791]: [QDG3Y-BY5TN] pathd state -> up : connect succeeded
Jan 31 23:05:03 FRR01 watchfrr[791]: [KWE5Q-QNGFC] all daemons up, doing startup-complete notify
Jan 31 23:05:03 FRR01 frrinit.sh[657]: Started watchfrr.
Jan 31 23:05:03 FRR01 systemd[1]: Started frr.service - FRRouting.
Jan 31 23:05:03 FRR01 zebra[879]: [V98V0-MTWPF] client 130 says hello and bids fair to announce only bgp routes vrf=0
Jan 31 23:05:06 FRR01 pathd[1502]: [NN4XW-E4M3V] IPv6 Router Id updated for VRF 0: fc00:192:168:55:20c:29ff:fee4:52f
Jan 31 23:05:06 FRR01 zebra[879]: [RHHTS-F96DR][EC 4043309149] ens33(2): Rx RA - our AdvManagedFlag (0) doesn't agree with fe80::58c1:>

FRR running config:

# vtysh -c "show run"
Building configuration...

Current configuration:
!
frr version 9.2-dev
frr defaults traditional
hostname FRR01
log syslog informational
no ip forwarding
no ipv6 forwarding
service integrated-vtysh-config
!
vrf blue
exit-vrf
!
vrf blue1
exit-vrf
!
vrf blue2
exit-vrf
!
vrf blue3
exit-vrf
!
vrf blue4
exit-vrf
!
vrf green
exit-vrf
!
vrf red
exit-vrf
!
vrf red1
exit-vrf
!
vrf red2
exit-vrf
!
vrf red3
exit-vrf
!
vrf red4
exit-vrf
!
vrf red5
exit-vrf
!
router bgp 65000
 bgp router-id 10.100.1.1
 no bgp suppress-duplicates
 bgp graceful-shutdown
 bgp graceful-restart
 neighbor MPLS-v4 peer-group
 neighbor MPLS-v4 remote-as 65000
 neighbor MPLS-v4 description Core Routers MPLS
...

Test with 11 VRFs:

# grep ^auto /etc/network/interfaces.d/01-vrf | wc -l
11

Output of networking.service and frr.service:

# systemctl status networking.service
● networking.service - ifupdown-ng networking initialization
     Loaded: loaded (/usr/lib/systemd/system/networking.service; enabled; preset: enabled)
     Active: active (exited) since Wed 2024-01-31 23:34:23 CET; 1min 28s ago
       Docs: man:interfaces(5)
             man:ifup(8)
             man:ifdown(8)
    Process: 730 ExecStart=/usr/share/ifupdown-ng/sbin/networking start (code=exited, status=0/SUCCESS)
   Main PID: 730 (code=exited, status=0/SUCCESS)
      Tasks: 1 (limit: 9435)
     Memory: 5.7M (peak: 8.5M)
        CPU: 749ms
     CGroup: /system.slice/networking.service
             └─1678 /usr/sbin/dhclient -pf /var/run/dhclient.ens33.pid ens33

Jan 31 23:34:23 FRR01 networking[753]: /usr/libexec/ifupdown-ng/dhcp
Jan 31 23:34:23 FRR01 networking[753]: /usr/libexec/ifupdown-ng/link
Jan 31 23:34:23 FRR01 networking[753]: /usr/libexec/ifupdown-ng/dhcp
Jan 31 23:34:23 FRR01 networking[753]: /bin/run-parts /etc/network/if-pre-up.d
Jan 31 23:34:23 FRR01 networking[753]: /usr/libexec/ifupdown-ng/link
Jan 31 23:34:23 FRR01 networking[753]: /usr/libexec/ifupdown-ng/dhcp
Jan 31 23:34:23 FRR01 networking[753]: /bin/run-parts /etc/network/if-up.d
Jan 31 23:34:23 FRR01 networking[753]: /usr/libexec/ifupdown-ng/link
Jan 31 23:34:23 FRR01 networking[753]: /usr/libexec/ifupdown-ng/dhcp
Jan 31 23:34:23 FRR01 systemd[1]: Finished networking.service - ifupdown-ng networking initialization.

# systemctl status frr.service
● frr.service - FRRouting
     Loaded: loaded (/usr/lib/systemd/system/frr.service; enabled; preset: enabled)
     Active: active (running) since Wed 2024-01-31 23:34:23 CET; 2min 22s ago
       Docs: https://frrouting.readthedocs.io/en/latest/setup.html
    Process: 717 ExecStart=/usr/lib/frr/frrinit.sh start (code=exited, status=0/SUCCESS)
   Main PID: 833 (watchfrr)
     Status: "FRR Operational"
      Tasks: 45 (limit: 9435)
     Memory: 124.7M (peak: 169.0M)
        CPU: 1.615s
     CGroup: /system.slice/frr.service
             ├─ 833 /usr/lib/frr/watchfrr -d -F traditional zebra mgmtd bgpd ospfd ospf6d isisd pimd pim6d ldpd nhrpd eigrpd pbrd stat>
             ├─ 929 /usr/lib/frr/zebra -d -F traditional -A 127.0.0.1 -s 90000000 -M snmp
             ├─ 979 /usr/lib/frr/mgmtd -d -F traditional -A 127.0.0.1
             ├─1060 /usr/lib/frr/bgpd -d -F traditional -A 127.0.0.1 -M snmp
             ├─1109 /usr/lib/frr/ospfd -d -F traditional -A 127.0.0.1 -M snmp
             ├─1153 /usr/lib/frr/ospf6d -d -F traditional -A ::1 -M snmp
             ├─1205 /usr/lib/frr/isisd -d -F traditional -A 127.0.0.1 -M snmp
             ├─1251 /usr/lib/frr/pimd -d -F traditional -A 127.0.0.1
             ├─1289 /usr/lib/frr/pim6d -d -F traditional -A ::1
             ├─1301 /usr/lib/frr/ldpd -L -u frr -g frr
             ├─1302 /usr/lib/frr/ldpd -E -u frr -g frr
             ├─1328 /usr/lib/frr/ldpd -d -F traditional -A 127.0.0.1 -M snmp
             ├─1353 /usr/lib/frr/nhrpd -d -F traditional -A 127.0.0.1
             ├─1378 /usr/lib/frr/eigrpd -d -F traditional -A 127.0.0.1
             ├─1405 /usr/lib/frr/pbrd -d -F traditional -A 127.0.0.1
             ├─1423 /usr/lib/frr/staticd -d -F traditional -A 127.0.0.1
             ├─1477 /usr/lib/frr/bfdd -d -F traditional -A 127.0.0.1
             ├─1504 /usr/lib/frr/vrrpd -d -F traditional -A 127.0.0.1
             └─1535 /usr/lib/frr/pathd -d -F traditional -A 127.0.0.1

Jan 31 23:34:24 FRR01 zebra[929]: [V98V0-MTWPF] client 130 says hello and bids fair to announce only bgp routes vrf=0
Jan 31 23:34:25 FRR01 staticd[1423]: [NZTXV-H7AQ5] route_notify_owner: Route ::/0 over-ridden by better route for table: 254
Jan 31 23:34:26 FRR01 pathd[1535]: [NN4XW-E4M3V] IPv6 Router Id updated for VRF 0: fc00:192:168:55:20c:29ff:fee4:52f
Jan 31 23:34:27 FRR01 zebra[929]: [RHHTS-F96DR][EC 4043309149] ens33(2): Rx RA - our AdvManagedFlag (0) doesn't agree with fe80::58c1:>
Jan 31 23:34:28 FRR01 zebra[929]: [WVJCK-PPMGD][EC 4043309093] netlink-dp (NS 0) error: No such device, type=RTM_NEWROUTE(24), seq=48,>
Jan 31 23:34:28 FRR01 staticd[1423]: [NZTXV-H7AQ5] route_notify_owner: Route 0.0.0.0/0 over-ridden by better route for table: 5000
Jan 31 23:34:28 FRR01 zebra[929]: [WVJCK-PPMGD][EC 4043309093] netlink-dp (NS 0) error: No such device, type=RTM_NEWROUTE(24), seq=51,>
Jan 31 23:34:28 FRR01 staticd[1423]: [NZTXV-H7AQ5] route_notify_owner: Route ::/0 over-ridden by better route for table: 5000
Jan 31 23:34:28 FRR01 zebra[929]: [VYKYC-709DP] green(26:5000):0.0.0.0/0: Route install failed
Jan 31 23:34:28 FRR01 zebra[929]: [VYKYC-709DP] green(26:5000):::/0: Route install failed

FRR running config:

frr version 9.2-dev
frr defaults traditional
hostname FRR01
log syslog informational
no ip forwarding
no ipv6 forwarding
service integrated-vtysh-config
!
ip route 0.0.0.0/0 blackhole 250
ip route 192.168.88.0/24 192.168.55.1
ipv6 route ::/0 blackhole 250
!
vrf red
 ip route 0.0.0.0/0 blackhole 250
 ipv6 route ::/0 blackhole 250
exit-vrf
!
vrf blue
 ip route 0.0.0.0/0 blackhole 250
 ipv6 route ::/0 blackhole 250
exit-vrf
!
vrf green
 ip route 0.0.0.0/0 blackhole 250
 ipv6 route ::/0 blackhole 250
exit-vrf
!
vrf blue1
exit-vrf
!
vrf blue2
exit-vrf
!
vrf blue3
exit-vrf
!
vrf red1
exit-vrf
!
vrf red2
exit-vrf
!
vrf red3
exit-vrf
!
vrf red4
exit-vrf
!
vrf red5
exit-vrf
!
router bgp 65000
 bgp router-id 10.100.1.1
 no bgp suppress-duplicates
 bgp graceful-shutdown
 bgp graceful-restart
 neighbor MPLS-v4 peer-group
...

Saved config that should be loaded at boot:

# cat /etc/frr/frr.conf
frr version 9.2-dev
frr defaults traditional
hostname FRR01
log syslog informational
no ip forwarding
no ipv6 forwarding
service integrated-vtysh-config
!
ip route 0.0.0.0/0 blackhole 250
ip route 192.168.88.0/24 192.168.55.1
ipv6 route ::/0 blackhole 250
!
vrf red
 ip route 0.0.0.0/0 blackhole 250
 ipv6 route ::/0 blackhole 250
exit-vrf
!
vrf blue
 ip route 0.0.0.0/0 blackhole 250
 ipv6 route ::/0 blackhole 250
exit-vrf
!
vrf green
 ip route 0.0.0.0/0 blackhole 250
 ipv6 route ::/0 blackhole 250
exit-vrf
!
router bgp 65000
 bgp router-id 10.100.1.1
 no bgp suppress-duplicates
 bgp graceful-shutdown
 bgp graceful-restart
 neighbor MPLS-v4 peer-group
 neighbor MPLS-v4 remote-as 65000
 neighbor MPLS-v4 description Core Routers MPLS
 neighbor MPLS-v4 bfd
 neighbor MPLS-v4 bfd check-control-plane-failure
 no neighbor MPLS-v4 enforce-first-as
 neighbor MPLS-v4 update-source lo
 neighbor MPLS-v4 capability extended-nexthop
 neighbor MPLS-v6 peer-group
 neighbor MPLS-v6 remote-as 65000
 neighbor MPLS-v6 description Core Routers MPLS
 neighbor MPLS-v6 bfd
 neighbor MPLS-v6 bfd check-control-plane-failure
 no neighbor MPLS-v6 enforce-first-as
 neighbor MPLS-v6 update-source lo
 neighbor MPLS-v6 capability extended-nexthop
 neighbor SPOKES-v4 peer-group
 neighbor SPOKES-v4 remote-as 65000
 neighbor SPOKES-v4 description SPOKES
 neighbor SPOKES-v4 bfd
 neighbor SPOKES-v4 bfd check-control-plane-failure
 neighbor SPOKES-v4 graceful-shutdown
 no neighbor SPOKES-v4 enforce-first-as
 neighbor SPOKES-v4 update-source lo
 neighbor SPOKES-v4 capability extended-nexthop
 neighbor SPOKES-v4 graceful-restart
 neighbor SPOKES-v6 peer-group
 neighbor SPOKES-v6 remote-as 65000
 neighbor SPOKES-v6 description SPOKES
 neighbor SPOKES-v6 bfd
 neighbor SPOKES-v6 bfd check-control-plane-failure
 neighbor SPOKES-v6 graceful-shutdown
 no neighbor SPOKES-v6 enforce-first-as
 neighbor SPOKES-v6 update-source lo
 neighbor SPOKES-v6 capability extended-nexthop
 neighbor SPOKES-v6 graceful-restart
 neighbor 10.100.2.1 peer-group MPLS-v4
 neighbor 10.100.2.1 description R02_VPNv4
 no neighbor 10.100.2.1 enforce-first-as
 neighbor fc00:0:0:2::1 peer-group MPLS-v6
 neighbor fc00:0:0:2::1 description R02_VPNv6
 neighbor fc00:0:0:2::1 bfd
 no neighbor fc00:0:0:2::1 enforce-first-as
 neighbor fc00:0:0:2::1 graceful-restart
 !
 address-family ipv4 unicast
  redistribute connected route-map VPN-export-GRT-connected
  no neighbor MPLS-v4 activate
  no neighbor MPLS-v6 activate
  no neighbor SPOKES-v4 activate
  no neighbor SPOKES-v6 activate
  label vpn export auto
  rd vpn export 43474:11001
  rt vpn import 43474:11000 43474:11999
  rt vpn export 43474:11000
  export vpn
  import vpn
 exit-address-family
 !
 address-family ipv4 multicast
  network 239.0.0.0/8
 exit-address-family
 !
 address-family ipv4 vpn
  neighbor MPLS-v4 activate
  neighbor SPOKES-v4 activate
  neighbor SPOKES-v4 soft-reconfiguration inbound
  neighbor SPOKES-v4 route-map to-SPOKES out
 exit-address-family
 !
 address-family ipv6 unicast
  redistribute connected route-map VPN-export-GRT-connected
  label vpn export auto
  rd vpn export 43474:11001
  rt vpn import 43474:11000 43474:11999
  rt vpn export 43474:11000
  export vpn
  import vpn
 exit-address-family
 !
 address-family ipv6 vpn
  neighbor MPLS-v6 activate
  neighbor SPOKES-v6 activate
  neighbor SPOKES-v6 soft-reconfiguration inbound
  neighbor SPOKES-v6 route-map to-SPOKES out
 exit-address-family
exit
!
router bgp 65000 vrf red
 bgp router-id 10.100.1.12
 !
 address-family ipv4 unicast
  redistribute connected route-map VPN-export-RED-connected
  redistribute static route-map VPN-export-RED-static
  label vpn export auto
  rd vpn export 65000:16001
  rt vpn import 65000:16000 65000:16100 65000:16200 65000:16800 65000:16999
  rt vpn export 65000:16000
  export vpn
  import vpn
 exit-address-family
 !
 address-family ipv6 unicast
  redistribute connected route-map VPN-export-RED-connected
  redistribute static route-map VPN-export-RED-static
  label vpn export auto
  rd vpn export 65000:16001
  rt vpn import 65000:16000 65000:16100 65000:16200 65000:16800 65000:16999
  rt vpn export 65000:16000
  export vpn
  import vpn
 exit-address-family
exit
!
router bgp 65000 vrf blue
 bgp router-id 10.100.1.16
 !
 address-family ipv4 unicast
  redistribute connected route-map VPN-export-BLUE-connected
  redistribute static route-map VPN-export-BLUE-static
  label vpn export auto
  rd vpn export 65000:12001
  rt vpn import 65000:12000 65000:12100 65000:12200 65000:12800 65000:12999
  rt vpn export 65000:12000
  export vpn
  import vpn
 exit-address-family
 !
 address-family ipv6 unicast
  redistribute connected route-map VPN-export-BLUE-connected
  redistribute static route-map VPN-export-BLUE-static
  label vpn export auto
  rd vpn export 65000:12001
  rt vpn import 65000:12000 65000:12100 65000:12200 65000:12800 65000:12999
  rt vpn export 65000:12000
  export vpn
  import vpn
 exit-address-family
exit
!
router bgp 65000 vrf green
 bgp router-id 10.100.1.11
 !
 address-family ipv4 unicast
  redistribute connected route-map VPN-export-GREEN-connected
  label vpn export auto
  rd vpn export 65000:14001
  rt vpn import 65000:14000 65000:14100 65000:14200 65000:14800 65000:14999
  rt vpn export 65000:14000
  export vpn
  import vpn
 exit-address-family
 !
 address-family ipv6 unicast
  redistribute connected route-map VPN-export-GREEN-connected
  label vpn export auto
  rd vpn export 65000:14001
  rt vpn import 65000:14000 65000:14100 65000:14200 65000:14800 65000:14999
  rt vpn export 65000:14000
  export vpn
  import vpn
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
exit
!
router ospf
 ospf router-id 10.100.1.1
 log-adjacency-changes
 compatible rfc1583
 maximum-paths 4
 flood-reduction
 fast-reroute ti-lfa
 area 10 stub no-summary
 mpls ldp-sync
exit
!
router ospf vrf iptv
 ospf router-id 10.100.1.1
 log-adjacency-changes
 compatible rfc1583
 maximum-paths 4
 neighbor 10.190.129.4
 default-information originate always
exit
!
router ospf6
 ospf6 router-id 10.100.1.1
exit
!
mpls ldp
 router-id 10.100.1.1
 dual-stack cisco-interop
 ordered-control
 neighbor 10.100.2.1 ttl-security disable
 neighbor 10.100.2.1 password ENPDW
 !
 address-family ipv4
  discovery transport-address 10.100.1.1
  !
  interface lan0.3001
  exit
  !
  interface wan0.3000
  exit
  !
 exit-address-family
 !
 address-family ipv6
  discovery transport-address fc00:0:0:1::1
  !
  interface lan0.3001
  exit
  !
  interface wan0.3000
  exit
  !
 exit-address-family
 !
exit
!
router isis EasyNet
 net 49.0001.0101.9000.0001.00
 fast-reroute load-sharing disable level-1
 fast-reroute lfa tiebreaker lowest-backup-metric index 1 level-1
 fast-reroute load-sharing disable level-2
exit
!
router isis EasyNet-A2
 net 49.0002.0101.9000.0001.00
 lsp-mtu 1400
exit
!
route-map to-SPOKES deny 60000
exit
!
route-map VPN-export-GRT-connected permit 1000
 match ip address prefix-list pl-EASYNET-subnets
 set extcommunity rt 65000:10999 65000:18999 65000:17999 65000:13999 65000:12999 65000:11000 65000:14999 65000:15999
exit
!
route-map VPN-export-GRT-connected permit 1100
 match ipv6 address prefix-list pl-EASYNET-subnets
 set extcommunity rt 65000:10999 65000:18999 65000:17999 65000:13999 65000:12999 65000:11000 65000:14999 65000:15999
exit
!
route-map VPN-export-GRT-connected permit 2000
 match ip address prefix-list pl-INTERCONNECT-subnets
 set extcommunity rt 65000:12999 65000:15999
exit
!
route-map VPN-export-GRT-connected permit 2100
 match ipv6 address prefix-list pl-INTERCONNECT-subnets
 set extcommunity rt 65000:12999 65000:15999
exit
!
route-map VPN-export-GRT-connected deny 10000
exit
!
route-map VPN-export-RED-connected permit 1000
 set extcommunity rt 65000:16100 65000:16800 65000:12999 65000:15999 65000:18999 65000:17999
exit
!
route-map VPN-export-RED-static permit 1000
 set extcommunity rt 65000:16100 65000:16800 65000:12999 65000:15999 65000:18999 65000:17999
exit
!
route-map VPN-export-BLUE-connected permit 1000
 match ip address prefix-list pl-EASYNET-subnets
 match source-protocol local
 set extcommunity rt 65000:12100 65000:10999 65000:14999 65000:15999 65000:16999 65000:17999
exit
!
route-map VPN-export-BLUE-connected permit 1100
 match ipv6 address prefix-list pl-EASYNET-subnets
 set extcommunity rt 65000:12100 65000:10999 65000:14999 65000:15999 65000:16999 65000:17999
exit
!
route-map VPN-export-BLUE-static deny 1000
 match ip address prefix-list pl-DEFAULT-ROUTE
exit
!
route-map VPN-export-BLUE-static deny 1100
 match ipv6 address prefix-list pl-DEFAULT-ROUTE
exit
!
route-map VPN-export-BLUE-static permit 2000
 set extcommunity rt 65000:12200 65000:12800 65000:10999 65000:11999 65000:14999 65000:15999 65000:16999
exit
!
route-map VPN-export-GREEN-connected permit 1000
 set extcommunity rt 65000:14100 65000:14800 65000:12999 65000:15999 65000:18999 65000:17999
exit
!
segment-routing
 traffic-eng
 exit
exit
!

Expected behavior

All static routes should be installed on boot.
To fix the issue I have to run vtysh -d mgmtd -f /etc/frr/frr.conf after booting.

Screenshots

Versions

  • OS Version: Debian 13 Trixie
  • Kernel: 6.5.0-5-amd64

Additional context
@EasyNetDev EasyNetDev added the triage Needs further investigation label Jan 31, 2024
@EasyNetDev
Copy link
Contributor Author

EasyNetDev commented Feb 2, 2024 •
edited
Loading

To solve the booting issue I've created a service like this:

/etc/systemd/system/frr-static-routes-fix.service:

[Unit]
Description=FRRouting static routes fix
Documentation=https://frrouting.readthedocs.io/en/latest/setup.html
Documentation=https://github.com/FRRouting/frr/issues/15267
Wants=network.target
Requires=frr.service
After=network-pre.target systemd-sysctl.service frr.service
Before=network.target

[Service]
Nice=-5
Type=oneshot
RemainAfterExit=yes
#NotifyAccess=all
StartLimitInterval=3m
StartLimitBurst=3
TimeoutSec=2m
WatchdogSec=60s
RestartSec=5
#Restart=always
ExecStart=/usr/bin/vtysh -d mgmtd -f "/etc/frr/frr.conf"

[Install]
WantedBy=multi-user.target

@idryzhov idryzhov mentioned this issue Feb 2, 2024
Merged
@idryzhov
Copy link
Contributor

idryzhov commented Feb 2, 2024 •
edited
Loading

Hi. Would be great if you could get and test the branch from this PR: #15286. I think it should fix your problem.

@EasyNetDev
Copy link
Contributor Author

Hi @idryzhov I will try to patch FRR with that PR and I will give a try. Thanks!

@EasyNetDev
Copy link
Contributor Author

@idryzhov I compiled today's git FRR with your patch. I had to change a little bit the patch to be able to applied on the latest dev.

Looks like is working now.

@idryzhov
Copy link
Contributor

idryzhov commented Feb 4, 2024

Perfect. Thanks for the testing!

@EasyNetDev
Copy link
Contributor Author

I will test on all devices tomorrow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
triage Needs further investigation
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix conflict in mgmtd on startup idryzhov/frr
2 participants
@idryzhov @EasyNetDev