sho ip ospf database ext 0.0.0.0 continues displaying LSA for default gateway in down state

[kpiq]

• FRR VERSION: 7.5.1
• OPERATING SYSTEM VERSION: OPNsense 23.1.7
• KERNEL VERSION: FREEBSD 13.1-RELEASE-p7 FreeBSD 13.1-RELEASE-p7 stable/23.1-n250430-7eb6eb035df SMP amd64

frr/ospf configuration

sho run
Building configuration...

Current configuration:
!
frr version 7.5.1
frr defaults traditional
hostname hostname.fqdn
log syslog notifications
log commands
!
interface igb2
ip ospf cost 1
!
router ospf
ospf router-id <router-id-1>
auto-cost reference-bandwidth 2000
redistribute kernel
redistribute connected
redistribute static
passive-interface igb1
passive-interface ix0
passive-interface lo0
passive-interface openvpn
passive-interface wg0
passive-interface wireguard
network 0.0.0.0/0 area 0.0.0.0
network 10.0.0.0/8 area 0.0.0.0
network 224.0.0.0/24 area 0.0.0.0
default-information originate metric 10
!
line vty
!
end

In Routing: OSPF, "Advertise default gateway" is checked/enabled, " Always advertise default gateway" is not checked/enabled.

---

Describe the bug

My network has two OPNsense firewalls with frr 7.5.1, for sites A (router-id-1) and B (router-id-2) The firewalls are directly connected to multilayer switches which take care of the InterVLAN routing, for sites A, B C, and D. They also run OSPF and are member of the same OSPF area, 0.0.0.0. All of the devices, firewalls and multilayer switches, appear to be talking OSPF. All the routes are being properly distributed across the multiple sites.

frr/OSPF is configured on both OPNsense firewalls as described above. I'm not on a high availability situation, thus am not using CARP. But I tried to test whether OSPF would recalculate and update the default gateway metric when I disable the WAN interface or disconnect its connection to the ISP, and it isn't. The metric for the default gateway that lost connectivity stays the same.

I checked the external Link states on the router for site A, the one I disconnected, and it stays the same as it was before the disconnection. Waited 1, 2, 3... 5 minutes, nothing, no updates to the metric. None of the other devices received updates either. With one fully functioning firewall running OSPF on site B, and all other devices having continued connectivity to site B's firewall, there was no re-routing, no updates on the OSPF external LSA database, anywhere.

sho ip ospf  database ext 0.0.0.0

     OSPF Router with ID (<router-id-1>)

              AS External Link States 

LS age: 367
Options: 0x2  : *|-|-|-|-|-|E|-
LS Flags: 0x6  
LS Type: AS-external-LSA
Link State ID: 0.0.0.0 (External Network Number)
Advertising Router: <router-id-1>
LS Seq Number: 80000384
Checksum: 0x34b4
Length: 36

Network Mask: /0
      Metric Type: 2 (Larger than any link state path)
      TOS: 0
      Metric: 10
      Forward Address: default-gateway-ip-address-1
      External Route Tag: 0

LS age: 76
Options: 0x2  : *|-|-|-|-|-|E|-
LS Flags: 0xb  
LS Type: AS-external-LSA
Link State ID: 0.0.0.0 (External Network Number)
Advertising Router: router-id-2
LS Seq Number: 80000029
Checksum: 0xb162
Length: 36

Network Mask: /0
      Metric Type: 2 (Larger than any link state path)
      TOS: 0
      Metric: 10
      Forward Address: default-gateway-ip-address-2
      External Route Tag: 0

• [X ] Did you check if this is a duplicate issue?
• Did you test it on the latest FRRouting/frr master branch?

To Reproduce

Disconnected site A's fibers to the WAN interface on the OPNsense frr/OSPF firewall. Tried pings to 1.1.1.1, 8.8.8.8, and others, from site A's firewall, all timeout as expected. Displayed the ospf external LSAs on the site A firewall. No change was observed. Displayed the ospf external LSAs on other routers, no change.

Expected behavior

It was expected that we would first see changes to the OSPF routing table and the external LSAs in site A's frr/OSPF device, then slowly those updates should have propagated to all other routers. Propagation happens during firewall start-up, but not when the WAN interface is disabled, or the corresponding fibers pulled.

Versions

• OPERATING SYSTEM VERSION: OPNsense 23.1.7
• KERNEL VERSION: FREEBSD 13.1-RELEASE-p7 FreeBSD 13.1-RELEASE-p7 stable/23.1-n250430-7eb6eb035df SMP amd64
• FRR Version:7.5.1

Additional context
Since I am unsure whether this belongs in FRRouting/frr or OPNsense/plugins I'm informing both teams (see opnsense/plugins#3445). Please review the frr log I placed there, and the notes about when I disabled the WAN interface and frr/ospf didn't even flinch.

[kpiq]

This is an extract of site C's multilayer switch routing table.  After 5 minutes it remain unchanged.    Same thing happens to MLS's on sites A and B, no change.

sh ip route

                                 IP Route Entries

  Destination        Gateway         VLAN Type      Sub-Type   Metric     Dist.
  ------------------ --------------- ---- --------- ---------- ---------- -----
  0.0.0.0/0          10.253.0.4      1401 ospf      External2  10         110  
  10.0.0.0/8         10.253.0.4      1401 ospf      External2  20         110  
  2.3.4.5/30     10.253.0.4      1401 ospf      IntraArea  3          110  
  6.7.8.9/30     10.253.0.4      1401 ospf      IntraArea  4          110

IP addresses have been sanitized, but the last two are the ones that represent the default gateways on sites A and B respectively.

[mjstapp]

That 7.5.x release is quite old at this point - can you recreate the problem with a newer release?

[kpiq]

hmmm... I am depending on what OPNsense community edition supports right now. Let me research what it would take to run a newer release. Thanks for your attention. Will keep you posted.

[kpiq]

That 7.5.x release is quite old at this point - can you recreate the problem with a newer release?

I checked with an experienced member of the OPNsense forum. Contrary to pfSense, OPNsense doesn't have the option to install newer packages. I'm limited to what's available in their list of plugins, and that's 7.5.1_4 at this time. Any ideas?

[fichtner]

@kpiq you can install newer versions from the FreeBSD ports tree at any time but the way they manage a package for every major release is inconvenient as a drop-in replacement. Perhaps @mimugmail has time for FFR upgrade to target for 23.7?

[mimugmail]

I think so, yes. Can you publish both pkg's in the repo so -devel can point to frr8?

[kpiq]

should this be reason to worry?

on opnsense/tools@68a43aa:

"This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository."

[mimugmail]

No, it was a testing branch which was deleted since its now in master :)

[fichtner]

A version tag was misplaced so the master branch received a force-push to amend the situation. Change is still there but commit hash changed.

[kpiq]

A version tag was misplaced so the master branch received a force-push to amend the situation. Change is still there but commit hash changed.

Folks

I just upgraded my lab with OPNsense 23.7, will be testing and observing it for a week or two before proposing the upgrade to the production firewalls that were previously not succeeding to failover.

Appreciate all the hard work, your time, and attention. Will keep you posted.

Regards

Pedro

[github-actions]

This issue is stale because it has been open 180 days with no activity. Comment or remove the autoclose label in order to avoid having this issue closed.

[frrbot]

This issue will be automatically closed in the specified period unless there is further activity.