[Solved] Basic Routing Not Working - Looking For Troubleshooting Ideas

[SweetSoccerMoves]

Hello Everyone,
I am new to FRR and am looking for assistance with understanding why my FRR topology is not working as expected. I will start with describing the topology and technology involved, describe the problem, and then provide details of my troubleshoot. Again, I am trying to learn this, so I apologize if my terminology or troubleshooting tactics are off.

The Topology and Technology:
• Three Alpine Linux Routers. Hub and Spoke Topology.
• Route Names:

• CE1-2-1
• PE1-1
• CE1-3-1
  • They're connected in the following way:
• CE1-2-1 --------- PE1-1 --------- CE1-3-1
• Each connection is a routed connection. CE1-2-1 to PE1-1 is 10.9.12.0/24. CE1-3-1 to PE1-1 is 10.9.13.0/24. A topology has been attached (FRRouting Issue - Basic Topology).
  • Each Router is running the following versions:
• Alpine Linux Version 3.16.2
• FRR Version 8.2.2

The Problem Description:
CE1-2-1 (10.9.12.2) cannot PING CE1-3-1 (10.9.13.2)

The Details:
This is a simple routing topology. CE1-2-1 and CE1-3-1 have IP addresses on their interfaces and a default gateway pointing to PE1-1. PE1-1 can PING it's own interfaces and both CE1-2-1 and CE1-3-1 (Picture attached to show (FRRouting Issue - PE1-1 PING)). CE1-2-1 can PING PE1-1's VLAN 118 interface (10.9.13.1) interface. And CE1-3-1 can PING PE1-1's VLAN 113 (10.9.12.1) interface. But trying to PING CE1-3-1 (10.9.13.2) from CE1-2-1 (10.9.12.2) does not work. From CE1-2-1, we can PING the VLAN 118 interface on PE1-1, but not the next hop router on the VLAN 118 subnet. Pictures of the topologies have been attached.
I've built this in an Ubuntu Docker environment and in VMware. Both result in this same issue.
I've made sure that the 'net.ipv4.conf.all.forwarding' is set to 1.

The Request:
Does anyone think my configurations are wrong? And does anyone have any tips on how to look at the traffic to find out where the communication breakdown is occurring? I do not know if CE1-2-1 and CE1-3-1 are even sending the ICMP traffic, or if PE1-1 is just not routing.

[AndreyBaycharov]

Try traceroute and tcpdump for diagnostic.
And show the routing table

[SweetSoccerMoves]

Hi,
Thanks for the ideas. I will have to post a tcpdump a bit later on. But here are the traceroute and route tables for these routers.

CE1-2-1:

PE1-1:

CE1-3-1:

[AndreyBaycharov]

Everything points to a problem in PE.
Check ip_forward on /etc/sysctl.conf or cat /proc/sys/net/ipv4/ip_forward and iptables (firewall)

[SweetSoccerMoves]

Thank you, Andrey. You pointed me in the right direction. I have updated the comments with the solution. This took a lot of research and understanding of iptables. But you got me on the right track. Thank you.

[SweetSoccerMoves]

Update: Issue has been resolved.

Solution: In the Alpine Linux VM, change 'iptables' FORWARD chain to default "ACCEPT" instead of "DROP"
Command: iptables --policy FORWARD ACCEPT

More details:
I ran the command "iptables -L -n -v" and noticed that the "FORWARD" chain was defaulting to "DROP". Below is that command (mine says "ACCEPT" because I have already corrected the issue.

The command "iptables --policy FORWARD ACCEPT" changed the default action to ACCEPT. This will not persist after a reboot. To make this persistent through a reboot, issue the following command twice. I don't know why twice. The first time creates a file. The second time saves a file. Odd, but twice works, once does not. I tried on all three routers.
/etc/init.d/iptables save
/etc/init.d/iptables save

Now CE1-2-1 can PING through PE1-1 to CE1-3-1 and vice versa.