L3VPN with strongswan IPSec and XFRM interface

[nitinraj92]

Hi All,
I am trying to bringup a site-site L3VPN with strongswan ipsec using xfrm interface.
I am using the xfrm section in here as reference.
https://docs.strongswan.org/docs/5.9/features/routeBasedVpn.html
But instead of static routes, I want bgp routes installed via frr to have the xfrm interface( xfrm_wan0_17) as nexthop instead of physical interface(wan0).

root@rtr-986aa800:/home/lavelle# ip link show xfrm_wan0_17
36: xfrm_wan0_17@wan0: <NOARP,UP,LOWER_UP> mtu 1600 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/none

Attaching the frr configuration as reference:

router bgp 64501
 bgp router-id 5.5.5.5
 no bgp ebgp-requires-policy
 neighbor 4.4.4.4 remote-as 64501
 neighbor 4.4.4.4 update-source lo0
 neighbor 6.6.6.6 remote-as 64501
 neighbor 6.6.6.6 update-source lo0
 !
 address-family ipv4 vpn
  neighbor 4.4.4.4 activate
  neighbor 6.6.6.6 activate
 exit-address-family
exit
!
router bgp 64501 vrf vrf-green
 bgp router-id 192.168.20.10
 no bgp ebgp-requires-policy
 neighbor 192.168.20.10 remote-as 63401
 neighbor 192.168.20.10 update-source lan0.100
 !
 address-family ipv4 unicast
  network 192.168.20.0/24
  redistribute connected
  label vpn export auto
  rd vpn export 100:100
  rt vpn both 100:100
  export vpn
  import vpn
 exit-address-family
exit
!
router bgp 64501 vrf vrf-black
 bgp router-id 192.168.21.10
 no bgp ebgp-requires-policy
 neighbor 192.168.21.10 remote-as 63411
 neighbor 192.168.21.10 update-source lan0.200
 !
 address-family ipv4 unicast
  network 192.168.21.0/24
  redistribute connected
  label vpn export auto
  rd vpn export 100:200
  rt vpn both 100:200
  export vpn
  import vpn
 exit-address-family
exit
!
router ospf
 ospf router-id 5.5.5.5
 network 0.0.0.0/0 area 0
exit
!
mpls ldp
 router-id 5.5.5.5
 neighbor 1.1.1.1 password test
 !
 address-family ipv4
  discovery transport-address 5.5.5.5
  label local allocate host-routes
  !
  interface wan0
  exit
  !
 exit-address-family
 !
exit
!

The routes currently installed using frr:

root@rtr-986aa800:/home/lavelle# ip route show
1.1.1.1 nhid 82 via 172.16.4.1 dev wan0 proto ospf metric 20
2.2.2.2 nhid 83  encap mpls  17 via 172.16.4.1 dev wan0 proto ospf metric 20
3.3.3.3 nhid 84  encap mpls  20 via 172.16.4.1 dev wan0 proto ospf metric 20
4.4.4.4 nhid 85  encap mpls  19 via 172.16.4.1 dev wan0 proto ospf metric 20
6.6.6.6 nhid 86  encap mpls  21 via 172.16.4.1 dev wan0 proto ospf metric 20
10.5.0.0/24 nhid 81 via 172.16.4.1 dev wan0 proto ospf metric 20
169.254.100.0/24 dev s5.mgmt proto kernel scope link src 169.254.100.1
172.16.1.0/30 nhid 81 via 172.16.4.1 dev wan0 proto ospf metric 20
172.16.2.0/30 nhid 81 via 172.16.4.1 dev wan0 proto ospf metric 20
172.16.3.0/30 nhid 81 via 172.16.4.1 dev wan0 proto ospf metric 20
172.16.4.0/30 dev wan0 proto kernel scope link src 172.16.4.2
172.16.5.0/30 nhid 81 via 172.16.4.1 dev wan0 proto ospf metric 20
172.16.6.0/30 nhid 81 via 172.16.4.1 dev wan0 proto ospf metric 20
root@rtr-986aa800:/home/lavelle# ip route show vrf vrf-green
unreachable default metric 4278198272
10.10.10.0/24 nhid 99  encap mpls  19/80 via 172.16.4.1 dev wan0 proto bgp metric 20
20.20.20.0/24 nhid 69 via 192.168.20.10 dev lan0.100 proto bgp metric 20
30.30.30.0/24 nhid 93  encap mpls  21/80 via 172.16.4.1 dev wan0 proto bgp metric 20
192.168.10.0/24 nhid 99  encap mpls  19/80 via 172.16.4.1 dev wan0 proto bgp metric 20
192.168.20.0/24 dev lan0.100 proto kernel scope link src 192.168.20.1
192.168.30.0/24 nhid 93  encap mpls  21/80 via 172.16.4.1 dev wan0 proto bgp metric 20
root@rtr-986aa800:/home/lavelle# ip route show vrf vrf-black
unreachable default metric 4278198272
11.11.11.0/24 nhid 101  encap mpls  19/81 via 172.16.4.1 dev wan0 proto bgp metric 20
21.21.21.0/24 nhid 70 via 192.168.21.10 dev lan0.200 proto bgp metric 20
31.31.31.0/24 nhid 95  encap mpls  21/81 via 172.16.4.1 dev wan0 proto bgp metric 20
192.168.11.0/24 nhid 101  encap mpls  19/81 via 172.16.4.1 dev wan0 proto bgp metric 20
192.168.21.0/24 dev lan0.200 proto kernel scope link src 192.168.21.1
192.168.31.0/24 nhid 95  encap mpls  21/81 via 172.16.4.1 dev wan0 proto bgp metric 20
root@rtr-986aa800:/home/lavelle#
root@rtr-986aa800:/home/lavelle# ip -f mpls route show
16 via inet 172.16.4.1 dev wan0 proto ldp
17 as to 17 via inet 172.16.4.1 dev wan0 proto ldp
18 as to 20 via inet 172.16.4.1 dev wan0 proto ldp
19 as to 19 via inet 172.16.4.1 dev wan0 proto ldp
20 as to 21 via inet 172.16.4.1 dev wan0 proto ldp
80 dev vrf-green proto bgp
81 dev vrf-black proto bgp

In these routes, if nexthop is present as xfrm_wan0_17, then the xfrm has policies installed to encrypt the traffic.

root@rtr-986aa800:/home/lavelle# ip xfrm policy
src 0.0.0.0/0 dst 0.0.0.0/0
        dir out priority 1
        tmpl src 172.16.4.2 dst 172.16.3.2
                proto esp spi 0xcf1b3dac reqid 7 mode tunnel
        if_id 0x11
src 0.0.0.0/0 dst 0.0.0.0/0
        dir fwd priority 1
        tmpl src 172.16.3.2 dst 172.16.4.2
                proto esp reqid 7 mode tunnel
        if_id 0x11
src 0.0.0.0/0 dst 0.0.0.0/0
        dir in priority 1
        tmpl src 172.16.3.2 dst 172.16.4.2
                proto esp reqid 7 mode tunnel
        if_id 0x11
src 0.0.0.0/0 dst 0.0.0.0/0
        socket in priority 0
src 0.0.0.0/0 dst 0.0.0.0/0
        socket out priority 0
src 0.0.0.0/0 dst 0.0.0.0/0
        socket in priority 0
src 0.0.0.0/0 dst 0.0.0.0/0
        socket out priority 0
src ::/0 dst ::/0
        socket in priority 0
src ::/0 dst ::/0
        socket out priority 0
src ::/0 dst ::/0
        socket in priority 0
src ::/0 dst ::/0
        socket out priority 0
root@rtr-986aa800:/home/lavelle# ip xfrm state
src 172.16.4.2 dst 172.16.3.2
        proto esp spi 0xcf1b3dac reqid 7 mode tunnel
        replay-window 0 flag nopmtudisc af-unspec
        aead rfc7539esp(chacha20,poly1305) 0x17993c64efe0d5e038f012252dcc4547f541d00f5514f5774658d52f84ce723aa55d3444 128
        encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
        if_id 0x11
src 172.16.3.2 dst 172.16.4.2
        proto esp spi 0xc1b48a5e reqid 7 mode tunnel
        replay-window 32 flag nopmtudisc af-unspec
        aead rfc7539esp(chacha20,poly1305) 0x23b5b08fccc58187c3cd21b7758bfbfc6bcbd83ec51aeaa33d449ea37cb3adebced75f2b 128
        encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
        if_id 0x11



root@rtr-986aa800:/home/lavelle# swanctl --list-sas
IKE_CONN_ROUTER_TO_SDWAN_num_0: #21, ESTABLISHED, IKEv2, d84cfcf2e1277fa7_i 78c231d428e24e56_r*
  local  'router1.lavelle' @ 172.16.4.2[4500]
  remote 'router1.lavelle' @ 172.16.3.2[4500]
  AES_CBC-128/HMAC_SHA2_256_128/PRF_AES128_XCBC/ECP_256
  established 57s ago, rekeying in 183s
  IPSEC_CONN_ROUTER_TO_SDWAN_xfrm_94338: #35, reqid 7, INSTALLED, TUNNEL-in-UDP, ESP:CHACHA20_POLY1305
    installed 57s ago, rekeying in 2343s
    in  c1b48a5e (-|0x00000011),      0 bytes,     0 packets
    out cf1b3dac (-|0x00000011),      0 bytes,     0 packets
    local  0.0.0.0/0
    remote 0.0.0.0/0
root@rtr-986aa800:/home/lavelle# swanctl --list-conns
IKE_CONN_ROUTER_TO_SDWAN_num_0: IKEv2, no reauthentication, rekeying every 240s, dpd delay 10s
  local:  172.16.4.2
  remote: 172.16.3.2
  local pre-shared key authentication:
    id: router1.lavelle
  remote pre-shared key authentication:
    id: router1.lavelle
  IPSEC_CONN_ROUTER_TO_SDWAN_xfrm_94338: TUNNEL, rekeying every 2400s, dpd action is clear
    local:  0.0.0.0/0
    remote: 0.0.0.0/0
    priority: 1

Let me know if we have some configuration if frr to modify the route nexthop interface. Let me know if there is some other way using PBR or route-maps I can achieve this.