firewall: T5493: Implement remote-group

[Embezzle]

Change summary

Initial implementation of firewall remote-group capabilities. This PR adds the 'remote-group' tag-node to the firewall, which utilises the vyos-domain-resolver to fetch & load a list of IPv4 networks/address's from an external URL.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Code style update (formatting, renaming)
• Refactoring (no functional changes)
• Migration from an old Vyatta component to vyos-1x, please link to related PR inside obsoleted component
• Other (please describe):

Related Task(s)

https://vyos.dev/T5493

Related PR(s)

How to test / Smoketest result

Test Steps

1. Configure a firewall remote-group:

set firewall group remote-group externallist url http://127.0.0.1/ip-list.txt

2. Check entries have been loaded into NFT correctly:

vyos@vyos# sudo nft list ruleset | grep 'R_externallist' -A 12
	set R_externallist {
		type ipv4_addr
		flags interval
		elements = { 127.0.0.1/32, 192.168.1.0/24 }
	}
--
	set R_externallist {
		type ipv4_addr
		flags interval
		elements = { 127.0.0.1/32, 192.168.1.0/24 }
	}
--
	set R_externallist {
		type ipv4_addr
		flags interval
	}

Smoke Test

vyos@vyos:~$ python3 /usr/libexec/vyos/tests/smoke/cli/test_firewall.py
test_bridge_firewall (__main__.TestFirewall.test_bridge_firewall) ... ok
test_cyclic_jump_validation (__main__.TestFirewall.test_cyclic_jump_validation) ... ok
test_flow_offload (__main__.TestFirewall.test_flow_offload) ... ok
test_geoip (__main__.TestFirewall.test_geoip) ... ok
test_gre_match (__main__.TestFirewall.test_gre_match) ... ok
test_groups (__main__.TestFirewall.test_groups) ... ok
test_ipsec_metadata_match (__main__.TestFirewall.test_ipsec_metadata_match) ... ok
test_ipv4_advanced (__main__.TestFirewall.test_ipv4_advanced) ... ok
test_ipv4_basic_rules (__main__.TestFirewall.test_ipv4_basic_rules) ... ok
test_ipv4_dynamic_groups (__main__.TestFirewall.test_ipv4_dynamic_groups) ... ok
test_ipv4_global_state (__main__.TestFirewall.test_ipv4_global_state) ... ok
test_ipv4_mask (__main__.TestFirewall.test_ipv4_mask) ... ok
test_ipv4_remote_group (__main__.TestFirewall.test_ipv4_remote_group) ... ok
test_ipv4_state_and_status_rules (__main__.TestFirewall.test_ipv4_state_and_status_rules) ... ok
test_ipv4_synproxy (__main__.TestFirewall.test_ipv4_synproxy) ... ok
test_ipv6_advanced (__main__.TestFirewall.test_ipv6_advanced) ... ok
test_ipv6_basic_rules (__main__.TestFirewall.test_ipv6_basic_rules) ... ok
test_ipv6_dynamic_groups (__main__.TestFirewall.test_ipv6_dynamic_groups) ... ok
test_ipv6_mask (__main__.TestFirewall.test_ipv6_mask) ... ok
test_nested_groups (__main__.TestFirewall.test_nested_groups) ... ok
test_source_validation (__main__.TestFirewall.test_source_validation) ... ok
test_sysfs (__main__.TestFirewall.test_sysfs) ... ok
test_timeout_sysctl (__main__.TestFirewall.test_timeout_sysctl) ... ok
test_zone_basic (__main__.TestFirewall.test_zone_basic) ... ok
test_zone_flow_offload (__main__.TestFirewall.test_zone_flow_offload) ... ok
test_zone_with_vrf (__main__.TestFirewall.test_zone_with_vrf) ... ok

----------------------------------------------------------------------
Ran 26 tests in 88.646s

OK

Checklist:

• I have read the CONTRIBUTING document
• I have linked this PR to one or more Phabricator Task(s)
• I have run the components SMOKETESTS if applicable
• My commit headlines contain a valid Task id
• My change requires a change to the documentation
• I have updated the documentation accordingly

[github-actions]

👍
No issues in PR Title / Commit Title

[sever-sever]

Which format is expected for the "externallist" ?

01

192.0.2.0/24
203.0.113.0/24

02

192.0.2.0-192.0.2.5
10.0.0.1-10.0.5.2
203.0.113.95
0.0.0.0
example.com

03

# foo
192.0.2.1
### bar
192.0.2.2

04

; Spamhaus DROP List 2025/01/30 - (c) 2025 The Spamhaus Project SLU
; https://www.spamhaus.org/drop/drop.txt
; Last-Modified: Thu, 30 Jan 2025 18:25:19 GMT
; Expires: Thu, 30 Jan 2025 19:37:54 GMT
1.10.16.0/20 ; SBL256894
1.19.0.0/16 ; SBL434604
1.32.128.0/18 ; SBL286275
2.56.192.0/22 ; SBL459831
2.57.122.0/24 ; SBL636050
2.57.168.0/24 ; SBL628448
2.57.232.0/23 ; SBL538946
2.57.234.0/23 ; SBL538947
5.42.92.0/24 ; SBL625300

Or something else? If some list contains my IP ASN addresses or some bogons' addresses, can I skip them?

[Embezzle]

@sever-sever in my latest commit all of the list format examples you gave would load correctly, however the FQDN entry in this example would be skipped.

192.0.2.0-192.0.2.5
10.0.0.1-10.0.5.2
203.0.113.95
0.0.0.0
example.com

The remote-group will accepts text-list's containing any mix of IPv4 addresses, CIDR’s, or ranges (127.0.0.1, 127.0.1.0/24, 127.0.2.5-127.0.2.10) delimitated by line breaks. We could add support in the future to resolve the IPv4 addresses of FQDN entries specified in lists.

[sarthurdev]

Some nitpicking, but otherwise a good PR. Thanks!

[Embezzle]

Some nitpicking, but otherwise a good PR. Thanks!

Thanks for the feedback, I've updated my commit as per your suggestions.

[github-actions]

CI integration ❌ failed!

Details

CI logs

• CLI Smoketests (no interfaces) 👍 passed
• CLI Smoketests (interfaces only) ❌ failed
• Config tests 👍 passed
• RAID1 tests 👍 passed
• TPM tests 👍 passed