-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bgpd, topotests: add bgp_rpki_topo1 and RPKI fixes #15034
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We had some discussions a couple of years ago about what RTR daemon we should use in CI. There were some candidates:
- Routinator
- GoRTR (StayRTR)
Having this custom Python implementation might be OK too (since we have the same with BMP), but I doubt if someone wants to spend some time implementing/fixing missing stuff in the future(?). The same is for BMP, but let's not judge here.
/cc @mwinter-osr
bgpd/bgp_rpki.c
Outdated
@@ -1326,15 +1326,18 @@ DEFPY (show_rpki_prefix_table, | |||
{ | |||
struct json_object *json = NULL; | |||
|
|||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: let's drop this empty line
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
|
||
def get_announcements4(self, serial=0): | ||
if serial > self.last_serial: | ||
return [(29134, "217.31.48.0/20", 20), (29134, "62.109.128.0/19", 19)] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are these ASNs and prefixes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changed to private TEST-2 and TEST-3 prefixes and to a private ASN
@@ -0,0 +1,271 @@ | |||
#!/usr/bin/python3 | |||
# | |||
# by Tomas Hlavacek ([email protected]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Btw, where did this script come from?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have added the source in comment
{ | ||
"prefixes":[ | ||
{ | ||
"prefix":"62.109.128.0", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please use private (or documentation) ranges and private ASNs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
ip address 192.168.1.2/24 | ||
! | ||
interface r2-eth1 | ||
ip address 192.168.2.2/24 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where is this used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed
@@ -0,0 +1,11 @@ | |||
! | |||
int lo | |||
ip address 192.0.2.2/32 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where is this used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed
8129be6
to
4bce1b2
Compare
Routinator and GoRTR are complex to set up. Maybe that is the reason nobody published an RPKI topotest. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
4bce1b2
to
0161058
Compare
@ton31337 |
0161058
to
fe1cc43
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good
@louis-6wind AFAIU we are waiting for relicensing (GPL3 -> GPL2)? |
Yes ... I've put the |
"show rpki XX json" should not return a void output because json.loads() considers it to be an incorrect JSON. > >>> json.loads("") > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/lib/python3.9/json/__init__.py", line 346, in loads > return _default_decoder.decode(s) > File "/usr/lib/python3.9/json/decoder.py", line 337, in decode > obj, end = self.raw_decode(s, idx=_w(s, 0).end()) > File "/usr/lib/python3.9/json/decoder.py", line 355, in raw_decode > raise JSONDecodeError("Expecting value", s, err.value) from None > json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) > >>> json.loads("{}") > {} Return "{}" instead in such a case. Link: FRRouting#15034 Fixes: dff41cc ("bgpd: Add JSON output for `show rpki prefix` and other show commands") Signed-off-by: Louis Scalbert <[email protected]>
"show rpki XX json" should not return a void output because json.loads() considers it to be an incorrect JSON. > >>> json.loads("") > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/lib/python3.9/json/__init__.py", line 346, in loads > return _default_decoder.decode(s) > File "/usr/lib/python3.9/json/decoder.py", line 337, in decode > obj, end = self.raw_decode(s, idx=_w(s, 0).end()) > File "/usr/lib/python3.9/json/decoder.py", line 355, in raw_decode > raise JSONDecodeError("Expecting value", s, err.value) from None > json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) > >>> json.loads("{}") > {} Return "{}" instead in such a case. Link: FRRouting#15034 Fixes: dff41cc ("bgpd: Add JSON output for `show rpki prefix` and other show commands") Signed-off-by: Louis Scalbert <[email protected]>
Add bgp_rpki_topo1 topotest to validate the RPKI feature. Use a RTR RPKI server from the above link. Link: FRRouting#15034 Link: https://raw.githubusercontent.com/SmartValidator/rtr-python/368a4e268aaae0de7d88d0c34665315661851d40/rtrd.py Signed-off-by: Louis Scalbert <[email protected]>
Hi! I am the original author of https://github.com/SmartValidator/rtr-python/. This part is now abandonware and there is a history of intellectual property disputes that ultimately contributed to shutdown of SmartValidator project and withdrawal of some parts from public repos. Since I needed this RTR mockup server for other purposes I created re-implementation and based on this re-licensing question I released it under GPL 2 here: It is not 1:1 equivalent - the main difference is that the current version expects to have vrps.csv file with real or synthetic VRPS and it loads and serves this content. Please see README for more details. |
I guess comment from SmartValidator/rtr-python#2 (comment) means that RTRD implementation should be refreshed in the FRR repository. |
Marked as |
Thank you for @tmshlvck for the new GPL-2 compatible version |
"show rpki XX json" should not return a void output because json.loads() considers it to be an incorrect JSON. > >>> json.loads("") > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/lib/python3.9/json/__init__.py", line 346, in loads > return _default_decoder.decode(s) > File "/usr/lib/python3.9/json/decoder.py", line 337, in decode > obj, end = self.raw_decode(s, idx=_w(s, 0).end()) > File "/usr/lib/python3.9/json/decoder.py", line 355, in raw_decode > raise JSONDecodeError("Expecting value", s, err.value) from None > json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) > >>> json.loads("{}") > {} Return "{}" instead in such a case. Link: FRRouting#15034 Fixes: dff41cc ("bgpd: Add JSON output for `show rpki prefix` and other show commands") Signed-off-by: Louis Scalbert <[email protected]>
b3fa113
to
0dc5ff2
Compare
@@ -0,0 +1 @@ | |||
bgp_table_rpki_valid.json |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Means symbolic link (used in the route map test)
$ ls -l tests/topotests/bgp_rpki_topo1/r2/bgp_table_rmap_rpki_valid.json
lrwxrwxrwx 1 user users 25 Jan 11 14:17 tests/topotests/bgp_rpki_topo1/r2/bgp_table_rmap_rpki_valid.json -> bgp_table_rpki_valid.json
$ git diff HEAD^ --name-only
tests/topotests/bgp_rpki_topo1/__init__.py
tests/topotests/bgp_rpki_topo1/r1/bgpd.conf
tests/topotests/bgp_rpki_topo1/r1/rtrd.py
tests/topotests/bgp_rpki_topo1/r1/staticd.conf
tests/topotests/bgp_rpki_topo1/r1/vrps.csv
tests/topotests/bgp_rpki_topo1/r1/zebra.conf
tests/topotests/bgp_rpki_topo1/r2/bgp_table_rmap_rpki_any.json
tests/topotests/bgp_rpki_topo1/r2/bgp_table_rmap_rpki_notfound.json
tests/topotests/bgp_rpki_topo1/r2/bgp_table_rmap_rpki_valid.json
tests/topotests/bgp_rpki_topo1/r2/bgp_table_rpki_any.json
tests/topotests/bgp_rpki_topo1/r2/bgp_table_rpki_notfound.json
tests/topotests/bgp_rpki_topo1/r2/bgp_table_rpki_valid.json
tests/topotests/bgp_rpki_topo1/r2/bgpd.conf
tests/topotests/bgp_rpki_topo1/r2/rpki_prefix_table.json
tests/topotests/bgp_rpki_topo1/r2/staticd.conf
tests/topotests/bgp_rpki_topo1/r2/zebra.conf
tests/topotests/bgp_rpki_topo1/test_bgp_rpki_topo1.py
bgpd/bgp_rpki.c
Outdated
vty_out(vty, "rpki is %s", | ||
listcount(cache_list) ? "Enabled" : "Disabled"); | ||
|
||
if (list_isempty(cache_list)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the list is empty (= disabled) we need to print a new line before returning.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
return 1; | ||
} | ||
if ((rpki_debug_conf || rpki_debug_term) && !running) { | ||
vty_out(vty, " BGP RPKI debugging is on\n"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should ensure this is omitted if we turn off no debug rpki
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now OK
r2# debug rpki
r2# show debugging
MGMT debugging status:
Zebra debugging status:
BGP debugging status:
BGP RPKI debugging is on
Staticd debugging status
r2# no debug rpki
r2# show debugging
MGMT debugging status:
Zebra debugging status:
BGP debugging status:
Staticd debugging status
json = json_object_new_object(); | ||
json_object_boolean_add(json, "enabled", | ||
!!listcount(cache_list)); | ||
json_object_int_add(json, "serversCount", listcount(cache_list)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be handy to list a cache server that is connected too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can do show rpki cache-connection
The command is for displaying the configured timers
bgpd/bgp_rpki.c
Outdated
@@ -1688,7 +1788,10 @@ DEFUN (no_debug_rpki, | |||
DEBUG_STR | |||
"Disable debugging for rpki\n") | |||
{ | |||
rpki_debug = false; | |||
if (vty->node == CONFIG_NODE) | |||
rpki_debug_conf = true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be false.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
bgpd/bgp_rpki.c
Outdated
if (vty->node == CONFIG_NODE) | ||
rpki_debug_conf = true; | ||
else | ||
rpki_debug_term = true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be false.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
"routerId": "192.0.2.2", | ||
"defaultLocPrf": 100, | ||
"localAS": 65002, | ||
"routes": { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe this is always a true match. I mean despite there being any routes under routes
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do that to avoid the issue:
expected_nb = len(expected.get("routes"))
output_nb = len(output.get("routes", {}))
if expected_nb != output_nb:
return {"error": "expected {} prefixes. Got {}".format(expected_nb, output_nb)}
"show rpki XX json" should not return a void output because json.loads() considers it to be an incorrect JSON. > >>> json.loads("") > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/lib/python3.9/json/__init__.py", line 346, in loads > return _default_decoder.decode(s) > File "/usr/lib/python3.9/json/decoder.py", line 337, in decode > obj, end = self.raw_decode(s, idx=_w(s, 0).end()) > File "/usr/lib/python3.9/json/decoder.py", line 355, in raw_decode > raise JSONDecodeError("Expecting value", s, err.value) from None > json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) > >>> json.loads("{}") > {} Return "{}" instead in such a case. Link: FRRouting#15034 Fixes: dff41cc ("bgpd: Add JSON output for `show rpki prefix` and other show commands") Signed-off-by: Louis Scalbert <[email protected]>
Add error messages to rpki JSON output instead of an empty JSON. Signed-off-by: Louis Scalbert <[email protected]>
Fix RPKI module compilation when rtrlib is compiled without SSH support, ie. with cmake option: > -D RTRLIB_TRANSPORT_SSH=No > bgpd/bgp_rpki.c: In function ‘config_write’: > bgpd/bgp_rpki.c:1062:3: error: enumeration value ‘SSH’ not handled in switch [-Werror=switch-enum] > 1062 | switch (cache->type) { > | ^~~~~~ > bgpd/bgp_rpki.c: In function ‘show_rpki_cache_connection_magic’: > bgpd/bgp_rpki.c:1598:3: error: enumeration value ‘SSH’ not handled in switch [-Werror=switch-enum] > 1598 | switch (cache->type) { > | ^~~~~~ > cc1: all warnings being treated as errors Signed-off-by: Louis Scalbert <[email protected]>
Add bgp_rpki_topo1 topotest to validate the RPKI feature. Use a RTR RPKI server from the above link with a black cleaning. Link: https://raw.githubusercontent.com/tmshlvck/pyrtr/90df586375396aae08b07069187308b5b7b8823b/pyrtr/__init__.py Signed-off-by: Louis Scalbert <[email protected]>
Log bgp_rpki_topo1 pyrtr output Signed-off-by: Louis Scalbert <[email protected]>
RPKI FRR module should not send any RPKI error packet during the tests. Exit rtrd when receiving error packet. Skip tests with errors if rtrd has stopped. Signed-off-by: Louis Scalbert <[email protected]>
Fix a crash when re-adding a rpki server: > r2# sh run bgpd > [...] > rpki > rpki retry_interval 5 > rpki cache 192.0.2.1 15432 preference 1 > exit > [...] > r2# conf t > r2(config)# rpki > r2(config-rpki)# no rpki cache 192.0.2.1 15432 preference 1 > r2(config-rpki)# do show rpki cache-connection > Cannot find a connected group. > r2(config-rpki)# rpki cache 192.0.2.1 15432 preference 1 > r2(config-rpki)# do show rpki cache-connection > vtysh: error reading from bgpd: Resource temporarily unavailable (11)Warning: closing connection to bgpd because of an I/O error! > #0 raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:50 > #1 0x00007f3fd2d16e57 in core_handler (signo=11, siginfo=0x7ffffd5931b0, context=0x7ffffd593080) at lib/sigevent.c:246 > #2 <signal handler called> > #3 0x00007f3fd26926b4 in tommy_list_head (list=0x2e322e302e323931) at /home/lscalber/git/rtrlib/./third-party/tommyds/tommylist.h:125 > FRRouting#4 0x00007f3fd2693812 in rtr_mgr_get_first_group (config=0x55fbf31d7f00) at /home/lscalber/git/rtrlib/rtrlib/rtr_mgr.c:409 > FRRouting#5 0x00007f3fd2ebef59 in get_connected_group () at bgpd/bgp_rpki.c:718 > FRRouting#6 0x00007f3fd2ec0b39 in show_rpki_cache_connection_magic (self=0x7f3fd2ec69c0 <show_rpki_cache_connection_cmd>, vty=0x55fbf31f9ef0, argc=3, argv=0x55fbf31f99d0, uj=0x0) > # at bgpd/bgp_rpki.c:1575 > FRRouting#7 0x00007f3fd2ebd4da in show_rpki_cache_connection (self=0x7f3fd2ec69c0 <show_rpki_cache_connection_cmd>, vty=0x55fbf31f9ef0, argc=3, argv=0x55fbf31f99d0) at ./bgpd/bgp_rpki_clippy.c:648 > FRRouting#8 0x00007f3fd2c8a142 in cmd_execute_command_real (vline=0x55fbf31f9990, vty=0x55fbf31f9ef0, cmd=0x0, up_level=0) at lib/command.c:978 > FRRouting#9 0x00007f3fd2c8a25c in cmd_execute_command (vline=0x55fbf31e5260, vty=0x55fbf31f9ef0, cmd=0x0, vtysh=0) at lib/command.c:1028 > FRRouting#10 0x00007f3fd2c8a7f1 in cmd_execute (vty=0x55fbf31f9ef0, cmd=0x55fbf3200680 "do show rpki cache-connection ", matched=0x0, vtysh=0) at lib/command.c:1203 > FRRouting#11 0x00007f3fd2d36548 in vty_command (vty=0x55fbf31f9ef0, buf=0x55fbf3200680 "do show rpki cache-connection ") at lib/vty.c:594 > FRRouting#12 0x00007f3fd2d382e1 in vty_execute (vty=0x55fbf31f9ef0) at lib/vty.c:1357 > FRRouting#13 0x00007f3fd2d3a519 in vtysh_read (thread=0x7ffffd5963c0) at lib/vty.c:2365 > FRRouting#14 0x00007f3fd2d2faf6 in event_call (thread=0x7ffffd5963c0) at lib/event.c:1974 > FRRouting#15 0x00007f3fd2cc238e in frr_run (master=0x55fbf2a0cd60) at lib/libfrr.c:1214 > FRRouting#16 0x000055fbf073de40 in main (argc=9, argv=0x7ffffd596618) at bgpd/bgp_main.c:510 Signed-off-by: Louis Scalbert <[email protected]>
RPKI configuration is not totally flushed when doing "no rpki". Timers remains to default values. > r2# sh run bgpd > [...] > rpki > rpki retry_interval 5 > rpki cache 192.0.2.1 15432 preference 1 > exit > [...] > r2# conf t > r2(config)# no rpki > r2(config)# do sh run > [...] > rpki > rpki retry_interval 5 > exit Reset the timers after doing "no rpki" Signed-off-by: Louis Scalbert <[email protected]>
remove double spaces when doing show running-config. Signed-off-by: Philippe Guibert <[email protected]> Signed-off-by: Louis Scalbert <[email protected]>
"show run" displays the default RPKI timers when at least one cache server is configured. Only display the RPKI timers that differs from the default values. Signed-off-by: Philippe Guibert <[email protected]> Signed-off-by: Louis Scalbert <[email protected]>
Add documentation about the new "show rpki configuration" command. Signed-off-by: Louis Scalbert <[email protected]>
when a plugin is attached, some debugs may be attached to that plugin. For that, add one hook that is interacting with vty: a boolean indicates what the usage is for: either for impacting the 'show running-config', or for impacting the 'show debugging' command. Signed-off-by: Philippe Guibert <[email protected]> Signed-off-by: Louis Scalbert <[email protected]>
Only include "debug rpki" in "show run" if it was requested from the configure mode but not it was from the enabled mode. Signed-off-by: Philippe Guibert <[email protected]> Signed-off-by: Louis Scalbert <[email protected]>
0dc5ff2
to
93f05b0
Compare
CI:rerun Retest for new ASAN Memory Leaks |
https://github.com/Mergifyio backport stable/9.1 |
✅ Backports have been created
|
"show rpki XX json" should not return a void output because json.loads() considers it to be an incorrect JSON. > >>> json.loads("") > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/lib/python3.9/json/__init__.py", line 346, in loads > return _default_decoder.decode(s) > File "/usr/lib/python3.9/json/decoder.py", line 337, in decode > obj, end = self.raw_decode(s, idx=_w(s, 0).end()) > File "/usr/lib/python3.9/json/decoder.py", line 355, in raw_decode > raise JSONDecodeError("Expecting value", s, err.value) from None > json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) > >>> json.loads("{}") > {} Return "{}" instead in such a case. Link: #15034 Fixes: dff41cc ("bgpd: Add JSON output for `show rpki prefix` and other show commands") Signed-off-by: Louis Scalbert <[email protected]> (cherry picked from commit 4011682)
"show rpki XX json" should not return a void output because json.loads() considers it to be an incorrect JSON. > >>> json.loads("") > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/lib/python3.9/json/__init__.py", line 346, in loads > return _default_decoder.decode(s) > File "/usr/lib/python3.9/json/decoder.py", line 337, in decode > obj, end = self.raw_decode(s, idx=_w(s, 0).end()) > File "/usr/lib/python3.9/json/decoder.py", line 355, in raw_decode > raise JSONDecodeError("Expecting value", s, err.value) from None > json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) > >>> json.loads("{}") > {} Return "{}" instead in such a case. Link: FRRouting#15034 Fixes: dff41cc ("bgpd: Add JSON output for `show rpki prefix` and other show commands") Signed-off-by: Louis Scalbert <[email protected]>
Add bgp_rpki_topo1 topotest to validate the RPKI feature. Use the RPKI pyrtr GPL-2.0 daemon
Misc fixes for the RPKI module.
This is a preparatory step to re-add feature for #14220