Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

isis_tilfa_topo1 rework #16187

Merged

Conversation

rwestphal
Copy link
Member

Summary of the changes:

  • Rework the IS-IS switchover test to be more reliable
  • Replace diff files with full JSON snapshots
  • Add method to update reference data by simply setting an environment variable: REGEN_DATA=true python3 ./test_isis_tilfa_topo1.py

Please see the commit messages for details.

Fixes #15704

@github-actions github-actions bot added the master label Jun 8, 2024
@donaldsharp
Copy link
Member

@rwestphal -> please rebase and force push. This will bring in the fix for the failed test.

In this topotest, steps 10-15 were added to test the IS-IS switchover
functionality. In short, two cases were tested: switchover after a
link down event and switchover after a BFD down event. Both cases
were tested in sequence on the same router, rt6. This involved the
following steps:
- Setting the SPF delay timer to 15 seconds
- Shutting down the eth-rt5 interface from the switch side
- Testing the post-switchover RIB and LIB (triggered by the link down
  event)
- Testing the post-SPF RIB and LIB
- Bringing the eth-rt5 interface back up
- Configuring a BFD session between rt6 and rt5
- Shutting down the eth-rt5 interface from the switch side once again
- Testing the post-switchover RIB and LIB (triggered by the BFD down
  event)
- Testing the post-SPF RIB and LIB

Since the time window to test the post-switchover RIB and LIB was too
narrow (10 seconds), these tests were having sporadic failures.

To resolve this problem, we can simplify the switchover test as follows:
- Setting the SPF delay timer to 60 seconds (not 15)
- Disabling "link-detect" on rt6's eth-rt5 interface
- Shutting down the eth-rt5 interface from the switch side
- On rt6, testing the post-switchover RIB and LIB (triggered by the
  BFD down event)
- On rt5, testing the post-switchover RIB and LIB (triggered by the
  link down event)

Notice how we can test both post-link-down and post-BFD-down switchover
cases simultaneously by having different "link-detect" configurations
on rt5 and rt6. Additionally, by using a larger SPF delay timer, the
time window to test the post-switchover RIB and LIB is much larger
and less prone to sporadic failures.

Signed-off-by: Renato Westphal <[email protected]>
The isis_tilfa_topo1 topotest is comprehensive and contains a large
amount of reference data. One problem is that, when changes occur,
updating this reference data can be difficult.

To address this problem, this commit introduces a method to
automatically regenerate the reference data by setting the `REGEN_DATA`
environment variable.

Usage:
$ REGEN_DATA=true python3 ./test_isis_tilfa_topo1.py

When `REGEN_DATA` is set, the topotest regenerates reference data
from the current run instead of comparing against existing reference
data. Note that regenerated data must be manually verified for
correctness.

This commit also simplifies the reference data by replacing all diff
files with complete JSON snapshots.

Signed-off-by: Renato Westphal <[email protected]>
@rwestphal rwestphal force-pushed the isis_tilfa_topo_rework branch from c5b8325 to dc354cb Compare June 10, 2024 15:28
@rwestphal
Copy link
Member Author

@rwestphal -> please rebase and force push. This will bring in the fix for the failed test.

@donaldsharp Done 🤞

@donaldsharp donaldsharp merged commit d0bdbd1 into FRRouting:master Jun 11, 2024
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

isis_tilfa_topo1 test has a timing issue
2 participants