Added connection tracking test plan.

- Added conntrack.md - Updated existing vnet.md and eni.md - Added link to the test plans in dash-sonic-hld.md Signed-off-by: Anton Putria <[email protected]>
sonic-net · Sep 27, 2022 · d65a77d · d65a77d
1 parent 4709e4c
commit d65a77d
Show file tree

Hide file tree

Showing 6 changed files with 137 additions and 34 deletions.
diff --git a/documentation/general/dash-sonic-hld.md b/documentation/general/dash-sonic-hld.md
@@ -625,7 +625,13 @@ The following commands shall be added :
 
 ## 3.5 Test Plan
 
-Refer DASH documentation for the test plan. 
+DASH testing shall include at least two test levels:
+1. SAI-level testing
+1. SONiC-level testing
+
+SAI-level testing includes:
+- SAI underlay API verification (to add test plan)
+- DASH Overlay API verification - [Overlay test plan](https://github.com/Azure/DASH/tree/main/test/docs/testplans/overlay.md)
 
 ## 3.6 Example configuration
 

diff --git a/test/docs/testplans/README.md b/test/docs/testplans/README.md
@@ -4,6 +4,8 @@
 |----------|-------------|
 | [Overlay Test Plans](./overlay.md) | Home page of all overlay test plans. |
 | [ENI Test Plan](./eni.md) | Detailed ENI test plan. |  
+| [Connection tracking](./conntrack.md) | Detailed connection tracking test plan. |
+| [VNET to VNET](./vnet.md) | Detailed VNET to VNET test plan. |
 
 
 You can start with the [Overlay Test Plans](./overlay.md). 
diff --git a/test/docs/testplans/conntrack.md b/test/docs/testplans/conntrack.md
@@ -0,0 +1,85 @@
+# Table of content
+
+1. [Objectives](#objectives)
+2. [Requirements](#requirements)
+3. [Automation](#automation)
+4. [Test Suites](#test-suites)
+    - [Basic](#eni-creation)
+    - [Ageing](#eni-removal)
+    - [Performance](#eni-scale)
+
+---
+
+# Objectives
+
+Verify proper functioning of the connection tracking mechanism: establishing, handling, closing connections.
+Connection per Second (CPS) is the most important attribute of the DASH products.
+
+# Requirements
+
+| Item |	Expected value
+|---|---
+| Active Connections/ENI | 1M (Bidirectional)
+| CPS per card | 4M+
+
+**To clarify**
+1. Ageing time
+
+# Automation
+
+Test cases are automated:
+1. Functional - using SAI PTF test framework.
+1. Scale/Performance - to be defined.
+
+## Automation notes
+
+1. Each test should be executed using TCP and UDP streams.
+1. Each test should be executed for Inbound and Outbound routing scenario.
+1. So far there is **no direct API to get active connection number** or CPS. Verification might be done using indirect ways:
+    -  Based on packets forwarding - check whether packets are passed or dropped
+    - Based on routing and ACL counters that are incremented on slow path. So when connection is established the counters should not be incremented.
+
+# Test suites
+
+## Basic
+
+| # | Test case | Test Class.Method
+| --- | --- | ---
+| 1 | Basic positive TCP session verification with counters check | -
+| 2 | Basic positive UDP session verification with counters check | -
+| 3 | TCP session verification with fragmented packets | -
+| 4 | UDP session verification with fragmented packets | -
+| 5 | ICMP traffic. **to clarify** no session is expected? | -
+| 6 | Same overlay IPs but different ENIs | -
+
+## Ageing
+
+| # | Test case | Test Class.Method
+| --- | --- | ---
+| 1 | Standard ageing (fully correct TCP session). | -
+| 2 | Standard ageing (fully correct UDP session). | -
+| 3 | Open TCP session but no data and no FIN | -
+| 4 | TCP session started from the middle (no SYN packet) | -
+
+## Integration
+
+| # | Test case | Test Class.Method
+| --- | --- | ---
+| 1 | Inbound and outbound configuration in parallel | -
+| 2 | Routes update during active session | -
+| 3 | ACL update during active session | -
+| 4 | Configuration removal during active session (Route, VNET, ENI removal) | -
+
+## Performance
+
+**TBD**
+
+| # | Test case | Test Class.Method
+| --- | --- | ---
+| 1 | CPS | -
+| 2 | Max sessions per ENI. (Verify resources clean up) | -
+| 3 | Max sessions per card (multiple ENIs). (Verify resources clean up) | -
+
+## Future
+
+1. Add IPv6 tests
diff --git a/test/docs/testplans/eni.md b/test/docs/testplans/eni.md
@@ -35,36 +35,38 @@ Test cases are automated using SAI PTF test framework.
 
 Verifies create operations, an association with VNI, MAC.
 
-| Test case | Test Class.Method
-| --- | ---
-| create inbound/outbound DASH ACL groups | CreateDeleteEniTest.createInOutAclGroupsTest
-| create VNET | CreateDeleteEniTest.createVnetTest
-| create ENI | CreateDeleteEniTest.createEniTest
-| create ENI Ether address map entry | CreateDeleteEniTest.createEniEtherAddressMapTest
-| create PA validation entry | CreateDeleteEniTest.createPaValidationTest
-| create Outbound routing entry | CreateDeleteEniTest.createOutboundRoutingEntryTest
-| verify ENI attributes getting/setting | CreateDeleteEniTest.eniAttributesTest
-| verify ENI Ether address map entry attributes getting/setting | CreateDeleteEniTest.eniEtherAddressMapAttributesTest
-| verify PA validation entry attributes getting/setting | CreateDeleteEniTest.paValidationEntryAttributesTest
-| verify Outbound routing entry attributes getting/setting | CreateDeleteEniTest.outboundRoutingEntryAttributesTest
+| # | Test case | Test Class.Method
+| --- | --- | ---
+| 1 | create inbound/outbound DASH ACL groups | CreateDeleteEniTest.createInOutAclGroupsTest
+| 2 | create VNET | CreateDeleteEniTest.createVnetTest
+| 3 | create ENI | CreateDeleteEniTest.createEniTest
+| 4 | create ENI Ether address map entry | CreateDeleteEniTest.createEniEtherAddressMapTest
+| 5 | create PA validation entry | CreateDeleteEniTest.createPaValidationTest
+| 6 | create Outbound routing entry | CreateDeleteEniTest.createOutboundRoutingEntryTest
+| 7 | create Inbound routing entry | CreateDeleteEniTest.createInboundRoutingEntryTest |
+| 8 | verify Inbound routing entry attributes getting/setting | CreateDeleteEniTest.inboundRoutingEntryAttributesTest |
+| 9 | verify ENI attributes getting/setting | CreateDeleteEniTest.eniAttributesTest
+| 10 | verify ENI Ether address map entry attributes getting/setting | CreateDeleteEniTest.eniEtherAddressMapAttributesTest
+| 11 | verify PA validation entry attributes getting/setting | CreateDeleteEniTest.paValidationEntryAttributesTest
+| 12 | verify Outbound routing entry attributes getting/setting | CreateDeleteEniTest.outboundRoutingEntryAttributesTest
 
 ## ENI removal
 
 Verifies remove operations.
 
-| Test case | Test Class.Method
-| --- | ---
-| normal delete:<br>verify deletion of: inbound/outbound DASH ACL groups, VNET, ENI, ENI Ether address map entry, PA validation entry, Outbound routing entry | CreateDeleteEniTest.deleteEniTest
-| error id mapped rules exist:<br>verify ENI cannot be deleted when map exist | CreateDeleteEniTest.deleteEniWhenMapExistTest
-| duplicated deletion<br>no errors | CreateDeleteEniTest.duplicatedEniDeletionTest 
-| normal bulk delete | -
-| bulk delete does not remove any if there is a mapping for some ENI | -
+| # | Test case | Test Class.Method
+| --- | --- | ---
+| 1 | normal delete:<br>verify deletion of: inbound/outbound DASH ACL groups, VNET, ENI, ENI Ether address map entry, PA validation entry, inbound/outbound routing entry | Is a part of CreateDeleteEniTest.destroy_teardown_obj
+| 2 | error id mapped rules exist:<br>verify ENI cannot be deleted when map exist | CreateDeleteEniTest.deleteEniWhenMapExistTest
+| 3 | duplicated deletion<br>no errors | CreateDeleteEniTest.duplicatedEniDeletionTest 
+| 4 | normal bulk delete | -
+| 5 | bulk delete does not remove any if there is a mapping for some ENI | -
 
 ## ENI scale. 
 
 Verifies basic ENI scale, create/remove/recreate maximum number of ENIs.
 
-| Test case | Test Class.Method
-| --- | ---
-| Create/remove a max number of ENI entries | EniScaleTest.eniScaleTest
-| Recreate (repeated creation/removal a max number of ENI entries) | EniScaleTest.eniScaleTest
+| # | Test case | Test Class.Method
+| --- | --- | ---
+| 1 | Create/remove a max number of ENI entries | EniScaleTest.eniScaleTest
+| 2 | Recreate (repeated creation/removal a max number of ENI entries) | EniScaleTest.eniScaleTest
diff --git a/test/docs/testplans/overlay.md b/test/docs/testplans/overlay.md
@@ -28,7 +28,7 @@ Two test frameworks are suggested for automation:
 |---|---
 | VNETs | 1024
 | ENI per card | 64
-| Routes per ENI | 100k (**to clarify** in some md docs it is 200k)
+| Routes per ENI | 100k
 | NSGs per ENI | 6
 | ACLs per ENI | 6x100K prefixes
 | ACLs per ENI | 6x10K SRC/DST ports
@@ -38,7 +38,7 @@ Two test frameworks are suggested for automation:
 ### Performance
 | Item |	Expected value
 |---|---
-| CPS per card | 4M+
+| CPS per card | 5M
 | Flows per ENI | 1M
 | Flows per card | 16M per 200G (**64M**)
 
@@ -52,7 +52,8 @@ More requirements may be found in [DASH SONiC HLD](https://github.com/Azure/DASH
 
 1. [ENI config](./eni.md)<br>
 Verifies base CRUD operations and scaling for Elastic Network Interface (ENI),
-1. Connection tracking
+1. [Connection tracking](./conntrack.md)<br>
+Verifies the connection tracking mechanism, ageing, scaling and performance.
 1. ACL
 1. [VNET-to-VNET](./vnet.md)<br>
 Verifies VM to VM communication in VNET, using an Appliance for rules and routing offload.

diff --git a/test/docs/testplans/vnet.md b/test/docs/testplans/vnet.md
@@ -31,7 +31,7 @@ The scenario allows the following:
 |---|---
 | VNETs | 1024
 | ENI per card | 64
-| Routes per ENI | 100k (**to clarify** in some md docs it is 200k)
+| Routes per ENI | 100k
 | NSGs per ENI | 6
 | ACLs per ENI | 6x100K prefixes
 | ACLs per ENI | 6x10K SRC/DST ports
@@ -65,12 +65,17 @@ More details may be found in [DASH SONiC HLD](https://github.com/Azure/DASH/blob
 # Automation
 
 Test cases are automated using SAI PTF test framework, except scale and performance tests.
+Scale and performance tests automation approach - to be defined.
 
 # Test suites
 
-**NOTE**: Each test has to send multiple traffic types:
-- Traffic that matches applied configuration (positive case)
-- Traffic that doesn't match applied configuration for each applied attribute (negative case).
+**Overall comments**
+1. Each scenario should be executed with and without underlay configuration:
+    - without underlay - same rx/tx port
+    - with underlay - use port to port traffic verification
+1. Each test should send multiple traffic types:
+    - Traffic that matches applied configuration (positive case)
+    - Traffic that doesn't match applied configuration for each applied attribute (negative case).
 
 ### **Outbound VNET routing**
 | # | Test case | Test Class.Method
@@ -91,9 +96,7 @@ Original table [link](https://github.com/Azure/DASH/blob/main/documentation/gene
 | --- | --- | ---
 | 1 | VNET2VNET routing with PA validation entry PERMIT.<br>SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP_PA_VALIDATE<br>SAI_PA_VALIDATION_ENTRY_ACTION_PERMIT| Vnet2VnetInboundTest.<br>vnet2VnetInboundPaValidatePermitTest
 | 2 | Direction lookup DENY action | Vnet2VnetInboundTest.<br/>vnet2VnetInboundDenyVniTest
-| 3 | Drop if CA DMAC does not match | Vnet2VnetInboundTest.<br/>vnet2VnetInboundInvalidEniMacTest
-| 4 | Drop if PA SIP does not match on PA validation | Vnet2VnetInboundTest.<br/>vnet2VnetInboundInvalidPaSrcIpTest
-| 5 | VNET2VNET routing without PA validation entry<br>SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP | -
+| 3 | VNET2VNET routing without PA validation entry<br>SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP | -
 
 
 ### **Integration**
@@ -103,6 +106,7 @@ Original table [link](https://github.com/Azure/DASH/blob/main/documentation/gene
 | 1 |Multiple inbound and outbound configurations at the same time. Send multiple allowed and forbidden traffic types. | -
 | 2 |Send non VXLAN traffic. (**to clarify** underlay routing?) | VnetRouteTest
 | 3 |Use multiple VIPs | -
+| 4 | Use same prefixes in CA and PA networks for outbound and inbound VNET at the same time | -
 
 ### **Negative**
 
@@ -111,6 +115,8 @@ Original table [link](https://github.com/Azure/DASH/blob/main/documentation/gene
 | 1 | Traffic with invalid VIP (Inbound and Outbound) | -
 | 2 | Traffic with valid VNI but no match to any ENI MAC | -
 | 3 | Invalid configurations:<br>- Multiple MACs for same ENI<br>- All different VNIs in ENI, direction lookup, vnet configuration.<br>- Add same VNI for different direction lookup entries. | -
+| 4 | Drop if CA DMAC does not match | Vnet2VnetInboundTest.<br/>vnet2VnetInboundInvalidEniMacTest
+| 5 | Drop if PA SIP does not match on PA validation | Vnet2VnetInboundTest.<br/>vnet2VnetInboundInvalidPaSrcIpTest
 
 ### **Scaling & Performance**
 
@@ -119,6 +125,7 @@ To be defined.
 
 ### **To clarify / Future**
 
+1. Items 5 and 7 in [other requirements](#other) are conflicting to each other.
 1. What is relation between vm_vni and vnet_id in ENI create?
 1. The lookup table is per ENI, but could be Global, or multiple Global lookup tables per ENIs. How to configure global lookup? Multiple lookups?
 1. In Encap and Decap rules we have: