Skip to content

Commit

Permalink
Added overlay test plan.
Browse files Browse the repository at this point in the history
- Overlay test plan page.
- ENI detaild test plan.

Signed-off-by: Anton Putria <[email protected]>
  • Loading branch information
aputriax committed Sep 9, 2022
1 parent 6d1c50a commit 023ca8c
Show file tree
Hide file tree
Showing 3 changed files with 150 additions and 0 deletions.
9 changes: 9 additions & 0 deletions test/docs/testplans/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Table of Contents - Test Plans

| Document | Description |
|----------|-------------|
| [Overlay Test Plans](./overlay.md) | Home page of all overlay test plans. |
| [ENI Test Plan](./eni.md) | Detailed ENI test plan. |


You can start with the [Overlay Test Plans](./overlay.md).
70 changes: 70 additions & 0 deletions test/docs/testplans/eni.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Table of content

1. [Objectives](#objectives)
2. [Requirements](#requirements)
3. [Automation](#automation)
4. [Test Suites](#test-suites)
- [ENI creation](#eni-creation)
- [ENI removal](#eni-removal)
- [ENI scale](#eni-scale)

---

# Objectives

Verify proper CRUD API operations and scaling for Elastic Network Interface (ENI).

# Requirements

| Item | Expected value
|---|---
| ENI per card | 64
| Bulk operations | Yes
| Admin state | When the ENI is admin-state down, the packets destined to this ENI shall be dropped.
| Remove | - During ENI delete, implementation must support ability to delete all mappings or routes in a single API call.<br>- Deleting an object that doesn't exists shall not return an error and shall not perform any force-deletions or delete dependencies implicitly. Sonic implementation shall validate the entire API as pre-checks before applying and return accordingly
| Memory | Flexible memory allocation for ENI and not reserve max scale during initial create. (To allow oversubscription)
| Error handling | Implementation must not have silent failures for APIs.

# Automation

Test cases are automated using SAI PTF test framework.

# Test suites

## ENI creation

Verifies create operations, an association with VNI, MAC.

| Test case | Test Class.Method
| --- | ---
| create inbound/outbound DASH ACL groups | CreateDeleteEniTest.createInOutAclGroupsTest
| create VNET | CreateDeleteEniTest.createVnetTest
| create ENI | CreateDeleteEniTest.createEniTest
| create ENI Ether address map entry | CreateDeleteEniTest.createEniEtherAddressMapTest
| create PA validation entry | CreateDeleteEniTest.createPaValidationTest
| create Outbound routing entry | CreateDeleteEniTest.createOutboundRoutingEntryTest
| verify ENI attributes getting/setting | CreateDeleteEniTest.eniAttributesTest
| verify ENI Ether address map entry attributes getting/setting | CreateDeleteEniTest.eniEtherAddressMapAttributesTest
| verify PA validation entry attributes getting/setting | CreateDeleteEniTest.paValidationEntryAttributesTest
| verify Outbound routing entry attributes getting/setting | CreateDeleteEniTest.outboundRoutingEntryAttributesTest

## ENI removal

Verifies remove operations.

| Test case | Test Class.Method
| --- | ---
| normal delete:<br>verify deletion of: inbound/outbound DASH ACL groups, VNET, ENI, ENI Ether address map entry, PA validation entry, Outbound routing entry | CreateDeleteEniTest.deleteEniTest
| error id mapped rules exist:<br>verify ENI cannot be deleted when map exist | CreateDeleteEniTest.deleteEniWhenMapExistTest
| duplicated deletion<br>no errors | CreateDeleteEniTest.duplicatedEniDeletionTest
| normal bulk delete | -
| bulk delete does not remove any if there is a mapping for some ENI | -

## ENI scale.

Verifies basic ENI scale, create/remove/recreate maximum number of ENIs.

| Test case | Test Class.Method
| --- | ---
| Create/remove a max number of ENI entries | EniScaleTest.eniScaleTest
| Recreate (repeated creation/removal a max number of ENI entries) | EniScaleTest.eniScaleTest
71 changes: 71 additions & 0 deletions test/docs/testplans/overlay.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Table of content

1. [Objectives](#objectives)
2. [Requirements](#requirements)
- [Scaling](#scaling)
- [Performance](#performance)
- [Other](#other)
3. [Test Suites](#test-suites)

---

# Objectives

The test plan designed to cover **overlay** related features for the **DASH SmartAppliances** use case.

The test suites should validate that the DASH devices satisfy the standard SONiC functional requirements. This is a black-box testing concerned with validating whether the device works as intended with SONiC.

Two test frameworks are suggested for automation:
- **SAI PTF** - Functional verification
- **sonic-mgmt** - System/Integration verification

---

# Requirements

### Scaling
| Item | Expected value
|---|---
| VNETs | 1024
| ENI per card | 64
| Routes per ENI | 100k (**to clarify** in some md docs it is 200k)
| NSGs per ENI | 6
| ACLs per ENI | 6x100K prefixes
| ACLs per ENI | 6x10K SRC/DST ports
| CA-PA Mappings | 10M
| Active Connections/ENI | 1M (Bidirectional)

### Performance
| Item | Expected value
|---|---
| CPS per card | 4M+
| Flows per ENI | 1M
| Flows per card | 16M per 200G

### Other

More requirements may be found in [DASH SONiC HLD](https://github.com/Azure/DASH/blob/main/documentation/general/design/dash-sonic-hld.md#15-design-considerations).

---

# Test suites

1. [ENI config](./eni.md)<br>
Verifies base CRUD operations and scaling for Elastic Network Interface (ENI),
1. Connection tracking
1. ACL
1. [VNET-to-VNET](./vnet.md)<br>
Verifies VM to VM communication in VNET, using an Appliance for rules and routing offload.
1. VNET Peering<br>
Virtual network peering connects two virtual networks seamlessly. Once peered, for connectivity purposes, the virtual networks appear as one. For background information, see Virtual network peering.
1. High Availability (HA)<br>
Useful for failure and failover events.
flow efficiently replicates to secondary card; Active/Passive (depending upon ENI policy) or can even have Active/Active; OR provision the same ENI over multiple devices w/o multiple SDN appliances – Primaries for a certain set of VMS can be on both
1. Load Balancer<br>
The feature that switches traffic from using VIP-to-VIP connectivity (which involves transiting SLB MUXes), into using a direct path between VMs (direct PA to PA path).
1. Service Tunnel & Private Link<br>
Service Tunnel prevents Internet access to specific services. Access is permitted only from a specific virtual network (VNET). The Service Tunnel feature provides this capability by encoding certain id's via packet transformation. Private Link feature is an extension to the Service Tunnel feature and enables customers to access public facing shared services via their private IP addresses within their VNET.
1. Encryption Gateway<br>
Express Route Gateway.
1. gNMI
1. Multiple DPUs device

0 comments on commit 023ca8c

Please sign in to comment.