-
Notifications
You must be signed in to change notification settings - Fork 129
HW Tests
This section will mainly focus on how to set-up and run hardware tests for reference projects. For clarity let us consider running the tests for reference_nic projects.
Hardware requirements
You should have a fully functional NetFPGA-10G machine. Please refer to the Getting started guide for more help.
Software requirements
Make sure you clone the latest version of the NetFPGA package. Please ensure that you have the necessary packages installed. The current HW testing infrastructure is Python based.
- All the tests for the project should be inside the specific project folder. For example all the tests inside the reference_nic project can be seen by going inside the test folder.
cd NetFPGA-10G-live/projects/reference_nic/test
The following shows the tree structure of the test folder.
test ├── connections │ ├── conn │ └── crossover ├── global │ └── setup ├── hw_external_crossover │ └── run.py ├── hw_external_payload │ └── run.py ├── hw_loopback_maxsize │ └── run.py ├── hw_loopback_minsize │ └── run.py ├── hw_loopback_random │ └── run.py
2.connections is the connections folder, where connections file for the project can be placed. A connection file specifies how the network interfaces are physically connected. The connection file is formatted with one connection per line, where the connection is specified by nfX:ethY, denoting that the interface nfX should be physically connected to ethY. For example for internal loopback, for all the nfX interfaces are connected to each other, the corresponding conn file will have the following configuration.
nf0:nf0 nf1:nf1 nf2:nf2 nf3:nf3
3.global is the folder which contains the setup script. The script is used to configure the IP address for the interfaces while running the hardware tests. The IP's are hard-coded in this file. But the user has to remain concious that they will have to change the file if their interfaces are different(having a different X in ethX) or if they want to configure different IP address for the interfaces.
4.All the tests for the project have a specific naming convention. Test directories should be named both_<major><minor> if they can be run in both simulation and hardware, hw<major><minor> if they can be run only on hardware and sim<major>_<minor> if they can be run only as simulation. Neither major or minor can have underscores in their name, nor can they be left blank. Inside each specific test folder, there is a run.py.
5.run.py is the executable scripts which runs the test. The run.py calls other functions in the NFTest library. It is important to know what arguments needs to be passed to the run.py along with the tests, so that the system can perform the tests as simulation only, hardware only or both.
EDA and SW environment setup.
- This section assumes that the system is capable of running impact in batch mode (Preferred).
1.To check if your tool chain is capable of running impact in batch mode, do the following.
source /your/xilinx/path/for/settings64.sh impact -batch
2.Use the impact_run.sh script to program the FGPA with the desired bitfile. This will remove the driver if already loaded, program the FPGA, does a PCI rescan and inserts the nf10 driver again.
cd NetFPGA-10G-live/tools/scripts ./impact_run.sh /path/to/bitfile
- This section assumes that you are using impact in GUI mode.
1.Program the FPGA with the desired bitfile by running impact
source /your/xilinx/path/for/settings64.sh impact
2.PCI rescanning for loading the configuration
cd NetFPGA-10G-live/tools/scripts ./pci_rescan_run.sh
3.Check if the FPGA is programmed
lspci -vxx | grep Xilinx
If you don't see the device, you need to reprogram the FPGA.
4.Ensure that the nf10 driver is loaded. For the NetFPGA, the generic driver can be found inside the reference_nic project (inside reference_nic/sw/host/driver). Do a make and you should be able to see the nf10.ko. Insert the driver by running the following command
cd /go/to/the/driver/path insmod nf10.ko
5.Run dmesg to confirm if the driver is properly loaded. If properly loaded, you will see nf10 device ready.
Hardware and physical connections setup
While doing the HW Tests users have to be very careful in getting the connections right. Else the test won't pass and will report an error. Following guidelines are suggested but man more can be included.
- Ensure that the SFP+ transceviers are properly connected to the SFP+ cage of the NetFPGA board.
- Check if the fiber connections are intact. Handle the fiber with care as any damage to them can result in failed tests.
1.Make sure the bashrc_addon_NetFPGA_10G is updated in your system.
2.Check if the variables like NF_ROOT,NF_DESIGN_DIR are set in the right path. Try to do an echo $NF_ROOT , echo $NF_DESIGN_DIR refer to the place you want
3.Running the test
The top level file nf_test.py file can be found inside NetFPGA-10G-live/tools/bin.
Tests are run using the nf_test.py command followed by the arguments indicating if it is a HW or sim test and what is the specific test that we would like to run. So when running the test, test mode should be specified (sim or hw). Other parameters include
- -- major <string>
- -- minor <string>
For instance : ./nf_test.py hw_--major loopback --minor maxsize . For a complete list of arguments type nf_test.py --help
The HW tests have been updated to include the regread_expect functionality.
- We have nf10_lib.c which has the following functions regread, regwrite and regread_expect. This can be found inside NetFPGA-10G-live/tools/lib . There is a Makefile which creates a shared library (nf10_lib.so) of nf10_lib.c and import them using ctypes in python Following are the use case for the functions in python inside the run.py
For asserting the reset_counter to 1 for clearing the registers
nf10_lib.regwrite(RESET_CNTRS(), 0x1)
For asseting the reset_counter to 0 for enable the counters to increment
nf10_lib.regwrite(RESET_CNTRS(), 0x0)
For regread_expect value of 0 in lut_hit and 10 in lut_miss registers
rres1= nf10_lib.regread_expect(SWITCH_OP_LUT_NUM_HITS_REG(), 0x0) rres2=nf10_lib.regread_expect(SWITCH_OP_LUT_NUM_MISSES_REG(), 0xa)
2.In the current implementation if the HW Test fails, we can track down if the errors were due to:
- packet loss and barriers related errors
- reg read errors
1.Set the design environment variables in bashrc (look at the bashrc_addon in the NetFPGA package for details). By default variables assume that the NetFPGA package is installed in root. Following are the default contents of the bashrc
export NF_ROOT=${HOME}/NetFPGA-10G-live
export NF_DESIGN_DIR=${NF_ROOT}/projects/reference_nic
export NF_WORK_DIR=/tmp/${USER}
export PYTHONPATH=${NF_ROOT}/lib/python:${NF_DESIGN_DIR}/lib/Python:
export LD_LIBRARY_PATH=${NF_ROOT}/lib/java/NetFPGAFrontEnd/bin:${LD_LIBRARY_PATH}
2.Ensure the bitfile is loaded properly.
3.Modify the conn file based on your system interfaces.
4.Modify setup file is available to configure the right IP for your interfaces. Following are the default IP address and the interface names.
"eth2","192.168.100.1" "eth3","192.168.101.1" "nf0","192.168.200.1" "nf1","192.168.201.1" "nf2","192.168.202.1" "nf3","192.168.203.1"
5.Make sure you do a make on the NetFPGA-10G-live/tools/lib so that the shared library (nf10_lib.so) is created. This important for running tests for reference_switch and reference_switch_lite.
6.Check if your hardware connections are intact.
7.Check if the setup and the run.py files are in the proper modes (executables). If not when you run the HW tests, it will give you a "permission denied" error and the "interface not UP" errors.