1. Connecting to AWS | 2. Makefile Flow | 3. GUI Flow | 4. Optimization Lab | 5. Using the RTL Kernel Wizard | 6. Hardware/ Software Debugging |
This lab is a continuation of the previous (RTL-Kernel Wizard Lab) lab. You will add ChipScope cores to monitor the activities taking place at the kernel interface level and perform software debugging using SDx debug capabilities.
After completing this lab, you will be able to:
- Add ChipScope cores at the kernel interface level
- Debug software application
- Verify functionality in hardware on F1
Please refer to the Appendix-II if you either want to run the lab in the command line mode or want to run the lab on your personally created instance (i.e. non-xilinx instance)
- Change to the working directory where the precompiled project is provided:
cd /home/centos/sources/debug_lab
- Since we will be executing application in System configuration mode, we need to start the SDx program with su. Execute the following commands to launch sdx
sudo sh export XILINX_SDX="/opt/Xilinx/SDx/2018.2.op2258646" source $XILINX_SDX/settings64.sh source /opt/xilinx/xrt/setup.sh sdx
- An Eclipse launcher window will appear asking you to select a directory as workspace. Click on the Browse… button, browse to /home/centos/sources/debug_lab, click OK twice
Review the Appendix-I section to understand how to add the ChipScope Debug bridge core. It is already added in the precompiled design
- From the Run menu, select Run Configurations
- Make sure that the Arguments tab shows ../binary_container_1.xclbin entry
- Make sure that the Environment tab shows /opt/xilinx/xrt/lib in the LD_LIBRARY_PATH field
- Click Run
The host application will start executing, loading bitstream, and pausing for the user input as coded on line 246
Paused execution
- In another terminal window, start a virtual jtag connection using following two commands. The Virtual JTAG XVC Server will start listening to TCP port 10201
source $XILINX_SDX/settings64.sh sudo fpga-start-virtual-jtag -P 10201 -S 0
Starting Virtual JTAG
- Start Vivado from another terminal window from the debug_lab directory
cd /home/centos/sources/debug_lab vivado
- Click on Open Hardware Manager link
- Click Open Target > Open New Target
- Click Next
- Click Next keeping default Local Server option
- Click on the Add Xilinx Virtual Cable (XVC) button
- Enter localhost in the Host name and 10201 in the Port fields, and click OK
Adding Virtual JTAG cable
The Open New Hardware Target form with scanned debug_bridge will appearScanned debug bridge
- Click Next and then Finish
The Vivado Hardware Manager will open showing Hardware, Waveform, Settings-hw, Trigger-Setup windows. The Hardware window also shows two ILA cores inserted in the designHardware Manager
- In the Hardware Device Properties view, click on the browse button beside Probes file, browse to /home/centos/sources/debug_lab/rtl_kernel_example/System , select top_sp.ltx entry and click OK
Notice four (Slot_0 to Slot_3) probes are filled in the Waveform window - Click on the Run Trigger immediate button and observe the waveform window is filled indicating the four channels are in Inactive state
Forced triggered waveform window
- Expand slot_1 : KVAdd_1_m01_axi : W Channel in the Waveform window, select the WVALID signal and drag it to the Trigger Setup - hw window
Adding a probe in the Trigger Setup window
- Click on drop-down button of the Value field and select trigger condition value as 1
Setting trigger condition
- Click on the Run trigger button and observe the hw_ila_1 probe is waiting for the trigger condition to occur
Waiting for the trigger condition to occur
- Switch to the SDx window and hit Enter key in the Console window for the program to continue execution
Observe that the program finishes execution displaying INFO: Test completed successfully message in the Console window - Switch back to Vivado and observed that since the trigger condition is met, the waveform window is displaying activities
Triggered waveform
- Expand Slot_0, slot_1, and slot_2 groups, zoom in region of about 450 to 1000, and observe the data transfer taking place on each channels. Also note the addresses from where data are read and results are written into
Zoomed view showing various activities
- Zoom in on one of the data beats and hover your mouse at each successive samples and notice the data content changing
- Close Vivado by selecting File > Exit
- Close the jtag probe by switching to its terminal window and pressing Ctrl-C
- Switch to the SDx GUI
- Comment out lines 246 and 247
- Save the file by typing Ctrl-S
- In the Assistant tab, right-click on System > Debug and select Debug Configuration
- Make sure that the Arguments tab shows ../binary_container_1.xclbin entry
- Make sure that the Environment tab shows /opt/xilinx/xrt/lib in the LD_LIBRARY_PATH field
- Click Debug
The host application will compile since we have modified it and a window will pop-up asking to switch to Debug perspective - Click Yes
The program will be downloaded and execution will begin, halting at main() entry point - In the main.c view scroll down to line 280 and double-click on the left border to set a breakpoint
At this point, three buffers would have been createdSetting a breakpoint
- Click on the Resume button or press F8
The execution will resume and stop at the breakpoint
At this point you can go to various tabs and see the contents in the current scope
SDx debug allows command queues and memory buffers to be examined as the program execution progresses - Click on the Step Over button or press F6
The execution will progress one statement at a time - Continue pressing F6 until you reach line number 344 at which point kernel would have finished execution
- Select the Memory Buffers tab
Notice that three buffers are allocated, their IDs, DDR memory address, and sizesMemory buffers allocated
- Select the Command Queue tab and notice that there no commands enqued. Lines 344-348 creates commands to read the data and results
Setting a breakpoint
- Press F6 to execute clEnqueueReadBuffer command to create a read buffer command for reading operand d_A
Notice the Command Queue tab shows one command submittedSetting a breakpoint
- Press F6 to execute clEnqueueReadBuffer command for reading operand d_B
Notice the Command Queue tab shows two commands submittedSetting a breakpoint
- Set a breakpoint at line 397 and press F8 to resume the execution
Notice that the Command Queue tab still shows entries - Press F6 to execute clReleaseKernel command
Notice the Memory Buffers tab is empty as all memories are released - Click F8 to complete the execution
- Close the SDx program
In this lab, you used ChipScope Debug bridge and cores to perform hardware debugging. You also performed software debugging using SDx debug perspective.
- In the Assistant tab, expand System > binary_container_1 > KVadd
- Select KVAdd, right-click and select Settings...
- In the Hardware Function Settings window, click on the ChipScope Debug option for the KVAdd kernel
Adding ChipScope Debug module
- Click Apply and OK
- In the Project Explorer tab, expand src > sdx_rtl_kernel > KVAdd and double-click on the main.c to open it in the editor window
- Go to line 246 and enter the following lines of code which will pause the host software execution after creating kernel but before allocating buffer
printf("\nPress ENTER to continue after setting up ILA trigger..."); getc(stdin);
Modifying code to stop its execution before kernel is executed to start Vivado Hardware manager
- Execute the following commands to setup the environments
cd ~/aws-fpga/ source sdaccel_setup.sh source $XILINX_SDX/settings64.sh
- Create a working directory and copy the necessary files
mkdir debug_lab cd debug_lab/ mkdir rundir cp ~/sources/debug_lab/rtl_kernel_example/System/rtl_kernel_example.exe rundir/. cp ~/sources/debug_lab/rtl_kernel_example/xclbin/binary_container_1.awsxclbin rundir/. cp ~/sources/debug_lab/rtl_kernel_example/System/sdaccel.ini rundir/. cp ~/sources/debug_lab/rtl_kernel_example/System/top_sp.ltx . cp -R ~/sources/debug_lab/rtl_kernel_example/src .
- Change to the rundir, execute the application in debug mode using the following commands
cd rundir sudo sh source /opt/xilinx/xrt/setup.sh /opt/Xilinx/SDx/2018.2.op2258646/bin/xgdb --args rtl_kernel_example.exe binary_container_1.awsxclbin
- At the GDB command line prompt, set a breakpoint at the main program entry, and run the application by executing the following commands
break main run
The program will run up to the main() function entry point
- Set another breakpoint at line 280 and then continue executing the program
break 280 continue
The program will execute and wait for you to hit the Enter key. At this stage, you can follow hardware debugging as listed in the Start Vivado Hardware Manager section. Note the path to the probes file is /home/centos/aws-fpga/debug_lab/
- Perform all the steps except the setting up of the hardware manager, if already not done, listed in the previous section
- When line 280 breakpoint is encountered, enter the following commands to see the command queues and the memory buffers
xprint queue xprint mem
Notice that both queue and memory buffers are empty
At this stage you can enter next to execute one statement at a time and/or continue to execute until the next breakpoint is encountered or the program executes the exit() function
You can set breakpoints where desired
You can execute list to show about 10 lines of source code
- Enter following commands to step over two steps and set a breapoint at 344
next next break 344 continue xprint mem xprint queue
Notice that three memory buffers are allocated but commands queue is empty
- Enter following commands to set a next breakpoint, continue to the breakpoint, execute two statements, examine command queues and memory buffers
break 397 continue next next xprint mem xprint queue
- Execute next and xprint mem. Notice memory buffers are empty
- Execute continue to execute till the end
- Type quit to exit the GDB