DPDK With SRSRAN #312

IbrahimBabiker87 · 2023-11-14T11:10:22Z

IbrahimBabiker87
Nov 14, 2023

I have switched to SRSRAN with DPDK and expected to get better/ more stable performance, However the opposite is happening, Is thee any special settings for DPDK with SRSRAN,
with out DPDK I have reached 1.4Gbps however with DPDK max is 700Mbps for one UE scenario.

root@bm-able-amd:/home/ad/srsRAN_Project/configs# sudo taskset -c 0-31  gnb -c N310_100MHz.yml
Lower PHY in quad executor mode.

--== srsRAN gNB (commit 374200dee) ==--

Connecting to AMF on 10.0.135.40:38412
Available radio types: uhd.
[INFO] [UHD] linux; GNU C++ version 11.4.0; Boost_107400; DPDK_20.11; UHD_4.6.0.0-0-g50fa3baa
[INFO] [LOGGING] Fastpath logging disabled at runtime.
Making USRP object with args 'use_dpdk=1,type=n3xx,mgmt_addr=10.0.64.241,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000'
EAL: Detected 32 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Detected shared linkage of DPDK
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-2048kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   using IOMMU type 1 (Type 1)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:01:00.2 (socket 0)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:01:00.3 (socket 0)
EAL: No legacy callbacks, legacy socket not created
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=10.0.64.241,type=n3xx,product=n310,serial=32000F1,name=ni-n3xx-32000F1,fpga=XG,claimed=False,use_dpdk=1,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000
[INFO] [MPM.PeriphManager] init() called with device args `fpga=XG,master_clock_rate=122.88e6,mgmt_addr=10.0.64.241,name=ni-n3xx-32000F1,product=n310,recv_frame_size=8000,second_addr=10.10.2.100,send_frame_size=8000,use_dpdk=1,clock_source=internal,time_source=internal'.
[INFO] [MULTI_USRP]     1) catch time transition at pps edge
[INFO] [MULTI_USRP]     2) set times next pps (synchronously)
[WARNING] [0/Radio#1] Attempting to set tick rate to 0. Skipping.
[WARNING] [0/Radio#0] Attempting to set tick rate to 0. Skipping.
[WARNING] [0/Radio#0] Attempting to set tick rate to 0. Skipping.
Cell pci=1, bw=100 MHz, dl_arfcn=632628 (n78), dl_freq=3489.42 MHz, dl_ssb_arfcn=629952, ul_freq=3489.42 MHz

andrepuschmann · 2023-11-14T11:15:02Z

andrepuschmann
Nov 14, 2023
Maintainer

That's UHD with DPDK. I think you have to go down the rabbit hole and benchmark UHD first with and w/o DPDK.

Also can you share a console trace/logs? Where do we lose the 700mbits?

4 replies

IbrahimBabiker87 Nov 14, 2023
Author

Here is the benchmark with DPDK:

root@bm-able-amd:/home/ad/srsRAN_Project/configs# /usr/local/lib/uhd/examples/benchmark_rate --rx_rate 6e6 --tx_rate 6e6 --args="use_dpdk=1,type=n3xx,mgmt_addr=10.0.64.241,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000"

[INFO] [UHD] linux; GNU C++ version 11.4.0; Boost_107400; DPDK_20.11; UHD_4.6.0.0-0-g50fa3baa
EAL: Detected 32 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Detected shared linkage of DPDK
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-2048kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: using IOMMU type 1 (Type 1)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:01:00.2 (socket 0)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:01:00.3 (socket 0)
EAL: No legacy callbacks, legacy socket not created
[00:00:00.000050] Creating the usrp device with: use_dpdk=1,type=n3xx,mgmt_addr=10.0.64.241,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=10.0.64.241,type=n3xx,product=n310,serial=32000F1,name=ni-n3xx-32000F1,fpga=XG,claimed=False,use_dpdk=1,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000
[INFO] [MPM.PeriphManager] init() called with device args `fpga=XG,master_clock_rate=122.88e6,mgmt_addr=10.0.64.241,name=ni-n3xx-32000F1,product=n310,recv_frame_size=8000,second_addr=10.10.2.100,send_frame_size=8000,use_dpdk=1,clock_source=internal,time_source=internal'.
Using Device: Single USRP:
Device: N300-Series Device
Mboard 0: n310
RX Channel: 0
RX DSP: 0
RX Dboard: A
RX Subdev: Magnesium
RX Channel: 1
RX DSP: 1
RX Dboard: A
RX Subdev: Magnesium
RX Channel: 2
RX DSP: 2
RX Dboard: B
RX Subdev: Magnesium
RX Channel: 3
RX DSP: 3
RX Dboard: B
RX Subdev: Magnesium
TX Channel: 0
TX DSP: 0
TX Dboard: A
TX Subdev: Magnesium
TX Channel: 1
TX DSP: 1
TX Dboard: A
TX Subdev: Magnesium
TX Channel: 2
TX DSP: 2
TX Dboard: B
TX Subdev: Magnesium
TX Channel: 3
TX DSP: 3
TX Dboard: B
TX Subdev: Magnesium

[00:00:03.80834986] Setting device timestamp to 0...
[WARNING] [MULTI_USRP] Could not set RX rate to 6.000 MHz. Actual rate is 6.144 MHz
[WARNING] [MULTI_USRP] Could not set RX rate to 6.000 MHz. Actual rate is 6.144 MHz
[WARNING] [MULTI_USRP] Could not set RX rate to 6.000 MHz. Actual rate is 6.144 MHz
[WARNING] [MULTI_USRP] Could not set RX rate to 6.000 MHz. Actual rate is 6.144 MHz
[00:00:03.83700823] Testing receive rate 6.144000 Msps on 1 channels
[WARNING] [MULTI_USRP] Could not set TX rate to 6.000 MHz. Actual rate is 6.144 MHz
[WARNING] [MULTI_USRP] Could not set TX rate to 6.000 MHz. Actual rate is 6.144 MHz
[WARNING] [MULTI_USRP] Could not set TX rate to 6.000 MHz. Actual rate is 6.144 MHz
[WARNING] [MULTI_USRP] Could not set TX rate to 6.000 MHz. Actual rate is 6.144 MHz
Setting TX spp to 1996
[00:00:03.84648116] Testing transmit rate 6.144000 Msps on 1 channels
[00:00:13.106354028] Benchmark complete.

Benchmark rate summary:
Num received samples: 61439176
Num dropped samples: 0
Num overruns detected: 0
Num transmitted samples: 61568616
Num sequence errors (Tx): 0
Num sequence errors (Rx): 0
Num underruns detected: 0
Num late commands: 0
Num timeouts (Tx): 0
Num timeouts (Rx): 0

Done!


and without DPDK:

root@bm-able-amd:/home/ad/srsRAN_Project/configs# /usr/local/lib/uhd/examples/benchmark_rate --rx_rate 6e6 --tx_rate 6e6 --args="type=n3xx,mgmt_addr=10.0.64.241,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000"

[INFO] [UHD] linux; GNU C++ version 11.4.0; Boost_107400; DPDK_20.11; UHD_4.6.0.0-0-g50fa3baa
[00:00:00.000160] Creating the usrp device with: type=n3xx,mgmt_addr=10.0.64.241,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=10.0.64.241,type=n3xx,product=n310,serial=32000F1,name=ni-n3xx-32000F1,fpga=XG,claimed=False,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000
[INFO] [MPM.PeriphManager] init() called with device args `fpga=XG,master_clock_rate=122.88e6,mgmt_addr=10.0.64.241,name=ni-n3xx-32000F1,product=n310,recv_frame_size=8000,second_addr=10.10.2.100,send_frame_size=8000,clock_source=internal,time_source=internal'.
Using Device: Single USRP:
Device: N300-Series Device
Mboard 0: n310
RX Channel: 0
RX DSP: 0
RX Dboard: A
RX Subdev: Magnesium
RX Channel: 1
RX DSP: 1
RX Dboard: A
RX Subdev: Magnesium
RX Channel: 2
RX DSP: 2
RX Dboard: B
RX Subdev: Magnesium
RX Channel: 3
RX DSP: 3
RX Dboard: B
RX Subdev: Magnesium
TX Channel: 0
TX DSP: 0
TX Dboard: A
TX Subdev: Magnesium
TX Channel: 1
TX DSP: 1
TX Dboard: A
TX Subdev: Magnesium
TX Channel: 2
TX DSP: 2
TX Dboard: B
TX Subdev: Magnesium
TX Channel: 3
TX DSP: 3
TX Dboard: B
TX Subdev: Magnesium

[00:00:03.670499316] Setting device timestamp to 0...
[WARNING] [MULTI_USRP] Could not set RX rate to 6.000 MHz. Actual rate is 6.144 MHz
[WARNING] [MULTI_USRP] Could not set RX rate to 6.000 MHz. Actual rate is 6.144 MHz
[WARNING] [MULTI_USRP] Could not set RX rate to 6.000 MHz. Actual rate is 6.144 MHz
[WARNING] [MULTI_USRP] Could not set RX rate to 6.000 MHz. Actual rate is 6.144 MHz
[00:00:03.672911343] Testing receive rate 6.144000 Msps on 1 channels
[WARNING] [MULTI_USRP] Could not set TX rate to 6.000 MHz. Actual rate is 6.144 MHz
[WARNING] [MULTI_USRP] Could not set TX rate to 6.000 MHz. Actual rate is 6.144 MHz
[WARNING] [MULTI_USRP] Could not set TX rate to 6.000 MHz. Actual rate is 6.144 MHz
[WARNING] [MULTI_USRP] Could not set TX rate to 6.000 MHz. Actual rate is 6.144 MHz
Setting TX spp to 1996
[00:00:03.673806925] Testing transmit rate 6.144000 Msps on 1 channels
[00:00:13.685212456] Benchmark complete.

Benchmark rate summary:
Num received samples: 61441251
Num dropped samples: 0
Num overruns detected: 0
Num transmitted samples: 61504744
Num sequence errors (Tx): 0
Num sequence errors (Rx): 0
Num underruns detected: 0
Num late commands: 0
Num timeouts (Tx): 0
Num timeouts (Rx): 0

Done!

andrepuschmann Nov 14, 2023
Maintainer

But this config is far away from 100MHz on 4 channels ;-)

IbrahimBabiker87 Nov 14, 2023
Author

Here you go
with DPDK rx/tx rate 122.88 mbps


root@bm-able-amd:/home/ad/srsRAN_Project/configs# /usr/local/lib/uhd/examples/benchmark_rate --rx_rate 122.88e6 --tx_rate 122.88e6  --args="use_dpdk=1,type=n3xx,mgmt_addr=10.0.64.241,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000"

[INFO] [UHD] linux; GNU C++ version 11.4.0; Boost_107400; DPDK_20.11; UHD_4.6.0.0-0-g50fa3baa
EAL: Detected 32 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Detected shared linkage of DPDK
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
MEMPOOL: Maximum number of mempool ops structs exceeded
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-2048kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   using IOMMU type 1 (Type 1)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:01:00.2 (socket 0)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:01:00.3 (socket 0)
EAL: No legacy callbacks, legacy socket not created
[00:00:00.000030] Creating the usrp device with: use_dpdk=1,type=n3xx,mgmt_addr=10.0.64.241,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=10.0.64.241,type=n3xx,product=n310,serial=32000F1,name=ni-n3xx-32000F1,fpga=XG,claimed=False,use_dpdk=1,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000
[INFO] [MPM.PeriphManager] init() called with device args `fpga=XG,master_clock_rate=122.88e6,mgmt_addr=10.0.64.241,name=ni-n3xx-32000F1,product=n310,recv_frame_size=8000,second_addr=10.10.2.100,send_frame_size=8000,use_dpdk=1,clock_source=internal,time_source=internal'.
Using Device: Single USRP:
  Device: N300-Series Device
  Mboard 0: n310
  RX Channel: 0
    RX DSP: 0
    RX Dboard: A
    RX Subdev: Magnesium
  RX Channel: 1
    RX DSP: 1
    RX Dboard: A
    RX Subdev: Magnesium
  RX Channel: 2
    RX DSP: 2
    RX Dboard: B
    RX Subdev: Magnesium
  RX Channel: 3
    RX DSP: 3
    RX Dboard: B
    RX Subdev: Magnesium
  TX Channel: 0
    TX DSP: 0
    TX Dboard: A
    TX Subdev: Magnesium
  TX Channel: 1
    TX DSP: 1
    TX Dboard: A
    TX Subdev: Magnesium
  TX Channel: 2
    TX DSP: 2
    TX Dboard: B
    TX Subdev: Magnesium
  TX Channel: 3
    TX DSP: 3
    TX Dboard: B
    TX Subdev: Magnesium

[00:00:03.114500248] Setting device timestamp to 0...
[00:00:03.116971915] Testing receive rate 122.880000 Msps on 1 channels
Setting TX spp to 1996
[00:00:03.118031787] Testing transmit rate 122.880000 Msps on 1 channels
[00:00:13.119593816] Benchmark complete.


Benchmark rate summary:
  Num received samples:     1228924445
  Num dropped samples:      0
  Num overruns detected:    0
  Num transmitted samples:  1228915244
  Num sequence errors (Tx): 0
  Num sequence errors (Rx): 0
  Num underruns detected:   0
  Num late commands:        0
  Num timeouts (Tx):        0
  Num timeouts (Rx):        0


Done!

without DPDK:


root@bm-able-amd:/home/ad/srsRAN_Project/configs# /usr/local/lib/uhd/examples/benchmark_rate --rx_rate 122.88e6 --tx_rate 122.88e6  --args="type=n3xx,mgmt_addr=10.0.64.241,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000"

[INFO] [UHD] linux; GNU C++ version 11.4.0; Boost_107400; DPDK_20.11; UHD_4.6.0.0-0-g50fa3baa
[00:00:00.000040] Creating the usrp device with: type=n3xx,mgmt_addr=10.0.64.241,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=10.0.64.241,type=n3xx,product=n310,serial=32000F1,name=ni-n3xx-32000F1,fpga=XG,claimed=False,addr=10.10.3.100,second_addr=10.10.2.100,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000
[INFO] [MPM.PeriphManager] init() called with device args `fpga=XG,master_clock_rate=122.88e6,mgmt_addr=10.0.64.241,name=ni-n3xx-32000F1,product=n310,recv_frame_size=8000,second_addr=10.10.2.100,send_frame_size=8000,clock_source=internal,time_source=internal'.
Using Device: Single USRP:
  Device: N300-Series Device
  Mboard 0: n310
  RX Channel: 0
    RX DSP: 0
    RX Dboard: A
    RX Subdev: Magnesium
  RX Channel: 1
    RX DSP: 1
    RX Dboard: A
    RX Subdev: Magnesium
  RX Channel: 2
    RX DSP: 2
    RX Dboard: B
    RX Subdev: Magnesium
  RX Channel: 3
    RX DSP: 3
    RX Dboard: B
    RX Subdev: Magnesium
  TX Channel: 0
    TX DSP: 0
    TX Dboard: A
    TX Subdev: Magnesium
  TX Channel: 1
    TX DSP: 1
    TX Dboard: A
    TX Subdev: Magnesium
  TX Channel: 2
    TX DSP: 2
    TX Dboard: B
    TX Subdev: Magnesium
  TX Channel: 3
    TX DSP: 3
    TX Dboard: B
    TX Subdev: Magnesium

[00:00:03.90613518] Setting device timestamp to 0...
Setting TX spp to 1996
[00:00:03.93081574] Testing receive rate 122.880000 Msps on 1 channels
[00:00:03.93922146] Testing transmit rate 122.880000 Msps on 1 channels
UU[00:00:13.94926543] Benchmark complete.


Benchmark rate summary:
  Num received samples:     1228800199
  Num dropped samples:      0
  Num overruns detected:    0
  Num transmitted samples:  1228833408
  Num sequence errors (Tx): 0
  Num sequence errors (Rx): 0
  Num underruns detected:   2
  Num late commands:        0
  Num timeouts (Tx):        0
  Num timeouts (Rx):        0


Done!

pgawlowicz Jan 23, 2024
Maintainer

@IbrahimBabiker87 any update on this issue?
You might be interested in checking out new tutorial on using srsRAN gNB with DPDK
https://docs.srsran.com/projects/project/en/latest/tutorials/source/dpdk/source/index.html

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DPDK With SRSRAN #312

{{title}}

Replies: 1 comment 4 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

DPDK With SRSRAN #312

IbrahimBabiker87 Nov 14, 2023

Replies: 1 comment · 4 replies

andrepuschmann Nov 14, 2023 Maintainer

IbrahimBabiker87 Nov 14, 2023 Author

andrepuschmann Nov 14, 2023 Maintainer

IbrahimBabiker87 Nov 14, 2023 Author

pgawlowicz Jan 23, 2024 Maintainer

IbrahimBabiker87
Nov 14, 2023

Replies: 1 comment 4 replies

andrepuschmann
Nov 14, 2023
Maintainer

IbrahimBabiker87 Nov 14, 2023
Author

andrepuschmann Nov 14, 2023
Maintainer

IbrahimBabiker87 Nov 14, 2023
Author

pgawlowicz Jan 23, 2024
Maintainer