Skip to content

Releases: JohannesDeml/NetworkBenchmarkDotNet

0.3.1

30 Oct 18:41
Compare
Choose a tag to compare

0.3.1 - Maintenance Release 👨‍🔧

  • Update ENet to 2.4.5
  • Update NetCoreServer to 3.0.21
  • Fix tasks not pausing due to unused Task.Delay()
  • Cleanup code, remove legacy code
BenchmarkDotNet=v0.12.1, OS=ubuntu 20.04
Intel Core i5-3570K CPU 3.40GHz (Ivy Bridge), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=3.1.403
  [Host]     : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
  Job-WOXISK : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT

Concurrent=False  Server=True  InvocationCount=1  
IterationCount=10  LaunchCount=1  UnrollFactor=1  
WarmupCount=1  
Method Library Mean Error StdDev
Benchmark1 ENet 5,270.0 ms 171.39 ms 101.99 ms
Benchmark2 ENet 908.3 ms 84.13 ms 55.64 ms
Benchmark1 NetCoreServer 8,826.7 ms 272.59 ms 180.30 ms
Benchmark2 NetCoreServer 9,389.2 ms 236.10 ms 140.50 ms
Benchmark1 LiteNetLib 13,276.9 ms 290.76 ms 192.32 ms
Benchmark2 LiteNetLib 2,985.0 ms 33.39 ms 22.09 ms

0.3.0

28 Oct 19:40
Compare
Choose a tag to compare

0.3.0 - BenchmarkDotNet 📊

  • Use BenchmarkDotNet for generating statistics
  • Change Benchmark 2 to a more realistic scenario.
  • Refactor code to have a more streamlined class structure.
BenchmarkDotNet=v0.12.1, OS=ubuntu 20.04
Intel Core i5-3570K CPU 3.40GHz (Ivy Bridge), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=3.1.403
  [Host]     : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
  Job-MXHSPZ : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT

Concurrent=False  Server=True  InvocationCount=1  
IterationCount=10  LaunchCount=1  UnrollFactor=1  
WarmupCount=1  
Method Library Mean Error StdDev
Benchmark1 ENet 5,304.1 ms 120.23 ms 71.55 ms
Benchmark2 ENet 898.7 ms 77.05 ms 50.97 ms
Benchmark1 NetCoreServer 8,490.2 ms 46.09 ms 30.49 ms
Benchmark2 NetCoreServer 8,683.1 ms 37.25 ms 24.64 ms
Benchmark1 LiteNetLib 13,248.2 ms 240.54 ms 143.14 ms
Benchmark2 LiteNetLib 2,961.5 ms 49.71 ms 32.88 ms

0.2.1

26 Oct 17:07
Compare
Choose a tag to compare

0.2.1 - Performance optimizations 🚀

  • Fixing #5
  • Adding Runtime-configuration options for GC
  • Tests run more stable now - remove early in development note.

Results

OS: Ubuntu 20.04.1 LTS
CPU: Intel® Core™ i5-3570K CPU @ 3.40GHz × 4
Mainboard:  Gigabyte Z77X-D3H Gb LAN (Atheros)
RAM: G.Skill 16GB (2 x 8 GB) DDR3-1600 (Part number: F3-1600C11-8GNT)
ENet LiteNetLib NetCoreServer
Benchmark 1 193,515 msg/s 65,142 msg/s 110,051 msg/s
Benchmark 2 2,149,574 msg/s 478,019 msg/s 118,773 msg/s

0.2.0

26 Oct 09:31
a6ec82f
Compare
Choose a tag to compare

0.2.0 - LiteNetLib support 💪

  • Add support for LiteNetLib 🎉
  • Improve output
  • Add option for MessageLoad
  • Cleanup code and get the coding conventions in sync with .Net Conventions

Notes

  • LiteNetLib does not run as performant and stable as the other libraries, maybe the implementation still has flaws.
  • LiteNetLib runs better on windows than it does on linux. This is a bit strange, since both other libraries perform a lot better on linux compared to windows.

Results

OS: Ubuntu 20.04.1 LTS
CPU: Intel® Core™ i5-3570K CPU @ 3.40GHz × 4
Mainboard:  Gigabyte Z77X-D3H Gb LAN (Atheros)
RAM: G.Skill 16GB (2 x 8 GB) DDR3-1600 (Part number: F3-1600C11-8GNT)
ENet LiteNetLib NetCoreServer
Benchmark 1 197,036 msg/s 1,768 msg/s 107,772 msg/s
Benchmark 2 1,908,836 msg/s 144,585 msg/s 118,974 msg/s

0.1.0

28 Sep 15:14
Compare
Choose a tag to compare
0.1.0 Pre-release
Pre-release

First build with some raw tests. There are probably still a lot of places performance can be optimized, take the results with two grains of salt!

Benchmark Results

Linux Setup

  • OS: Ubuntu 20.04.1 LTS
  • CPU: Intel® Core™ i5-3570K CPU @ 3.40GHz × 4
  • Mainboard: Gigabyte Z77X-D3H Gb LAN (Atheros)
  • RAM: G.Skill 16GB (2 x 8 GB) DDR3-1600 (Part number: F3-1600C11-8GNT)

Benchmark 1 (v 0.1.0)

  • OS: Linux 5.4.0-48-generic #52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020 X64
  • Framework: .NET Core 3.1.8
  • Test: PingPong
  • Address: 127.0.0.1, Port: 3333
  • Number of clients: 1000
  • Parallel messages per client: 1
  • Message size: 32 bytes
  • Defined duration: 60 seconds

ENet

Duration: 60.014 s
Messages sent by clients: 11,753,452
Messages server received: 11,753,455
Messages sent by server: 11,753,455
Messages clients received: 11,753,452

Total data: 358.69 MB
Data throughput: 5.98 MB/s
Message throughput: 195,845 msg/s
Message latency: 5.106 μs

NetCoreServer

Duration: 60.006 s
Messages sent by clients: 6,547,612
Messages server received: 6,546,729
Messages sent by server: 6,546,728
Messages clients received: 6,546,620

Total data: 199.79 MB
Data throughput: 3.33 MB/s
Message throughput: 109,100 msg/s
Message latency: 9.166 μs

Benchmark 2 (v 0.1.0)

  • OS: Linux 5.4.0-48-generic #52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020 X64
  • Framework: .NET Core 3.1.8
  • Test: PingPong
  • Address: 127.0.0.1, Port: 3333
  • Number of clients: 100
  • Parallel messages per client: 1,000
  • Message size: 32 bytes
  • Defined duration: 60 seconds

ENet

Duration: 60.009 s
Messages sent by clients: 130,768,872
Messages server received: 130,770,536
Messages sent by server: 130,770,536
Messages clients received: 130,768,872

Total data: 3990.75 MB
Data throughput: 66.50 MB/s
Message throughput: 2,179,168 msg/s
Message latency: 0.459 μs

NetCoreServer

Duration: 60.602 s
Messages sent by clients: 7,222,248
Messages server received: 7,124,206
Messages sent by server: 7,124,205
Messages clients received: 7,123,023

Total data: 217.38 MB
Data throughput: 3.59 MB/s
Message throughput: 117,538 msg/s
Message latency: 8.508 μs

Windows Setup

  • OS: Windows 10 Pro 2004 x64 (v10.0.19041)
  • CPU: AMD Ryzen 7 3700X 3.6 GHz 8-Core
  • Mainboard: MSI MAG X570 TOMAHAWK WIFI ATX AM4 Lan: 2.5 Gbit/s
  • RAM: G.Skill Aegis 32 GB (2 x 16 GB) DDR4-3200 CL16
    Complete Build

Benchmark 1 (v 0.1.0)

  • OS: Microsoft Windows 10.0.19041 X64
  • Framework: .NET Core 3.1.8
  • Test: PingPong
  • Address: 127.0.0.1, Port: 3333
  • Number of clients: 1000
  • Parallel messages per client: 1
  • Message size: 32 bytes
  • Defined duration: 60 seconds

ENet

Duration: 60,004 s
Messages sent by clients: 1.584.735
Messages server received: 1.585.734
Messages sent by server: 1.585.734
Messages clients received: 1.584.735

Total data: 48,36 MB
Data throughput: 0,81 MB/s
Message throughput: 26.411 msg/s
Message latency: 37,864 µs

NetCoreServer

Duration: 60,040 s
Messages sent by clients: 4.156.889
Messages server received: 4.156.127
Messages sent by server: 4.156.126
Messages clients received: 4.155.989

Total data: 126,83 MB
Data throughput: 2,11 MB/s
Message throughput: 69.221 msg/s
Message latency: 14,447 µs

Benchmark 2 (v 0.1.0)

  • OS: Microsoft Windows 10.0.19041 X64
  • Framework: .NET Core 3.1.8
  • Test: PingPong
  • Address: 127.0.0.1, Port: 3333
  • Number of clients: 100
  • Parallel messages per client: 1.000
  • Message size: 32 bytes
  • Defined duration: 60 seconds

ENet

Duration: 60,015 s
Messages sent by clients: 83.344.879
Messages server received: 83.344.880
Messages sent by server: 83.344.880
Messages clients received: 83.344.879

Total data: 2543,48 MB
Data throughput: 42,38 MB/s
Message throughput: 1.388.738 msg/s
Message latency: 0,720 µs

NetCoreServer

Duration: 60,385 s
Messages sent by clients: 4.451.909
Messages server received: 4.352.375
Messages sent by server: 4.352.374
Messages clients received: 4.352.163

Total data: 132,82 MB
Data throughput: 2,20 MB/s
Message throughput: 72.074 msg/s
Message latency: 13,875 µs