Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rtl8169: Kernel panic on FreeBSD 14 when loading if_re kernel module #131

Open
LekKit opened this issue Mar 19, 2024 · 23 comments
Open

rtl8169: Kernel panic on FreeBSD 14 when loading if_re kernel module #131

LekKit opened this issue Mar 19, 2024 · 23 comments
Labels
bug Something isn't working guest issue Issue caused by the guest investigating Working to determine what's going on

Comments

@LekKit
Copy link
Owner

LekKit commented Mar 19, 2024

Steps to reproduce

Panic backtrace
root@freebsd:~ # kldload if_re
re0: <RealTek 8169/8169S/8169SB(L)/8110S/8110SB(L) Gigabit Ethernet> mem 0x40000000-0x400000ff irq 7 at device 1.0 on pci0
re0: Chip rev. 0x00800000
re0: MAC rev. 0x00000000
   t[0]: 0x0000000000000201
   t[1]: 0xffffffc06440b35c
   t[2]: 0xffffffffa3ce9d4f
   t[3]: 0xffffffc000400e76
   t[4]: 0x000000000c938ce0
   t[5]: 0xffffffc045368200
   t[6]: 0xffffffc045368200
   s[0]: 0xffffffc045368270
   s[1]: 0x0000000000008802
   s[2]: 0xffffffd001c64e00
   s[3]: 0xffffffc0453682a8
   s[4]: 0xffffffd0021c6810
   s[5]: 0xffffffc045368348
   s[6]: 0xffffffffffffffff
   s[7]: 0xffffffc0009b18d8
   s[8]: 0xffffffc0007f5cc2
   s[9]: 0xffffffc05afea130
  s[10]: 0xffffffc00071c230
  s[11]: 0x0000000000000801
   a[0]: 0xffffffd001c6f800
   a[1]: 0xffffffc0453682a8
   a[2]: 0x0000000000000000
   a[3]: 0x00000000c0306938
   a[4]: 0xffffffd0021c6820
   a[5]: 0x0000000000000020
   a[6]: 0x0000000000000020
   a[7]: 0x0000000000000800
     ra: 0xffffffc000400f7a
     sp: 0xffffffc045368240
     gp: 0xffffffc0007f4fa0
     tp: 0xffffffc0009c4600
   sepc: 0x0000000000000000
sstatus: 0x0000000a00000120
stval  : 0x0000000000000000
panic: Fatal page fault at 0: 0
cpuid = 0
time = 1710864605
KDB: stack backtrace:
#0 0xffffffc00034110a at kdb_backtrace+0x4a
#1 0xffffffc0002fc03a at vpanic+0x116
#2 0xffffffc0002fbf20 at panic+0x26
#3 0xffffffc0005ac980 at page_fault_handler+0x21a
#4 0xffffffc0005ac0a4 at do_trap_supervisor+0x56
#5 0xffffffc00059c934 at cpu_exception_handler_supervisor+0x74
#6 0xffffffc0003f8256 at if_ioctl+0xa
#7 0xffffffc0004b4dc6 at dump_iface+0x130
#8 0xffffffc0004b4742 at rtnl_handle_ifevent+0x7c
#9 0xffffffc0004b4a40 at rtnl_handle_ifattach+0x48
#10 0xffffffc0003f1f2a at if_attach_internal+0x36c
#11 0xffffffc0003f1bb2 at if_attach+0xe
#12 0xffffffc0003fc58c at ether_ifattach+0x34
#13 0xffffffc06440694a at .Lpcrel_hi96+0x48
#14 0xffffffc000331d26 at device_attach+0x368
#15 0xffffffc0003319a2 at device_probe_and_attach+0x3e
#16 0xffffffc00012e82a at pci_driver_added+0x102
#17 0xffffffc00032f736 at devclass_driver_added+0x34
Uptime: 47s

Workarounds

  • Put if_re_load="YES" into /boot/loader.conf and reboot instead of a kldload
  • Downgrading to FreeBSD 13.3

@LekKit LekKit added bug Something isn't working guest issue Issue caused by the guest investigating Working to determine what's going on labels Mar 19, 2024
@LekKit
Copy link
Owner Author

LekKit commented Jun 18, 2024

UPD: Still happening on FreeBSD 14.1

https://download.freebsd.org/snapshots/VM-IMAGES/14.1-STABLE/riscv64/Latest/FreeBSD-14.1-STABLE-riscv-riscv64.raw.xz

root@freebsd:~ # kldload if_re
re0: <RealTek 8169/8169S/8169SB(L)/8110S/8110SB(L) Gigabit Ethernet> mem 0x40000000-0x400000ff irq 7 at device 1.0 on pci0
re0: Chip rev. 0x00800000
re0: MAC rev. 0x00000000
   t[0]: 0x0000000000000800
   t[1]: 0xffffffc06fe0b28c
   t[2]: 0x0a9cefdd868226ea
   t[3]: 0xffffffc0003f01fc
   t[4]: 0x0a45b27c50c94637
   t[5]: 0x639e09a0a6608882
   t[6]: 0x5baaff10f28a8661
   s[0]: 0xffffffc050c94280
   s[1]: 0xffffffd0025ba100
   s[2]: 0xffffffc050c942b8
   s[3]: 0x0000000000000000
   s[4]: 0xffffffd0024b4810
   s[5]: 0xffffffc050c94358
   s[6]: 0xffffffc00099f9d8
   s[7]: 0xffffffc0007e3d08
   s[8]: 0xffffffc0686dd138
   s[9]: 0xffffffc0686e0130
  s[10]: 0xffffffc00070a2d0
  s[11]: 0x0000000000000801
   a[0]: 0xffffffd0025c1800
   a[1]: 0xffffffc050c942b8
   a[2]: 0xffffffd0025ba100
   a[3]: 0x0000000000000000
   a[4]: 0x00000000c0306938
   a[5]: 0x0000000000000010
   a[6]: 0x0000000000000020
   a[7]: 0x0000000000000016
     ra: 0xffffffc0003f02ec
     sp: 0xffffffc050c94250
     gp: 0xffffffc0007e3218
     tp: 0xffffffc0009b2700
   sepc: 0x0000000000000000
sstatus: 0x0000000a00000120
stval  : 0x0000000000000000
panic: Fatal page fault at 0: 0
cpuid = 0
time = 1718785949
KDB: stack backtrace:
#0 0xffffffc000331076 at kdb_backtrace+0x4a
#1 0xffffffc0002ec40c at vpanic+0x10c
#2 0xffffffc0002ec2fc at panic+0x26
#3 0xffffffc00059a1fc at page_fault_handler+0x210
#4 0xffffffc000599930 at do_trap_supervisor+0x52
#5 0xffffffc00058a194 at cpu_exception_handler_supervisor+0x74
#6 0xffffffc0003e7584 at if_ioctl+0xc
#7 0xffffffc0004a3e0c at dump_iface+0x130
#8 0xffffffc0004a3780 at rtnl_handle_ifevent+0x7c
#9 0xffffffc0004a3a86 at rtnl_handle_ifattach+0x48
#10 0xffffffc0003e1256 at if_attach_internal+0x36c
#11 0xffffffc0003e0ede at if_attach+0xe
#12 0xffffffc0003eb92e at ether_ifattach+0x34
#13 0xffffffc06fe06910 at .Lpcrel_hi96+0x46
#14 0xffffffc000321ce2 at device_attach+0x36a
#15 0xffffffc00032195c at device_probe_and_attach+0x3e
#16 0xffffffc000121278 at pci_driver_added+0x102
#17 0xffffffc00031f714 at devclass_driver_added+0x34

@LekKit
Copy link
Owner Author

LekKit commented Jun 19, 2024

@clausecker
Copy link

I just stumbled upon the same issue and reported it up stream: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=284283

@clausecker
Copy link

As a workaround, load the module through /boot/loader.conf instead of through kldload.

@LekKit
Copy link
Owner Author

LekKit commented Feb 12, 2025

@LekKit
Copy link
Owner Author

LekKit commented Feb 14, 2025

@clausecker FreeBSD guest networking stutter is likely fixed by 6093bc1, but /boot/loader.conf workaround is still needed.

Judging the relevant FreeBSD bug tickets, kldload crash is kernel module initialization order issue, and the fact that ifmedia ioctl()'s are not protected from NULL dereference.

@clausecker
Copy link

Thank you for looking into this! I did not expect an interrupt handling issue.

The kernel panic is not that critical of a problem as there is an easy workaround. What is more confusing is why the driver doesn't attach automatically on boot—maybe a problem with the device tasting?

@LekKit
Copy link
Owner Author

LekKit commented Feb 14, 2025

To be fair, rtl8169 is not the only device which has a FreeBSD driver but not probed on boot automatically.

RVVM has an IDE ATA controller, which is attached via -ata disk.img, and it is not found on FreeBSD boot automatically. When doing kldload atapci kernel crashes too... Although here i'm more confident it's ATA emulation bug (And i'm not sure whether it's worth investing time to debug this legacy device)

@clausecker
Copy link

Any reason why you went for IDE over AHCI?

@LekKit
Copy link
Owner Author

LekKit commented Feb 14, 2025

It was very long ago when RVVM didn't even have PCI, so @cerg2010cerg2010 implemented fdt-based ATA PIO over MMIO and it was the only disk controller RVVM had back then.
Then after PCI introduction ATA was promoted to a PCI UDMA IDE controller, and soon NVMe was imlemented, which superseded ATA. U-Boot has no ATA support on RISC-V (driver has hardcoded x86 stuff), so the only way to use ATA is either as a secondary storage device or with direct kernel boot (-k linux_payload)

@clausecker
Copy link

Makes sense. I guess ATA on a non-x86 device is just rare and poorly supported. AHCI may work a lot better.

@LekKit
Copy link
Owner Author

LekKit commented Feb 14, 2025

I think it is a non-issue anymore now that we have NVMe. It is possible that someone contributes it, though.

@LekKit
Copy link
Owner Author

LekKit commented Feb 14, 2025

@clausecker I tried to pass my host Realtek NIC via VFIO, and FreeBSD guest still doesn't recognize the card on boot and crashes on kldload if_re. So i think it is safe to conclude this is a guest issue and not related to RVVM rtl8169 implementation.

To pass a host device, Linux + root access is needed on the host. Note that I had to use ACS override patch to overcome IOMMU limitation, which has potential to allow VM escape.

rvvm ... -nonet -vfio_pci 09:00.0 <- put your host card PCI address here
Boot & crash log
Loading kernel...
/boot/kernel/kernel text=0x6016a0 text=0x1a6a80 data=0x128220 data=0xf50+0x286b48 0x8+0x131508+0x8+0x11c9b8\
Loading configured modules...
/boot/entropy size=0x1000
/etc/hostid size=0x25
Using DTB provided by EFI at 0xfdec8000.
Kernel entry at 0xf5e00030...
Kernel args: (null)
Loading splash ok
---<<BOOT>>---
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2025 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 15.0-CURRENT #0 main-n275087-cdacb12065e4: Thu Jan 30 06:58:00 UTC 2025
    [email protected]:/usr/obj/usr/src/riscv.riscv64/sys/GENERIC riscv
FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
WARNING: WITNESS option enabled, expect reduced performance.
VT: init without driver.
SBI: OpenSBI v1.4
SBI Specification Version: 2.0
CPU 0  : Vendor=Unspecified Core=Unknown (Hart 0)
  marchid=0x5256564d, mimpid=0xe9c85a7d
  MMU: 0x1<Sv39>
  ISA: 0x112f<Atomic,Compressed,Double,Float,Mult/Div>
  S-mode Extensions: 0x1<Sstc>
real memory  = 2147483648 (2048 MB)
avail memory = 2058039296 (1962 MB)
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
CPU 1  : Vendor=Unspecified Core=Unknown (Hart 1)
CPU 2  : Vendor=Unspecified Core=Unknown (Hart 2)
CPU 3  : Vendor=Unspecified Core=Unknown (Hart 3)
random: unblocking device.
random: entropy device external interface
kbd0 at kbdmux0
ofwbus0: <Open Firmware Device Tree>
simplebus0: <Flattened device tree simple bus> on ofwbus0
sbi0: <RISC-V Supervisor Binary Interface>
intc0: <RISC-V Local Interrupt Controller> on ofwbus0
sbi_ipi0: <RISC-V SBI Inter-Processor Interrupts> on sbi0
plic0: <RISC-V PLIC> mem 0xc000000-0xfffffff irq 8,9,10,11,12,13,14,15 on simplebus0
timer0: <RISC-V Timer>
Timecounter "RISC-V Timecounter" frequency 10000000 Hz quality 1000
Event timer "RISC-V Eventtimer" frequency 10000000 Hz quality 1000
riscv_syscon0: <RISC-V syscon> mem 0x100000-0x100fff on simplebus0
rcons0: <RISC-V console>
cpulist0: <Open Firmware CPU Group> on ofwbus0
cpu0: <Open Firmware CPU> on cpulist0
pcib0: <Generic PCI host controller> mem 0x30000000-0x3fffffff on simplebus0
pci0: <PCI bus> on pcib0
pci0: <network, ethernet> at device 1.0 (no driver attached)
nvme0: <Generic NVMe Device> mem 0x40004000-0x40007fff,0x40008000-0x40008fff irq 22 at device 2.0 on pci0
nvme0: unable to allocate MSI-X
nvme0: unable to allocate MSI
iicoc0: <OpenCores I2C master controller> mem 0x10030000-0x10030fff irq 16 on simplebus0
goldfish_rtc0: <Goldfish RTC> mem 0x101000-0x101fff irq 17 on simplebus0
goldfish_rtc0: registered as a time-of-day clock, resolution 1.000000s
uart0: <Non-standard ns8250 class UART with FIFOs> mem 0x10000000-0x10000fff irq 18 on simplebus0
uart0: console (125000,n,8,1)
syscon_power0: <Syscon poweroff> on ofwbus0
syscon_power1: <Syscon reboot> on ofwbus0
Timecounters tick every 1.000 msec
usb_needs_explore_all: no devclass
nvme0: temperature threshold not supported
nvme0: SET_FEATURES (09) sqid:0 cid:15 nsid:0 cdw10:0000000b cdw11:0000001d
nvme0: INVALID_FIELD (00/02) crd:0 m:0 dnr:0 p:1 sqid:0 cid:15 cdw0:0
nvme0: ASYNC_EVENT_REQUEST (0c) sqid:0 cid:15 nsid:0 cdw10:00000000 cdw11:00000000
nvme0: INVALID OPCODE (00/01) crd:0 m:0 dnr:0 p:1 sqid:0 cid:15 cdw0:0
iicbus0: <OFW I2C bus> on iicoc0
sbi_ipi0: using for IPIs
Release APs
Trying to mount root from ufs:/dev/gpt/rootfs [rw]...
WARNING: WITNESS option enabled, expect reduced performance.
nda0 at nvme0 bus 0 scbus0 target 0 lun 1
nda0: <NVMe Storage R1905 R3JN05KFUGW6>
nda0: Serial Number R3JN05KFUGW6
nda0: nvme version 1.4
nda0: 7200MB (14746715 512 byte sectors)
Setting hostuuid: 4e4ac1cd-812e-4c35-b09a-3776523c5d07.
Setting hostid: 0xdf6b547a.
Starting file system checks:
/dev/gpt/rootfs: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/gpt/rootfs: clean, 134290 free (114 frags, 16772 blocks, 0.0% fragmentation)
/dev/gpt/efiboot0: FILESYSTEM CLEAN; SKIPPING CHECKS
Mounting local filesystems:.
Setting up harvesting: [CALLOUT],[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
Feeding entropy: .
Setting hostname: freebsd.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat/pkg
lo0: link state changed to UP
Starting Network: lo0.
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        groups: lo
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
Starting devd.
route: message indicates error: File exists
add host 127.0.0.1: gateway lo0 fib 0: route already in table
route: message indicates error: File exists
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Clearing /tmp (X related).
Creating and/or trimming log files.
Updating /var/run/os-release done.
Updating motd:.
Recovering vi editor sessions:.
Starting syslogd.
No core dumps found.
Mounting late filesystems:.
Starting cron.
Starting background file system checks in 60 seconds.

Fri Feb 14 17:47:22 UTC 2025

FreeBSD/riscv (freebsd) (ttyu0)

login: root
Feb 14 17:47:24 freebsd login[854]: ROOT LOGIN (root) ON ttyu0
Last login: Fri Feb 14 17:46:06 on ttyu0
FreeBSD 15.0-CURRENT (GENERIC) #0 main-n275087-cdacb12065e4: Thu Jan 30 06:58:00 UTC 2025

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List:        https://www.FreeBSD.org/lists/questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

To change this login announcement, see motd(5).
You have new mail.
root@freebsd:~ # kldload if_re
re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> mem 0x40000000-0x40000fff,0x40001000-0x40001fff irq 21 at device 1.0 on pci0
re0: ASPM disabled
re0: Chip rev. 0x54000000
re0: MAC rev. 0x00100000
   t[0]: 0xffffffd005cd9000
   t[1]: 0xffffffc08860b3ec (.L0  + 0x450)
   t[2]: 0xffffffc000ae0890 (w_locklistdata + 0x42710)
   t[3]: 0xffffffc00042ae32 (ifmedia_ioctl)
   t[4]: 0xdc172a9837a9a4bc
   t[5]: 0xffffffc053164230 (_end + 0x5260b458)
   t[6]: 0xd1170264b893ee46
   s[0]: 0xffffffc0531642a0 (_end + 0x5260b4c8)
   s[1]: 0xffffffd005aaf300
   s[2]: 0xffffffc0531642e0 (_end + 0x5260b508)
   s[3]: 0x0000000000000000
   s[4]: 0xffffffd0181dc800
   s[5]: 0xffffffd005cd902c
   s[6]: 0xffffffc000a94e00 (__stack_chk_guard)
   s[7]: 0xffffffc0008d1b3a (in_mcast_maxgrpsrc + 0x2)
   s[8]: 0xffffffc0884c8000 (_end + 0x8796f228)
   s[9]: 0x0000000000000801
  s[10]: 0xffffffc0884cc130 (_end + 0x87973358)
  s[11]: 0xffffffc0007bfd58 (pci_find_cap_desc)
   a[0]: 0xffffffd0181dc800
   a[1]: 0xffffffc0531642e0 (_end + 0x5260b508)
   a[2]: 0xffffffd005aaf300
   a[3]: 0x0000000000000000
   a[4]: 0x00000000c0306938
   a[5]: 0x0000000000000010
   a[6]: 0x0000000000000020
   a[7]: 0x0000000000000016
     ra: 0xffffffc00042af22 (ifmedia_ioctl + 0xf0)
     sp: 0xffffffc053164270 (_end + 0x5260b498)
     gp: 0xffffffc0008d1b40 (__global_pointer$)
     tp: 0xffffffc000b55740 (__pcpu + 0x300)
   sepc: 0x0000000000000000
sstatus: 0x0000000200000120
stval  : 0x0000000000000000
panic: Fatal page fault at 0: 0
cpuid = 3
time = 1739555247
KDB: stack backtrace:
db_trace_self() at db_trace_self
db_trace_self_wrapper() at db_trace_self_wrapper+0x36
kdb_backtrace() at kdb_backtrace+0x2c
vpanic() at vpanic+0x16e
panic() at panic+0x26
page_fault_handler() at page_fault_handler+0x22a
do_trap_supervisor() at do_trap_supervisor+0x6c
cpu_exception_handler_supervisor() at cpu_exception_handler_supervisor+0x74
--- exception 12, tval = 0
(null)() at 0
if_ioctl() at if_ioctl+0xc
dump_iface() at dump_iface+0x10e
rtnl_handle_ifevent() at rtnl_handle_ifevent+0x60
rtnl_handle_ifattach() at rtnl_handle_ifattach+0x48
if_attach_internal() at if_attach_internal+0x344
if_attach() at if_attach+0xe
ether_ifattach() at ether_ifattach+0x32
.L0 () at .L0 +0xa
device_attach() at device_attach+0x40c
device_probe_and_attach() at device_probe_and_attach+0x44
pci_driver_added() at pci_driver_added+0x102
devclass_driver_added() at devclass_driver_added+0x34
devclass_add_driver() at devclass_add_driver+0xfc
driver_module_handler() at driver_module_handler+0x6a
module_register_init() at module_register_init+0xa8
linker_load_module() at linker_load_module+0x9ee
kern_kldload() at kern_kldload+0x14e
sys_kldload() at sys_kldload+0x52
do_trap_user() at do_trap_user+0x1e4
cpu_exception_handler_user() at cpu_exception_handler_user+0x72
--- syscall (304, FreeBSD ELF64, kldload)
KDB: enter: panic
[ thread pid 857 tid 100062 ]
Stopped at      kdb_enter+0x3a: sd      zero,1190(s1)

@LekKit
Copy link
Owner Author

LekKit commented Feb 14, 2025

With the /boot/loader.conf workaround the passed through host NIC works fine.

@clausecker
Copy link

I've tested the current version with your Realtek patches, and it seems to perform better. However, something is still wrong and some times connections just won't get established at all. Perhaps there is still something missing in the emulated device.

@LekKit
Copy link
Owner Author

LekKit commented Feb 18, 2025

It might be related to tap_user userspace networking backend and/or networking socket library.

@clausecker
Copy link

@LekKit Please consider just using libslirp instead of hand rolling userspace networking. That library solves all the various edge cases.

@LekKit
Copy link
Owner Author

LekKit commented Feb 18, 2025

RVVM userland networking was supposed to be superior to SLIRP in terms of performance (uses epoll/kqueue, multi-queue planned), portability, and being able to work under seccomp/pledge. It is also supposed to work in Emscripten at some point.

I will likely introduce SLIRP as a runtime option after some internal refactoring, but tap_user is definitely not going anywhere and should be fixed if it has any actual issues. SLIRP then would be a "choose and test for your needs" option.

@clausecker
Copy link

clausecker commented Feb 18, 2025

I welcome your desire for faster user space networking. I hope it's going to work out.

Maybe it would be better to integrate this effort into the existing slirp codebase? Or at least, make it a standalone library?

@LekKit
Copy link
Owner Author

LekKit commented Feb 18, 2025

Maybe it would be better to integrate this effort into the existing slirp codebase?

SLIRP afaik is an older version of FreeBSD networking stack, ported into userspace. RVVM tap_user is a ground-up implementation which is only like 1000 lines so it's pretty easy to fuzz, rework or fix, which I very much like (And having more control and integration in the codebase that way, too).

Or at least, make it a standalone library?

I am not sure if it's needed (At least at this point).
The whole librvvm is like <=400kb and by design aims to also solve various VM-related usecases other than purely running RISC-V machines. Some kind of splitting might happen in future if it becomes a bloated monster, but I don't foresee it yet.

@LekKit
Copy link
Owner Author

LekKit commented Feb 19, 2025

Some fixes made to rtl8169 in 9d8ee02. I also added a link to piece of documentation the FreeBSD folks supposedly base their driver on.

Made a very primitive network performance test with a FreeBSD 15-CURRENT guest using netcat (duh):

# RX to guest - Scored >=40M (~320 Mbit) on my machine

# On host
dd if=/dev/zero bs=1M status=progress | nc -l 12345

# In guest:
nc -N [HOST IP] 12345 | dd of=/dev/null bs=1M status=progress
# TX from guest - Scored >=15M (~120 Mbit) on my machine

# On host
nc -l 12345 | dd of=/dev/null bs=1M status=progress 

# In guest:
dd if=/dev/zero bs=1M status=progress | nc -N [HOST IP] 12345

I have not observed any stutter or failed connections yet. Though, those results seem at least 3x worse than with a Linux guest (And even weirder that TX is so much slower), but seem okay for some basic use.

It would be also nice if FreeBSD supported full RISC-V AIA (IMSIC+APLIC) as MSI-X interrupts could be used instead which are more optimal than legacy INTx too.

@clausecker
Copy link

Thank you for looking into this. I'll restart my builds soon and will let you know if this changed anything about the problems I had observed.

@LekKit
Copy link
Owner Author

LekKit commented Feb 19, 2025

If any networking issues still presist, please also share an image or software/script which would reliably reproduce them. It would simplify fixing this a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working guest issue Issue caused by the guest investigating Working to determine what's going on
Projects
None yet
Development

No branches or pull requests

2 participants