Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Segmentation fault in dracut --sysroot due to recursive kernel module installs #2680

Open
rcorujo opened this issue Jan 5, 2025 · 7 comments
Open

Segmentation fault in dracut --sysroot due to recursive kernel module installs #2680

rcorujo opened this issue Jan 5, 2025 · 7 comments
Labels
bug Our bugs

Comments

@rcorujo
Copy link

rcorujo commented Jan 5, 2025

Describe the bug
A clear and concise description of what the error is.

I am running SLES15SP6 on the host and have a RHEL 8.10 chroot environment for which I would like to create an initramfs.img.

Run this command:

dracut --debug --sysroot /opt/clmgr/image/images/rhel8.10 --kver 4.18.0-553.el8_10.x86_64 --kmoddir /opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64 --drivers "usb-storage" --force /tmp/my-initramfs.img

The debug output shows that there is a recursive loop that leads to a segmentation fault.

dracut-install: Handle module 'usb_storage'
dracut-install: dracut_install '/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz' '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz'
dracut-install: dracut_install('/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz', '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz')
dracut-install: dest dir '/opt/clmgr/image/images/rhel8.10/var/tmp/dracut.cHd7Tg/initramfs/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage' does not exist
dracut-install: dracut_install('/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage', '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage')
dracut-install: dest dir '/opt/clmgr/image/images/rhel8.10/var/tmp/dracut.cHd7Tg/initramfs/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb' does not exist
dracut-install: dracut_install('/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb', '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb')
dracut-install: mkdir '/opt/clmgr/image/images/rhel8.10/var/tmp/dracut.cHd7Tg/initramfs/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb'
dracut-install: mkdir '/opt/clmgr/image/images/rhel8.10/var/tmp/dracut.cHd7Tg/initramfs/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage'
dracut-install: dracut_install ret = 0
dracut-install: cp '/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz' '/opt/clmgr/image/images/rhel8.10/var/tmp/dracut.cHd7Tg/initramfs/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz'
dracut-install: dracut_install ret = 0
dracut-install: dracut_install 'usb_storage' OK
dracut-install: dracut_install('/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz', '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz')
dracut-install: dracut_install ret = 0
dracut-install: cp '/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz' '/opt/clmgr/image/images/rhel8.10/var/tmp/dracut.cHd7Tg/initramfs/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz'
dracut-install: dracut_install ret = 0
dracut-install: dracut_install '/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz' '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz' OK
dracut-install: dracut_install('/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz', '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz')
dracut-install: hash hit items for '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz'
dracut-install: dracut_install '/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz' '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz' OK
dracut-install: dracut_install('/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz', '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz')
dracut-install: hash hit items for '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz'
dracut-install: dracut_install '/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz' '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz' OK
dracut-install: dracut_install('/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz', '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz')
dracut-install: hash hit items for '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz'
dracut-install: dracut_install '/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz' '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz' OK
dracut-install: dracut_install('/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz', '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz')
dracut-install: hash hit items for '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz'
dracut-install: dracut_install '/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz' '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz' OK
dracut-install: dracut_install('/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz', '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz')
...
dracut-install: dracut_install('/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz', '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz')
dracut-install: hash hit items for '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz'
dracut-install: dracut_install '/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz' '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz' OK
dracut-install: dracut_install('/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz', '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz')
dracut-install: hash hit items for '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz'
dracut-install: dracut_install '/opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz' '/lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz' OK
/opt/clmgr/image/images/rhel8.10/usr/lib/dracut/dracut-init.sh: line 999: 1654880 Segmentation fault      (core dumped) $DRACUT_INSTALL ${initdir:+-D "$initdir"} ${loginstall:+-L "$loginstall"} ${hostonly:+-H} ${omit_drivers:+-N "$omit_drivers"} ${srcmods:+--kerneldir "$srcmods"} ${_optional:+-o} ${_silent:+--silent} -m "$@"

Here are the dependencies in the chroot environment. I'm not sure why the debug output shows that the uas kernel module is being installed, since the usb-storage kernel module does not have a dependency on it. It is the uas kernel module that has a dependency on usb-storage, but I didn't request for the uas kernel module to be installed.

# grep usb-storage /opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64/modules.dep
kernel/drivers/usb/storage/uas.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz
kernel/drivers/usb/storage/usb-storage.ko.xz:
kernel/drivers/usb/storage/ums-alauda.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz
kernel/drivers/usb/storage/ums-cypress.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz
kernel/drivers/usb/storage/ums-datafab.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz
kernel/drivers/usb/storage/ums-eneub6250.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz
kernel/drivers/usb/storage/ums-freecom.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz
kernel/drivers/usb/storage/ums-isd200.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz
kernel/drivers/usb/storage/ums-jumpshot.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz
kernel/drivers/usb/storage/ums-karma.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz
kernel/drivers/usb/storage/ums-onetouch.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz
kernel/drivers/usb/storage/ums-realtek.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz
kernel/drivers/usb/storage/ums-sddr09.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz
kernel/drivers/usb/storage/ums-sddr55.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz
kernel/drivers/usb/storage/ums-usbat.ko.xz: kernel/drivers/usb/storage/usb-storage.ko.xz

Distribution used
Which distribution was this behaviour seen in?

SLES15SP6 in the host and RHEL 8.10 in the chroot environment

Dracut version
Which dracut version was this behaviour seen in?

# dracut --version
dracut 059+suse.521.g8412a1c0-150600.1.3

Init system
Which init system is being used?

# ps -p 1 -o comm=
systemd

To Reproduce
Steps or code to reproduce the behavior.

See description of the bug.

Expected behavior
A clear and concise description of what you expected to happen.

No recursive loop and no segmentation fault.

Additional context
Add any other context you like about the problem here.
@rcorujo rcorujo added the bug Our bugs label Jan 5, 2025
@rcorujo
Copy link
Author

rcorujo commented Jan 5, 2025

Update: If I omit the uas driver, there are no more recursive module installs and no segmentation fault.

dracut --debug --sysroot /opt/clmgr/image/images/rhel8.10 --kver 4.18.0-553.el8_10.x86_64 --kmoddir /opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64 --drivers "usb-storage" --omit-drivers "uas" --force /tmp/my-initramfs.img

However, if I specify that I want the uas driver included, or both the uas and usb-storage drivers included, then the recursion and the segmentation fault is back.

dracut --debug --sysroot /opt/clmgr/image/images/rhel8.10 --kver 4.18.0-553.el8_10.x86_64 --kmoddir /opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64 --drivers "uas" --force /tmp/my-initramfs.img

dracut --debug --sysroot /opt/clmgr/image/images/rhel8.10 --kver 4.18.0-553.el8_10.x86_64 --kmoddir /opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64 --drivers "uas usb-storage" --force /tmp/my-initramfs.img

@jozzsi
Copy link

jozzsi commented Jan 5, 2025

CC @mwilck and @aafeijoo-suse for SLES

@rcorujo
Copy link
Author

rcorujo commented Jan 8, 2025

The problem does not occur if I explicitly chroot into the directory where the OS image is, and then run dracut without the --sysroot option.

chroot /opt/clmgr/image/images/rhel8.10 dracut --debug --kver 4.18.0-553.el8_10.x86_64 --kmoddir /lib/modules/4.18.0-553.el8_10.x86_64 --drivers "usb-storage" --force /tmp/my-initramfs.img

# lsinitrd /tmp/my-initramfs.img | grep "\.ko"
-rw-r--r--   1 root     root        67608 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/gpu/drm/drm_kms_helper.ko.xz
-rw-r--r--   1 root     root       187948 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/gpu/drm/drm.ko.xz
-rw-r--r--   1 root     root         6856 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/gpu/drm/drm_shmem_helper.ko.xz
-rw-r--r--   1 root     root        21016 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/gpu/drm/mgag200/mgag200.ko.xz
-rw-r--r--   1 root     root         5168 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/i2c/algos/i2c-algo-bit.ko.xz
-rw-r--r--   1 root     root        20884 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/scsi/sg.ko.xz
-rw-r--r--   1 root     root        11308 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/uas.ko.xz
-rw-r--r--   1 root     root        29428 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/usb/storage/usb-storage.ko.xz
-rw-r--r--   1 root     root         3312 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/video/fbdev/core/syscopyarea.ko.xz
-rw-r--r--   1 root     root         3684 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/video/fbdev/core/sysfillrect.ko.xz
-rw-r--r--   1 root     root         2804 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/drivers/video/fbdev/core/sysimgblt.ko.xz
-rw-r--r--   1 root     root         2472 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/net/802/stp.ko.xz
-rw-r--r--   1 root     root       113020 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/net/bridge/bridge.ko.xz
-rw-r--r--   1 root     root        10996 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/net/bridge/br_netfilter.ko.xz
-rw-r--r--   1 root     root         4784 Jan 15  2024 lib/modules/4.18.0-553.el8_10.x86_64/kernel/net/llc/llc.ko.xz

It seems that the problem may be related to having to specify the full path to the kernel modules when the --sysroot option is used. I think the leading /opt/clmgr/image/images/rhel8.10 is tripping it up.

--kmoddir /opt/clmgr/image/images/rhel8.10/lib/modules/4.18.0-553.el8_10.x86_64

@mwilck
Copy link
Contributor

mwilck commented Jan 9, 2025

Here are the dependencies in the chroot environment. I'm not sure why the debug output shows that the uas kernel module is being installed, since the usb-storage kernel module does not have a dependency on it.

We have this soft dependency on (open)SUSE:

# cat /usr/lib/modprobe.d/70-softdep-usb_storage.conf
# uas devices can be unpredictably a fallback for both drivers must be present
# (bnc#862397)
# but avoid circular dependency (bsc#1168661)
softdep usb_storage post: uas

The bug that this is supposed to solve is ancient (2014). It's well possible that this softdep isn't necessary any more. But as usual with things like that, it's hard to get rid of them, especially after such a long time, because we can't be sure whether users still rely on it.

But it doesn't cause endless recursion on plain SUSE systems. So I suppose this happens if SLE and RHEl are mixed together like in your case.
(We actually did observe recursion as the comment above says, but we solved by turning a previous hard dependency into a softdep. It's difficult to say why this doesn't work with the RHEL kernel).

I have no experience with this kind of setup. But as you're essentially building for RHEL, I suppose you should be using the soft dependencies that are valid and tested for RHEL, rather than those of the host system.

@rcorujo
Copy link
Author

rcorujo commented Jan 10, 2025

@mwilck Thank you for your response. The recursion does not occur if instead of using the dracut --sysroot <chroot-dir> option, I simply run chroot <chroot-dir> dracut ....

@mwilck
Copy link
Contributor

mwilck commented Jan 13, 2025

Ok, can this be closed then?

@rcorujo
Copy link
Author

rcorujo commented Jan 13, 2025

Yes. I thought that --sysroot causes dracut to operate strictly within the chroot environment, but apparently it does not. I guess it was a misunderstanding on my part. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Our bugs
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mwilck @rcorujo @jozzsi