No NVENC support?

[4evrplan]

Describe the bug

I installed bazzite-nvidia, but NVENC support is missing. Running the following:
nvcc --version
returns
bash: nvcc: command not found

When trying to run ALVR to connect to a Quest 2, I get the following error in the ALVR log:
12:22:57.187652889 [ERROR] Encoder: Provided device doesn't support required NVENC features
And SteamVR shows the following error:
SteamVR failed initialization with error code VRInitError_IPC_ServerInitFailed: "VR Server Init Failed (300)" Please verify SteamVR is properly installed and try again.

This was working on the previous install of Bazzite right out of the box, but a non-related issue forced me to re-install Bazzite from scratch (using the exact same ISO), and I can't think what I've done differently, other than I removed a potentially bad RAM stick from my system, so instead of 16GB of memory, I now have 8. I've tried completely removing and reinstalling SteamVR. I've also tried using the following launch options for SteamVR:
VK_ICD_FILENAMES="/usr/share/vulkan/icd.d/nvidia_icd.json" %command%

I'm running KDE-Plasma in X11 mode, and Steam is running correctly and launches and runs non-VR games without issue.

What did you expect to happen?

I expected NVENC support would be present, SteamVR and ALVR would both run correctly, connect to my Quest 2 VR headset, and allow me to run PCVR games from my Steam library.

Output of rpm-ostree status

State: idle
Deployments:
  ostree-image-signed:docker://ghcr.io/ublue-os/bazzite-nvidia:latest
                   Digest: sha256:3fc5f0ab5d9349c9edbf6b03a6e33a5701174233a10d85d5c6bbe49cab404766
                  Version: 39.20240116.0 (2024-02-05T09:03:16Z)
                     Diff: 27 upgraded, 26 removed, 4 added
                Initramfs: '"-I /etc/crypttab /usr/lib/modprobe.d/nvidia.conf"' 

● ostree-image-signed:docker://ghcr.io/ublue-os/bazzite-nvidia:latest
                   Digest: sha256:6b8f9409ca74161bf28640b9c8ea0b0eb6f96762ac9a29124e64402cf183d0e9
                  Version: 39.20240116.0 (2024-02-04T00:09:55Z)
                Initramfs: '"-I /etc/crypttab /usr/lib/modprobe.d/nvidia.conf"' 

  ostree-unverified-image:docker://ghcr.io/ublue-os/bazzite-nvidia:latest
                   Digest: sha256:de20300ba7dbe398b3cc58ecced829d822bab3c1a5afa4c462135ea1671b8020
                  Version: 39.20240116.0 (2024-02-02T02:34:29Z)
                Initramfs: '"-I /etc/crypttab /usr/lib/modprobe.d/nvidia.conf"'

Hardware

MSI GF63 Thin laptop
Intel i5-11400
Nvidia Geforce GTX 1650 max-q
8GB RAM at 3200 MHz
1TB SSD

Extra information or context

This may or may not be related, but when running Steam through the provided shortcut on the task bar, it would not run games correctly. I had to unpin it, and pin my own version that I saved in ~/.local/share/applications. This copy is identical to the original, except that I've commented out the following 2 lines.

#PrefersNonDefaultGPU=true
#X-KDE-RunOnDiscreteGpu=true

I'm fairly certain that with this modified version, the games are still running on my discreet GPU, because Deep Rock Galactic (for example) runs at a smooth framerate.

EDIT: I just thought of another difference. I had SecureBoot disabled in my BIOS for both this and the previous install of Bazzite, but for this latest install, I did the 'enroll MOK' step (hopefully correctly - I used the provided "ublue-os" password, but I don't really understand how SB or MOK work). For the previous install, MOK timed out, or I cancelled out of it (can't remember which), and oddly enough, that's the install that worked.

[81reap]

EDIT: I just thought of another difference. I had SecureBoot disabled in my BIOS for both this and the previous install of Bazzite, but for this latest install, I did the 'enroll MOK' step (hopefully correctly - I used the provided "ublue-os" password, but I don't really understand how SB or MOK work). For the previous install, MOK timed out, or I cancelled out of it (can't remember which), and oddly enough, that's the install that worked.

@4evrplan are you saying enabling SecureBoot made nvcc work?

Describe the bug

• Steam is able to install and run games successfully.
• nvidia-smi works and shows a cuda version of 12.3
• AMD iGPU is disabled in the BIOS
• SecureBoot is disabled in the BIOS
• nvcc not found on a fresh install

Attempted Fixes

1. Disable nouveau
   This didn't work, but I was able to login to a working DE.

sudo rpm-ostree kargs --append=rd.driver.blacklist=nouveau --append=modprobe.blacklist=nouveau --append=nvidia-drm.modeset=1 initcall_blacklist=simpledrm_platform_driver_init

2. Install cuda
   This didn't work and also gave me a black screen w/ cursor in the middle after logging in. no input.

sudo rpm-ostree install xorg-x11-drv-nvidia-cuda xorg-x11-drv-nvidia-cuda-libs --allow-inactive

Hardware

B550I Aurus Pro AX
AMD Ryzen 7 5700G
Nvidia RTX 4000 SFF Ada Gen
2x32GB @ 3200 MHz
2TB NVME Drive

[81reap]

Oh also to answer your question.

SecureBoot checks your boot loader and OS against a set of approved certs to make sure your system hasn't been compromised. These certs have to be approved by a central authority (Microsoft) so they don't contain the certs you would need to run a custom os like Bazzite.

MOK is a utility that allows you to add 3rd party or unsigned software to the list of approved certs. Adding the MOK for Bazzite lets the bootloader and Linux kernel to run.

[81reap]

I have found a workaround. This was tested on a fresh install of the bazzite-nvidia branch.

Notes

• I had to enable the iGPU to resolve some wayland issues. So now the AMD iGPU handles the display and os, while the Nvidia GPU handles Steam Games and PyTorch.
• SafeBoot is still disabled in the BIOS.
• Still having some issues getting nvcc working in podman containers. nvidia-smi works fine though.

Workaround

$ nvidia-smi
# this shows the correct output and says that cuda 12.3 is installed
$ nvcc --version
# this should fail to find nvcc
$ ls /etc/local
# this output does not contain cuda which confirms that the cuda toolkit is not installed

$ wget https://developer.download.nvidia.com/compute/cuda/12.3.2/local_installers/cuda_12.3.2_545.23.08_linux.run
$ sudo sh cuda_12.3.2_545.23.08_linux.run
# this will require you to accept the licence first. You should only be installing the cuda drivers as the system already has nvidia drivers.
$ ls /etc/local
# now we have the cuda toolkit, but nvcc will still fail as it is not on your path

# add this to your ~/.bashrc so that it is loaded every boot
$ export PATH=/usr/local/cuda-12.3/bin${PATH:+:${PATH}}
$ export LD_LIBRARY_PATH=/usr/local/cuda-12.3/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
$ nvcc --version 
# nvcc now works

[4evrplan]

@4evrplan are you saying enabling SecureBoot made nvcc work?

No, I'm saying that not adding the 3rd party cert made it work. Adding the 3rd party cert for Bazzite broke it. At least, that's the only difference I can think of between the working install and the broken install (that and the amount of RAM). Both installs had SB disabled.

Oh also to answer your question...

Oh, thank you for explaining! That makes so much more sense now. I've read someone else's explanation, but it didn't really click until I read yours.

I have found a workaround...

That's awesome! You are the best!

Unfortunately, I've discovered that SteamVR has issues on Linux that are no fault of Linux or the distro. The movement is unstable and jumpy. I've had to switch back to Windows to get SteamVR to play nice. It's a shame really, as I really enjoyed Bazzite, and I am impressed by everything you have done. Maybe some day I'll come back, once all the app and driver vendors are supporting Linux right out of the box. You have solved the original issue, so I'm closing it. Thank you.

[81reap]

Sorry to hear about your issues with SteamVR, hopefully linux gets stable drivers soon!

I have reopened a new issue here as the workaround is not a full fix but rather a stop gap. The true fix would be if the preinstalled cuda libs would work out of the box or be removed so that end users can install cuda safely through rpm-ostree.

[KyleGospo]

Can you open this against ublue-os/nvidia? That's upstream for our Nvidia driver implementation.

[81reap]

moved :: ublue-os/hwe#198