prime-select on-demand then reboot not get DynamicPowerManagement=2 in next boot

[alex-tu-cc]

the tested combination:
nvidia-prime : 8fbbffe
ubuntu-drivers-common : 2dc8f91
nvidia-driver-450 : 450.66-0ubuntu0.20.04.1
01:00.0 3D controller [0302]: NVIDIA Corporation TU117GLM [Quadro T2000 Mobile / Max-Q] [10de:1fb8] (rev a1)
Subsystem: Dell TU117GLM [Quadro T2000 Mobile / Max-Q] [1028:097e]
/var/log/gpu-manager.log

steps:

1. prime-select on-demand
2. reboot
3. cat /proc/driver/nvidia/params | pastebinit
   https://paste.ubuntu.com/p/HpcwQ5vbcR/
   FAILED here DynamicPowerManagement: 0

checked:

• $ cat /lib/modprobe.d/nvidia-runtimepm.conf
  options nvidia "NVreg_DynamicPowerManagement=0x02"
• manually unload nvidia then reload nvida can get DynamicPowerManagement=0x02 be set correctly.

[tseliot]

If you can disable that, and reproduce the problem, I would like to see your /var/log/gpu-manager.log

[alex-tu-cc]

here's the simulate of a user doing prime-select on-demand.

I do the prime-select nvidia to get the default environment:

$ prime-select nvidia
/lib/modprobe.d$ cat nvidia*
blacklist nouveau
blacklist lbm-nouveau
alias nouveau off
alias lbm-nouveau off

# This file was generated by nvidia-prime
# Set value to 1 to enable modesetting
options nvidia-drm modeset=1

# to simulate user update kernel
$ sudo update-initramfs -k `uname -r `

# check content in initrd
$ unmkinitramfs initrd.img-5.6.0-1027-oem .
ubuntu@u-Precision-5550:~/tmp/alber-testing$ find . -name "nvi*"
./main/usr/lib/modprobe.d/nvidia-graphics-drivers.conf
./main/usr/lib/modprobe.d/nvidia-kms.conf
./main/usr/lib/modules/5.6.0-1027-oem/updates/dkms/nvidia.ko
./main/usr/lib/modules/5.6.0-1027-oem/updates/dkms/nvidia-modeset.ko
./main/usr/lib/modules/5.6.0-1027-oem/updates/dkms/nvidia-uvm.ko
./main/usr/lib/modules/5.6.0-1027-oem/updates/dkms/nvidia-drm.ko
./main/usr/lib/modules/5.6.0-1027-oem/kernel/drivers/net/ethernet/nvidia

$ cat ./main/usr/lib/modprobe.d/nvidia*
blacklist nouveau
blacklist lbm-nouveau
alias nouveau off
alias lbm-nouveau off

# This file was generated by nvidia-prime
# Set value to 1 to enable modesetting
options nvidia-drm modeset=1

Then this is the /var/log/gpu-manager.log in next boot
https://paste.ubuntu.com/p/jFH2Rs8GTr/

this is the glxinfo in next boot which should show it's rendering by Intel GPU. [FAILED]
https://paste.ubuntu.com/p/bHWt5PcPzj/

Then I do the prime-select on-demand

$ sudo prime-select on-demand
$ cd /lib/udev/rules.d
$ ls | grep nvi
50-pm-nvidia.rules
71-nvidia.rules

/lib/udev/rules.d$ cat 50-pm-nvidia.rules
# Enable runtime PM for NVIDIA VGA/3D controller devices on driver bind
ACTION=="bind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030000", TEST=="power/control", ATTR{power/control}="auto"
ACTION=="bind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030200", TEST=="power/control", ATTR{power/control}="auto"

# Disable runtime PM for NVIDIA VGA/3D controller devices on driver unbind
ACTION=="unbind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030000", TEST=="power/control", ATTR{power/control}="on"
ACTION=="unbind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030200", TEST=="power/control", ATTR{power/control}="on"

/lib/udev/rules.d$ cat 71-nvidia.rules 
# Tag the device as master-of-seat so that logind is happy
# (see LP: #1365336)
SUBSYSTEM=="pci", ATTRS{vendor}=="0x10de", DRIVERS=="nvidia", TAG+="seat", TAG+="master-of-seat"

# Start and stop nvidia-persistenced on power on and power off
# respectively
ACTION=="add", DEVPATH=="/bus/pci/drivers/nvidia", TAG+="systemd", ENV{SYSTEMD_WANTS}="nvidia-persistenced.service"

# Load and unload nvidia-modeset module
ACTION=="add", DEVPATH=="/bus/pci/drivers/nvidia", RUN+="/sbin/modprobe nvidia-modeset"
ACTION=="remove", DEVPATH=="/bus/pci/drivers/nvidia", RUN+="/sbin/modprobe -r nvidia-modeset"

# Load and unload nvidia-drm module
ACTION=="add", DEVPATH=="/bus/pci/drivers/nvidia", RUN+="/sbin/modprobe nvidia-drm"
ACTION=="remove", DEVPATH=="/bus/pci/drivers/nvidia", RUN+="/sbin/modprobe -r nvidia-drm"

# Load and unload nvidia-uvm module
ACTION=="add", DEVPATH=="/bus/pci/drivers/nvidia", RUN+="/sbin/modprobe nvidia-uvm"
ACTION=="remove", DEVPATH=="/bus/pci/drivers/nvidia", RUN+="/sbin/modprobe -r nvidia-uvm"

# This will create the device nvidia device nodes
ACTION=="add", DEVPATH=="/bus/pci/drivers/nvidia", RUN+="/sbin/ub-device-create"

# Create the device node for the nvidia-uvm module
ACTION=="add", DEVPATH=="/module/nvidia_uvm", SUBSYSTEM=="module", RUN+="/sbin/ub-device-create"

$ sudo reboot

Then this is the /var/log/gpu-manager.log in next boot
https://paste.ubuntu.com/p/j4CsQtnNcT/

Then this is the /proc/driver/nvidia/params in next boot
https://paste.ubuntu.com/p/nmtkVqQW2D/
[FAILED] DynamicPowerManagement: 0

[tseliot]

I have fixed the pci detection code and force-pushed to master. Things work correctly on 10.101.46.225.

[alex-tu-cc]

I got a similar issue from the latest testing: https://bugs.launchpad.net/somerville/+bug/1904762
Could you please share me the git hash of your pushing , so that I can check if the patch be included by my testing.