Unable to capture any window using nvidia prime

[arazadaz]

Describe the bug
I can not capture any vulkan or openGL application with obs-vkcapture currently. It's just black. It doesn't even show up in obs unless I add the environmental variable OBS_VKCAPTURE_GLVULKAN=1. This used to all work fine. I blame Nvidia, but perhaps you can help me solve this.

It doesn't matter if I run obs on the igpu or nvidia gpu with prime. Same result.

Screenshots or video
If applicable, add screenshots or video to help explain your problem.

System (please complete the following information):

• OS: Arch Linux
• GPU: Gtx 1660 ti
• Mesa version:
• obs-vkcapture version: 1.4.2.1

OBS log:

qt.qpa.plugin: Could not find the Qt platform plugin "wayland" in ""
debug: Found portal inhibitor
debug: Attempted path: share/obs/obs-studio/locale/en-US.ini
debug: Attempted path: /usr/share/obs/obs-studio/locale/en-US.ini
debug: Attempted path: share/obs/obs-studio/locale.ini
debug: Attempted path: /usr/share/obs/obs-studio/locale.ini
debug: Attempted path: share/obs/obs-studio/themes/Yami.qss
debug: Attempted path: /usr/share/obs/obs-studio/themes/Yami.qss
info: Using EGL/X11
info: CPU Name: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
info: CPU Speed: 3099.976MHz
info: Physical Cores: 6, Logical Cores: 12
info: Physical Memory: 31823MB Total, 9413MB Free
info: Kernel Version: Linux 6.5.2-zen1-1-zen
info: Distribution: "EndeavourOS" Unknown
info: Desktop Environment: KDE (KDE)
info: Session Type: wayland
info: Window System: X11.0, Vendor: The X.Org Foundation, Version: 1.23.2
info: Qt Version: 6.5.2 (runtime), 6.5.1 (compiled)
info: Portable mode: false
info: OBS 29.1.3-1 (linux)
info: ---------------------------------
info: ---------------------------------
info: audio settings reset:
        samples per sec: 48000
        speakers:        2
        max buffering:   960 milliseconds
        buffering type:  dynamically increasing
info: ---------------------------------
info: Initializing OpenGL...
info: Loading up OpenGL on adapter Intel Mesa Intel(R) UHD Graphics 630 (CFL GT2)
info: OpenGL loaded successfully, version 4.6 (Core Profile) Mesa 23.1.7-arch1.1, shading language 4.60
info: ---------------------------------
info: video settings reset:
        base resolution:   1920x1080
        output resolution: 1280x720
        downscale filter:  Bicubic
        fps:               30/1
        format:            NV12
        YUV mode:          Rec. 709/Partial
info: NV12 texture support not available
info: P010 texture support not available
info: Audio monitoring device:
        name: Default
        id: default
info: ---------------------------------
warning: Failed to load 'en-US' text for module: 'decklink-captions.so'
warning: Failed to load 'en-US' text for module: 'decklink-output-ui.so'
libDeckLinkAPI.so: cannot open shared object file: No such file or directory
warning: A DeckLink iterator could not be created.  The DeckLink drivers may not be installed
warning: Failed to initialize module 'decklink.so'
info: [pipewire] Available captures:
info: [pipewire]     - Desktop capture
info: [pipewire]     - Window capture
warning: v4l2loopback not installed, virtual camera disabled
info: [linux-vkcapture] plugin loaded successfully (version 1.4.2)
info: NVENC supported
error: VAAPI: Failed to initialize display in vaapi_device_h264_supported
info: VAAPI: API version 1.19
info: FFmpeg VAAPI H264 encoding supported
error: VAAPI: Failed to initialize display in vaapi_device_hevc_supported
info: FFmpeg VAAPI HEVC encoding supported
info: ---------------------------------
info:   Loaded Modules:
info:     text-freetype2.so
info:     rtmp-services.so
info:     obs-x264.so
info:     obs-vst.so
info:     obs-transitions.so
info:     obs-outputs.so
info:     obs-libfdk.so
info:     obs-filters.so
info:     obs-ffmpeg.so
info:     linux-vkcapture.so
info:     linux-v4l2.so
info:     linux-pulseaudio.so
info:     linux-pipewire.so
info:     linux-jack.so
info:     linux-capture.so
info:     linux-alsa.so
info:     image-source.so
info:     frontend-tools.so
info:     decklink-output-ui.so
info:     decklink-captions.so
info: ---------------------------------
info: ==== Startup complete ===============================================
info: All scene data cleared
info: ------------------------------------------------
info: pulse-input: Server name: 'PulseAudio (on PipeWire 0.3.79) 15.0.0'
info: pulse-input: Audio format: s32le, 48000 Hz, 2 channels
info: pulse-input: Started recording from 'alsa_output.pci-0000_00_1f.3.analog-stereo.monitor' (default)
info: [Loaded global audio device]: 'Desktop Audio'
info: pulse-input: Server name: 'PulseAudio (on PipeWire 0.3.79) 15.0.0'
info: pulse-input: Audio format: s16le, 48000 Hz, 1 channels
info: pulse-input: Started recording from 'alsa_input.usb-BLUE_MICROPHONE_Blue_Snowball_797_2018_11_06_69254-00.mono-fallback' (default)
info: [Loaded global audio device]: 'Mic/Aux'
info: Switched to scene 'Scene'
info: ------------------------------------------------
info: Loaded scenes:
info: - scene 'Scene':
info:     - source: 'Game Capture' (vkcapture-source)
info: ------------------------------------------------
info: adding 21 milliseconds of audio buffering, total audio buffering is now 21 milliseconds (source: Desktop Audio)

info: [linux-vkcapture] Client 1 connected (pid=22807)
info: [linux-vkcapture] Creating texture from dmabuf 854x480 modifier:216172782120099860
info: [linux-vkcapture]  [0] fd:40 stride:3456 offset:0
error: Cannot create EGLImage: Arguments are inconsistent (for example, a valid context requires buffers not supplied by a valid surface).
warning: [linux-vkcapture] Asking client to create texture no modifiers

[nowrep]

Please also post game log.

[nowrep]

Also probably duplicate of #157

[arazadaz]

Also probably duplicate of #157

I don't think so. I saw that one before making posting this issue. He got his working by running obs on the same gpu as the game. That solution doesn't work for me. He also had a very buggy output in his broken scenario where as my output is just a black box.

Please also post game log.

Is there any game in particular you'd like to to post a log for. Proton games work? How about glxgears?

[nowrep]

Is there any game in particular you'd like to to post a log for. Proton games work? How about glxgears?

Anything that doesn't work. So if glxgears doesn't work, please post it.

Is this only OpenGL applications, or do you have the same issue with Vulkan too?

[Sorrydough]

I'm having this issue too now with an AMD 5700XT. It was working in the past, it's just randomly stopped working. In the terminal when I run vkcube it says that it successfully starts vulkan capture. In my OBS scene when I hover over the vkcapture source it shows an outline of the window's resolution. It's just black though. I have no errors or whatever.

This behavior occurs on both wayland and X11. I only have one GPU in my system, not even an iGPU. I'm also on Arch linux. I have this problem with every application.

[nowrep]

Please open new issue and post logs.

[fan87]

try starting obs with prime-run too? that fixed my issue

[Kimiblock]

Well I don't need to use prime-run obs to capture GTAV, NieR running on my discrete graphics card. I'm on Arch Linux, with NVIDIA driver version 535.113.01-1 (545 breaks a lot of things for me)