GStreamer, pipewiresrc, can't capture full screen

sblantipodi · January 24, 2025, 11:54pm

Hi there.
I’m using GStreamer and pipewire to capture my screen.

It works awesome until I put some apps like YouTube, steam or whatever full screen.
Once an app is full screen GStreamer continue to correctly capture for four or five seconds and then it stops. No screen is captured, if I then exit the full screen mode, screen capture restarts no problem.

This isn’t a pipewire problem because pipewire correctly works when combined with OBS.

It should be an issue with GStreamer pipewiresrc.

Is there someone who can help me understanding how to workaround it?

I opened a bug report on gitlab but no one is answering from the pipewire side.

Is pipewire project still alive?

sblantipodi · January 27, 2025, 3:35pm

Using Nvidia RTX4090
Ubuntu 24.10 + GNOME 47 + Wayland (using latest kernel from the distro 6.11.0-13-generic)
I even tried
Arch linux with Linux 6.12.9-zen1-1-zen kernel. GNOME 47.3 , WM Mutter(wayland)
and same issue.

ndufresne · January 27, 2025, 5:51pm

There isn’t much material to help. Do you have some test application we can use ? Steps to reproduce, some GStreamer logs showing some warning/error or some other conditions changing ?

sblantipodi · January 27, 2025, 8:25pm

hi @ndufresne, thanks for the answer, I appreciate it.

You can use this simple python app to reproduce the problem.

Launch the python app with no args.
It will ask you to grent the permission to capture the screen, select full screen.

you’ll see that it will start capturing the screen.

Now open Chrome browser and surf to youtube,
the app is still capturing,
now put a video in full screen and CTRL+TAB to the capturing app,
you’ll notice that it was capturing a black frame and when you switch to the capturing app it restarts the capture correctly.

When this problem happen, no error is thrown, gstreamer simply stops capturing with a black frame.

This happen on most full screen app like youtube on Chrome, Steam app, videogames and full screen app in general.

Please let me know if you need more infos.

sblantipodi · January 27, 2025, 9:22pm

I created another Python program to make debugging easier.

gist.github.com

https://gist.github.com/sblantipodi/4f8854d9fd9b5012ab6baa31e2a7d361

gistfile1.txt

#!/usr/bin/python3

import re
import signal
import dbus
from gi.repository import GLib
from dbus.mainloop.glib import DBusGMainLoop

import gi
gi.require_version('Gst', '1.0')

This file has been truncated. show original

This one contains a counter that prints something like:
Frame captured #: 1
Frame captured #: 2
Frame captured #: 3
Frame captured #: n++

for every captured frame.
If you run the program, you’ll notice that the counter increments very quickly.

Now, read the counter before putting the YouTube video in full screen and wait a minute more or less.

Under normal conditions, the counter should continue to grow quickly even when the youtube video is in full screen.

In reality, the counter will continue to increment for 4 or 5 seconds and then stop; the screen capture stops.

Once you exit the YouTube full screen mode, you’ll notice that the counter remains almost the same and hasn’t incremented during the full-screen video.

Counter will continue to grow as you exit the full screen mode.

this problem happens with most full screen apps like (Chrome + Youtube, Steam, videogames, full screen java apps, etc.)

ndufresne · January 27, 2025, 9:57pm

Interesting and strange, since it only happens with Chrome (don’t have any steam game). If you do a dual monitor, as soon as your mouse quit the monitor where the fullscreen window is, it pause. As soon as you move the mouse it resumes for me.

ndufresne · January 27, 2025, 9:59pm

Ok, so it happens for any fullscreen X11 application. That’s why I could not reproduce with Showtime or waylandsink (which do DMABuf offloading and all). That looks like a compositor bug though. Are you also on Gnome Shell ? (cc @rmader )

Reproduction:

python ./xdp-screen-cast.py &
GST_GL_WINDOW=x11 gst-launch-1.0 videotestsrc ! glimagesink

Simply maximize the GL window on the screen you are recording, then move you cursor onto the other display.

ndufresne · January 27, 2025, 10:04pm

On possible workaround, is to install a wayland enabled Chrome browser (or use Firefox).

sblantipodi · January 27, 2025, 10:16pm

Yes I’m using gnome-shell too.
Using Firefox is not a workaround.

I have an app (dynamic bias light) that must record the screen regardless of the app being used in full screen mode, chrome, steam, videogame, ecc…

mvvvv · January 28, 2025, 1:11pm

I think the problem arrived with the 24.10, I also had a test that worked previously

sblantipodi · January 28, 2025, 1:14pm

yes I agree, it’s something arrived lately but it doesn’t happen on Ubuntu only.
it happen on Arch Linux too and on every distros that uses gnome-shell more in general.

I haven’t understood if it’s something that needs to be addressed by the GStreamer pipewire plugin or not but it’s not a pipewire problem because OBS uses pipewire and does not have this problem.

ndufresne · January 28, 2025, 6:12pm

There is no indication of a GStreamer bug, even pipewiresrc bug is not clear (which is not part of GStreamer fyi). The following simply stopped happening when the condition are met:

pipewiresrc gstpipewiresrc.c:574:dequeue_buffer:<pipewiresrc0> got new buffer 0x7fe1dc019990

And that is triggered by Gnome Shell pushing frames into pipewire (in the pipewire sender).

ndufresne · January 28, 2025, 6:14pm

I confirm it works in OBS strangely. Perhaps it works if you don’t use DMABuf.

ndufresne · January 28, 2025, 6:15pm

Of course we don’t use dmabuf in the test, since its using xvimagesink…

ndufresne · January 28, 2025, 6:32pm

Its actually the other way around. If I replaced xvimagesink with glimagesink, it works:

    pipeline = Gst.parse_launch('pipewiresrc fd=%d path=%u ! glimagesink'%(fd, node_id))

If I force non-dmabuf negotiation, we are back to the issue we had.

    pipeline = Gst.parse_launch('pipewiresrc fd=%d path=%u ! video/x-raw ! glimagesink'%(fd, node_id))

A very important thing to notice is that scale factor changing. Once I move my cursor out of the second screen, in the dmabuf case, I can see a 25% scale up of the image (which ended up being cropped). This does not happen if the app running fullscreen is a wayland app. This all points to Gnome Shell XWayland integration. This last issue is also visible through OBS.

sblantipodi · January 29, 2025, 8:45pm

Hi @ndufresne
thanks for the accurate analysis, I appreciate it and I hope that this will move towards a solution.

in the mean time, is there a way to change my current pipeline

pipewiresrc fd={1} path={2} ! videorate drop-only=true ! queue ! videoscale ! queue ! videoconvert
video/x-raw,width=3840,height=2160

to something that can workaround this issue?

I don’t need a “sink” step because the output of the screen capture (the GPU buffer) is processed inside a callback called every frame in my code.

I don’t know if this make sense but how can I make that pipeline “dmabuf compatible” and workaround the problem?

ndufresne · January 30, 2025, 12:40am

I’ve been experimenting and I think something like this should do, let us know what final pipeline you ended with if it worked:

pipewiresrc ! glupload ! glcolorconvert ! gldowload ! ....

Using vapostproc is a plausible alternative.

sblantipodi · January 30, 2025, 7:37am

I tried this:

pipewiresrc fd={1} path={2} ! videorate ! glupload ! glcolorconvert ! gldownload
video/x-raw,width=3840,height=2160

but it doesn’t work at all…
I get this error:

0:00:03.212201916 13111 0x76dbd8009f90 ERROR              glcontext gstglcontext_egl.c:1287:gst_gl_context_egl_activate:<glcontextegl0> Failed to bind context to the current rendering thread: EGL_SUCCESS
0:00:03.212236906 13111 0x76dc24002320 WARN            glbasefilter gstglbasefilter.c:607:gst_gl_base_filter_find_gl_context_unlocked:<gluploadelement0> error: Failed to activate the GL Context
0:00:03.226319353 13111 0x76dbd80066f0 ERROR              glcontext gstglcontext_egl.c:1287:gst_gl_context_egl_activate:<glcontextegl1> Failed to bind context to the current rendering thread: EGL_SUCCESS
0:00:03.226353243 13111 0x76dc24002320 WARN            glbasefilter gstglbasefilter.c:607:gst_gl_base_filter_find_gl_context_unlocked:<glcolorconvertelement0> error: Failed to activate the GL Context
0:00:03.238975710 13111 0x76dbd8005060 ERROR              glcontext gstglcontext_egl.c:1287:gst_gl_context_egl_activate:<glcontextegl2> Failed to bind context to the current rendering thread: EGL_SUCCESS
0:00:03.239003150 13111 0x76dc24002320 WARN            glbasefilter gstglbasefilter.c:607:gst_gl_base_filter_find_gl_context_unlocked:<gluploadelement0> error: Failed to activate the GL Context
0:00:03.239048210 13111 0x76dc24002320 WARN           basetransform gstbasetransform.c:1600:gst_base_transform_default_query:<gluploadelement0:sink> no caps can be handled by this pad
0:00:03.239056800 13111 0x76dc24002320 WARN             pipewiresrc gstpipewiresrc.c:1014:gst_pipewire_src_negotiate:<pipewiresrc0> error: No supported formats found
0:00:03.239059250 13111 0x76dc24002320 WARN             pipewiresrc gstpipewiresrc.c:1014:gst_pipewire_src_negotiate:<pipewiresrc0> error: This element does not have formats in common with the peer
*** pw_stream_set_error called from wrong context, check thread and locking: Operation not permitted
0:00:03.239103290 13111 0x76dc24002320 WARN             pipewiresrc gstpipewiresrc.c:704:on_state_changed:<pipewiresrc0> error: stream error: No supported formats found
0:00:03.239135300 13111 0x76dc24002320 WARN                 basesrc gstbasesrc.c:3177:gst_base_src_loop:<pipewiresrc0> error: Internal data stream error.
0:00:03.239139500 13111 0x76dc24002320 WARN                 basesrc gstbasesrc.c:3177:gst_base_src_loop:<pipewiresrc0> error: streaming stopped, reason not-negotiated (-4)

what am I doing wrong?

ndufresne · January 30, 2025, 2:37pm

If the resolution changes, you need glcolorscale probably. Also, from doing more testing, videorate break pipewiresrc, you can workaround with drop-only property, or moving the rate adjustement after pipewire buffer has been copied.

sblantipodi · January 30, 2025, 3:58pm

I tried this:
pipewiresrc fd={1} path={2} ! videorate drop-only=true ! glupload ! glcolorscale ! glcolorconvert ! gldownload

but it doesn’t work same error:


0:00:03.206836585  8948 0x703dd000a0a0 ERROR              glcontext gstglcontext_egl.c:1287:gst_gl_context_egl_activate:<glcontextegl0> Failed to bind context to the current rendering thread: EGL_SUCCESS
0:00:03.206873325  8948 0x703e1c0129b0 WARN            glbasefilter gstglbasefilter.c:607:gst_gl_base_filter_find_gl_context_unlocked:<gluploadelement0> error: Failed to activate the GL Context
0:00:03.222870062  8948 0x703dd0007c00 ERROR              glcontext gstglcontext_egl.c:1287:gst_gl_context_egl_activate:<glcontextegl1> Failed to bind context to the current rendering thread: EGL_SUCCESS
0:00:03.222899562  8948 0x703e1c0129b0 WARN            glbasefilter gstglbasefilter.c:607:gst_gl_base_filter_find_gl_context_unlocked:<glcolorconvertelement1> error: Failed to activate the GL Context
0:00:03.225109453  8948 0x703dd0007c00 ERROR              glcontext gstglcontext_egl.c:1287:gst_gl_context_egl_activate:<glcontextegl2> Failed to bind context to the current rendering thread: EGL_SUCCESS
0:00:03.225129583  8948 0x703e1c0129b0 WARN            glbasefilter gstglbasefilter.c:607:gst_gl_base_filter_find_gl_context_unlocked:<glcolorconvertelement0> error: Failed to activate the GL Context
0:00:03.232922006  8948 0x703dd0007c00 ERROR              glcontext gstglcontext_egl.c:1287:gst_gl_context_egl_activate:<glcontextegl3> Failed to bind context to the current rendering thread: EGL_SUCCESS
0:00:03.232947916  8948 0x703e1c0129b0 WARN            glbasefilter gstglbasefilter.c:607:gst_gl_base_filter_find_gl_context_unlocked:<gluploadelement0> error: Failed to activate the GL Context
0:00:03.232989596  8948 0x703e1c0129b0 WARN           basetransform gstbasetransform.c:1600:gst_base_transform_default_query:<gluploadelement0:sink> no caps can be handled by this pad
0:00:03.233000176  8948 0x703e1c0129b0 WARN             pipewiresrc gstpipewiresrc.c:1014:gst_pipewire_src_negotiate:<pipewiresrc0> error: No supported formats found
0:00:03.233003026  8948 0x703e1c0129b0 WARN             pipewiresrc gstpipewiresrc.c:1014:gst_pipewire_src_negotiate:<pipewiresrc0> error: This element does not have formats in common with the peer
*** pw_stream_set_error called from wrong context, check thread and locking: Operation not permitted
0:00:03.233026446  8948 0x703e1c0129b0 WARN             pipewiresrc gstpipewiresrc.c:704:on_state_changed:<pipewiresrc0> error: stream error: No supported formats found
0:00:03.233046166  8948 0x703e1c0129b0 WARN                 basesrc gstbasesrc.c:3177:gst_base_src_loop:<pipewiresrc0> error: Internal data stream error.
0:00:03.233049086  8948 0x703e1c0129b0 WARN                 basesrc gstbasesrc.c:3177:gst_base_src_loop:<pipewiresrc0> error: streaming stopped, reason not-negotiated (-4)