H264 Pipewire issues

Application Development
1

I’m experimenting with H264 encoding via software and NVIDIA hardware decoding and I’m running into various problems I cannot explain.

I am on Ubuntu in wayland and I’ve launched a screen recording session using this script: Immersed-Linux-Virtual-Monitors/scripts/wayland-scripts/dbus_screen_cast.py at main · augustoicaro/Immersed-Linux-Virtual-Monitors · GitHub

gst-launch-1,0 -v pipewiresrc fd={fd} path={path} do-timestamp-true ! videoconvert ! autovideosink

This works and it works quickly as I expect.

gst-launch-1,0 -v pipewiresrc fd={fd} path={path} do-timestamp-true ! videoconvert ! x264enc ! h264parse ! avdec_h264 ! autovideosink

This takes anywhere between 30 seconds to a full minute before it even produces one frame!? After the first frame it takes between 5-10 seconds to produce another. I believe this might just be because of software encoding but is it really that slow?

gst-launch-1,0 -v pipewiresrc fd={fd} path={path} do-timestamp-true ! videoconvert ! x264enc ! h264parse ! nvh264dec ! autovideosink

This again works but takes 30 seconds to a minute to produce a frame.

gst-launch-1,0 -v pipewiresrc fd={fd} path={path} do-timestamp-true ! videoconvert ! nvh264enc ! h264parse ! nvh264dec ! autovideosink

This doesn’t work at all. I get

[error]: Setting pipeline to PAUSED ... <-- it sits here for about 15 seconds

[error]: Failed to set pipeline to PAUSED.

[error]: Setting pipeline to NULL ...

[error]: 0:00:31.299491137 16792 0x7e9e38000d80 WARN                 basesrc gstbasesrc.c:3187:gst_base_src_loop:<pipewiresrc0> error: Internal data stream error.
0:00:31.299512898 16792 0x7e9e38000d80 WARN                 basesrc gstbasesrc.c:3187:gst_base_src_loop:<pipewiresrc0> error: streaming stopped, reason not-negotiated (-4)

[error]: ERROR: from element /GstPipeline:pipeline0/GstPipeWireSrc:pipewiresrc0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3187): gst_base_src_loop (): /GstPipeline:pipeline0/GstPipeWireSrc:pipewiresrc0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.

[error]: Freeing pipeline ...

When run with -v I can see the caps coming from pipewiresrc are

[error]: /GstPipeline:pipeline0/GstPipeWireSrc:pipewiresrc0.GstPad:src: caps = video/x-raw, format=(string)BGRx, width=(int)2560, height=(int)1440, framerate=(fraction)0/1, max-framerate=(fraction)15715677/262144

Which as I understand should be accepted by nvh264 enc as

video/x-raw:
         format: { NV12, Y444, VUYA, RGBA, RGBx, BGRA, BGRx }
          width: [ 160, 4096 ]
         height: [ 64, 4096 ]
 interlace-mode: progressive

As a sanity check, with the test stream

gst-launch-1.0 videotestsrc ! nvh264enc ! h264parse ! nvh264dec ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'nvh264dec0': gst.cuda.context=context, gst.cuda.context=(GstCudaContext)"\(GstCudaContext\)\ cudacontext1", cuda-device-id=(uint)0;
Redistribute latency...
Redistribute latency...
0:00:01.377618710 17164 0x71fa14000b90 WARN              video-info video-info.c:240:validate_chroma_site: chroma-site only makes sense for YUV formats, ENCODED is none
0:00:01.377637345 17164 0x71fa14000b90 WARN              video-info video-info.c:549:gst_video_info_from_caps: invalid chroma-site, using default
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
0:00:05.476903053

This displays a picture correctly so clearly the nvidia plugins are both capable of working correctly.

There’s no indication of what’s going wrong when using this with pipewiresrc and I’m not sure where to go next.

2

I installed PopOS and did sudo apt install gstreamer1.0-plugins-bad and my pipeline works correctly with pipewiresrc. I forgot about this and installed gstreamer with cerbero under /opt on my Ubuntu install. I’m going to try doing the same on Ubuntu and see if my pipeline works that way.

3

Nope, I did sudo apt install gstreamer1.0-plugins-bad in Ubuntu and it doesn’t include the NVIDIA requirements so nvh264enc isn’t installed like it was in PopOS.

4

I believe this might just be because of software encoding but is it really that slow?

No, it’s just that you haven’t configured it for low-latency. Try x264enc tune=zerolatency speed-preset=superfast instead.

To make nvh264enc work you need to use nvvidconv (sometimes called nvvideoconvert) instead of videoconvert before the nvh264enc. I believe what this does is move the video into the nvidia hardware pipeline (video memory). In your example you will probably need to do the same on the decoding side as well.

5

To make nvh264enc work you need to use nvvidconv (sometimes called nvvideoconvert) instead of videoconvert before the nvh264enc.

I don’t think so otherwise this would not work.

gst-launch-1.0 videotestsrc ! nvh264enc ! h264parse ! nvh264dec ! autovideosink

The docs show that nvh264enc can accept the raw frames output by pipewiresrc and my exact command unchanged works perfectly in PopOS but not in Ubuntu. The only difference is with Ubuntu 25 I have to build libgstnvcodec myself as the repository bad plugins package doesn’t include it: Bug #2109413 “Plugin nvcodec is missing on plucky amd64” : Bugs : gst-plugins-bad1.0 package : Ubuntu

I did a clean install of Ubuntu and built gstreamer from version 1.26.0. I took the libgstnvcodec and copied it into my system install of gstreamer and I get the same behavior I mentioned in the OP. I then updated pipewire to 1.4.5 and tried again only to find the same behavior and nothing had changed so I’m out of ideas.

ChatGPT doesn’t seem to think nvvidconv can be installed on x86 systems anyway and I can’t find any sort of guide for even attempting this. I’m going to have to go back to Ubuntu 24 and see if it works there I guess.

6

nvvidconv is pre-installed on Nvidia Jetson boards. In Nvidia Deepstream (which you would typically use on a regular computer) it’s usually called nvvideoconvert. And yes, I don’t think you can get them via apt or similar. I only know them from pre-made Nvidia contexts (jetpack or deepstream containers).

7

nvh264enc is not the encoder provided by Nvidia but the upstream encoder and does not work with Nvidia provided nvvidconv or nvvideoconvert.

8

This works properly on Ubuntu 24.04 without any changes. I’ve given up on Ubuntu 25 because there are clearly serious issues with pipewire going on there.