I’m experimenting with H264 encoding via software and NVIDIA hardware decoding and I’m running into various problems I cannot explain.
I am on Ubuntu in wayland and I’ve launched a screen recording session using this script: Immersed-Linux-Virtual-Monitors/scripts/wayland-scripts/dbus_screen_cast.py at main · augustoicaro/Immersed-Linux-Virtual-Monitors · GitHub
gst-launch-1,0 -v pipewiresrc fd={fd} path={path} do-timestamp-true ! videoconvert ! autovideosink
This works and it works quickly as I expect.
gst-launch-1,0 -v pipewiresrc fd={fd} path={path} do-timestamp-true ! videoconvert ! x264enc ! h264parse ! avdec_h264 ! autovideosink
This takes anywhere between 30 seconds to a full minute before it even produces one frame!? After the first frame it takes between 5-10 seconds to produce another. I believe this might just be because of software encoding but is it really that slow?
gst-launch-1,0 -v pipewiresrc fd={fd} path={path} do-timestamp-true ! videoconvert ! x264enc ! h264parse ! nvh264dec ! autovideosink
This again works but takes 30 seconds to a minute to produce a frame.
gst-launch-1,0 -v pipewiresrc fd={fd} path={path} do-timestamp-true ! videoconvert ! nvh264enc ! h264parse ! nvh264dec ! autovideosink
This doesn’t work at all. I get
[error]: Setting pipeline to PAUSED ... <-- it sits here for about 15 seconds
[error]: Failed to set pipeline to PAUSED.
[error]: Setting pipeline to NULL ...
[error]: 0:00:31.299491137 16792 0x7e9e38000d80 WARN basesrc gstbasesrc.c:3187:gst_base_src_loop:<pipewiresrc0> error: Internal data stream error.
0:00:31.299512898 16792 0x7e9e38000d80 WARN basesrc gstbasesrc.c:3187:gst_base_src_loop:<pipewiresrc0> error: streaming stopped, reason not-negotiated (-4)
[error]: ERROR: from element /GstPipeline:pipeline0/GstPipeWireSrc:pipewiresrc0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3187): gst_base_src_loop (): /GstPipeline:pipeline0/GstPipeWireSrc:pipewiresrc0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
[error]: Freeing pipeline ...
When run with -v I can see the caps coming from pipewiresrc are
[error]: /GstPipeline:pipeline0/GstPipeWireSrc:pipewiresrc0.GstPad:src: caps = video/x-raw, format=(string)BGRx, width=(int)2560, height=(int)1440, framerate=(fraction)0/1, max-framerate=(fraction)15715677/262144
Which as I understand should be accepted by nvh264 enc as
video/x-raw:
format: { NV12, Y444, VUYA, RGBA, RGBx, BGRA, BGRx }
width: [ 160, 4096 ]
height: [ 64, 4096 ]
interlace-mode: progressive
As a sanity check, with the test stream
gst-launch-1.0 videotestsrc ! nvh264enc ! h264parse ! nvh264dec ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'nvh264dec0': gst.cuda.context=context, gst.cuda.context=(GstCudaContext)"\(GstCudaContext\)\ cudacontext1", cuda-device-id=(uint)0;
Redistribute latency...
Redistribute latency...
0:00:01.377618710 17164 0x71fa14000b90 WARN video-info video-info.c:240:validate_chroma_site: chroma-site only makes sense for YUV formats, ENCODED is none
0:00:01.377637345 17164 0x71fa14000b90 WARN video-info video-info.c:549:gst_video_info_from_caps: invalid chroma-site, using default
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
0:00:05.476903053
This displays a picture correctly so clearly the nvidia plugins are both capable of working correctly.
There’s no indication of what’s going wrong when using this with pipewiresrc and I’m not sure where to go next.