Hello gstreamer experts!
On a Xilinx ZynqMP running petalinux 2024.1, which have gstreamer 1.20.15 as original, have backported 1.22.12 which is latest available from amd/xilinx (from petalinux 2025.2).
We have a simple pipeline running nicely as
$ GST_DEBUG=4 GST_DEBUG_DUMP_DOT_DIR=/tmp/logs gst-launch-1.0 -e
v4l2src name=vi1source device=/dev/video1 io-mode=4 num-buffers=100 !
capsfilter name=vi1caps caps=video/x-raw,pixel-aspect-ratio=1/1,width=1920,height=1080,format=NV16_10LE32,framerate=30/1 !
tee name=vp1tee
vp1tee. !
queue name=vo1queue !
capsfilter name=vo1caps caps=video/x-raw,pixel-aspect-ratio=1/1,width=1920,height=1080,format=NV16_10LE32,framerate=30/1 !
kmssink name=vo1sink bus-id=fd4a0000.display connector-id=49 plane-id=39 \
/tmp/logs/stdout.log 2>/tmp/logs/stderr.log
$
which takes a v4l2src video in (starts with a a xlnx csi2rx block) and output to the kmssink displayport of zynqmp.
we can also do similar for a sdi out pipeline :
$ GST_DEBUG=4 GST_DEBUG_DUMP_DOT_DIR=/tmp/logs gst-launch-1.0 -e
v4l2src name=vi1source device=/dev/video1 io-mode=4 num-buffers=100 !
capsfilter name=vi1caps caps=video/x-raw,pixel-aspect-ratio=1/1,width=1920,height=1080,format=NV16_10LE32,framerate=30/1 !
tee name=vp1tee
vp1tee. !
queue name=vo1queue !
capsfilter name=vo1caps caps=video/x-raw,pixel-aspect-ratio=1/1,width=1920,height=1080,format=NV16_10LE32,framerate=30/1 !
kmssink name=vo1sink bus-id=801f0000.v_mix connector-id=44 plane-id=36 \
/tmp/logs/stdout.log 2>/tmp/logs/stderr.log
$
which takes same v4l2src and outputs to a kmssink SDI out. And to be specific, either of these are running not both at same time of course.
Now, we would like to combine these to a pipeline that takes same v4l2src and output on both of the above kmssinks, like this :
$ GST_DEBUG=4 GST_DEBUG_DUMP_DOT_DIR=/tmp/logs gst-launch-1.0 -e
v4l2src name=vi1source device=/dev/video1 io-mode=4 !
capsfilter name=vi1caps caps=video/x-raw,pixel-aspect-ratio=1/1,width=1920,height=1080,format=NV16_10LE32,framerate=30/1 !
tee name=vp1tee
vp1tee. !
queue name=vo1queue !
capsfilter name=vo1caps caps=video/x-raw,pixel-aspect-ratio=1/1,width=1920,height=1080,format=NV16_10LE32,framerate=30/1 !
kmssink name=vo1sink bus-id=fd4a0000.display connector-id=49 plane-id=39 sync=false skip-vsync=true
vp1tee. !
queue name=vo2queue !
capsfilter name=vo2caps caps=video/x-raw,pixel-aspect-ratio=1/1,width=1920,height=1080,format=NV16_10LE32,framerate=30/1 !
kmssink name=vo2sink bus-id=801f0000.v_mix connector-id=44 plane-id=36 sync=false skip-vsync=true \
/tmp/logs/stdout.log 2>/tmp/logs/stderr.log
$
And this is where our problem begins… 
The pipeline fails and the errors are like :
$ grep ERR logs/stderr.log
0:00:01.621392440 15373 0xaaaaebc41c60 INFO GST_ERROR_SYSTEM gstelement.c:2281:gst_element_message_full_with_details:<vo1sink> posting message: GStreamer encountered a general resource error.
ERROR: from element /GstPipeline:pipeline0/GstKMSSink:vo1sink: GStreamer encountered a general resource error.
0:00:01.625441860 15373 0xaaaaebc41c60 INFO GST_ERROR_SYSTEM gstelement.c:2308:gst_element_message_full_with_details:<vo1sink> posted error message: GStreamer encountered a general resource error.
0:00:01.625681090 15373 0xaaaaebc41c00 INFO GST_ERROR_SYSTEM gstelement.c:2281:gst_element_message_full_with_details:<vi1source> posting message: Internal data stream error.
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:vi1source: Internal data stream error.
0:00:01.628427900 15373 0xaaaaebc41c00 INFO GST_ERROR_SYSTEM gstelement.c:2308:gst_element_message_full_with_details:<vi1source> posted error message: Internal data stream error.
0:00:01.628539870 15373 0xaaaaebc41c00 INFO GST_ERROR_SYSTEM gstelement.c:2281:gst_element_message_full_with_details:<vo1queue> posting message: Internal data stream error.
ERROR: from element /GstPipeline:pipeline0/GstQueue:vo1queue: Internal data stream error.
0:00:01.631165150 15373 0xaaaaebc41c00 INFO GST_ERROR_SYSTEM gstelement.c:2308:gst_element_message_full_with_details:<vo1queue> posted error message: Internal data stream error.
If I instead of io-mode dmabuf (4) uses mmap(2) then the pipeline runs but with poor performance (high latency, frame drops) which is to be expected since we cannot cope with copy frames here…
So we would need to get to a decent zero-copy pipeline. Ultimately we should record it at same time via omxh26Xenc… (which we can do with one kmssink)
Even more ultimately would be to use live video streaming in fpga all-together so that we don’t touch memory at all for the video frames.
So would be nice to have some idea howto progress this further, if anyone have experience running similar with xilinx (or perhaps other fpga) hw. I assume we have issues since our older gst level don’t have DMA_DRM format concept which some talks indicates would be good step forward when doing dmabuf pipelines, judging references like https://www.youtube.com/watch?v=wAQiJLDGuRQ
But unsure if this would help just by itself..
GST doc points to start tee branches with queue, to get threading at those points so the branches don’t block each other. But the queue seems to make that we do copying of frames…
Any other way to have branches threaded without queues?
Can we enforce zero-copy somehow in the pipeline? (setting io-mem to dmabuf in itself wouldnt require zero-copy, right? that is more like saying we want to use dma for copying instead of cpu-cycles…)