Using new unixfdsrc/sink with fallbackswitch to fallback to black and silence

Application Development
1

Hello GStreamer guru’s.
I have a application which needs a feed to stay live not to cause the whole pipeline to EOS. fallbackswitch is a great offering to use however I’d thought I’ll be smart and use the new unixfdsrc/sink elements over using shmsink and shmsrc as these are new an use DMA with zero copy?

The shm arrangement requires to know the buffer size etc. Furthermore, I wanting to split the sender and receiver over different thread boundaries working toward having to desperate programs.

I’m using Ubuntu 24.04 LTS with gstreamer 1.24.2.

I’ve been experimenting with gst-launch-1.0 to test my theory however, after closing the sender, the receiver EOS and closes.

Any tips on making the receiver switch to a blank screen (black) with audio silence?

SENDER

gst-launch-1.0 -v multifilesrc location=starwars.ts loop=true ! identity sync=true ! tsdemux name=demux demux. ! h264parse ! nvh264dec ! videoconvert ! video/x-raw,format=I420,width=1920,height=1080,framerate=25/1 ! queue ! unixfdsink socket-path=/tmp/video_HD1 demux. ! decodebin ! audioconvert ! audio/x-raw,format=S16LE,rate=48000,channels=2 ! queue ! unixfdsink socket-path=/tmp/audio_HD1

RECEIVER

gst-launch-1.0 -v
unixfdsrc socket-path=/tmp/video_HD1 ! identity sync=true ! fallbackswitch name=v ! videoconvert ! video/x-raw,format=I420,width=1920,height=1080,framerate=25/1 !
queue ! autovideosink
unixfdsrc socket-path=/tmp/audio_HD1 ! identity sync=true ! fallbackswitch name=a ! audioconvert ! audio/x-raw,format=S16LE,rate=48000,channels=2 ! queue ! autoaudiosink
videotestsrc pattern=black is-live=true ! video/x-raw,format=I420,width=1920,height=1080,framerate=25/1 ! v.
audiotestsrc wave=silence is-live=true ! audio/x-raw,format=S16LE,rate=48000,channels=2 ! a.

What’s my issue here?

2

With unixfd, supported events are now serialized and deserialized over the Unix socket. This is a great advantage for GstVideoMeta notably, allowing a lot more zero-copy of these high bandwidth stream. Though, EOS events are also passed over.

The solution would be to drop the EOS event. One way to do so would be to add padprobes to unixfdsink sink pads. Then drop the data if it’s an Eos event. You will have to track the EOS state on each sink, when all sinks have received EOS, you can emit an EOS message, that will mimic (in a simpler fashion) what is normally done by sinks and pipeline.

Future thinking, we could add a property to not forward EOS in unixfdsink.

3

Hello @ndufresne.

Thanks you for your feedback… I was hoping some outcome to use in gst-launch-1.0 as a prototype however, to so checking of EOS needs to be constructed in code. I’ve been using Go and go-gst/go-gst package to implement my gstreamer pipelines and shall play with unixfdsrc/sink with it. I have had partial success with the shm once the buffer size was correctly set. With fallbackswitch I can’t seem to get to EOS from destroying the pipeline when the sender is closed. Soon as the thread is exited my receiver exits.
Another method is to use a compositor with identity sync=true to effectively monitor the pipeline for EOS and keep the pipeline running and signal a switch zorder change to keep a black screen from EOS the rest of the pipeline.

I’m sure this method has been used considerably over the years however, I just can’t find how to implement such in Go…

4

unixfdsink and src don’t seem to work on different instances. I’m having better luck with shmsrc and shmsink as long as the buffersize is correctly sized.

Anyone using shm before and has any pointers?

5

Did you configure the socket patch correctly?

6

Hi @ndufresne ,
Unfortunately I abandoned using unixfdsink/src due to lack of information on how to use it correctly? Not sure if I’m able to be used across different instances?
Being new to the suite, may take time before anyone has any practical implementation.

I’ve gone back to using shmsink and shmsrc for the moment.

7

I am sorry it didn’t work for you. I do wonder what did not work for you in the instruction found in the documentation ?

https://gstreamer.freedesktop.org/documentation//unixfd/unixfdsink.html

8

Hello @ndufresne ,

After sending a reply, I again had another play with unixfdsink and unixfdsrc elements.

The example from the gstreamer.freedesktop.org doesn’t work on Linux. Need updating to include a video cap.

 gst-launch-1.0 -v videotestsrc ! unixfdsink socket-path=/tmp/blah
 gst-launch-1.0 -v unixfdsrc socket-path=/tmp/blah ! autovideosink

After adding video caps it works!

Start in first instance.

 gst-launch-1.0 -v videotestsrc !  video/x-raw,format=I420,width=1920,height=1080,framerate=25/1 ! queue ! unixfdsink socket-path=/tmp/blah

Start the second instance.

 gst-launch-1.0 -v unixfdsrc socket-path=/tmp/blah  !  video/x-raw,format=I420,width=1920,height=1080,framerate=25/1 ! queue ! autovideosink
9

The caps are being serialized and passed through the socket (that and most metadata now) so this should not be required, I’ll check again tomorrow.

10

I tried here and got a caps nego issue on the receiving pipeline. After adding a videoconvert before the sink, it worked (without specifying caps).

11

Strangely A444_16LE was negotiated on the sender pipeline. That seems a bit odd :slight_smile: