Using new unixfdsrc/sink with fallbackswitch to fallback to black and silence

Application Development
1

Hello GStreamer guru’s.
I have a application which needs a feed to stay live not to cause the whole pipeline to EOS. fallbackswitch is a great offering to use however I’d thought I’ll be smart and use the new unixfdsrc/sink elements over using shmsink and shmsrc as these are new an use DMA with zero copy?

The shm arrangement requires to know the buffer size etc. Furthermore, I wanting to split the sender and receiver over different thread boundaries working toward having to desperate programs.

I’m using Ubuntu 24.04 LTS with gstreamer 1.24.2.

I’ve been experimenting with gst-launch-1.0 to test my theory however, after closing the sender, the receiver EOS and closes.

Any tips on making the receiver switch to a blank screen (black) with audio silence?

SENDER

gst-launch-1.0 -v multifilesrc location=starwars.ts loop=true ! identity sync=true ! tsdemux name=demux demux. ! h264parse ! nvh264dec ! videoconvert ! video/x-raw,format=I420,width=1920,height=1080,framerate=25/1 ! queue ! unixfdsink socket-path=/tmp/video_HD1 demux. ! decodebin ! audioconvert ! audio/x-raw,format=S16LE,rate=48000,channels=2 ! queue ! unixfdsink socket-path=/tmp/audio_HD1

RECEIVER

gst-launch-1.0 -v
unixfdsrc socket-path=/tmp/video_HD1 ! identity sync=true ! fallbackswitch name=v ! videoconvert ! video/x-raw,format=I420,width=1920,height=1080,framerate=25/1 !
queue ! autovideosink
unixfdsrc socket-path=/tmp/audio_HD1 ! identity sync=true ! fallbackswitch name=a ! audioconvert ! audio/x-raw,format=S16LE,rate=48000,channels=2 ! queue ! autoaudiosink
videotestsrc pattern=black is-live=true ! video/x-raw,format=I420,width=1920,height=1080,framerate=25/1 ! v.
audiotestsrc wave=silence is-live=true ! audio/x-raw,format=S16LE,rate=48000,channels=2 ! a.

What’s my issue here?

2

With unixfd, supported events are now serialized and deserialized over the Unix socket. This is a great advantage for GstVideoMeta notably, allowing a lot more zero-copy of these high bandwidth stream. Though, EOS events are also passed over.

The solution would be to drop the EOS event. One way to do so would be to add padprobes to unixfdsink sink pads. Then drop the data if it’s an Eos event. You will have to track the EOS state on each sink, when all sinks have received EOS, you can emit an EOS message, that will mimic (in a simpler fashion) what is normally done by sinks and pipeline.

Future thinking, we could add a property to not forward EOS in unixfdsink.

3

Hello @ndufresne.

Thanks you for your feedback… I was hoping some outcome to use in gst-launch-1.0 as a prototype however, to so checking of EOS needs to be constructed in code. I’ve been using Go and go-gst/go-gst package to implement my gstreamer pipelines and shall play with unixfdsrc/sink with it. I have had partial success with the shm once the buffer size was correctly set. With fallbackswitch I can’t seem to get to EOS from destroying the pipeline when the sender is closed. Soon as the thread is exited my receiver exits.
Another method is to use a compositor with identity sync=true to effectively monitor the pipeline for EOS and keep the pipeline running and signal a switch zorder change to keep a black screen from EOS the rest of the pipeline.

I’m sure this method has been used considerably over the years however, I just can’t find how to implement such in Go…

4

unixfdsink and src don’t seem to work on different instances. I’m having better luck with shmsrc and shmsink as long as the buffersize is correctly sized.

Anyone using shm before and has any pointers?

5

Did you configure the socket patch correctly?

6

Hi @ndufresne ,
Unfortunately I abandoned using unixfdsink/src due to lack of information on how to use it correctly? Not sure if I’m able to be used across different instances?
Being new to the suite, may take time before anyone has any practical implementation.

I’ve gone back to using shmsink and shmsrc for the moment.

7

I am sorry it didn’t work for you. I do wonder what did not work for you in the instruction found in the documentation ?

https://gstreamer.freedesktop.org/documentation//unixfd/unixfdsink.html

8

Hello @ndufresne ,

After sending a reply, I again had another play with unixfdsink and unixfdsrc elements.

The example from the gstreamer.freedesktop.org doesn’t work on Linux. Need updating to include a video cap.

 gst-launch-1.0 -v videotestsrc ! unixfdsink socket-path=/tmp/blah
 gst-launch-1.0 -v unixfdsrc socket-path=/tmp/blah ! autovideosink

After adding video caps it works!

Start in first instance.

 gst-launch-1.0 -v videotestsrc !  video/x-raw,format=I420,width=1920,height=1080,framerate=25/1 ! queue ! unixfdsink socket-path=/tmp/blah

Start the second instance.

 gst-launch-1.0 -v unixfdsrc socket-path=/tmp/blah  !  video/x-raw,format=I420,width=1920,height=1080,framerate=25/1 ! queue ! autovideosink
9

The caps are being serialized and passed through the socket (that and most metadata now) so this should not be required, I’ll check again tomorrow.

10

I tried here and got a caps nego issue on the receiving pipeline. After adding a videoconvert before the sink, it worked (without specifying caps).

11

Strangely A444_16LE was negotiated on the sender pipeline. That seems a bit odd :slight_smile:

12

I know this thread is old.

If the same caps are used at source (sink) and end points (source) both unixfdsink and unixfdsrc work.

13

I’ve been using unixfd plugin for a large projects and multiple services (GThreads) 1-5 connecting to one unixfdsink endpoint.
Works however after a 24 hours the application crashes with >1024 fd

After extensive investigation, I am seeing retained memfd:gst-shm file descriptors in the producer process after a unixfdsrc client disconnects from unixfdsink.

Environment:

  • GStreamer: 1.28.3

  • Platform: Linux x86_64

  • Plugin: unixfd, unixfdsink / unixfdsrc

  • Use case: long-running producer pipeline serving MPEG-TS over unixfdsink socket-path=/tmp/channel-1-internet.ts.ufd

  • Client: short-lived gst-launch-1.0 unixfdsrc ... ! fakesink

The issue appears on the producer side. After the client exits, the client process is gone, but the producer process still holds hundreds of memfd:gst-shm FDs.

Minimal reproducer pattern:

Producer is a long-running pipeline ending in something equivalent to:

... ! queue ! unixfdsink socket-path=/tmp/channel-1-internet.ts.ufd sync=false async=false

Client attach/detach test:

pid=$(pidof datv-repeater)

echo -n "before gst-shm="
for f in /proc/$pid/fd/*; do readlink "$f" 2>/dev/null; done | grep -c "memfd:gst-shm"

timeout --signal=INT 10 gst-launch-1.0 -e -q \
  unixfdsrc socket-path=/tmp/channel-1-internet.ts.ufd \
  ! queue \
  ! fakesink sync=false

sleep 5

echo -n "after gst-shm="
for f in /proc/$pid/fd/*; do readlink "$f" 2>/dev/null; done | grep -c "memfd:gst-shm"

Observed with the stock unixfd plugin:

before gst-shm=2
after gst-shm=439

In longer repeated testing the producer reached more than 700 total FDs, mostly:

memfd:gst-shm

The socket count returned to normal, and no gst-launch, ffplay, ffprobe, or vlc clients remained running, so the retained FDs were still owned by the producer process.

Expected behaviour:

After the last unixfdsrc client disconnects, unixfdsink should release any per-client buffer references and drop/flush any shm/memfd allocator pool memory that is only useful while clients are connected. The producer process should return close to its baseline FD count.

Tested local fix:

I built an out-of-tree patched libgstunixfd.so based on 1.28.3. The effective changes were:

  1. If there are no clients connected and wait-for-connection=false, drop incoming buffers before copying/exporting them into shm/memfd.

  2. On last client disconnect, flush/drop the unixfd shm allocator pool.

  3. If a send failure removes the last client, also flush/drop the allocator pool.

  4. On element stop/finalize, explicitly flush/drop allocator-held memory.

  5. Ensure disconnect/HUP/ERR/NVAL paths remove the client and free its buffer tracking structures.

With the patched plugin installed:

Version: 1.28.3-fd-hygiene-v3

The same attach/detach test returns to baseline:

before gst-shm=0
after gst-shm=0

During an active RTMP remux client test, the producer held a small live working set:

gst-shm=27/28

After the client was stopped cleanly with Ctrl+C, it returned to:

gst-shm=0

This suggests the retained FDs are caused by unixfdsink/allocator state not being released when the client disconnects, rather than by unixfdsrcto a point where my application quits.

It may be useful to hook this cleanup to the existing client tracking / num-clients state so that when the client count transitions to zero, unixfdsink releases stale client buffers and allocator-held memfd/shm memory.

I can provide the out-of-tree patch used for testing if useful.