I’m trying to do something seemingly simple by muxing 2 live camera video sources (v4l2src) and a video file source (uridecodebin/playbin) and encoding the output.
I have something like this:
v4l2src device1 ! tee name=t1
v4l2src device2 ! tee name=t2
t1. ! queue ! videoconvert ! appsink name=appsink1 //used to preview device 1
t2. ! queue ! videoconvert ! appsink name=appsink2 //used to preview device 2
input-selector name=selector ! x264enc ! h264parse ! dashsink //encode the output of the selector
t1. ! queue ! videoconvert ! capsfilter <device1 caps == device2 caps> ! selector.sink_0 //hook device1 into selector input
t2. ! queue ! videoconvert ! capsfilter <device1 caps == device2 caps> ! selector.sink_1 //hook device2 into selector input
playbin or decodebin uri ! tee name=tf //video file source
tf. ! queue ! videoconvert ! appsink name=appsinkF //used to preview video file source
tf. ! queue ! videoconvert ! capsfilter <device1 caps == device2 caps> ! selector.sink_2 //hook video file source into selector input
this works if I use uridecodebin (all previews are live, selector output is being encoded, and active-pad can be swapped between any of the 3 inputs), but I’d like to be able to pause the video file source playback if it’s not the active-pad on the input-selector.
if I try swapping the uridecodebin to a playbin to get play/pause capabilities, I have to use something like gstinterpipe’s interpipesrc/interpipesink instead of tee as playbin requires a sink. however I tried that and when I build my pipeline and start it, my entire pipeline freezes/stalls. the device1 and device2 previews render a single frame of video and is frozen while the playbin preview is empty.
I’m not really sure how to debug this or progress past this. I’ve tried looking at the debug logs (GST_DEBUG=4) and also looking at the dot map image (I can see with uridecodebin the pipeline looks correct and with playbin the playbin branch looks segmented) but I’m not exactly sure what it is I’m looking for. If anyone has any ideas I’d really appreciate it.
(I am using gst1-java-core bindings, gstreamer 1.24.2 and isobmff/gstinterpipe plugins)