Hello, trying to use the pipeline below
gst-launch-1.0 -f \
uridecodebin3 name=decode uri=file:///app/vid.mp4 ! queue2 ! videoconvert ! videoscale ! video/x-raw,width=1920,height=1080 ! \
compositor name=comp ! videoconvert ! x264enc bitrate=5120 tune=zerolatency speed-preset=veryfast key-int-max=30 ! queue2 ! flvmux name=mux streamable=TRUE ! rtmp2sink location=rtmp://localhost:1936/live/live \
decode. ! queue2 ! audioconvert ! avenc_aac ! aacparse ! mux. \
wpesrc location=http://localhost draw-background=FALSE ! video/x-raw,width=1920,height=1080,framerate=\(fraction\)30/1 ! videoconvert ! queue2 ! comp.
The main idea is that I’m trying to stream a video that has audio content while adding a web source overlay. So we combine the video branch from the decodebin with the wpesrc using the compositor element. However when passing the video + audio content to the flvmux, we get the following:
0:00:00.206039225 4444 0x60c6bc5968c0 FIXME aggregator gstaggregator.c:1497:gst_aggregator_loop:<mux> Subclass should call gst_aggregator_selected_samples() from its aggregate implementation.
0:00:00.260199404 4444 0x60c6bc5968c0 WARN flvmux gstflvmux.c:1296:gst_flv_mux_buffer_to_tag_internal:<mux:video> Got backwards dts! (0:00:00.133000000 < 0:00:00.139000000)
0:00:00.325690836 4444 0x60c6bc5968c0 WARN flvmux gstflvmux.c:1296:gst_flv_mux_buffer_to_tag_internal:<mux:video> Got backwards dts! (0:00:00.200000000 < 0:00:00.208000000)
0:00:00.393012018 4444 0x60c6bc5968c0 WARN flvmux gstflvmux.c:1296:gst_flv_mux_buffer_to_tag_internal:<mux:video> Got backwards dts! (0:00:00.266000000 < 0:00:00.278000000)
0:00:00.434020596 4444 0x60c6bc5968c0 WARN flvmux gstflvmux.c:1296:gst_flv_mux_buffer_to_tag_internal:<mux:video> Got backwards dts! (0:00:00.300000000 < 0:00:00.301000000)
0:00:00.495217744 4444 0x60c6bc5968c0 WARN flvmux gstflvmux.c:1296:gst_flv_mux_buffer_to_tag_internal:<mux:video> Got backwards dts! (0:00:00.366000000 < 0:00:00.371000000)
0:00:00.556334828 4444 0x60c6bc5968c0 WARN flvmux gstflvmux.c:1296:gst_flv_mux_buffer_to_tag_internal:<mux:video> Got backwards dts! (0:00:00.433000000 < 0:00:00.441000000)
0:00:00.637664059 4444 0x60c6bc5968c0 WARN flvmux gstflvmux.c:1296:gst_flv_mux_buffer_to_tag_internal:<mux:video> Got backwards dts! (0:00:00.500000000 < 0:00:00.510000000)
0:00:00.661593407 4444 0x60c6bc5968c0 WARN flvmux gstflvmux.c:1296:gst_flv_mux_buffer_to_tag_internal:<mux:video> Got backwards dts! (0:00:00.533000000 < 0:00:00.534000000)
0:00:00.692337634 4444 0x60c6bc5968c0 WARN flvmux gstflvmux.c:1296:gst_flv_mux_buffer_to_tag_internal:<mux:video> Got backwards dts! (0:00:00.566000000 < 0:00:00.580000000)
0:00:00.722230563 4444 0x60c6bc5968c0 WARN flvmux gstflvmux.c:1296:gst_flv_mux_buffer_to_tag_internal:<mux:video> Got backwards dts! (0:00:00.600000000 < 0:00:00.603000000)
0:00:00.768306195 4444 0x60c6bc5968c0 WARN flvmux gstflvmux.c:1296:gst_flv_mux_buffer_to_tag_internal:<mux:video> Got backwards dts! (0:00:00.633000000 < 0:00:00.650000000)
I believe the muxer is not able to properly combine the video and audio content with the correct timestamps after the video+websource got passed to the compositor? Any idea how can I navigate this?
Edit:
I was able to solve this by delaying linking the wpe branch to the compositor for a few seconds in my rust code. Though the same problem happens when the video finishes playing and I switch to another video by updating the uri
parameter on the uridecodebin3
on the about-to-finish
signal. Thoughts?
snippet of the signal handler
uridecodebin.set_property("instant-uri", true);
let uridecodebin = Arc::new(uridecodebin);
let decodebin_clone = Arc::clone(&uridecodebin);
uridecodebin.connect("about-to-finish", true, move |data| {
println!(
"================= About to finish ================ {:?}",
data
);
decodebin_clone.set_property("uri", FILE2_PATH);
None
});