Hello,
I am conducting research on GStreamer but have encountered some confusion regarding stream mixing. For example, with the following stream mixing code:
"
gst-launch-1.0 -ve videomixer name=m sink_1::xpos=880 sink_1::ypos=570 sink_2::xpos=880 sink_2::ypos=420 ! videoconvert ! x264enc tune=zerolatency threads=0 bitrate=5000 speed-preset=superfast ! mux. flvmux name=mux streamable=true ! rtmp2sink sync=false location=rtmp://0.0.0.0/live/stream
udpsrc uri=udp://0.0.0.0:5005 buffer-size=524288 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! avdec_h264 ! videoconvert ! videoscale ! video/x-raw,width=1080,height=720 ! queue ! m.
udpsrc uri=udp://0.0.0.0:5003 buffer-size=524288 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! avdec_h264 ! videoconvert ! videoscale ! video/x-raw,width=200,height=150 ! queue ! m.
adder name=mix ! queue ! audiorate ! audioconvert ! voaacenc ! mux.audio
udpsrc uri=udp://0.0.0.0:5002 name=thehost ! application/x-rtp, payload=98, encoding-name=OPUS ! rtpopusdepay ! opusdec ! queue ! audiorate ! mix.
udpsrc uri=udp://0.0.0.0:5004 name=voicechat ! application/x-rtp, payload=98, encoding-name=OPUS ! rtpopusdepay ! opusdec ! queue ! audiorate ! mix.
"
I want to run each udpsrc in an asynchronous thread to maximize CPU efficiency. At the same time, I want to adjust the layout of the videomixer’s sink_# in real-time during operation when adding or removing video udpsrc, ensuring that changes in each udpsrc do not affect the mixing effect.
I have tried various methods:
1. gst_bin_add(GST_BIN(mainPipeline), subpipeline);
2. ghost_pad + gst_pad_link
However, none have been successful. Recently, I noticed the “GStreamer Daemon + GstInterpipe” solution, but I am unsure if it is theoretically feasible. Are there any reference solutions available for learning?
Thank you very much for your support, and thank you again!