Hi,
I have a custom board based on AM5728 SOC with omnivision 0v5640 camera installed on it running linux. I want to support maximum 4 party conference streams using gstreamer.
POC done for three party conference streaming is as follows:-
Three custom boards namely A, B and C were taken.
On A, a gstreamer pipeline was run to open self camera feed and its self video and audio streams were sent to both B and C. B and C were able to receive the video and audio streams as receiving pipelines were run on each.
Also, on B and C pipelines were run to send their respective video and audio streams to A.
It is needed to display the feeds of both B and C at A. How can that be achieved?
The gstreamer pipelines used are as below:-
######################
Sender Pipeline at A
######################
B=192.168.8.139 #(IP address of party B)
C=192.168.8.140 #(IP address of party C)
A_BVidSendPort=5000
A_BAudSendPort=5001
A_CVidSendPort=5002
A_CAudSendPort=5003
gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=4 ! \
'video/x-raw,format=(string)YUY2,width=1280,height=720,framerate=(fraction)30/1' ! \
vpe num-input-buffers=8 ! \
queue ! ducatih264enc intra-interval=1 ! \
h264parse ! \
rtph264pay mtu=200 ! \
multiudpsink clients=$B:$A_BVidSendPort,$C:$A_CVidSendPort alsasrc device='hw:0,0' ! \
audioconvert ! \
'audio/x-raw,channels=1,depth=16,width=16,rate=16000,encoding-name=(string)L16' ! \
rtpL16pay ! \
multiudpsink clients=$B:$A_BAudSendPort,$C:$A_CAudSendPort sync=false
######################
Receiver Pipeline at A
######################
#TBD
#####################
Sender Pipeline at B
#####################
B=192.168.8.138 # (IP address of party A)
A=192.168.8.139 # (IP address of party B)
A_BVidSendPort=5004
A_BAudSendPort=5005
gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=4 ! \
'video/x-raw,format=(string)YUY2,width=1280,height=720,framerate=(fraction)30/1' ! \
vpe num-input-buffers=8 ! \
queue ! ducatih264enc intra-interval=1 ! \
h264parse ! \
rtph264pay mtu=200 ! \
udpsink host=$B port=$A_BVidSendPort alsasrc device='hw:0,0' ! \
audioconvert ! \
'audio/x-raw,channels=1,depth=16,width=16,rate=16000,encoding-name=(string)L16' ! \
rtpL16pay ! \
udpsink host=$B port=$A_BAudSendPort sync=false
########################
#Receiver Pipeline at B
########################
B_AVidSendPort=5000
B_AAudSendPort=5001
gst-launch-1.0 -v udpsrc port=$B_AVidSendPort ! \
'application/x-rtp,media=(string)video, clock-rate=(int)90000, payload=(int)96' ! \
rtph264depay ! \
h264parse ! \
ducatih264dec ! \
vpe ! \
'video/x-raw,format=(string)NV12, width=1280,height=1232' ! \
videoscale sharpen=1 ! \
kmssink sync=false \
udpsrc port=$B_AAudSendPort ! \
"application/x-rtp,media=(string)audio,clock-rate=(int)16000, width=16, height=16,\
encoding-name=(string)L16,\
encoding-params=(string)1, channels=(int)1, channel-positions=(int)1,\
payload=(int)96" !\
rtpL16depay ! audioconvert ! alsasink sync=false
#######################
#Sender pipeline at C
######################
B=192.168.8.138 # (IP address of party A)
A=192.168.8.140 # (IP address of party C)
A_BVidSendPort=5006
A_BAudSendPort=5007
gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=4 ! \
'video/x-raw,format=(string)YUY2,width=1280,height=720,framerate=(fraction)30/1' ! \
vpe num-input-buffers=8 ! \
queue ! ducatih264enc intra-interval=1 ! \
h264parse ! \
rtph264pay mtu=200 ! \
udpsink host=$B port=$A_BVidSendPort alsasrc device='hw:0,0' ! \
audioconvert ! \
'audio/x-raw,channels=1,depth=16,width=16,rate=16000,encoding-name=(string)L16' ! \
rtpL16pay ! \
udpsink host=$B port=$A_BAudSendPort sync=false
########################
#Receiver pipeline at C
#######################
B_AVidSendPort=5002
B_AAudSendPort=5003
gst-launch-1.0 -v udpsrc port=$B_AVidSendPort ! \
'application/x-rtp,media=(string)video, clock-rate=(int)90000, payload=(int)96' ! \
rtph264depay ! \
h264parse ! \
ducatih264dec ! \
vpe ! \
'video/x-raw,format=(string)NV12, width=1280,height=1232' ! \
videoscale sharpen=1 ! \
kmssink sync=false \
udpsrc port=$B_AAudSendPort ! \
"application/x-rtp,media=(string)audio,clock-rate=(int)16000, width=16, height=16,\
encoding-name=(string)L16,\
encoding-params=(string)1, channels=(int)1, channel-positions=(int)1,\
payload=(int)96" !\
rtpL16depay ! audioconvert ! alsasink sync=false
This is the POC for 3 party conference streaming. Since there are multiple piplelines already in 3 party POC, 4 party conference will have furthermore pipelines added to these.
Wouldn’t it affect the system performance as the number of pipelines are increased?
Also can the sender and receiver pipelines be improved using gstreamer’s mux feature?