I’ve asked this on SO (c++ - Enable RED on webrtcbin for audio - Stack Overflow), but got nowhere, so I’m trying here.
General description:
I’ve got a media server that receives SDP offers from endpoints, then acts as a middleman between them all, so I’ll have a pipeline a bit like this:
endpoint1 <--> server <--> endpoint2
--> WatcherOfEndpoint1
So my server has components matches up to the various endpoints, which include a webrtcbin that it listens to, gets streams from, decodes them to video/x-raw or audio/x-raw and then the components communicate with each other via appsink/appsrc.
However, sometimes our audio from one of the endpoints (say endpoint1) isn’t great. The other endpoints can’t hear it very well, even though endpoint1 can hear everyone else just fine. So I would like to try and beef up the audio on that one endpoint a bit more, using RED (as suggested here - RED: Improving Audio Quality with Redundancy - webrtcHacks).
Making dot diagrams of my pipelines suggest that webrtcbin should be capable of doing RED internally, without me adding elements to my pipelines (though I’ve tried that, see the SO post). The trouble is, even if I munge the SDP offer from my Vivaldi browser so that the red line is above the opus line, my answer prefers opus. The only way I can get red to be the number one choice in my answer is by removing opus entirely from the m-line (although I wonder if that causes problems with it understanding what’s within the red-encoded stream?)
If I get-transceivers, initially kind isn’t set, so I can’t target just the audio, and if I set both of them to use fec-type and do-nack, and then revert the video one back, I get the webrtcbin emitting just the video pad-added signal, then complaining that it can’t stream because it’s not negotiated. If I do nothing, or set the audio transceiver when I finally have the kind, then I get video, but the audio pad-added signal never fires.
Any suggestions welcome, I’m pretty much out of ideas!