Webrtcbin video latency increases when an audio source is added

Application Development
1

webrtcbin video latency increases when an audio source is added.Is there an overall increase in latency due to audio and video synchronization issues? How can this delay be reduced？
(1)When there is only video streaming in webrtcbin, the latency is low.
gst_parse_launch(“webrtcbin bundle-policy=max-bundle name=sendrecv "
“d3d11screencapturesrc name=screencap ! capsfilter caps=”
DIRECTX_PAD”,framerate=60/1 name=capsfilter ! " QUEUE
"d3d11convert ! “DIRECTX_PAD”,format=NV12 ! " QUEUE
"nvd3d11h265enc name=videoencoder ! " QUEUE
"rtph265pay name=rtp ! " QUEUE
RTP_CAPS_VIDEO "H265 ! sendrecv. "
, &error);
(2)When adding audio streams to webrtcbin, the latency increases significantly.
gst_parse_launch(“webrtcbin bundle-policy=max-bundle name=sendrecv "
“d3d11screencapturesrc name=screencap ! capsfilter caps=”
DIRECTX_PAD”,framerate=60/1 name=capsfilter ! " QUEUE
"d3d11convert ! “DIRECTX_PAD”,format=NV12 ! " QUEUE
"nvd3d11h265enc name=videoencoder ! " QUEUE
"rtph265pay name=rtp ! " QUEUE
RTP_CAPS_VIDEO "H265 ! sendrecv. "
“wasapisrc loopback=true name=audiocapsrc !” QUEUE
"audioconvert ! " QUEUE
"audioresample ! " QUEUE
"opusenc name=audioencoder ! " QUEUE
"rtpopuspay ! " QUEUE
RTP_CAPS_AUDIO "OPUS ! sendrecv. ", &error);

2

What is “significantly”?

Have you tried configuring the audio source latency via its properties?

3

1、I configured wasapisrc’s low_latency parameter with the following code.
core->audio_sound_encoder = gst_bin_get_by_name(GST_BIN(core->pipeline), “audioencoder”);
g_object_set(core->audio_sound_source, “low_latency”, TRUE, NULL);
2、I configured the webrtcbin build below
g_object_set(core->webrtcbin,“latency”, 0, NULL);
3、I don’t know what configuration to do with webrtcbin, and I can’t find anything about it in the parameters exposed by the webrtcbin build.I found someone on stack overflow who had the same problem, but no one had a solution.
[gstreamer - webrtcbin video latency increases when an audio source is added - Stack Overflow](https://webrtcbin video latency increases when an audio source is added)

4

I have solve this problem.I use autoaudiosink element in client,the problem is disappear when i set the “sync” paramenter to “false” of autoaudiosink.

static void
handle_audio_stream (GstPad * pad, 
                     PoleisWebrtcClient* core)
{
    GstElement* pipeline = core->pipeline;

    core->audio_convert = gst_element_factory_make ("audioconvert", NULL);
    core->audio_resample = gst_element_factory_make ("audioresample", NULL);
    core->audio_sink =    gst_element_factory_make ("autoaudiosink", NULL);

    /* Might also need to resample, so add it just in case.
    * Will be a no-op if it's not required. */
    gst_bin_add_many (GST_BIN (core->pipeline), 
        core->audio_convert, 
        core->audio_resample, 
        core->audio_sink, NULL);

    gst_element_sync_state_with_parent (core->audio_convert);
    gst_element_sync_state_with_parent (core->audio_resample);
    gst_element_sync_state_with_parent (core->audio_sink);

    g_object_set(core->audio_sink, "sync", FALSE, NULL);

    gst_element_link_many ( 
        core->audio_queue_convert, 
        core->audio_convert, 
        core->audio_queue_resample,
        core->audio_resample,
        core->audio_queue_sink,
        core->audio_sink, NULL);

    GstPad* queue_pad = gst_element_get_static_pad (core->audio_queue_convert, "sink");
    GstPadLinkReturn ret = gst_pad_link (pad, queue_pad);
    g_assert_cmphex (ret, ==, GST_PAD_LINK_OK);
}