Selecting audio track from MKV file fails in playbin3

General Discussion
1

Hey!

I’m using playbin3 to play MKV file.

What i have found is that playbin3 is unable to select an audio stream from this file. Externally it looks like property current-audio (I used other approach in C++ with message handling and selecting streams with gst_event_new_select_streams) works well. But no data is ever move over the pipeline and if I look into pipeline structure i see that internally only one particular audio track is ever read, and internally pipeline in not connected.

See the picture (decodebin3-0/multiqueue0, sink_1/src_1 and sink_3/src4):

image
image1920×822 175 KB

I’m not seeing anything suspicious here. But audio track selection does not work at all.

Is there anything more I can look into?

2

What GStreamer version are you using?

Does it work with the gst-play-1.0 command line player? You can switch tracks using the a and v keys. You may need to pass --use-playbin3 to make it use playbin3 if your GStreamer version is older.

3

Thanks for the reply!

I’m using GStreamer 1.26.6 customly built.

Interestingly, gst-play works and is able to select all the streams. I have no video though (a bit of misconfiguration, GStreamer is unable to configure video sink). This made me to try another approach and …

When i only extract audio track and send it to WebRTC i have all the streams accessible!

And this is what happens internally:

image
image1920×1244 202 KB

I’m starting to suspect that the issue is with video track starting many seconds before audio, thus filling and blocking some queues while having others empty.

My pipeline looks like this (it is generated, not manually created):

playbin3 name=pb uri=file:///home/akanashin/WORK/video/mfl.mkv

capsfilter name=stream_1_v1 caps=video/x-raw
capsfilter name=stream_1_a1 caps=audio/x-raw

webrtcbin name=webrtcbin stun-server=stun://stun.l.google.com:19302 bundle-policy=max-bundle

stream_1_v1.
! queue ! videoconvertscale ! videorate ! video/x-raw,format=I420,format=I420,width=3584,height=2160,framerate=24000/1001
! x264enc bitrate=3000 speed-preset=ultrafast tune=zerolatency key-int-max=15
! rtph264pay mtu=1000 config-interval=-1 aggregate-mode=zero-latency pt=96
! application/x-rtp,a-mid=“f_v_v1”
! webrtcbin.

stream_1_a1.
! queue ! audioconvert ! audioresample ! audio/x-raw,format=S16LE,rate=48000,channels=2,layout=interleaved
! opusenc perfect-timestamp=true
! rtpopuspay mtu=1000 pt=97
! application/x-rtp,a-mid=“f_a_a1”
! webrtcbin.

This pipeline just never actually starts after all required negotiations. Handling only video or only audio makes it work.

4

So, by the end it looks like audio stream starts lately and this makes webrtcbin not be able to configure it’s internal pipeline. The issue is solved by adding audiomixer between audioresample and capsfilter.