Whipclientsink The SDP negotiation for simultaneously pushing audio and video streams is in chaos

[mifan]

When I use the following statement to push streams and capture packets, during SDP negotiation, sometimes the video port is 0 and sometimes the audio port is 0

gst-launch-1.0 d3d11screencapturesrc ! video/x-
raw,width=1920,height=1080,framerate=30/1 ! queue ! videoconvert ! x264enc speed-
preset=ultrafast tune=zerolatency ! video/x-h264,profile=baseline ! queue ! 
whipclientsink name=whip_sink signaller::whip-endpoint==$WHIP_ENDPOIN wasapisrc 
loopback=true low-latency=true ! audioconvert ! audioresample ! opusenc ! queue ! 
whip_sink. 

I’ve spent many hours bashing my head into the same limited set of problems, and am bewildered, and frustrated by the limited documentation available. Any insight would be appreciated.

[Sanchayan]

Please note the bundle attribute at the session level. Both audio and video are bundled together. Either of the below two should not be a problem.

a=group:BUNDLE audio0 video1
a=group:BUNDLE video0 audio1

Please see RFC8866 and RFC 8843 which states the following,

updates RFC 3264 [RFC3264], to also allow assigning a zero port value to an “m=” section in cases where the media described by the “m=” section is not disabled or rejected.

Port being 0 should not be a problem.

There should also a bundle-only attribute in that SDP which is probably clipped in the screenshot. See Section 1.4 of RFC8866 for the same.

[mifan]

This is the complete SDP. Please take a look at that side

[Sanchayan]

Can you clarify more about the problems you see aside from the SDP?

I see ZLMediaKit as the server. Do you have problems on the WHEP side or pulling stream from ZLMediaKit?

[mifan]

When transmitting the stream, the audio stream was not penetrated. The audio bitrate is 0 when pulling the stream

[Sanchayan]

It is possible that the stream being pulled from ZLMediaKit is the problem due to how it handles OPUS. See related issue.

[BUG] WHIP->SRT with Opus creates invalid MPEG-TS

There is a fix in GStreamer upstream for the same with this MR.

You might want to try another server like mediamtx and see if the problem is reproducible.

[mifan]

Thank you for your guidance, but I still think there is a problem with the command