GstForceKeyUnit in "webrtcbin"

yuyou · August 21, 2024, 12:11pm

Hi,

As the original discussion is closed at

github.com/centricular/gstwebrtc-demos

Example code of how to force keyframe with GstForceKeyUnit?

opened 08:46PM - 05 May 20 UTC

closed 09:06AM - 28 Apr 21 UTC

gicu8ab2

The problem persists that the video stream from browser back to sendrecv demo is… corrupted when network packet loss occurs. The previously identified solution to this problem of forcing keyframes to be sent was given by Matthew Waters below. However, I'm having trouble implementing this approach. Could you provide a sample code snippet that's compatible with webrtc-sendrecv.c or webrtc-sendrecv.py that implements this suggested solution? This would go a long way to making this demo workable. ------------------------------------------------------------------------------------------------------------------ One can force a keyframe by sending a GstForceKeyUnit request upstream on the src pad of the video stream output by webrtcbin. This is converted internally by GStreamer's rtp implementation into a fir/pli rtcp message that is sent to the peer: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/blob/master/gst/rtpmanager/rtpsession.c#L4435 . https://gstreamer.freedesktop.org/documentation/design/keyframe-force.html#upstream _Originally posted by @ystreet in https://github.com/centricular/gstwebrtc-demos/issues/61#issuecomment-438915464_

The remaining question seems to be unclear, whether or not the latest “webrtcbin” or “rtpsession” in 1.22 or 1.24 has ever implemented the RTCP PLI or FIR feedback and issue “GstForceKeyUnit” event to upstream like encoders to generate a new key frame?

If not, so it will leave the logic to the application to publish the event. That would be fine but just need to know it is the case or not. Also it would be grateful if a pointer to the source code would be available to show the implementation if it exists.

Thanks.

Yu

slomo · August 21, 2024, 1:39pm

It’s supported in general, but that was always the case. What kind of behaviour are you looking for specifically?

yuyou · August 21, 2024, 2:04pm

is it possible to share any direct link to the source code in Gstreamer mono repo where the implementation is done? There’s nothing specifically but just want to verify the feature has been supported by GStreamer WebRTC stack., e.g. emitting a key frame event when a PLI or FIR event from a browser (e.g. Chrome).

slomo · August 21, 2024, 2:23pm

That’s supported since before WebRTC even existed. Check commit a61bb9e94b23f6dbed0319d6f018a8d857b2edcb.

yuyou · August 22, 2024, 5:52am

Thanks for the pointer. It is exactly what I was looking for.

One more question, the key unit request was triggered in the case of GST_RTCP_PSFB_TYPE_PLI only. How about the GST_RTCP_PSFB_TYPE_FIR (Full Intra Request Command)?

It is fine to response to PLI type. Just curious to the FIR type.

ystreet00 · August 22, 2024, 5:59am

Both PLI and FIR are supported and will result in GstForceKeyUnit events being produced.

slomo · August 22, 2024, 6:14am

That was added shortly afterwards. Look at the code / git history, it’s all there