WebRTC: webrtcsink statistics

Application Development Embedded
1

Hello,
this post is about using webrtcsink/src plugins (version 0.11.4-8335bdb0) on a imx8mp system and paired with the Rust signalling server we got great results.

I’m now trying to retrieve some statistics, in short: I’m querying the plugin property stats, parsing things and forward it to a GRPC stream client for visualization/chat.
Everything works on my computer and Raspberry but on nxp imx8mp I get this:

Jun 19 13:08:40 imx8mp-XXXXX[1754]: GetWebRTCStatsStream
Jun 19 13:08:40 imx8mp-XXXXX[1754]: thread '<unnamed>' panicked at net/webrtc/src/webrtcsink/imp.rs:958:24:
Jun 19 13:08:40 imx8mp-XXXXX[1754]: not implemented: Factory vpuenc_h264 is currently not supported
Jun 19 13:08:40 imx8mp-XXXXX[1754]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Easy to say that this has to do with the imx encoder, but streaming works fine, so I don’t understand, please take care that I’m not using “GoogleCongestion Control algorithm”

Thanks.

2

I’ll cross build , test and see if this is enough…

diff --git a/net/webrtc/src/webrtcsink/imp.rs b/net/webrtc/src/webrtcsink/imp.rs
index cb7f8ebd..43181776 100644
--- a/net/webrtc/src/webrtcsink/imp.rs
+++ b/net/webrtc/src/webrtcsink/imp.rs
@@ -949,7 +949,7 @@ impl VideoEncoder {
     fn bitrate(&self) -> i32 {
         match self.factory_name.as_str() {
             "vp8enc" | "vp9enc" => self.element.property::<i32>("target-bitrate"),
-            "x264enc" | "nvh264enc" | "vaapih264enc" | "vaapivp8enc" | "qsvh264enc" => {
+            "x264enc" | "nvh264enc" | "vaapih264enc" | "vaapivp8enc" | "qsvh264enc" | "vpuenc_h264"=> {
                 (self.element.property::<u32>("bitrate") * 1000) as i32
             }
             "nvv4l2h264enc" | "nvv4l2vp8enc" | "nvv4l2vp9enc" => {
@@ -976,7 +976,7 @@ impl VideoEncoder {
     pub(crate) fn set_bitrate(&mut self, element: &super::BaseWebRTCSink, bitrate: i32) {
         match self.factory_name.as_str() {
             "vp8enc" | "vp9enc" => self.element.set_property("target-bitrate", bitrate),
-            "x264enc" | "nvh264enc" | "vaapih264enc" | "vaapivp8enc" | "qsvh264enc" => self
+            "x264enc" | "nvh264enc" | "vaapih264enc" | "vaapivp8enc" | "qsvh264enc" | "vpuenc_h264" => self
                 .element
                 .set_property("bitrate", (bitrate / 1000) as u32),
             "nvv4l2h264enc" | "nvv4l2vp8enc" | "nvv4l2vp9enc" => {
3

Not that it changes anything in your specific case, but the IMX encoder/decoder elements of gstreamer-imx are generally working better than the ones from gst-fsl-plugins.

Can you create an MR that adds support for those encoders to webrtcsink?

4

Sure, no problem.
Let me test this on a couple of imx8 targets and in different scenarios.

1 Like
5

@slomo looks great.

Just tested measurement from congestion control gccbwe too:

rtpgccbwe net/rtp/src/gcc/imp.rs:942:gstrsrtp::gcc::imp::State::set_bitrate:<rtpgccbwe0>e[00m Delay: 2100.84kbps => 1995.80kbps (Decrease("Over use detected Network Usage: Over. Effective bitrate: 3318.44kbps - Measure: 13.033114ms Estimate: 5.169552ms threshold 12.807984ms - overuse_estimate 25.84776ms")) - effective bitrate: 3318.44kb

Regarding the merge request, my account doesn’t allow to push on branch or create fork.

6

Check here: Contributing to GStreamer

Specifically this paragraph:

Please note that new gitlab accounts do not have permission to create forks by default, so you will have to first file an issue and fill in the user verification template as mentioned above in order to get forking rights. This should be a fairly quick process.

1 Like
7

This should work.