Gstreamer RTP Header Extension

General Discussion
1

Hi,

Gstreamer: 1.22.12

I have an upstream pipeline and a downstream pipeline; both gstreamer ones.
The upstream pipeline processes video data and sends it to a udp multicast using rtpbin and udpsink. I am setting the following in the payloader on the upstream.

hdr_ext = GstRtp.RTPHeaderExtension.create_from_uri("urn:ietf:params:rtp-hdrext:ntp-64")
hdr_ext.set_id(1)
payloader.emit("add-extension", hdr_ext)

I am able to confirm that header extensions are seen on these RTP packets using pcap analysis.

On the downstream, I was originally using udpsrc and rtpbin with add-reference-timestamp-meta enabled on the rtpbin and I can see the timestamps come through on the downstream pipeline and the buffer meta updated with this reference timestamp.

But I noticed that rtpbin (specifically rtpjitterbuffer) was adding so much overhead that the CPU usage skyrocketed. The way my pipelines are setup, we have about 500 pipelines on one machine. So, obviously this doesn’t scale. I have read about issues and articles on how this is an issue with rtpjitterbuffer.

Regardless, I am fine with forgoing the rtpbin/jitterbuffer for my implementation. But I need to be able to parse and extract the rtp header to extract the timestamp and populate the buffer metadata.

I see that the rtpdepay has abilities to read header extensions. But I don’t see this happening despite me trying a few things.

  1. setting extmap-1=(string)urn:ietf:params:rtp-hdrext:ntp-64 in the caps on the udpsrc.
  2. calling this on the depay
    hdr_ext = GstRtp.RTPHeaderExtension.create_from_uri(“urn:ietf:params:rtp-hdrext:ntp-64”)
    hdr_ext.set_id(1)
    self.depay.emit(“add-extension”, hdr_ext)

Neither of these approaches seem to be triggering the downstream depayloader to parse the RTP header extension. (I don’t see logs on increasing log level on the depay)

Is my understanding of this wrong? Should I instead add a custom plugin/element that can call the low level rtp header parsing methods that gstreamer uses and populate the reference meta on the buffers?

I appreciate your inputs.

Thanks

2

Doing this should make the depayloader create a parser for the header extension and have the header extension added as GstReferenceTimestampMeta to the buffers.

You’ll have to check what happens in the depayloader base class (does it actually create the extension parser?) and then what happens inside the parser.

Note that rtpbin / rtpjitterbuffer are generally a good idea to use for synchronization purposes and handling of packet loss / reordering.