How to pass RTP packets to appsrc?

General Discussion
1

I’m currently using webrtc.rs where I’m trying to pass RTP packets to a GStreamer pipeline like so:
appsrc name=videosrc is-live=true format=time do-timestamp=true ! decodebin3 ! queue name=videopadsrc ! waylandsink name=videosink sync=true async=false

I’m pushing the packets like so by manually constructing the packet like so that I got from Gemini, which is most likely wrong, like so:

while let Ok((packet, _)) =
track_remote.read_rtp().await
{
    let mut rtp_packet_bytes =
        Vec::with_capacity(12 + packet.payload.len());
    rtp_packet_bytes.extend_from_slice(&[
        (2 << 6)
        | (packet.header.padding as u8) << 5
        | (packet.header.extension as u8) << 4
        | (packet.header.csrc.len() as u8), // Version, Padding, Extension, CSRC Count
        (packet.header.marker as u8) << 7
        | packet.header.payload_type, // Marker, Payload Type
        (packet.header.sequence_number >> 8) as u8, // Sequence Number (High Byte)
        packet.header.sequence_number as u8, // Sequence Number (Low Byte)
        (packet.header.timestamp >> 24) as u8, // Timestamp (Byte 3)
        (packet.header.timestamp >> 16) as u8, // Timestamp (Byte 2)
        (packet.header.timestamp >> 8) as u8, // Timestamp (Byte 1)
        packet.header.timestamp as u8, // Timestamp (Byte 0)
        (packet.header.ssrc >> 24) as u8, // SSRC (Byte 3)
        (packet.header.ssrc >> 16) as u8, // SSRC (Byte 2)
        (packet.header.ssrc >> 8) as u8, // SSRC (Byte 1)
        packet.header.ssrc as u8, // SSRC (Byte 0)
    ]);
    for csrc in &packet.header.csrc {
        rtp_packet_bytes
            .extend_from_slice(&csrc.to_be_bytes());
    }
    rtp_packet_bytes.extend_from_slice(&packet.payload);
    let gst_buffer =
        gstreamer::Buffer::from_slice(rtp_packet_bytes);
    if let Err(e) = videoappsrc.push_buffer(gst_buffer)
    {
        println!("Error {:?}", e);
    }
}

I’m getting the following error with GST_DEBUG=3

0:00:05.421632375 78709 0x600000f7caa0 WARN        rtpbasedepayload gstrtpbasedepayload.c:971:gst_rtp_base_depayload_handle_buffer:<rtpvp8depay0> warning: Received invalid RTP payload, dropping

What’s the correct way of pushing RTP packets? Is there a helper function/struct through which I can construct the packet?
Since I can access the payload directly, is there a simpler way to avoid the packet reconstruction logic?

2

Does your appsrc have any caps set on it with appropriate application/x-rtp caps?

3

Yup. I’m currently doing it like this:

let encoding_name = if mime_type.contains("vp8") {
    "VP8"
} else if mime_type.contains("vp9") {
    "VP9"
} else {
    "H264"
};
let caps = Caps::builder("application/x-rtp")
    .field("media", "video")
    .field("encoding-name", encoding_name)
    .field("payload", track_remote.payload_type() as i32)
    .field(
        "clock-rate",
        track_remote.codec().capability.clock_rate as i32,
    )
    .build();
videoappsrc.set_caps(Some(&caps));
4

That looks fine.

You would need to figure out why your packets are not liked by rtpvp8depay.

You can use a rust crate for constructing RTP packets like crates.io: Rust Package Registry

5

You were right! The parse() method from rtp_types was failing as the packet size was small.
I was able to get it to work after I discovered the webrtc_util crate which exposed the Marshal trait.

tokio::spawn(async move {
    let mut b = vec![0u8; 1500];
    while let Ok((rtp_packet, _)) =
    track_remote.read(&mut b).await
    {
        let n = rtp_packet.marshal_to(&mut b).unwrap();
        let gst_buffer =
            gstreamer::Buffer::from_slice(b[..n].to_vec();
        if let Err(e) = videoappsrc.push_buffer(gst_buffer)
        {
            println!("Error {:?}", e);
        }
    }
});

Thanks a lot!