Capturing output of rtp*depay with appsink

Application Development
1

I want to open an RTSP stream and capture the encoded media directly using an appsink. (I explicitly don’t want to use a decodebin, I want to deal with the encoding myself)

I am doing this by attaching to the pad-added signal of rtspsrc, and attaching my appsink to that pad. The appsink has a new-sample callback added, and emit-signals is turned on for that sink.

This captures the full RTP payload properly, but whenever I try to add a depayloader between the RTP source and the sink, the sink no longer emits the new-sample signal. I’ve tried this with several different depayloders - they link correctly, but seem to emit nothing.

I’m guessing I’m missing something with how the RTP depayloaders are supposed to work. Using the appsink without the depayloader, I get byte arrays made of the RTP header and the encoded payload. My expectation is from the depayloader I should get the same, minus the RTP header?

This works to capture the raw RTP stream:

def on_pad_added(src, pad):
    caps = pad.query_caps()
    structure = caps.get_structure(0)
    media = structure.get_value('media')
    
    # attempt the link
    if media == "audio":
        sinkpad = audio_sink.get_static_pad("sink")
        ret = pad.link(sinkpad)
        if not ret == Gst.PadLinkReturn.OK:
            print("could not link pad")
        else:
            print("link works")
        return

This doesn’t work. Everything links correctly, but the appsink seems not to get anything:

def on_pad_added(src, pad):
    caps = pad.query_caps()
    structure = caps.get_structure(0)
    media = structure.get_value('media')
    
    # attempt the link
    if media == "audio":
        dp = Gst.ElementFactory.make("rtpmp4gdepay", "mp4dp")
        pipeline.add(dp)
        dp_sink = dp.get_static_pad("sink")
        dp_src = dp.get_static_pad("src")
        dp_caps = dp_sink.query_caps()

        ret = pad.link(dp_sink)
        if not ret == Gst.PadLinkReturn.OK:
            print(f"could not link pad: {ret}")
        else:
            print("link works")

        sinkpad = audio_sink.get_static_pad("sink")
        ret = dp_src.link(sinkpad)
        if not ret == Gst.PadLinkReturn.OK:
            print(f"could not link pad {ret}")
        else:
            print("link works")
        return

set-up code omitted for brevity, happy to share the full example if necessary

2

What you’re doing (wait for pad-added on rtspsrc and then add the RTP depayloader) should work fine in principle and should give you the output you’re after.

Looking at your code, you might be missing a dp.set_state(Gst.State.PLAYING) or somesuch after you add it to the pipeline. The pipeline won’t bring up the newly-added element to the pipeline state automatically.

If that doesn’t make it work you may want to check if you’re plugging the right depayloader (although if it was the wrong one you would probably get an internal streaming error with a not-negotiated flow).

You could also use the urisourcebin element to plug the right depayloaders and parsers for you, e.g. gst-launch-1.0 urisourcebin uri=rtsp://127.0.0.1:8554/test parse-streams=true ! fakesink -v | grep fakesink should show you that you get an elementary stream at the fakesink.