RTSP H.264 pipeline latency gradually increases from 150ms to 300ms+ over time, resets only after pipeline restart

General Discussion
1

Hi all,

I’m using a GStreamer pipeline to render a live RTSP H.264 stream from an IP camera. Initial latency is around 100–150ms, which is acceptable. However, after running for a while, the latency gradually increases to 300ms+ and keeps growing. Restarting the pipeline (reconnect) immediately brings latency back to 100–150ms. This strongly suggests some internal buffering or timestamp drift is accumulating over time.

My pipeline:
rtspsrc name=src location=<rtsp_url>
protocols=tcp
latency=100
drop-on-latency=true
buffer-mode=0
do-retransmission=false
! rtph264depay
! queue name=queue0 max-size-buffers=30 max-size-bytes=0 max-size-time=0
! h264parse config-interval=-1
! queue name=queue1 max-size-buffers=30 max-size-bytes=0 max-size-time=0
! avdec_h264 max-threads=2
! videoconvert
! videoscale
! queue name=queue2 max-size-buffers=10 max-size-bytes=0 max-size-time=0
! ximagesink name=videosink sync=false drop=true**

What I’ve already tried:**

  • latency=100, drop-on-latency=true, buffer-mode=0, do-retransmission=false

  • sync=false on the sink

  • Queue sizes are bounded, not unlimited

Questions:

  1. Is the internal rtpjitterbuffer inside rtspsrc likely accumulating depth over time? How can I monitor current-level-time with GST_DEBUG?

  2. Does drop-on-latency=true only affect rtspsrc output, or does it also force the internal jitter buffer to drop?

  3. Could the camera’s RTP timestamp clock drift relative to the system clock cause the jitter buffer to grow slowly?

  4. Is avdec_h264 max-threads=2 adding frame-level latency that compounds with the jitter buffer?

  5. What is the recommended way to force-drop old buffers in a live pipeline so latency does not accumulate? Would a queue leaky=downstream help?

Any debugging commands (e.g. GST_DEBUG=*rtpjitterbuffer*:5, *rtspsrc*:5) or pipeline modifications to isolate the cause would be greatly appreciated.

Thanks!

2

What GStreamer version is this with?

Does your rtspsrc have a tcp-timestamp property? If so, you could try setting that to true to see if that helps.