Gst element to mitigate wrong camera timestamping


Given following pipeline where we recieve rtsp data from ip camera with camera timestamping:
rtspsrc add-reference-timestamp-meta=true buffer_mode=0 ! rtph264depay ! h264parse ! tee name=tp tp. ! queue ! splitmuxsink tp. ! queue ! avdec_h264 ! jpegenc ! tee name=tp2 tp2. ! appsink

On receiver side, after appsink, we observe following picture (see img. below, where x-axis is frame number and y-axis is camera timestamp):

Is there any gst element which can handle ordering issues, namely some buffer to handle proper ordering based on camera timestamps? Can rtpjitterbuffer work with camera timestamps and reorder based on them? If not, what else can be done to mitigate timestamping issues? Tried h264timestamper but without any success.

Is it possible at all given h264 encoding? From what I see, we get raw image from camera (probably convert it to jpeg), timestamp it, and then send to h264 encoding phase and I’m not sure how decoder will work given possible reorderings based on timestamping issues.

Thanks in advance.

Any ideas? Is it solvable? Can new element solve this issue?

This is a really strange behaviour. I think your best bet is to use a pad probe, and modify the timestamps of the buffer with some heuristics.

1 Like

Thank you so much for reply.

1)What element should I use for apd probing, queue?

  1. Really weired situation – I doubt that camera reorders frames, or they arrive in different order. Rather it seems to me like camera clock jitter for some periods and it is not clear how to deal with that…
  1. You can add pad probe on whatever element’s pad you would like. You do not need a special ‘pad probe element’.
1 Like

rtpjitterbuffer is already used by rtspsrc element, so there’s probably not much to expect from adding it.
Not sure, but you may also give a try to setting rtspsrc property ntp-sync to true.

1 Like

Thank you for reply!

Won’t it break camera timestamping mechanism?