Hello,
After profiling our gstreamer pipeline, it seems that the h265parse’s latency is about one frame (17ms with 60fps, 33ms with 30fps, 67ms with 15fps, and 167ms with 6fps). Has this been seen by others? And what may be causing it? Any recommended ways of reducing this latency?
The relevant structure of our pipeline is … >> nvv4l2h265enc >> h265parse >> nvv4l2decoder >> …
Thank you for the help!
It depends on the conversion your doing, but if you convert from NALu to AU, you need to get the first NALu of the next AU before you know the previous one is complete, so you introduce a frame of latency.
Only if the producer of the stream (nvv4l2h264enc) does not put the MARKER flag on the last buffer of an AU (since 1.18 IIRC).
Which I would guess is exactly the problem here. Either a too old GStreamer is used, or the NVIDIA encoder does not use the marker flag. Otherwise there should be no latency introduced by the parser.
Thank you @slomo! We’re using an old version indeed (GStreamer 1.16.3), so that may be the issue.
Just to make sure I understood, before GStreamer 1.18, the h265parse element doesn’t support the marker flag?
IIRC it did not make use of it back then, correct. But even if it makes use of it, this would require the NVIDIA encoder to actually set the flag correctly too. I don’t know if it does that 
Here nvv4lh265enc outputs video/x-h265, stream-format=(string)byte-stream, alignment=(string)au anyway, so I don’t know why h265parse goes into that mode. I don’t think it should need to add any latency here in the alignment=au case.
Yes I think that was also part of the changes related to the marker flag handling around the same time.