Hello,
I want to decode a live video stream as fast as possible. It is encoded in h265/h264 and does not contain any B Frames. For decoding, I tried the decoding plugins: vaapi, va and msdk.
An example pipeline looks like:
appsrc → h265parse → vah265dec → vapostproc → appsink
I push new frames in the pipeline with gst_app_src_push_buffer
and get the decoded ones via new_sample
callback. The problem is now the high decoding latency. For 1024x768@30fps its around 75ms and for 1024x768@60fps its around 37 ms.
The interesting thing is that it seems to keep at least two frames in the pipeline, so it doesn’t seem to be a performance issue but rather something buffering. To verify it i tried a program which does a similar thing but uses ffmpeg and also ueses vaapi as backend, I took arround 1,2 ms to decode a frame. So the hardware seems to be fine. I then tried to trace it with latency(flags=element+pipeline+reported)
(https://pastebin.com/sQj2jvxL). Which also indicates, if I interpret it correctly, that decoding should be significantly faster. The sum of h265parse, vah265dec and vapostproc always seems to be smaller than 2 ms.
And now I ask myself what I am missing?