H265/h264 skip-frame decoding to reduce load

Application Development
1

Suppose I would like to build something that functions similar to an NVR using gstreamer, where I will have many incoming rtsp streams (h264/h265) and I would like to record all of them at their original quality/resolution/framerate, but display a sample/preview of them at a much more infrequent rate…maybe even where the rate decreases in proportion to the number of streams being previewed.

Is it realistic to perform reference-frame only decoding as a way to reduce the compute load? And related to that, is it possible to even skip decoding some of the reference frames depending on how often I want to give a preview?
The reason I ask this is that I’m assuming (maybe incorrectly?) that if I can only decode one out of every N frames per stream, then I could display preview/snapshots of many more streams at a time, also assuming that writing the already-encoded streams to disk is not a concern for load as long as I don’t re-encode.

If this is possible, what are some things I need to watch out for when doing this? Is it possible via any existing plugins, or would I need to do it with a pad probe or custom plugin?

Any help or pointers is greatly appreciated. Thanks!

2

Generally speaking that should work I think, at least for rtsp feeds.

You can drop non-keyframes with identity drop-buffer-flags=delta, but that won’t get you to some kind of minimum distance between frames, for that you might need something else.

Buffer probes can do the job too, yes.

Addendum: some decoders, like avdec_h264 also support skip frame decoding via the skip-frame property, but that property has changed in the main branch and it also sounds like you’re looking for a bigger hammer in your case.

(Seek events can also be passed trick mode flags and a min interval between frames, but I don’t think that helps you for the RTSP use case where you want to receive the full stream.)