Hello, I’m trying to display a graphic overlay over a video. After some investigating I ran across the cairooverlay element and added it to my pipeline. It displays the overlay well and maintains a good framerate for the video with only a few drawn elements. However, when I try to display more elements in the overlay the framerate starts decreasing until it crashes from reaching 0 fps. I was thinking of working around it by running the update function in a separate thread and using an older version of the overlay while the new version is being created. Is it possible to do this and if it is whats the best way to go about doing this?
The cairooverlay element has a slight flaw that unfortunately affects performance quite massively - it always provides the full frame to draw on even if you only need to put something small into a corner.
There’s also an overlaycomposition element, that lets you “draw” overlays by providing a GstOverlayComposition mini object (made up of multiple `OverlayCompositionRectangles’. That way, you can use whatever API you want (incl. cairo) to draw/create the overlays, and then make them only as big as needed and position them however way you want.
This approach has multiple advantages, including lower overhead if used with hardware-accelerated video decoders.
I’m trying to use the overlaycomposition element right now but the framerate is very bad if the overlay size is increased. I’m also having difficulty getting the GstBuffer created from the cairo to display correctly.
It will depend on the size of the overlay of course.
Another alternative is to do the overlaying via a video mixer such as compositor or glvideomixer.