Using imxcompositor_g2d causes framerate issues

Application Development
1

I’m using an imx8mp on a variscite dart board and a basler camera and I’m trying to display a full graphic overlay on top of the video. When I use the overlay on the video with the imxcompositor_g2d the framerate drops significantly. This issue can be seen with the following 2 commands.

gst-launch-1.0 imxcompositor_g2d background=3 name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::keep-ratio=true sink_1::xpos=160 sink_1::ypos=120 ! waylandsink sync=true v4l2src device=/dev/video2 ! video/x-raw, width=320, height=240, framerate=60/1 ! queue ! comp.sink_0 videotestsrc ! video/x-raw,format=BGRA,width=320,height=240 ! queue ! comp.sink_1

The previous command creates a testsrc over the video and the framerate is 60fps which is desired but when I increase the size of the testsrc in the following command the framerate drops:

gst-launch-1.0 imxcompositor_g2d background=3 name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::keep-ratio=true sink_1::xpos=160 sink_1::ypos=120 ! waylandsink sync=true v4l2src device=/dev/video2 ! video/x-raw, width=1920, height=1080, framerate=60/1 ! queue ! comp.sink_0 videotestsrc ! video/x-raw,format=BGRA,
width=1920,height=1080
! queue ! comp.sink_1

Is there any way to display a full graphic overlay over the video without making the frameate drop?

2

The videotestsrc element will generate new frames every time, it’s not surprising that it would be slow. You might be able to get a better performance by getting video from the camera, or by using the GPU accelerated gltestsrc. If you just want to test the overlay, you can also do something like “videotestsrc ! imagefreeze” which will freeze to a single image. Or you can try reducing the framerate of the overlay.

3

Hi ocrete, thank you for your response. We are using a video from the camera using v4l2src. The videotestsrc in the provided pipelines are being overlayed ontop of our camera video as a replicable replacement for our overlay. The main concern we’ve been having is that from my understanding the imxcompositor_g2d should combine the pipelines with the fastest framerate of the 2 but the framerate we’re getting back is slower than the video running on its own. It looks like the merging of the video and the overlay could be decreasing the framerate.

4

It’s possible, I’m not familiar with the limitations of the g2d hardware on that platform. You should ask that kind of question to the NXP support (or forums?)