Graphics API Interoperability and zero copy memory

This is more of a theoretical question…

If I have a pipeline that looks something like:

gst-launch-1.0.exe <src_element> ! parsebin ! d3d11h264dec ! glupload ! qml6glsink

How is the GPU data output from d3d11h264dec handled when passed to glupload? Does this do a GPU → CPU → GPU copy? If so, is there any way to avoid this redundant copy?

Is there any high-level information (or reference to implementation code) that you can point me to on how this is handled in general for different graphics APIs? For example, if I replace glupload ! qml6glsink with vulkanupload ! appsink how can I check that it is “zero-copy”?

Thanks!

In general GPU APIs do not interop in GStreamer so the d3d11h264dec ! glupload case will perform a GPU->CPU->GPU copy. There are ways to avoid this copy however the potential gain for the amount of infrastructure code that would need to be written skews this very much in favour of ‘just use the same GPU API’ in your pipeline.

The only case I know where there is some GPU API interop is with the nvidia decoder (and encoder?) elements and some CUDA->GL interop.

For vulkan specifically, currently vulkanupload does not have any ‘zero-copy’ support even if the underlying API my support some scenarios e.g. dma-buf, GPU API interop extensions, etc.

There isn’t really any design documents for this within GStreamer either and in general one needs to look at the code for the various *upload elements to figure out what kind of zero copy scenario are currently available.

Hope that helps.

1 Like