Graphics API Interoperability and zero copy memory

In general GPU APIs do not interop in GStreamer so the d3d11h264dec ! glupload case will perform a GPU->CPU->GPU copy. There are ways to avoid this copy however the potential gain for the amount of infrastructure code that would need to be written skews this very much in favour of ‘just use the same GPU API’ in your pipeline.

The only case I know where there is some GPU API interop is with the nvidia decoder (and encoder?) elements and some CUDA->GL interop.

For vulkan specifically, currently vulkanupload does not have any ‘zero-copy’ support even if the underlying API my support some scenarios e.g. dma-buf, GPU API interop extensions, etc.

There isn’t really any design documents for this within GStreamer either and in general one needs to look at the code for the various *upload elements to figure out what kind of zero copy scenario are currently available.

Hope that helps.

1 Like