cv::cuda::GpuMat to GstVideoFrame

I am trying to implement a plugin that is a subclass of GstVideoAggregator for a DeepStream pipeline. I need to implement the function prepare_frame of GstVideoAggregatorPad in which the GstBuffer of the pad is wrapped with a cv::cuda::GpuMat and processed with cv::cuda::remap. Now, I am struggling on how to populate the GstVideoFrame* prepared_frame with the resulted cv::cuda::GpuMat. I would be grateful if you could provide me with some guidelines!

Not sure for your case, but if using Jetson you may have a look to (for a probably sub-optimal solution, … just for hobbyist):

and for remap you may see:

If this can help your case…