Full-screen video rendering on Windows

Hi there,

first of all thanks for creating GStreamer and obviously putting a lot of work into it!

I need your help with a seemly simple use case. I need my pipeline to render full-screen on Windows x64.
I’m very new to GStreamer and struggling with this now for several hours and probably miss something.

My complete use case is a continuous looped video that will be rotated by arbitrary angles and rendered full screen. The looping and rotation (gltransformation) I made work nicely.

Now I need to find a way to render full-screen. I looked into the below approaches:

  1. extracting the the texture out of the glupload element and independently rendering that using native OpenGL commands. The rendering loop and and gliib main loop need to be integrated here
  • this gets really complex. I didn’t manage to make it full work yet. Also I think the time synchronization of the frames will get lost
  1. How GstVideoOverlay can be used with a Windows HWND of a full screen window
    I didn’t manage to find documentation here

  2. do this directly using glimagesink?
    My understanding is that this is not possible

  3. Use GTK to make the full screen window

  • there are some examples how to use GstVideoOverlay with GTK. But it would require building GTK from source with VS2022, in turn requiring installation of special build systems

I wonder do I miss something obvious? What would be a good way forward here?

Best regards

d3dvideosink is likely the best choice at the moment. Though, it had a bug in regard to maximize and full screen. A fix has been backported and will be part of the next stable release. Meanwhile, see:

Thanks a lot for the reply!

If I read it correctly that would then mean that the rotation of the video by the arbitrary angle would need to happen using “rotate” which happens in the CPU (as opposed gltranslattion doing it in the GPU). Would that be fast enough on standard PC with FullHD resolution from experience to not loose frames?

I tested that now. Unfortunately the CPU based rotate is indeed to slow. It produces a lot of frame drops and the video gets choppy.

I looked more into this. Would there be a way using “draw-on-shared-texture” and then doing the rotation with d3d?

fwiw, d3d11videosink supports predefined set of rotation methods d3d11videosink but gltransformation like element does not exist in d3d11 plugin.

Thanks for the pointer. I checked that. Unfortunately I need rotation by arbitrary and dynamically changing angle.

So I looked into the examples here: subprojects/gst-plugins-bad/tests/examples/d3d11 · main · GStreamer / gstreamer · GitLab

Which would be best one to build this upon?

I would imagine I need to rotate the texture before it is rendered.

there are multiple options.

  • (recommended) Write new d3d11 element for the transform
  • Get gstreamer rendered texture via appsink (e.g., … ! d3d11convert ! “video/x-raw(memory:D3D11Memory),format=RGBA” ! appsink) → rotate it in your app → render using appsrc and d3d11videosink (appsrc ! queue ! d3d11videosink). d3d11decoder-appsink.cpp and d3d11videosink-appsrc.cpp are related examples
  • Use present signal d3d11videosink. d3d11videosink-present.cpp is a related example but you need to copy the swapchain’s backbuffer into your own texture first for shader input. Then redraw rotated image to the swapchain’s backbuffer using your shader or other APIs such as d2d. you will need to carefully calculate viewport in this case.

thanks for your proposals. I’ll dig deeper based on that.

Or if you can use main branch, d3d11overlay element is an option as well (see d3d11overlay.cpp). Perform rotation to your own render target and copyback to the original texture.

I implemented 3D rotation and scale support in d3d11videosink (with anti-aliasing) d3d11videosink: Add support for transform and MSAA (!5532) · Merge requests · GStreamer / gstreamer · GitLab

1 Like

Wow! Thank you so much. I’ll check out your MR and look into how to build it here to directly test it.
Thanks again for this immediate great help!