I’m trying to composite two D3D11 video streams in GStreamer on Windows using d3d11compositor

Application Development
1

I’m trying to composite two D3D11 video streams in GStreamer on Windows using d3d11compositor.

Pipeline structure:

  • Main stream:
    UDP RTP H265 → rtpjitterbuffer → rtph265depay → h265parse → d3d11h265dec → d3d11convert → d3d11compositor

  • Overlay stream:
    videotestsrc → videoconvert → d3d11upload → d3d11compositor

Then compositor output goes to d3d11videosink.

Problem:

  • If I use H264 instead of H265, the compositor works correctly and both streams are displayed properly.

  • If only the UDP stream is connected to compositor, video works.

  • If only videotestsrc is connected, it works.

  • When both are connected, either:

    • only videotestsrc appears,

    • or main stream disappears,

    • or negotiation fails with not-negotiated,

    • or compositor chooses wrong output caps (e.g. 250x250@30 instead of 1920x1080@60)

GStreamer version:

* Windows

* GStreamer 1.24.10

I need to composite an H265 video stream with an OSD text overlay entirely on the GPU using d3d11compositor.

2

Can you supply us with some additional information?

  1. Is this being run on the command line or is it part of an application? If it is an application, what language is it written in?

  2. I have done something similar to this but I have not used the D3D11 package. There are a significant number of properties on the various elements and compositor sink pads that I would set. I would also be using caps filters and queues in a few different places. Can you share what you are setting or are you running this as is?

3

The application is written in C++.

I have already resolved the issue. It turned out that the H.265 stream was negotiating with interlace-mode=mixed by default. I added a capsfilter to force interlace-mode=progressive, and after that the compositor started handling the video correctly.

Thanks for your suggestions.