Disconnection of camera which composited with d3d11compositor

Application Development
1

Hello

I have created a new topic related to this topic.
#763 D3D11compositor with different frame rates

Below is the camera multi-view pipeline using d3d11compositor.

gst-launch-1.0 d3d11compositor name=m `
		sink_0::xpos=0 sink_0::ypos=0 sink_0::width=640 sink_0::height=320 `
		sink_1::xpos=640 sink_1::ypos=0 sink_1::width=640 sink_1::height=320 `
		sink_2::xpos=0 sink_2::ypos=320 sink_2::width=640 sink_2::height=320 `
		sink_3::xpos=640 sink_3::ypos=320 sink_3::width=640 sink_3::height=320 `
		! capsfilter ! d3d11videosink `
	rtspsrc location=***** ! rtph265depay ! h265parse ! d3d11h265dec ! videocrop ! d3d11convert ! capsfilter ! m.sink_0 `
	rtspsrc location=***** ! rtph265depay ! h265parse ! d3d11h265dec ! videocrop ! d3d11convert ! capsfilter ! m.sink_1 `
	rtspsrc location=***** ! rtph265depay ! h265parse ! d3d11h265dec ! videocrop ! d3d11convert ! capsfilter ! m.sink_2 `
	rtspsrc location=***** ! rtph265depay ! h265parse ! d3d11h265dec ! videocrop ! d3d11convert ! capsfilter ! m.sink_3

I am running with source code.

I found that the problem occurs when one camera is powered off.
In this case, other views will also not be displayed.
Or, other views are displayed, but all views are terminated when the connection to the powered off camera times out.

As a solution, I’m considering separating the pipeline so that the state of one camera doesn’t affect other views or “d3d11compositor ! d3d11videosink”.

I separated the pipeline with appsink/appsrc as shown below, and passed d3d11testsrc to appsrc instead until the camera video arrived, and it was working fine, but the CPU and GPU usage rates increased. It’s gone. In particular, the GPU has doubled.

 ... capsfilter ! appsink  /  appsrc ! m.sink_0
 ... capsfilter ! appsink  /  appsrc ! m.sink_1
 ... capsfilter ! appsink  /  appsrc ! m.sink_2
 ... capsfilter ! appsink  /  appsrc ! m.sink_3

I thought appsink/appsrc could be zero-copied, but in the case of video/x-raw(memory:D3D11Memory), will a copy or d3d11download/d3d11upload occur?

I would like to know whether appsink/appsrc should be OK, whether I need to use other elements such as d3d11ipcsrc/d3d11ipcsink of Ver.1.23, or whether it can be solved without separating the pipeline.

Best regards,
Kamiya.

*Sorry, I may not be able to reply during the New Year holidays.

2

appsrc/appsink itself should work in zero-copy manner but If GstD3D11Device object is not shared across pipelines, data copy can happen. Increased CPU/GPU usage makes sense if it’s the case.

There are couple of examples related to the device context sharing here (see need-context/have-context SYNC bus message handler)

Or, single pipeline with fallbacksrc in -rs plugins could be an alternative to separate pipeline approach.

1 Like
3

Hello seungha.

Thank you for your useful information!
I will try learning it.

Best regards,
Kamiya.

4

Hello seungha.

Thanks to your advice, I was able to share the d3d11videosink context with other d3d11 elements in different pipeline.

CPU and GPU usage have become also the same as before separating the pipelines.

I achieved what I wanted.

Thank you very much.
Kamiya.