I am trying to create a combined Neural Network detector (combined with opencv) and propagate the output via RTSP stream. I am using python if that’s not evident by the tag by the way.
My approach works to a degree satisfactory, so I am not going to start by scratch here. My pipeline is this one:
appsrc name=source is-live=true block=false format=GST_FORMAT_TIME caps=video/x-raw,format=BGR,width=384,height=288,framerate=25/1 ! videoconvert ! video/x-raw,format=I420 ! x264enc speed-preset=fast tune=zerolatency key-int-max=30 bitrate=500 ! rtph264pay config-interval=1 name=pay0 pt=96
My main problem for now is that I need to have 2 clients (at least) for comsuming the RTSP stream which does not seem to work. The first client can see the video stream just fine (playing around with the speed-preset I get some delay in the beginning of the display to be honest) but the second one cannot. It is displayed (in the second one) only for a moment after quite a delay. I am even impressed it is displaying a frame because I would expect either to not dispay anything or display all the time.
If I insert the queue option there seem to be some progress and a second client can display the stream for some time. The problem is that is short lived the display and often is frozen (in the second client) to a certain frame. I have overlayed a frame counter on the frames (a simple increasing number) and I can tell if it is stuck in a certain frame. Other times the outcome is worse throwing the error:
Assertion fctx->async_lock failed at libavcodec/pthread_frame.c:167
or even this one (a second message before is added in this case):
[h264 @ 0x1ae95b00] Got unexpected packet size after a partial decode
Assertion fctx->async_lock failed at libavcodec/pthread_frame.c:167
which breaks the whole module.
The second pipeline is this one:
appsrc name=source is-live=true block=false format=GST_FORMAT_TIME caps=video/x-raw,format=BGR,width=384,height=288,framerate=25/1 ! queue ! videoconvert ! video/x-raw,format=I420 ! x264enc speed-preset=fast tune=zerolatency key-int-max=30 bitrate=500 ! rtph264pay config-interval=1 name=pay0 pt=96
In a question in another site there were speculation that not all sources (like appsrc or filesrc) can support multiple clients. Is this the case here?
I can provide my whole code if necessary (just the RTSP streaming code I mean) if necessary. Also, I am using GstRtspServer.RTSPMediaFactory