GStreamer 1.16.3 Setting RTSP Pipeline to NULL

ConnorM · November 14, 2023, 5:18pm

I have tried several different methods to properly stop my GStreamer pipeline, however, no matter what I do it seems that things hang/freeze when trying to set the state to NULL.
I am using a pipeline with an rtspsrc and would like to simply set the pipeline to NULL, and restart it when an error occurs. The pipeline in this case is:

rtspsrc → h264depay → h264parse → nvv4l2decoder → nvvideoconvert → capsfilter → nvv4l2h264enc → rtph264pay → udpsink

for this simple test I am disconnecting the source rtsp camera connected to rtspsrc and waiting to recieve the SENDER TIMEOUT signal. Then I simply call:

def catch_on_sender_timeout(manager: Gst.Bin, ssrc, udata):
    rtspsrc: Gst.Element = manager.parent
    camera_bin: Gst.Bin = rtspsrc.parent
    pipeline: Gst.Pipeline = camera_bin.parent
    print(pipeline.name)
    print('setting to NULL')
    pipeline.set_state(Gst.State.NULL)

From here it just hangs/freezes indefinitely.

Here is the output log produced when pipeline.set_state(Gst.State.NULL) is called while listening to STATE_CHANGED on the bus and by GST_DEBUG = “rtspsrc:7”

0:00:41.935402597 14586 0x7f013c0b6000 DEBUG                rtspsrc gstrtspsrc.c:3498:on_ssrc_active:<rtspsrc_0> source in session 0 is active
0:00:42.342144672 14586 0x7f00fc01d520 DEBUG                rtspsrc gstrtspsrc.c:3004:gst_rtspsrc_handle_src_query:<rtspsrc_0> pad rtspsrc_0:recv_rtp_src_0_1268174119_96 received query duration
0:00:43.841846077 14586 0x7f00fc01d520 DEBUG                rtspsrc gstrtspsrc.c:3004:gst_rtspsrc_handle_src_query:<rtspsrc_0> pad rtspsrc_0:recv_rtp_src_0_1268174119_96 received query duration
SENDER TIMEOUT
stream-pipeline_0
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'udpsink_0',
 'element_parent': <Gst.Bin object at 0x7f0142689200 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002b80)>,
 'ts': '2023-11-14 16:35:52.773161',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'rtppay_0',
 'element_parent': <Gst.Bin object at 0x7f0142689bc0 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002c00)>,
 'ts': '2023-11-14 16:35:52.773951',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'h26encode_0',
 'element_parent': <Gst.Bin object at 0x7f0142689200 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002c80)>,
 'ts': '2023-11-14 16:35:52.774550',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'capsfilt_rtspsink0',
 'element_parent': <Gst.Bin object at 0x7f0142689340 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002d00)>,
 'ts': '2023-11-14 16:35:52.775125',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'nvvidconv_rtspsink0',
 'element_parent': <Gst.Bin object at 0x7f0142689200 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002d80)>,
 'ts': '2023-11-14 16:35:52.775714',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'sink-bin-00',
 'element_parent': <Gst.Pipeline object at 0x7f014505d600 (GstPipeline at 0x788a0f0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002e00)>,
 'ts': '2023-11-14 16:35:52.776342',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'capsfilt_rtspsrc0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002e80)>,
 'ts': '2023-11-14 16:35:52.776948',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'nvvidconv_rtspsrc0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002f00)>,
 'ts': '2023-11-14 16:35:52.777532',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'nvv4l2decoder_0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002b00)>,
 'ts': '2023-11-14 16:35:52.778129',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'h26parse_0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002980)>,
 'ts': '2023-11-14 16:35:52.778505',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'rtpdepay_0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7a92d20)>,
 'ts': '2023-11-14 16:35:52.778684',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'rtspsrc_0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7a91090)>,
 'ts': '2023-11-14 16:35:52.778881',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'source-bin-00',
 'element_parent': <Gst.Pipeline object at 0x7f014505d600 (GstPipeline at 0x788a0f0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7a91690)>,
 'ts': '2023-11-14 16:35:52.779051',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'stream-pipeline_0',
 'element_parent': None,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f013c1715c0)>,
 'ts': '2023-11-14 16:35:52.779223',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'udpsink_0',
 'element_parent': <Gst.Bin object at 0x7f0142689200 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7ad0840)>,
 'ts': '2023-11-14 16:35:52.779436',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'rtppay_0',
 'element_parent': <Gst.Bin object at 0x7f0142689cc0 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00fc024ea0)>,
 'ts': '2023-11-14 16:35:52.779616',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'h26encode_0',
 'element_parent': <Gst.Bin object at 0x7f0142689200 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00fc024f20)>,
 'ts': '2023-11-14 16:35:52.780164',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'capsfilt_rtspsink0',
 'element_parent': <Gst.Bin object at 0x7f0142689bc0 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e8002820)>,
 'ts': '2023-11-14 16:35:52.780379',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'nvvidconv_rtspsink0',
 'element_parent': <Gst.Bin object at 0x7f0142689200 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e80028a0)>,
 'ts': '2023-11-14 16:35:52.780570',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'sink-bin-00',
 'element_parent': <Gst.Pipeline object at 0x7f014505d600 (GstPipeline at 0x788a0f0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e8002b20)>,
 'ts': '2023-11-14 16:35:52.780750',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'capsfilt_rtspsrc0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e8002ba0)>,
 'ts': '2023-11-14 16:35:52.780945',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'nvvidconv_rtspsrc0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e8332440)>,
 'ts': '2023-11-14 16:35:52.781137',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'nvv4l2decoder_0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e832b030)>,
 'ts': '2023-11-14 16:35:52.782724',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'h26parse_0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e8332340)>,
 'ts': '2023-11-14 16:35:52.782941',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'rtpdepay_0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e83323c0)>,
 'ts': '2023-11-14 16:35:52.783124',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'rtpptdemux0',
 'element_parent': <__gi__.GstRtpBin object at 0x7f0141d67840 (GstRtpBin at 0x7f013c084060)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7a92e20)>,
 'ts': '2023-11-14 16:35:52.783376',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'rtpjitterbuffer0',
 'element_parent': <__gi__.GstRtpBin object at 0x7f0141d67840 (GstRtpBin at 0x7f013c084060)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002e00)>,
 'ts': '2023-11-14 16:35:52.783582',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'rtpssrcdemux0',
 'element_parent': <__gi__.GstRtpBin object at 0x7f0141d67840 (GstRtpBin at 0x7f013c084060)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002d80)>,
 'ts': '2023-11-14 16:35:52.783783',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'rtpstorage0',
 'element_parent': <__gi__.GstRtpBin object at 0x7f0141d67840 (GstRtpBin at 0x7f013c084060)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002d00)>,
 'ts': '2023-11-14 16:35:52.783983',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'rtpsession0',
 'element_parent': <__gi__.GstRtpBin object at 0x7f0141d67840 (GstRtpBin at 0x7f013c084060)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002c80)>,
 'ts': '2023-11-14 16:35:52.784181',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from playing to paused.',
 'element_name': 'manager',
 'element_parent': <__gi__.GstRTSPSrc object at 0x7f0142906680 (GstRTSPSrc at 0x7894440)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002c00)>,
 'ts': '2023-11-14 16:35:52.784376',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'rtpptdemux0',
 'element_parent': <__gi__.GstRtpBin object at 0x7f0141d67840 (GstRtpBin at 0x7f013c084060)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002b80)>,
 'ts': '2023-11-14 16:35:52.784575',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'rtpjitterbuffer0',
 'element_parent': <__gi__.GstRtpBin object at 0x7f0141d67840 (GstRtpBin at 0x7f013c084060)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f4001c80)>,
 'ts': '2023-11-14 16:35:52.784808',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'rtpssrcdemux0',
 'element_parent': <__gi__.GstRtpBin object at 0x7f0141d67840 (GstRtpBin at 0x7f013c084060)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e832b4b0)>,
 'ts': '2023-11-14 16:35:52.785010',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'rtpstorage0',
 'element_parent': <__gi__.GstRtpBin object at 0x7f0141d67840 (GstRtpBin at 0x7f013c084060)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e832b430)>,
 'ts': '2023-11-14 16:35:52.785212',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
0:00:55.320449284 14586 0x7f013c053f00 DEBUG                rtspsrc gstrtspsrc.c:5942:gst_rtspsrc_loop_send_cmd:<rtspsrc_0> sending cmd WAIT
0:00:55.320499831 14586 0x7f013c053f00 DEBUG                rtspsrc gstrtspsrc.c:5967:gst_rtspsrc_loop_send_cmd:<rtspsrc_0> cancel previous request LOOP
0:00:55.320520086 14586 0x7f013c053f00 DEBUG                rtspsrc gstrtspsrc.c:5974:gst_rtspsrc_loop_send_cmd:<rtspsrc_0> connection flush busy LOOP
0:00:55.320545782 14586 0x7f013c053f00 DEBUG                rtspsrc gstrtspsrc.c:5093:gst_rtspsrc_connection_flush:<rtspsrc_0> set flushing 1
0:00:55.320561628 14586 0x7f013c053f00 DEBUG                rtspsrc gstrtspsrc.c:5096:gst_rtspsrc_connection_flush:<rtspsrc_0> connection flush
0:00:55.320681046 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:5682:gst_rtspsrc_loop_udp:<rtspsrc_0> got interrupted
0:00:55.320715541 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:6040:gst_rtspsrc_loop:<rtspsrc_0> pausing task, reason flushing
0:00:55.320733628 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:5942:gst_rtspsrc_loop_send_cmd:<rtspsrc_0> sending cmd WAIT
0:00:55.320750233 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:5974:gst_rtspsrc_loop_send_cmd:<rtspsrc_0> connection flush busy LOOP
0:00:55.320766574 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:5093:gst_rtspsrc_connection_flush:<rtspsrc_0> set flushing 1
0:00:55.320927093 14586 0x7f013c053f00 DEBUG                rtspsrc gstrtspsrc.c:5942:gst_rtspsrc_loop_send_cmd:<rtspsrc_0> sending cmd PAUSE
0:00:55.320963748 14586 0x7f013c053f00 DEBUG                rtspsrc gstrtspsrc.c:5979:gst_rtspsrc_loop_send_cmd:<rtspsrc_0> not interrupting busy cmd WAIT
0:00:55.321068494 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:8783:gst_rtspsrc_thread:<rtspsrc_0> got command PAUSE
0:00:55.321106592 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:5093:gst_rtspsrc_connection_flush:<rtspsrc_0> set flushing 0
0:00:55.321132220 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:5096:gst_rtspsrc_connection_flush:<rtspsrc_0> connection flush
0:00:55.321183574 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:8559:gst_rtspsrc_pause:<rtspsrc_0> PAUSE...
0:00:55.321256829 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:522:default_before_send:<rtspsrc_0> default handler
0:00:55.321294629 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:522:default_before_send:<rtspsrc_0> default handler
0:00:55.321335464 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:6371:gst_rtspsrc_try_send:<rtspsrc_0> sending message
0:00:55.321362384 14586      0x794f6a0 LOG                  rtspsrc gstrtspsrc.c:9408:gst_rtspsrc_print_rtsp_message:<rtspsrc_0> --------------------------------------------
0:00:55.321397017 14586      0x794f6a0 LOG                  rtspsrc gstrtspsrc.c:9411:gst_rtspsrc_print_rtsp_message:<rtspsrc_0> RTSP request message 0x7f0128b8dc00
0:00:55.321422084 14586      0x794f6a0 LOG                  rtspsrc gstrtspsrc.c:9412:gst_rtspsrc_print_rtsp_message:<rtspsrc_0>  request line:
0:00:55.321457910 14586      0x794f6a0 LOG                  rtspsrc gstrtspsrc.c:9413:gst_rtspsrc_print_rtsp_message:<rtspsrc_0>    method: 'PAUSE'
0:00:55.321479592 14586      0x794f6a0 LOG                  rtspsrc gstrtspsrc.c:9415:gst_rtspsrc_print_rtsp_message:<rtspsrc_0>    uri:    'rtsp://my_IP:8554/test/'
0:00:55.321501861 14586      0x794f6a0 LOG                  rtspsrc gstrtspsrc.c:9416:gst_rtspsrc_print_rtsp_message:<rtspsrc_0>    version: '1.0'
0:00:55.321516817 14586      0x794f6a0 LOG                  rtspsrc gstrtspsrc.c:9418:gst_rtspsrc_print_rtsp_message:<rtspsrc_0>  headers:
0:00:55.321540210 14586      0x794f6a0 LOG                  rtspsrc gstrtspsrc.c:9391:dump_key_value:<rtspsrc_0>    key: 'User-Agent', value: 'GStreamer/1.16.3'
0:00:55.321569802 14586      0x794f6a0 LOG                  rtspsrc gstrtspsrc.c:9420:gst_rtspsrc_print_rtsp_message:<rtspsrc_0>  body:
0:00:55.321599112 14586      0x794f6a0 LOG                  rtspsrc gstrtspsrc.c:9500:gst_rtspsrc_print_rtsp_message:<rtspsrc_0> --------------------------------------------
0:00:55.330098584 14586 0x7f013c053f00 DEBUG                rtspsrc gstrtspsrc.c:5942:gst_rtspsrc_loop_send_cmd:<rtspsrc_0> sending cmd CLOSE
0:00:55.330115607 14586 0x7f013c053f00 DEBUG                rtspsrc gstrtspsrc.c:5967:gst_rtspsrc_loop_send_cmd:<rtspsrc_0> cancel previous request LOOP
0:00:55.330122681 14586 0x7f013c053f00 DEBUG                rtspsrc gstrtspsrc.c:5974:gst_rtspsrc_loop_send_cmd:<rtspsrc_0> connection flush busy PAUSE
0:00:55.330128464 14586 0x7f013c053f00 DEBUG                rtspsrc gstrtspsrc.c:5093:gst_rtspsrc_connection_flush:<rtspsrc_0> set flushing 1
0:00:55.330133323 14586 0x7f013c053f00 DEBUG                rtspsrc gstrtspsrc.c:5096:gst_rtspsrc_connection_flush:<rtspsrc_0> connection flush
0:00:55.330165239 14586      0x794f6a0 WARN                 rtspsrc gstrtspsrc.c:6326:gst_rtsp_src_receive_response:<rtspsrc_0> receive interrupted
0:00:55.330174200 14586      0x794f6a0 WARN                 rtspsrc gstrtspsrc.c:6424:gst_rtspsrc_try_send:<rtspsrc_0> receive interrupted
0:00:55.330180276 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:6527:gst_rtspsrc_send:<rtspsrc_0> got error -3
0:00:55.330186388 14586      0x794f6a0 WARN                 rtspsrc gstrtspsrc.c:8672:gst_rtspsrc_pause:<rtspsrc_0> PAUSE interrupted
0:00:55.330202949 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:8783:gst_rtspsrc_thread:<rtspsrc_0> got command CLOSE
0:00:55.330216061 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:5093:gst_rtspsrc_connection_flush:<rtspsrc_0> set flushing 0
0:00:55.330224716 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:5096:gst_rtspsrc_connection_flush:<rtspsrc_0> connection flush
0:00:55.330237866 14586      0x794f6a0 DEBUG                rtspsrc gstrtspsrc.c:7994:gst_rtspsrc_close:<rtspsrc_0> TEARDOWN...
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'rtspsrc_0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e83322c0)>,
 'ts': '2023-11-14 16:35:52.883325',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'source-bin-00',
 'element_parent': <Gst.Pipeline object at 0x7f014505d600 (GstPipeline at 0x788a0f0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e8312320)>,
 'ts': '2023-11-14 16:35:52.883747',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from paused to ready.',
 'element_name': 'stream-pipeline_0',
 'element_parent': None,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e8332140)>,
 'ts': '2023-11-14 16:35:52.883974',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from ready to null.',
 'element_name': 'udpsink_0',
 'element_parent': <Gst.Bin object at 0x7f0142689340 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e83321c0)>,
 'ts': '2023-11-14 16:35:52.884160',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from ready to null.',
 'element_name': 'rtppay_0',
 'element_parent': <Gst.Bin object at 0x7f0142689bc0 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e83320c0)>,
 'ts': '2023-11-14 16:35:52.884358',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from ready to null.',
 'element_name': 'h26encode_0',
 'element_parent': <Gst.Bin object at 0x7f0142689340 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f4001c00)>,
 'ts': '2023-11-14 16:35:52.889289',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from ready to null.',
 'element_name': 'capsfilt_rtspsink0',
 'element_parent': <Gst.Bin object at 0x7f0142689cc0 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f4001b80)>,
 'ts': '2023-11-14 16:35:52.889509',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from ready to null.',
 'element_name': 'nvvidconv_rtspsink0',
 'element_parent': <Gst.Bin object at 0x7f0142689340 (GstBin at 0x788d2d0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f013c135680)>,
 'ts': '2023-11-14 16:35:52.889716',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from ready to null.',
 'element_name': 'sink-bin-00',
 'element_parent': <Gst.Pipeline object at 0x7f014505d600 (GstPipeline at 0x788a0f0)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00f0002a00)>,
 'ts': '2023-11-14 16:35:52.889909',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from ready to null.',
 'element_name': 'capsfilt_rtspsrc0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e8002c20)>,
 'ts': '2023-11-14 16:35:52.890103',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from ready to null.',
 'element_name': 'nvvidconv_rtspsrc0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e832b230)>,
 'ts': '2023-11-14 16:35:52.890296',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from ready to null.',
 'element_name': 'nvv4l2decoder_0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e832b2b0)>,
 'ts': '2023-11-14 16:35:52.892979',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from ready to null.',
 'element_name': 'h26parse_0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e832b330)>,
 'ts': '2023-11-14 16:35:52.893195',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
STATE CHANGED:
{'debug': 'GSt STATE_CHANGED: Pipeline state changed from ready to null.',
 'element_name': 'rtpdepay_0',
 'element_parent': <Gst.Bin object at 0x7f0145072400 (GstBin at 0x788d130)>,
 'err': 'NA',
 'gst_msg': <Gst.Message object at 0x7f0141d4d5e0 (GstMessage at 0x7f00e832b3b0)>,
 'ts': '2023-11-14 16:35:52.893381',
 'type': <flags GST_MESSAGE_STATE_CHANGED of type Gst.MessageType>}
0:00:55.430298588 14586 0x7f013c053f00 WARN                 rtspsrc gstrtspsrc.c:6001:gst_rtspsrc_loop_send_cmd_and_wait:<rtspsrc_0> Timed out waiting for TEARDOWN to be processed.

Am I doing something wrong when trying to stop my pipeline?
Even following the process:

pause the pipeline
flush it
set the state to NULL

It still hangs.

Edit: To add to this I have tested with element videotestsrc and GLib.timeout_add_seconds() and the pipeline successfully goes to NULL. Is this a known bug with Gst 1.16.3 when using rtspsrc? I want to confirm whether or not it is my process/code.

Joe · November 14, 2023, 7:18pm

1.16.3 is kind of old. I am using 1.22.5 on Linux and stop my RTSP pipeline in steps:

check the current state
if the state is playing, pause first; then flush it
set state to null.
No problems.

ConnorM · November 14, 2023, 7:32pm

Thanks, I expect it has to do with the version I am using too. NVIDIA DeepStream build using GSt 1.16.3. I’ll keep working on trying to get my code running with DeepStream + GSt 1.22+.

Joe · November 14, 2023, 7:44pm

I had more memory leaks with the old versions and had to upgrade gstreamer.

omertal88 · November 15, 2023, 8:24am

Hey @ConnorM

I’m so glad you brought this issue up and I caught it on time.
This issue was a major pain for me for a while.
As far as I understand it’s an issue with the nvv4l2decoder.
I also found this post on the same issue: nvv4l2decoder Can't exit gracefully when stopping gsteamer pipeline - Jetson Nano - NVIDIA Developer Forums

What happens is that the nvv4l2deocer just deadlocks when trying to set its state to NULL.
NVIDIA suggests to try Jetpack r32.2 but someone wrote in the comments that it didn’t fix the issue for him on Xavier NX.

I did notice (and take what I’m saying with a grain of salt), that when I removed the h264parse before the decoder, the issue reproduces WAY less.

Can you try it and let me know?
Also, I’m using gst_element_send_event(gst_event_new_eos()) to stop the pipeline.
Maybe it helps.

Again - the issue DOES reproduce for me from time to time, but that’s the best I could come up with at the moment.

Also,
Just out of curiosity, what are you trying to achieve with this pipeline?
Why do you have to decode, then encode again?

ConnorM · November 15, 2023, 2:34pm

In my case I am using:
Ubuntu 20.04
DeepStream 6.2
GStreamer 1.16.3
GPU: RTX3060

When I switch to software decoding (avdec_h264) I still get the same issue trying to NULL the pipeline after recieving SENDER_TIMEOUT from rtpsession.
I’m going to see if my code works after trying to upgrade GStreamer with DeepStream…

This issue occurs every time I attempt to run and then NULL the pipeline, it isn’t intermittent. May be there is a difference when running on the Jetson line of Nvidia products.

omertal88 · November 15, 2023, 2:58pm

Sorry I was under the impression it was Jetson

ConnorM · November 15, 2023, 3:09pm

All good! I’ll report back with what I end up finding. I suspect it may be related to this: rtspsrc: deadlock on set_state(NULL) (#900) · Issues · GStreamer / gst-plugins-good · GitLab

zhouyuchong · June 24, 2024, 1:39am

Hi, there. Thanks for your work. But how you finally solve this problem? Since it’s been fixed in future gstreamer but deepstream not support yet.

ConnorM · June 24, 2024, 12:57pm

Hey @zhouyuchong, unfortunately I didn’t have much luck finding a solid solution. I haven’t tried with the latest GStreamer + DeepStream builds though.

Uku · August 19, 2024, 9:01am

Hi, ConnorM!

Have you managed to find a solution for this?
I try to set the state of rtspsrc to NULL in a dynamic context and have run into the same issue. As a workaround I have just ignored the best practices, and set the rtspsrc to PAUSED and remove it from the pipeline. I get a few CRITICAL errors, however nothing breaks and it seems to work as expected.

After the rtsp provider has reconnected, I create a new rtspsrc, add it to the pipeline, link it and sync the state with the parent.

This is the code I use to set it to NULL:

logger.info("Setting state to NULL")
if self.source_bin.set_state(Gst.State.NULL) != Gst.StateChangeReturn.SUCCESS:
    logger.error(f"Couldn't set state to NULL for {self.source_bin.get_name()}")
logger.info("Set state to NULL")

And this is the log output:

[INFO] [1724057693.373689326] [devstream_64c7e222517a46b67ba10b89]: Setting state to NULL
0:00:19.280784491    39 0xfffe5400ed80 WARN                 rtspsrc gstrtspsrc.c:6208:gst_rtspsrc_loop_send_cmd_and_wait:<rtspsrc0> Timed out waiting for TEARDOWN to be processed.

As you can see, the rtspsrc times out and the code hangs.

This is the error when I remove a PAUSED rtspsrc from the pipeline

(python3:39): GStreamer-CRITICAL **: 08:57:37.658: 
Trying to dispose element rtspsrc0, but it is in PAUSED instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

Funnily enough, despite the error, the element gets removed from the pipeline. This is happening on a Jetson Orin with DS 6.2 and a source build Gstreamer 1.20