Gstreamer not working in gazebo

Plugin and library development
1

Hi,
I’m experiencing an issue with video streaming from the plane in Gazebo.

ubuntu 22.04
gazebo sim, version 8.9.0
ROS 2 Humble

There are no errors in operation, startup, or monitoring. Only the camera does not work.

camera code :

<sensor name="camera" type="camera">
        <pose>0.2 0 0 0 0 0</pose> 
        <camera>
          <horizontal_fov>1.047</horizontal_fov>
          <image>
            <width>640</width>
            <height>480</height>
          </image>
          <clip>
            <near>0.1</near>
            <far>100</far>
          </clip>
        </camera>
        <always_on>1</always_on>
        <update_rate>30</update_rate>
        <visualize>true</visualize>
     <plugin name="GstCameraPlugin"
           filename="GstCameraPlugin">
          <udp_host>127.0.0.1</udp_host>
          <udp_port>5600</udp_port>
          <use_basic_pipeline>true</use_basic_pipeline>
          <use_cuda>false</use_cuda>
        </plugin>
      </sensor>

Camera output when operating in a gazebo:

GstCameraPlugin: attached to sensor [camera]
GstCameraPlugin: streaming video to 127.0.0.1:5600
[SystemManager.cc:80] Loaded system [GstCameraPlugin] for entity [30]

GST:

gst-launch-1.0 -v udpsrc address=127.0.0.1 port=5600 caps=“application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96” ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! autovideosink sync=false

The page continues on these outputs and there is no live video streaming :

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstUDPSrc:udpsrc0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96
/GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96
2

Are you sure data is actually making it to the udpsrc? (You can check the GST_DEBUG=*src:6 log)

Have you tried using wireshark to see if it can parse the RTP/H.264 packets coming in?

Does it work with other receivers? (e.g. VLC)

3

When using GST_DEBUG=*src:6

There are many lines, but the most important ones are:

The pipeline is running and reaches the PLAYING stage successfully:

Setting pipeline to PLAYING ...
New clock: GstSystemClock

udpsrc is not receiving any data!

gst_udpsrc_fill:<udpsrc0> doing select, timeout -1

In Wireshark, I’m trying to output this, but I didn’t find anything on the port.

wireshark
** (wireshark:7156) 16:52:11.397121 [Capture MESSAGE] -- Capture Start ...
** (wireshark:7156) 16:52:11.499554 [Capture MESSAGE] -- Error message from child: "Couldn't run /usr/bin/dumpcap in child process: Permission denied", ""
** (wireshark:7156) 16:52:12.830235 [Capture MESSAGE] -- Capture stopped.
** (wireshark:7156) 16:53:36.145825 [Capture MESSAGE] -- Capture Start... 
** (wireshark:7156) 16:53:36.225326 [Capture MESSAGE] -- Error message from child: "Couldn't run /usr/bin/dumpcap in child process: Permission denied", "" 
** (wireshark:7156) 16:53:36.912030 [Capture MESSAGE] -- Capture stopped. 
** (wireshark:7156) 16:53:57.811870 [Capture MESSAGE] -- Capture Start... 
** (wireshark:7156) 16:53:57.892555 [Capture MESSAGE] -- Error message from child: "Couldn't run /usr/bin/dumpcap in child process: Permission denied", "" 
** (wireshark:7156) 16:53:59.020405 [Capture MESSAGE] -- Capture stopped.

Using Wireshark, I did not see any signals on UDP or anything else. Unfortunately, my problem is that I do not know the solution, even though everything is correct (I think).

4

Re. wireshark sounds like you need more permissions, and/or add your user to the right group.

But if udpsrc doesn’t receive anything the problem is likely outside of GStreamer :slight_smile:

5

Does Gstreamer support streaming from Gazebo Sim?

Write here so I can know the reason and the solution, if you can help me,
I entered Matria-gstreamer

6

You could publish images from Gazebo to ROS and then stream it to the web using the webrtc-video capability we’ve built: WebRTC Video | Transitive Robotics

Otherwise, I’d check whether the GstCameraPlugin really encodes the stream in h264 as your pipeline seems to suggest – you are using rtph264depay but if the rtp stream (is it even RTP?) doesn’t contain a h264 stream then that won’t work.