TL;DR I want to measure the latency when using UDP when transferring frames from my Raspberry PI → Linux laptop → Back to RPI
Context
I’m running Buster on my Raspberry PI with a project that I inherited at work and I’ve been able to send the frames from the Raspberry PI to the Linux laptop (for image processing) and back to the Raspberry PI just fine.
But for what it’s worth, to make my life a little easier I’m relying on the RPI terminal to send the frames over with the following terminal commands, which I know SHOULD be used for debugging purposes and not for any real-time applications.
I’m also stuck using GStreamer 1.14 on the Raspberry PI since I’m running Buster.
Below are the pipelines that I’m currently using for my application.
GStreamer pipelines being used
#################### On the raspberry pi side ####################
# Send captured frames to Linux laptop
# irCamera
raspivid -t 0 -n -w 640 -h 480 -fps 30 -b 2000000 -o - -cs 0 | \
gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=10 pt=96 ! \
udpsink host=xxx.xx.xxx.xxx port=5000 &
# visibleCamera
raspivid -t 0 -n -w 640 -h 480 -fps 30 -b 2000000 -o - -cs 1 | \
gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=10 pt=96 ! \
udpsink host=xxx.xx.xxx.xxx port=5001 &
# Recieve processed frames from Linux laptop
gst-launch-1.0 -v udpsrc address=xxx.xx.xxx.xxx port=5002 caps=application/x-rtp ! \
rtph264depay ! h264parse ! queue ! v4l2h264dec ! autovideosink sync=false &
# visibleCameraPipeline
gst-launch-1.0 -v udpsrc address=xxx.xx.xxx.xxx port=5003 caps=application/x-rtp ! \
rtph264depay ! h264parse ! queue ! v4l2h264dec ! autovideosink sync=false &
######################################################
#################### On the Linux laptop side ####################
// Recieve incoming frames from RPI
std::string irRecieveCameraPipeline = "udpsrc port=5000 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! decodebin ! videoconvert ! appsink sync=false drop=true";
std::string visibleRecieveFromRPICameraPipeline = "udpsrc port=5001 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! decodebin ! videoconvert ! appsink sync=false drop=true";
// Send processed frames back to RPI
std::string irSendIRCameraPipe = "appsrc ! videoconvert ! queue ! openh264enc bitrate=2000000 complexity=2 ! h264parse ! rtph264pay ! udpsink host=xxx.xx.xxx.xxx port=5002";
std::string visibleSendIRCameraPipe = "appsrc ! videoconvert ! queue ! openh264enc bitrate=2000000 complexity=2 ! h264parse ! rtph264pay ! udpsink host=xxx.xx.xxx.xxx port=5003";
What I’m trying to accomplish
I’m very aware of the GST_TRACERS and how they can be used to measure the latency within the pipeline, but when it comes to measuring the latency between the actual transfer from the RPI → laptop and vice versa has me completely stumped.
I attempted to monitor the wireless traffic between both devices using Wireshark but after sifting through the data I couldn’t tell when a frame started/stopped or which data packets corresponded to the actual image data.
I’m open to any suggestions that y’all may have