We’re using pion for webrtc stuff and gstreamer for media processing.
Here are the H264 pipelines:
ximagesrc name=videosrc display-name=":0.0" remote=1 blocksize=16384 use-damage=0 show-pointer=false ! video/x-raw,name=videocaps ! queue leaky=downstream flush-on-eos=true max-size-time=40000000 max-size-buffers=0 max-size-bytes=0 ! videoconvert ! video/x-raw,format=I420 ! x264enc name=vcodec threads=8 aud=false b-adapt=false key-int-max=0 sliced-threads=true byte-stream=true tune=zerolatency speed-preset=veryfast bitrate=20000 pass=17 ! video/x-h264,stream-format=byte-stream,profile=high ! appsink name=appsink
appsrc format=time is-live=true do-timestamp=true name=src ! application/x-rtp ! queue leaky=downstream flush-on-eos=true max-size-time=40000000 max-size-buffers=0 max-size-bytes=0 ! rtph264depay ! queue leaky=downstream flush-on-eos=true max-size-time=40000000 max-size-buffers=0 max-size-bytes=0 ! decodebin ! glimagesink name=glsink
Example:
Logs:
0:00:00.203085819 3319 0x7fadd8000b90 WARN videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.001855188 deadline:0:00:00.001855188 earliest_time:0:00:00.198538446
0:00:00.211246655 3319 0x7fadd8000b90 WARN videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.001895521 deadline:0:00:00.001895521 earliest_time:0:00:00.198538446
0:00:00.214320923 3319 0x7fadd8000b90 WARN videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.001931616 deadline:0:00:00.001931616 earliest_time:0:00:00.198538446
0:00:00.215463204 3319 0x7fadd8000b90 WARN videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.001960999 deadline:0:00:00.001960999 earliest_time:0:00:00.198538446
0:00:00.217387286 3319 0x7fadd8000b90 WARN videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.019110044 deadline:0:00:00.019110044 earliest_time:0:00:00.198538446
0:00:00.218385377 3319 0x7fadd8000b90 WARN videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.060143218 deadline:0:00:00.060143218 earliest_time:0:00:00.198538446
0:00:00.219368241 3319 0x7fadd8000b90 WARN videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.100068833 deadline:0:00:00.100068833 earliest_time:0:00:00.198538446
0:00:00.225658181 3319 0x7fadd8000b90 WARN videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.138886049 deadline:0:00:00.138886049 earliest_time:0:00:00.198538446
0:00:00.267029074 3319 0x7fadd8000b90 WARN videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.179997493 deadline:0:00:00.179997493 earliest_time:0:00:00.198538446
0:00:00.986247775 3319 0x7fadd8000b90 WARN videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.899323117 deadline:0:00:00.899323117 earliest_time:0:00:00.899737790
0:00:05.986087304 3319 0x7fadd8000b90 WARN videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:05.899003020 deadline:0:00:05.899003020 earliest_time:0:00:05.904278051
0:00:06.007291704 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14108, expected 14172, gap 64 <= max_reorder (100), dropping!
0:00:06.007309075 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14109, expected 14172, gap 63 <= max_reorder (100), dropping!
0:00:06.007317811 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14110, expected 14172, gap 62 <= max_reorder (100), dropping!
0:00:06.007324112 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14111, expected 14172, gap 61 <= max_reorder (100), dropping!
0:00:06.007333579 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14112, expected 14172, gap 60 <= max_reorder (100), dropping!
0:00:06.007341984 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14113, expected 14172, gap 59 <= max_reorder (100), dropping!
0:00:06.007348586 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14114, expected 14172, gap 58 <= max_reorder (100), dropping!
0:00:06.007356140 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14115, expected 14172, gap 57 <= max_reorder (100), dropping!
0:00:06.007363944 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14116, expected 14172, gap 56 <= max_reorder (100), dropping!
0:00:06.007370847 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14117, expected 14172, gap 55 <= max_reorder (100), dropping!
0:00:06.007377579 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14118, expected 14172, gap 54 <= max_reorder (100), dropping!
0:00:06.007384201 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14119, expected 14172, gap 53 <= max_reorder (100), dropping!
0:00:06.007390823 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14120, expected 14172, gap 52 <= max_reorder (100), dropping!
0:00:06.007397545 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14121, expected 14172, gap 51 <= max_reorder (100), dropping!
0:00:06.007404177 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14122, expected 14172, gap 50 <= max_reorder (100), dropping!
0:00:06.007411520 3319 0x7fadd8000b90 WARN rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14123, expected 14172, gap 49 <= max_reorder (100), dropping!
What can we do to improve the quality? We’re running both client and server in the same local network, so I’d expect not to see so many dropped packets.