Video stream over webrtc has too many artifacts

We’re using pion for webrtc stuff and gstreamer for media processing.
Here are the H264 pipelines:

ximagesrc name=videosrc display-name=":0.0" remote=1 blocksize=16384 use-damage=0 show-pointer=false ! video/x-raw,name=videocaps ! queue leaky=downstream flush-on-eos=true max-size-time=40000000 max-size-buffers=0 max-size-bytes=0 ! videoconvert ! video/x-raw,format=I420 ! x264enc name=vcodec threads=8 aud=false b-adapt=false key-int-max=0 sliced-threads=true byte-stream=true tune=zerolatency speed-preset=veryfast bitrate=20000 pass=17 ! video/x-h264,stream-format=byte-stream,profile=high ! appsink name=appsink
appsrc format=time is-live=true do-timestamp=true name=src ! application/x-rtp ! queue leaky=downstream flush-on-eos=true max-size-time=40000000 max-size-buffers=0 max-size-bytes=0 ! rtph264depay ! queue leaky=downstream flush-on-eos=true max-size-time=40000000 max-size-buffers=0 max-size-bytes=0 ! decodebin ! glimagesink name=glsink

Example:

Logs:

0:00:00.203085819  3319 0x7fadd8000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.001855188 deadline:0:00:00.001855188 earliest_time:0:00:00.198538446
0:00:00.211246655  3319 0x7fadd8000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.001895521 deadline:0:00:00.001895521 earliest_time:0:00:00.198538446
0:00:00.214320923  3319 0x7fadd8000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.001931616 deadline:0:00:00.001931616 earliest_time:0:00:00.198538446
0:00:00.215463204  3319 0x7fadd8000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.001960999 deadline:0:00:00.001960999 earliest_time:0:00:00.198538446
0:00:00.217387286  3319 0x7fadd8000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.019110044 deadline:0:00:00.019110044 earliest_time:0:00:00.198538446
0:00:00.218385377  3319 0x7fadd8000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.060143218 deadline:0:00:00.060143218 earliest_time:0:00:00.198538446
0:00:00.219368241  3319 0x7fadd8000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.100068833 deadline:0:00:00.100068833 earliest_time:0:00:00.198538446
0:00:00.225658181  3319 0x7fadd8000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.138886049 deadline:0:00:00.138886049 earliest_time:0:00:00.198538446
0:00:00.267029074  3319 0x7fadd8000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.179997493 deadline:0:00:00.179997493 earliest_time:0:00:00.198538446
0:00:00.986247775  3319 0x7fadd8000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:00.899323117 deadline:0:00:00.899323117 earliest_time:0:00:00.899737790
0:00:05.986087304  3319 0x7fadd8000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:00:05.899003020 deadline:0:00:05.899003020 earliest_time:0:00:05.904278051
0:00:06.007291704  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14108, expected 14172, gap 64 <= max_reorder (100), dropping!
0:00:06.007309075  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14109, expected 14172, gap 63 <= max_reorder (100), dropping!
0:00:06.007317811  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14110, expected 14172, gap 62 <= max_reorder (100), dropping!
0:00:06.007324112  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14111, expected 14172, gap 61 <= max_reorder (100), dropping!
0:00:06.007333579  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14112, expected 14172, gap 60 <= max_reorder (100), dropping!
0:00:06.007341984  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14113, expected 14172, gap 59 <= max_reorder (100), dropping!
0:00:06.007348586  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14114, expected 14172, gap 58 <= max_reorder (100), dropping!
0:00:06.007356140  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14115, expected 14172, gap 57 <= max_reorder (100), dropping!
0:00:06.007363944  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14116, expected 14172, gap 56 <= max_reorder (100), dropping!
0:00:06.007370847  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14117, expected 14172, gap 55 <= max_reorder (100), dropping!
0:00:06.007377579  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14118, expected 14172, gap 54 <= max_reorder (100), dropping!
0:00:06.007384201  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14119, expected 14172, gap 53 <= max_reorder (100), dropping!
0:00:06.007390823  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14120, expected 14172, gap 52 <= max_reorder (100), dropping!
0:00:06.007397545  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14121, expected 14172, gap 51 <= max_reorder (100), dropping!
0:00:06.007404177  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14122, expected 14172, gap 50 <= max_reorder (100), dropping!
0:00:06.007411520  3319 0x7fadd8000b90 WARN        rtpbasedepayload gstrtpbasedepayload.c:850:gst_rtp_base_depayload_handle_buffer:<rtph264depay0> got old packet 14123, expected 14172, gap 49 <= max_reorder (100), dropping!

What can we do to improve the quality? We’re running both client and server in the same local network, so I’d expect not to see so many dropped packets.

Another example:

AV1 pipelines:

ximagesrc name=videosrc display-name=":0.0" remote=1 blocksize=16384 use-damage=0 show-pointer=false ! video/x-raw,name=videocaps ! queue leaky=downstream flush-on-eos=true max-size-time=40000000 max-size-buffers=0 max-size-bytes=0 ! videoconvert ! video/x-raw,format=I420 ! av1enc name=vcodec cpu-used=10 end-usage=cbr keyframe-max-dist=30 lag-in-frames=0 overshoot-pct=10 row-mt=True usage-profile=realtime tile-columns=2 tile-rows=2 threads=8 target-bitrate=20000 ! video/x-av1,parsed=True,stream-format=obu-stream ! appsink name=appsink
appsrc format=time is-live=true do-timestamp=true name=src ! application/x-rtp,encoding-name=AV1,payload=96 ! queue ! rtpav1depay ! queue ! decodebin ! glimagesink name=glsink

The issue here is that video constantly freezes.

0:00:00.095630325  7219 0x7f768c0010d0 FIXME                default gstutils.c:4088:gst_element_decorate_stream_id_internal:<src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:00.099881419  7219 0x7f768c000b90 FIXME           videodecoder gstvideodecoder.c:1193:gst_video_decoder_drain_out:<av1dec0> Sub-class should implement drain()
0:00:00.099900325  7219 0x7f768c000b90 FIXME           videodecoder gstvideodecoder.c:1193:gst_video_decoder_drain_out:<av1dec0> Sub-class should implement drain()
0:00:00.162207232  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:00.000738035 deadline:0:00:00.000738035 earliest_time:0:00:00.087590971
0:00:00.165198372  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:00.000766699 deadline:0:00:00.000766699 earliest_time:0:00:00.087590971
0:00:00.168140051  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:00.000772299 deadline:0:00:00.000772299 earliest_time:0:00:00.087590971
0:00:00.171690240  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:00.000778952 deadline:0:00:00.000778952 earliest_time:0:00:00.087590971
0:00:00.178615400  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:00.039372817 deadline:0:00:00.039372817 earliest_time:0:00:00.087590971
0:00:00.218440565  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:00.079890501 deadline:0:00:00.079890501 earliest_time:0:00:00.087590971
0:00:01.176103270  7219 0x7f768c000b90 WARN                basesink gstbasesink.c:3147:gst_base_sink_is_too_late:<sink> warning: A lot of buffers are being dropped.
0:00:01.176119681  7219 0x7f768c000b90 WARN                basesink gstbasesink.c:3147:gst_base_sink_is_too_late:<sink> warning: There may be a timestamping problem, or this computer is too slow.
0:00:01.216983063  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:01.078782715 deadline:0:00:01.078782715 earliest_time:0:00:01.092399070
0:00:01.297033623  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:01.159670765 deadline:0:00:01.159670765 earliest_time:0:00:01.174060363
0:00:01.376457527  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:01.239517213 deadline:0:00:01.239517213 earliest_time:0:00:01.253200644
0:00:01.456774306  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:01.319681115 deadline:0:00:01.319681115 earliest_time:0:00:01.334089685
0:00:01.536073726  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:01.400436426 deadline:0:00:01.400436426 earliest_time:0:00:01.415512511
0:00:01.617135694  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:01.479900837 deadline:0:00:01.479900837 earliest_time:0:00:01.494611796
0:00:01.695934045  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:01.559653799 deadline:0:00:01.559653799 earliest_time:0:00:01.573163815
0:00:01.776517725  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:01.638746471 deadline:0:00:01.638746471 earliest_time:0:00:01.652389337
0:00:01.856397604  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:01.720311242 deadline:0:00:01.720311242 earliest_time:0:00:01.735978324
0:00:01.935812239  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:01.799771714 deadline:0:00:01.799771714 earliest_time:0:00:01.815072199
0:00:02.017748766  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:01.879326142 deadline:0:00:01.879326142 earliest_time:0:00:01.894162926
0:00:02.097603828  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:01.959394434 deadline:0:00:01.959394434 earliest_time:0:00:01.972948102
0:00:02.178315567  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:02.039925875 deadline:0:00:02.039925875 earliest_time:0:00:02.053982517
0:00:02.259521213  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:02.120379530 deadline:0:00:02.120379530 earliest_time:0:00:02.133771204
0:00:02.358131717  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:02.199687455 deadline:0:00:02.199687455 earliest_time:0:00:02.213056618
0:00:02.417259197  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:02.280027848 deadline:0:00:02.280027848 earliest_time:0:00:02.292460522
0:00:02.498247645  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:02.360192781 deadline:0:00:02.360192781 earliest_time:0:00:02.374482759
0:00:02.577270186  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:02.442832387 deadline:0:00:02.442832387 earliest_time:0:00:02.458636215
0:00:02.657696069  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:02.519697961 deadline:0:00:02.519697961 earliest_time:0:00:02.533668521
0:00:02.738027535  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:02.599564575 deadline:0:00:02.599564575 earliest_time:0:00:02.613521959
0:00:02.858640009  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:02.858657682  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Reference buffer frame ID mismatch)
0:00:02.903928360  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:02.903951233  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:02.937090475  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:02.937110823  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:02.976947809  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:02.976969099  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.016049195  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:03.016074943  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.056102457  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:03.056122674  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.096188048  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:03.096209368  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.136676806  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:03.136713215  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.176700042  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:03.176729337  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.216702628  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:03.216732334  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.257466983  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:03.257487852  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.297500197  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:03.297524883  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.337858470  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:03.337887855  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.377201219  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:03.377221718  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.417174120  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:03.417200159  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.457159394  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:03.457175654  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.523743712  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4787:_gst_video_decoder_error:<av1dec0> error: Failed to decode frame
0:00:03.523761545  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:4789:_gst_video_decoder_error:<av1dec0> error: Corrupt frame detected (details: Keyframe / intra-only frame required to reset decoder state)
0:00:03.578313283  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:03.439983794 deadline:0:00:03.439983794 earliest_time:0:00:03.507314484
0:00:03.618964876  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:03.480501316 deadline:0:00:03.480501316 earliest_time:0:00:03.507314484
0:00:03.697673718  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:03.560349744 deadline:0:00:03.560349744 earliest_time:0:00:03.573220141
0:00:03.777434151  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:03.641495977 deadline:0:00:03.641495977 earliest_time:0:00:03.656924111
0:00:03.857904536  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:03.720433097 deadline:0:00:03.720433097 earliest_time:0:00:03.735413941
0:00:03.938585144  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:03.800780651 deadline:0:00:03.800780651 earliest_time:0:00:03.816982274
0:00:04.018095979  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:03.880572061 deadline:0:00:03.880572061 earliest_time:0:00:03.893959014
0:00:04.097379346  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:03.959662086 deadline:0:00:03.959662086 earliest_time:0:00:03.972407837
0:00:04.184429303  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:04.040781138 deadline:0:00:04.040781138 earliest_time:0:00:04.055047894
0:00:04.258066960  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:04.122043988 deadline:0:00:04.122043988 earliest_time:0:00:04.133411493
0:00:04.337948020  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:04.201363614 deadline:0:00:04.201363614 earliest_time:0:00:04.217704279
0:00:04.416771605  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:04.280189003 deadline:0:00:04.280189003 earliest_time:0:00:04.293962261
0:00:04.498899548  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:04.360434746 deadline:0:00:04.360434746 earliest_time:0:00:04.375280898
0:00:04.578491575  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:04.440605388 deadline:0:00:04.440605388 earliest_time:0:00:04.453350449
0:00:04.657264596  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:04.520434680 deadline:0:00:04.520434680 earliest_time:0:00:04.533000325
0:00:04.777143932  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:04.625972145 deadline:0:00:04.625972145 earliest_time:0:00:04.675370127
0:00:04.779784736  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:04.640096008 deadline:0:00:04.640096008 earliest_time:0:00:04.675370127
0:00:04.859412940  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:04.721228474 deadline:0:00:04.721228474 earliest_time:0:00:04.740133369
0:00:04.937138314  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:04.800460294 deadline:0:00:04.800460294 earliest_time:0:00:04.816789447
0:00:05.019243274  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:04.902707024 deadline:0:00:04.902707024 earliest_time:0:00:04.939583568
0:00:05.058164661  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:04.921168043 deadline:0:00:04.921168043 earliest_time:0:00:04.939583568
0:00:05.138323510  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:05.000508828 deadline:0:00:05.000508828 earliest_time:0:00:05.016129831
0:00:05.217655668  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:05.082004554 deadline:0:00:05.082004554 earliest_time:0:00:05.098657805
0:00:05.297934121  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:05.160495936 deadline:0:00:05.160495936 earliest_time:0:00:05.174882796
0:00:05.377451397  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:05.241277372 deadline:0:00:05.241277372 earliest_time:0:00:05.258053492
0:00:05.457602081  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:05.340912654 deadline:0:00:05.340912654 earliest_time:0:00:05.375390499
0:00:05.497406264  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:05.359878179 deadline:0:00:05.359878179 earliest_time:0:00:05.375390499
0:00:05.578082342  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:05.439868521 deadline:0:00:05.439868521 earliest_time:0:00:05.453939619
0:00:05.658276909  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:05.519635195 deadline:0:00:05.519635195 earliest_time:0:00:05.532016432
0:00:05.737908289  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:05.600446077 deadline:0:00:05.600446077 earliest_time:0:00:05.615805110
0:00:05.816888306  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:05.680253908 deadline:0:00:05.680253908 earliest_time:0:00:05.695345640
0:00:05.925164255  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:05.760167136 deadline:0:00:05.760167136 earliest_time:0:00:05.774288888
0:00:05.977912297  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:05.839944979 deadline:0:00:05.839944979 earliest_time:0:00:05.905938108
0:00:06.018356590  7219 0x7f768c000b90 WARN            videodecoder gstvideodecoder.c:3661:gst_video_decoder_clip_and_push_buf:<av1dec0> Dropping frame due to QoS. start:0:00:05.879704558 deadline:0:00:05.879704558 earliest_time:0:00:05.905938108

Both client and server have custom-built version of gstreamer. Basically, it’s archlinux’s 1.24.4-1 with gstreamer/-/merge_requests/7008, gstreamer/-/merge_requests/7040 and gstreamer/-/issues/3622#note_2455492 (had to remove gitlab.freedesktop prefix due to new user restrictions)
gst-plugins-rs are built from Sebastian Dröge / gst-plugins-rs · GitLab

For AV1, can you give it a try with latest git master of GStreamer and gst-plugins-rs? There were a couple more relevant fixes. Also av1parse: Do not return error when expectedFrameId mismatch (!7047) · Merge requests · GStreamer / gstreamer · GitLab should be included.

In addition, please share your sender / receiver code and how it integrates with Pion. As discussed on GitLab, you should add support for requesting keyframes on the sender side so this can be mitigated to some degree.

Also, 20MBit/s is quite a bit of bandwidth and would require Pion to do the networking bits correctly. I assume a lower bitrate works better?

@slomo
I tried building master, but couldn’t get it to work with gst-plugins-rs-git and gstreamer-git AUR packages, unfortunately.
gst-plugins-rs fail here:

warning: use of deprecated method `gtk4::prelude::WidgetExt::show`: Since 4.10
   --> video/gtk4/src/sink/imp.rs:736:20
    |
736 |             window.show();
    |                    ^^^^
    |
    = note: `#[warn(deprecated)]` on by default

error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/usr/lib64/rustlib/x86_64-unknown-linux-gnu/bin:/usr/bin:/home/user/go/bin:/home/user/.bin/:/home/user/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/user/.local/bin:/home/user/.local/bin" VSLANG="1033" "cc" "-Wl,--version-script=/tmp/rustcT6HVnQ/list" "-Wl,--no-undefined-version" "-m64" "/tmp/rustcT6HVnQ/symbols.o" "/home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/gstquinn.gstquinn.293b0500a7a9fb6c-cgu.12.rcgu.o" "-Wl,--as-needed" "-L" "/home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps" "-L" "/home/user/build/gst-plugins-rs-git/src/build/target/release/deps" "-L" "/home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/build/ring-273c02d6802beba9/out" "-L" "/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/tmp/rustcT6HVnQ/libring-4abba42067ace997.rlib" "/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-46939b2cf47f5496.rlib" "-Wl,-Bdynamic" "-lgstreamer-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstbase-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstreamer-1.0" "-lgobject-2.0" "-lgobject-2.0" "-lglib-2.0" "-lgio-2.0" "-lglib-2.0" "-lgobject-2.0" "-lglib-2.0" "-lgobject-2.0" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/libgstquinn.so" "-Wl,--gc-sections" "-shared" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-Wl,-soname,libgstquinn.so"
  = note: /usr/bin/ld: /home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/gstquinn.gstquinn.293b0500a7a9fb6c-cgu.12.rcgu.o: in function `ring::aead::aes_gcm::aes_gcm_seal':
          gstquinn.293b0500a7a9fb6c-cgu.12:(.text._ZN4ring4aead7aes_gcm12aes_gcm_seal17h27eab254072510c5E+0x1a4): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/gstquinn.gstquinn.293b0500a7a9fb6c-cgu.12.rcgu.o: in function `ring::cpu::intel::Feature::available':
          /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65:(.text._ZN4ring4aead7aes_gcm12aes_gcm_seal17h27eab254072510c5E+0x332): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65:(.text._ZN4ring4aead7aes_gcm12aes_gcm_open17h34b84628c6461ba5E+0x1b3): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65:(.text._ZN4ring4aead7aes_gcm12aes_gcm_open17h34b84628c6461ba5E+0x3d7): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65:(.text._ZN4ring4aead3gcm7Context12update_block17h6e2056f3728839deE+0xb): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/gstquinn.gstquinn.293b0500a7a9fb6c-cgu.12.rcgu.o:/home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65: more undefined references to `ring_core_0_17_8_OPENSSL_ia32cap_P' follow
          /usr/bin/ld: /tmp/rustcT6HVnQ/libring-4abba42067ace997.rlib(5322d8ad5a33c2e7-chacha-x86_64-elf.o): relocation R_X86_64_PC32 against undefined hidden symbol `ring_core_0_17_8_OPENSSL_ia32cap_P' can not be used when making a shared object
          /usr/bin/ld: final link failed: bad value
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `gst-plugin-quinn` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
warning: `gst-plugin-gtk4` (lib) generated 1 warning
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/usr/lib64/rustlib/x86_64-unknown-linux-gnu/bin:/usr/bin:/home/user/go/bin:/home/user/.bin/:/home/user/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/user/.local/bin:/home/user/.local/bin" VSLANG="1033" "cc" "-Wl,--version-script=/tmp/rustcta88BH/list" "-Wl,--no-undefined-version" "-m64" "/tmp/rustcta88BH/symbols.o" "/home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/gstaws.gstaws.a098888bfba7f8e7-cgu.11.rcgu.o" "-Wl,--as-needed" "-L" "/home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps" "-L" "/home/user/build/gst-plugins-rs-git/src/build/target/release/deps" "-L" "/home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/build/ring-273c02d6802beba9/out" "-L" "/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/tmp/rustcta88BH/libring-4abba42067ace997.rlib" "/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-46939b2cf47f5496.rlib" "-Wl,-Bdynamic" "-lgobject-2.0" "-lglib-2.0" "-lgstbase-1.0" "-lgstreamer-1.0" "-lgstaudio-1.0" "-lgstreamer-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstbase-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstreamer-1.0" "-lgobject-2.0" "-lgobject-2.0" "-lglib-2.0" "-lgio-2.0" "-lglib-2.0" "-lgobject-2.0" "-lglib-2.0" "-lgobject-2.0" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/libgstaws.so" "-Wl,--gc-sections" "-shared" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-Wl,-soname,libgstaws.so"
  = note: /usr/bin/ld: /home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/gstaws.gstaws.a098888bfba7f8e7-cgu.11.rcgu.o: in function `ring::aead::aes_gcm::aes_gcm_seal':
          gstaws.a098888bfba7f8e7-cgu.11:(.text._ZN4ring4aead7aes_gcm12aes_gcm_seal17h27eab254072510c5E+0x1a4): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/gstaws.gstaws.a098888bfba7f8e7-cgu.11.rcgu.o: in function `ring::cpu::intel::Feature::available':
          /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65:(.text._ZN4ring4aead7aes_gcm12aes_gcm_seal17h27eab254072510c5E+0x332): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65:(.text._ZN4ring4aead7aes_gcm12aes_gcm_open17h34b84628c6461ba5E+0x1b3): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65:(.text._ZN4ring4aead7aes_gcm12aes_gcm_open17h34b84628c6461ba5E+0x3d7): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65:(.text._ZN4ring4aead3gcm7Context12update_block17h6e2056f3728839deE+0xb): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/gstaws.gstaws.a098888bfba7f8e7-cgu.11.rcgu.o:/home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65: more undefined references to `ring_core_0_17_8_OPENSSL_ia32cap_P' follow
          /usr/bin/ld: /tmp/rustcta88BH/libring-4abba42067ace997.rlib(5322d8ad5a33c2e7-chacha-x86_64-elf.o): relocation R_X86_64_PC32 against undefined hidden symbol `ring_core_0_17_8_OPENSSL_ia32cap_P' can not be used when making a shared object
          /usr/bin/ld: final link failed: bad value
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `gst-plugin-aws` (lib) due to 1 previous error
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/usr/lib64/rustlib/x86_64-unknown-linux-gnu/bin:/usr/bin:/home/user/go/bin:/home/user/.bin/:/home/user/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/user/.local/bin:/home/user/.local/bin" VSLANG="1033" "cc" "-Wl,--version-script=/tmp/rustcUrys9g/list" "-Wl,--no-undefined-version" "-m64" "/tmp/rustcUrys9g/symbols.o" "/home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/gstrswebrtc.gstrswebrtc.a503a138cc8e034c-cgu.00.rcgu.o" "-Wl,--as-needed" "-L" "/home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps" "-L" "/home/user/build/gst-plugins-rs-git/src/build/target/release/deps" "-L" "/home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/build/ring-273c02d6802beba9/out" "-L" "/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/tmp/rustcUrys9g/libring-4abba42067ace997.rlib" "/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-46939b2cf47f5496.rlib" "-Wl,-Bdynamic" "-lgstreamer-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstnet-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstbase-1.0" "-lgstreamer-1.0" "-lgstaudio-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstbase-1.0" "-lgstreamer-1.0" "-lgstwebrtc-1.0" "-lgstreamer-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstsdp-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstbase-1.0" "-lgstreamer-1.0" "-lgstvideo-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstbase-1.0" "-lgstreamer-1.0" "-lgstapp-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstbase-1.0" "-lgstreamer-1.0" "-lgstrtp-1.0" "-lgstreamer-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstbase-1.0" "-lssl" "-lcrypto" "-lgobject-2.0" "-lglib-2.0" "-lgstreamer-1.0" "-lgobject-2.0" "-lgobject-2.0" "-lglib-2.0" "-lgio-2.0" "-lglib-2.0" "-lgobject-2.0" "-lglib-2.0" "-lgobject-2.0" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/libgstrswebrtc.so" "-Wl,--gc-sections" "-shared" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-Wl,-soname,libgstrswebrtc.so"
  = note: /usr/bin/ld: /home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/gstrswebrtc.gstrswebrtc.a503a138cc8e034c-cgu.00.rcgu.o: in function `jsonwebtoken::encoding::encode':
          gstrswebrtc.a503a138cc8e034c-cgu.00:(.text._ZN12jsonwebtoken8encoding6encode17h2665b0626461823bE+0x3b3b): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/gstrswebrtc.gstrswebrtc.a503a138cc8e034c-cgu.00.rcgu.o: in function `ring::cpu::intel::Feature::available':
          /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65:(.text._ZN4ring4aead7aes_gcm12aes_gcm_seal17h27eab254072510c5E+0x1a4): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65:(.text._ZN4ring4aead7aes_gcm12aes_gcm_seal17h27eab254072510c5E+0x332): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65:(.text._ZN4ring4aead7aes_gcm12aes_gcm_open17h34b84628c6461ba5E+0x1b3): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65:(.text._ZN4ring4aead7aes_gcm12aes_gcm_open17h34b84628c6461ba5E+0x3d7): undefined reference to `ring_core_0_17_8_OPENSSL_ia32cap_P'
          /usr/bin/ld: /home/user/build/gst-plugins-rs-git/src/build/target/x86_64-unknown-linux-gnu/release/deps/gstrswebrtc.gstrswebrtc.a503a138cc8e034c-cgu.00.rcgu.o:/home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.8/src/cpu/intel.rs:65: more undefined references to `ring_core_0_17_8_OPENSSL_ia32cap_P' follow
          /usr/bin/ld: /tmp/rustcUrys9g/libring-4abba42067ace997.rlib(5322d8ad5a33c2e7-chacha-x86_64-elf.o): relocation R_X86_64_PC32 against undefined hidden symbol `ring_core_0_17_8_OPENSSL_ia32cap_P' can not be used when making a shared object
          /usr/bin/ld: final link failed: bad value
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `gst-plugin-webrtc` (lib) due to 1 previous error
Error: CliError { error: Some(3 jobs failed), exit_code: 101 }
FAILED: libgstrsaudiofx.so libgstclaxon.so libgstlewton.so libgstspotify.so libgstrsfile.so libgstoriginalbuffer.so libgstthreadshare.so libgstrsinter.so libgstmp4.so libgstfmp4.so libgstaws.so libgsthlssink3.so libgstndi.so libgstrsonvif.so libgstraptorq.so libgstreqwest.so libgstrsrtsp.so libgstrsrtp.so libgstwebrtchttp.so libgstrswebrtc.so libgsttextahead.so libgstjson.so libgstregex.so libgsttextwrap.so libgstrstracers.so libgsturiplaylistbin.so libgstcdg.so libgstrsclosedcaption.so libgstdav1d.so libgstffv1.so libgstrsflv.so libgstgif.so libgsthsv.so libgstrspng.so libgstrav1e.so libgstrsvideofx.so libgstgopbuffer.so libgstquinn.so libgstrswebp.so libgstsodium.so libgstgtk4.so libgstfallbackswitch.so libgstlivesync.so libgsttogglerecord.so 
env SODIUM_USE_PKG_CONFIG=1 NASM=/usr/bin/nasm PKG_CONFIG=/usr/bin/pkg-config 'RUSTC=rustc -C linker=cc' /usr/bin/python /home/user/build/gst-plugins-rs-git/src/gst-plugins-rs/cargo_wrapper.py build /home/user/build/gst-plugins-rs-git/src/build /home/user/build/gst-plugins-rs-git/src/gst-plugins-rs /home/user/build/gst-plugins-rs-git/src/build release /usr lib --packages gst-plugin-audiofx gst-plugin-claxon gst-plugin-lewton gst-plugin-spotify gst-plugin-file gst-plugin-originalbuffer gst-plugin-threadshare gst-plugin-inter gst-plugin-mp4 gst-plugin-fmp4 gst-plugin-aws gst-plugin-hlssink3 gst-plugin-ndi gst-plugin-onvif gst-plugin-raptorq gst-plugin-reqwest gst-plugin-rtsp gst-plugin-rtp gst-plugin-webrtchttp gst-plugin-webrtc gst-plugin-textahead gst-plugin-json gst-plugin-regex gst-plugin-textwrap gst-plugin-tracers gst-plugin-uriplaylistbin gst-plugin-cdg gst-plugin-closedcaption gst-plugin-dav1d gst-plugin-ffv1 gst-plugin-flavors gst-plugin-gif gst-plugin-hsv gst-plugin-png gst-plugin-rav1e gst-plugin-videofx gst-plugin-gopbuffer gst-plugin-quinn gst-plugin-webp gst-plugin-sodium gst-plugin-gtk4 gst-plugin-fallbackswitch gst-plugin-livesync gst-plugin-togglerecord --depfile gst-plugins-rs.dep --lib-suffixes so --features glib/v2_74 gio/v2_74 gst-plugin-rav1e/asm gst-app/v1_22 gst/v1_22 gst-rtp/v1_22 gst-webrtc/v1_22 gst-video/v1_22 gst-sdp/v1_22 gst/v1_22 clap serde serde_json clap wayland x11egl x11glx dmabuf gtk_v4_14 gst/v1_20 --disable-doc
ninja: build stopped: subcommand failed.
==> ERROR: A failure occurred in build().
    Aborting...

gstreamer:

gstreamer/subprojects/gst-plugins-bad/gst-libs/gst/cuda/meson.build:95:29: ERROR: C header 'nvbufsurface.h' not found

It’s quite painful to build them without PKGBUILDS, so I’d rather apply necessary patches manually on top of stable branches.

I think all the required fixes are in the upcoming 1.24.5, if that helps anything.

Sure, I built gstreamer as of commit aa5092dabff3e696f541b746dae3a4127cda6eda and gst-plugins-rs as of commit 59bada0a9f743aaca1932f659339083b1ee2c957

You’re right, it’s a bit better with 2 Mbps, but still not good enough.

Sorry, I cannot publish the source code. Do you need a minimal example to test? I probably can write it.

There is already support for keyframes on the sender side. Receiver requests them every 1 minute, but Chromium doesn’t and everything works better (H264 only though as AV1 doesn’t work in Chromium).

But it basically is pion/webrtc/tree/master/examples/play-from-disk-renegotiation + gstreamer-src for the server and pion/example-webrtc-applications/blob/997c32f46b3a902eadc350dd186b660198299030/gstreamer-receive as a client. GStreamer integration is in pion/example-webrtc-applications/tree/997c32f46b3a902eadc350dd186b660198299030/internal

@slomo I tried to rewrite this from pion to webrtcbin, but it doesn’t seem to work. Can you please take a look at Webrtcbin with custom signaling doesn't produce video?

1 Like