I was able to integrate gstreamer webrtc into my test app. But it was dropping a lot of frames. So I decided to use hardware accelerating by swapping autovideosink with vaapisink.
But it kept failing and I wasn’t able to trace why it was failing as the program was giving me no clue.
So I tried running the example in the main repo with vaapisink instead of autovideosink. I got a segmentation fault this time.
I ran it with GST_DEBUG=5 and I got the following logs:
0:00:04.614956839 74162 0x7f0aa8006760 DEBUG GST_PADS gstpad.c:4115:check_sticky:<rtpfunnel0:src> pushing all sticky events
0:00:04.614980679 74162 0x7f0a54002300 DEBUG bin gstbin.c:4029:gst_bin_handle_message_func:<pipeline0> posting message upward
0:00:04.615068959 74162 0x7f0a54002300 DEBUG GST_BUS gstbus.c:338:gst_bus_post:<bus3> [msg 0x7f0a50001e20] posting on bus element message: 0x7f0a50001e20, time 99:99:99.999999999, seq-num 710, element 'vaapisink0', prepare-window-handle;
0:00:04.615069191 74162 0x7f0aa8006760 DEBUG GST_PADS gstpad.c:4039:push_sticky:<rtpfunnel0:src> event stream-start was already received
0:00:04.615102018 74162 0x7f0aa8006760 DEBUG GST_PADS gstpad.c:4039:push_sticky:<rtpfunnel0:src> event caps was already received
0:00:04.615112439 74162 0x7f0a54002300 DEBUG GST_BUS gstbus.c:377:gst_bus_post:<bus3> [msg 0x7f0a50001e20] dropped
0:00:04.615126570 74162 0x7f0a54002300 DEBUG GST_BUS gstbus.c:377:gst_bus_post:<bus2> [msg 0x7f0a50001e20] dropped
0:00:04.615138371 74162 0x7f0a54002300 DEBUG GST_BUS gstbus.c:377:gst_bus_post:<bus18> [msg 0x7f0a50001e20] dropped
0:00:04.615145763 74162 0x7f0aa8006760 DEBUG GST_EVENT gstpad.c:5860:gst_pad_send_event_unchecked:<rtpbin:send_rtp_sink_0> have event type segment event: 0x7f0a8c005190, time 99:99:99.999999999, seq-num 47, GstEventSegment, segment=(GstSegment)"segment, flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE, rate=(double)1, applied-rate=(double)1, format=(GstFormat)time, base=(guint64)0, offset=(guint64)0, start=(guint64)0, stop=(guint64)18446744073709551615, time=(guint64)0, position=(guint64)0, duration=(guint64)18446744073709551615;";
0:00:04.615168119 74162 0x7f0aa8006760 DEBUG GST_PADS gstpad.c:4115:check_sticky:<send_rtp_sink_0:proxypad21> pushing all sticky events
0:00:04.615180966 74162 0x7f0aa8006760 DEBUG GST_PADS gstpad.c:4039:push_sticky:<send_rtp_sink_0:proxypad21> event stream-start was already received
0:00:04.615193475 74162 0x7f0aa8006760 DEBUG GST_PADS gstpad.c:4039:push_sticky:<send_rtp_sink_0:proxypad21> event caps was already received
Segmentation fault
How do I tweak it to work with hardware acceleration?
I was reading about hardware acceleration here. It says that plugin with a higher rank will be used. But I noticed that when I tried to play a 4k 60 FPS video with playbin it was struggling to do so. But when I explicitly set the video-sink property to vaapisink, it started to play smoothly. So I kept autovideosink in the pipeline. But set the rank to PRIMARY + 1. I got error:
** (webrtc-app:79790): CRITICAL **: 00:53:07.862: gst_vaapi_window_wayland_new: assertion 'GST_VAAPI_IS_DISPLAY_WAYLAND (display)' failed
** (webrtc-app:79790): CRITICAL **: 00:53:07.863: gst_vaapi_window_wayland_new: assertion 'GST_VAAPI_IS_DISPLAY_WAYLAND (display)' failed
** (webrtc-app:79790): CRITICAL **: 00:53:07.866: gst_vaapi_window_wayland_new: assertion 'GST_VAAPI_IS_DISPLAY_WAYLAND (display)' failed
** (webrtc-app:79790): CRITICAL **: 00:53:07.866: gst_vaapi_window_wayland_new: assertion 'GST_VAAPI_IS_DISPLAY_WAYLAND (display)' failed
** (webrtc-app:79790): CRITICAL **: 00:53:07.866: gst_vaapi_window_wayland_new: assertion 'GST_VAAPI_IS_DISPLAY_WAYLAND (display)' failed
** (webrtc-app:79790): CRITICAL **: 00:53:07.866: gst_vaapi_window_wayland_new: assertion 'GST_VAAPI_IS_DISPLAY_WAYLAND (display)' failed
Error: Error from element /GstPipeline:pipeline0/GstWebRTCBin:webrtcbin/TransportReceiveBin:transportreceivebin0/GstNiceSrc:nicesrc0: Internal data stream error. (../libs/gst/base/gstbasesrc.c(3132): gst_base_src_loop (): /GstPipeline:pipeline0/GstWebRTCBin:webrtcbin/TransportReceiveBin:transportreceivebin0/GstNiceSrc:nicesrc0:
streaming stopped, reason not-negotiated (-4))
Also, if I use elements for example like decodebin, will it use vaapidecodebin if it has a higher rank?
