Gst and OpenCV, read mp4 on arm64. Strange messages

General Discussion
1

Hi.

Here is my pipeline to read:

"filesrc location=file.mp4 ! qtdemux ! queue ! h264parse ! omxh264dec ! 
nvvidconv ! video/x-raw,format=BGRx ! queue ! videoconvert !
queue ! video/x-raw, format=BGR ! appsink

This pipeline is launched on machine with jetson.

I have following messaes and warnings:

nvbuf_utils: Could not get EGL display connection 

(dotnet6:5968): GStreamer-CRITICAL **: 22:53:19.057: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed

(dotnet6:5968): GStreamer-CRITICAL **: 22:53:19.057: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed

(dotnet6:5968): GStreamer-CRITICAL **: 22:53:19.057: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed

(dotnet6:5968): GStreamer-CRITICAL **: 22:53:19.058: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed

(dotnet6:5968): GStreamer-CRITICAL **: 22:53:19.058: gst_structure_get_string: assertion 'structure != NULL' failed

(dotnet6:5968): GStreamer-CRITICAL **: 22:53:19.058: gst_mini_object_unref: assertion 'mini_object != NULL' failed

also these warnings:

[ WARN:0] global /src/cap_gstreamer.cpp (898) open OpenCV | GStreamer warning: unable to query duration of stream
[ WARN:0] global /src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=4, duration=-1

Pipeline works, it reads mp4 file into OpenCv Mat types. No problem with that.
But I wonder what is the reason behind gst-critical messages and OpenCv warnings?
Also, how to omit EGL message in the beggining?

Thanks in advance.

2

Opencv warnings are harmless and expected. A live video has no duration, so its current position cannot be computed.

Also note that omx plugins are deprecated since a few JetPack releases. Try using nvv4l2decoder instead.

For the remaining, probably your Jetson is headless. You may first try with a local monitor and GUI and set DISPLAY env variable.
When ok you may try installing NoMachine or TeamViewer for remote access.

1 Like
3

Thank you so much for reply. But it is not live video, it is mp4 file. Within gst I can’t event set it’s position – pipeline goes to link error state.

4

Sorry I missed that you’re using a file source, usually this warning is seen with live sources.
A mp4 file should be ok… in your case what gives:

gst-discoverer-1.0 -v file.mp4
5 
Topology:
  container: video/quicktime, variant=(string)iso
    video: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4, profile=(string)high, width=(int)1920, height=(int)1080, framerate=(fraction)25000/1661, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
      Tags:
        video codec: H.264 / AVC
        maximum bitrate: 12829259
        bitrate: 12829259
        datetime: 2023-12-27T15:53:41Z
        encoder: Lavf58.76.100
        container format: ISO MP4/M4A

      Codec:
        video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4, profile=(string)high, width=(int)1920, height=(int)1080, framerate=(fraction)25000/1661, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
      Additional info:
        None
      Stream ID: e98c5fb330772a100e43680b2d1da2424908362c1585cd2f805c297187cd8622/001
      Width: 1920
      Height: 1080
      Depth: 24
      Frame rate: 25000/1661
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 12829259
      Max bitrate: 12829259

Properties:
  Duration: 0:00:20.134000000
  Seekable: yes
  Live: no
  Tags:
      video codec: H.264 / AVC
      maximum bitrate: 12829259
      bitrate: 12829259
      datetime: 2023-12-27T15:53:41Z
      encoder: Lavf58.76.100
      container format: ISO MP4/M4A

Also in code, when I try
_videoCapture.PosFrames =0;
it throws but as seen above stream (mp4 file) is seekable.

6

Your mp4 file looks ok to me.
Are you able to play with gst-play-1.0 file.mp4 and pause with space bar, then rewind with left arrow key then unpause with space bar ?

I have been able to reproduce your issue with Jetson AGX Orin running R36.2, gst 1.20.3 and OpenCv 4.8.0. Playing with gst-play worked fine, the issue looks to be within opencv.
You may debug looking at videoio/cap_gstreamer or file an issue to opencv github.

You may alternatively use gst events such as in Basic tutorial 4: Time management, and if you need a cv mat or GpuMat you may try using a src pad probe from an element in the pipeline for getting the buffer into a cv mat or GpuMat.

Someone more skilled may better help.

7

Hi.

Your mp4 file looks ok to me.
Are you able to play with gst-play-1.0 file.mp4 and pause with space bar, then rewind with left arrow key then unpause with space bar ?

I connected to ssh to host, so maybe following output is expected or maybe not:

tx2:/home/data/tmp$ gst-play-1.0 video_00.mp4
Press 'k' to see a list of keyboard shortcuts.
Now playing /home/data/tmp/video_00.mp4
nvbuf_utils: Could not get EGL display connection
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvxBaseWorkerFunction[2575] comp OMX.Nvidia.std.iv_renderer.overlay.yuv420 Error -2147479552
NvxBaseWorkerFunction[2575] comp OMX.Nvidia.std.iv_renderer.overlay.yuv420 Error -2147479552
NvxBaseWorkerFunction[2575] comp OMX.Nvidia.std.iv_renderer.overlay.yuv420 Error -2147479552
NvxBaseWorkerFunction[2575] comp OMX.Nvidia.std.iv_renderer.overlay.yuv420 Error -2147479552
NvxBaseWorkerFunction[2575] comp OMX.Nvidia.std.iv_renderer.overlay.yuv420 Error -2147479552
ERROR Internal data stream error. for file:///home/data/tmp/video.mp4
ERROR debug information: qtdemux.c(6073): gst_qtdemux_loop (): /GstPlayBin:playbin/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0:
streaming stopped, reason error (-5)
NvxBaseWorkerFunction[2575] comp OMX.Nvidia.std.iv_renderer.overlay.yuv420 Error -2147479552
Reached end of play list.
8

Yes, possibly this is related to remote connexion. You would first try locally logging with monitor attached.
BTW, what is your Jetson model and L4T release (cat /etc/nv_tegra_release) ?

9

R32 (release), REVISION: 5.0, GCID: 25531747, BOARD: t186ref, EABI: aarch64, DATE: Fri Jan 15 23:21:05 UTC 2021

Btw, I use so lib which depends on:
libgstreamer-1.0.so.0 => /usr/lib/aarch64-linux-gnu/libgstreamer-1.0.so.0 (0x0000007fa3f19000)

For example, my code which set PosFrames = 0
throws error in pipeline (as I stated above).

10

Note that R32.5 with TX2 is quite old now.
You may also tell what gstreamer release you’re working with:

gst-inspect-1.0 --version

That being said, for summarizing your case:

  1. Did you try local logging with monitor attached ? Please provide the error logs for such case, ruling out remote connection issues (gst-play-1.0 may use nvorverlaysink with your L4T release).
  2. Once you can try local case, you may see if using nvv4l2decoder can better work:
gst-launch-1.0 filersc location=<where_ever_it_is>/file.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! autovideosink
  1. As stated before, even with newer hardware and newer opencv, this might not be supported so try to debug it and file an issue to opencv or NVIDIA (but don’t expect so much from the latter as you’re running a deprecated release).
1 Like