Onnxinference plugin problem

Application Development
1

Hello! I am trying to use the onnxinference plugin. I managed to build the plugin but when i use the sample pipeline from the gstreamer documentation :

GST_DEBUG=ssdobjectdetector:5
gst-launch-1.0 filesrc location=onnx-models/images/bus.jpg !
jpegdec ! videoconvert ! onnxinference execution-provider=cpu model-file=onnx-models/models/ssd_mobilenet_v1_coco.onnx !
ssdobjectdetector label-file=onnx-models/labels/COCO_classes.txt ! videoconvert ! imagefreeze ! autovideosink

It runs, but i only get the image, without the bounding boxes on the detected object. With debug enabled i saw a warning that might be of use but i dont know how to solve it:
0:00:05.638282612 3839 0x6168776cc000 WARN ssdobjectdetector gstssdobjectdetector.c:559:gst_ssd_object_detector_process: missing tensor meta

Can someone help me solve this issue?
I mention that i have tried the modify_onnx_metadata.py script from the gitlab.collabora.com repository and still the same outcome: the image without any object detection bounding boxes

1 Like
2

Hi, to visualize the bounding boxes you also need to have objectdetectionoverlay after the tensordecoder (in your case ssdobjectdetector).

You can see a pipeline using this overlay here:

But you have another problem because the log report missing tensor meta. Most likely an issue with the tensor group-id and tensor-id.

Which Gstreamer version are you using?

Or are you building from main?

Can you share the JSON file you used with the script?

Try with this file that already has the group-id and tensor-id embedded:

There’s also this repository where tensor-id come from :

3

Hello and thank you for your answer!

  1. I am building from main and my GStreamer version is 1.26.9;
  2. The model i’m trying to use is from that gitlab repo you’ve said;
  3. This is my json file with the model’s metadata extracted with the help of that script (modify_onnx_metadata.py)
{

"group-id": "ssd-mobilenet-v1-variant-1-out",

"detection_boxes:0": "ssd-mobilenet-v1-variant-1-out-boxes",

"detection_classes:0": "ssd-mobilenet-v1-variant-1-out-classes",

"detection_scores:0": "ssd-mobilenet-v1-variant-1-out-scores",

"num_detections:0": "generic-variant-1-out-count"

}
4

Use GST_DEBUG=onnxinference:6 run the pipeline and share the kog

Also can you share gst-inspect-1.0 ssdobjectdetector

5

Hello, I am having the same issue, but on 1.24. I have verified that the onnxinference is not giving an error, I see the “Num tensors: 4” trace print so that inference looks ok. I was also seeking some more info about the script and the output names, because in the header file on gstssdobjectdetection, you need to make outputs match with group-id, but but it’s not obvious what string or part to use there.

Best regards

6

So by trying some stuff out, I found that you need to use this json file with the “add” of the python script:

{
    "group-id": "ssd-object-detector",
    "detection_boxes:0": "Gst.Model.ObjectDetector.Boxes",
    "detection_classes:0": "Gst.Model.ObjectDetector.Classes",
    "detection_scores:0": "Gst.Model.ObjectDetector.Scores",
    "num_detections:0": "Gst.Model.ObjectDetector.NumDetections"
}

save it to for example “test.json” and then you can run:

python3 onnx-models/scripts/modify_onnx_metadata.py onnx-models/models/ssd_mobilenet_v1_coco.onnx add test.json

Then it will work and you should see log output of the detections. For the visualisation, you need to add the objectdetectionoverlay as suggested by dmorin.

Enjoy!