Gstreamer pull sample takes too long

General Discussion
1

I have made the following code to retrieve the buffer data from a stream:

cv::Mat VideoFrameProcessing::getFrameFromGST()
{
cv::Mat _temp_frame;

int w, h;
int num, den;
GstStructure* structure;
GstMapInfo map; 

current_gst.sink = gst_bin_get_by_name(GST_BIN(current_gst.pipeline), "stream");
if (!current_gst.sink)
    logNeuralAndPrint("ERROR: Sink es nulo");   

current_gst.appsink = GST_APP_SINK(current_gst.sink);
if (!current_gst.appsink)
{
    logNeuralAndPrint("ERROR: Appsink es nulo");
    gst_object_unref(current_gst.sink);
}

current_gst.sample = gst_app_sink_pull_sample(current_gst.appsink);
if (!current_gst.sample)
{
    logNeuralAndPrint("ERROR: Sample es nulo");
    gst_object_unref(current_gst.sink);
}


current_gst.buffer = gst_sample_get_buffer(current_gst.sample);
if (!current_gst.buffer)
{
    logNeuralAndPrint("ERROR: El buffer es nulo");
    gst_sample_unref(current_gst.sample);
    gst_object_unref(current_gst.sink);
}

current_gst.caps = gst_sample_get_caps(current_gst.sample);
structure = gst_caps_get_structure(current_gst.caps, 0);

if(gst_structure_get_fraction(structure, "framerate", &num, &den))
    current_gst.framecount = static_cast<double>(num) / static_cast<double>(den);

memset(&map, 0, sizeof(map));

if (gst_buffer_map(current_gst.buffer, &map, GST_MAP_READ))
{
    gst_structure_get_int(structure, "width", &w);
    gst_structure_get_int(structure, "height", &h);

    _temp_frame = cv::Mat(h, w, CV_8UC3, (void*)map.data, w * 3).clone();
    gst_buffer_unmap(current_gst.buffer, &map);
}
else
    logNeuralAndPrint("ERROR: No se puede mapear el buffer");

gst_sample_unref(current_gst.sample);
gst_object_unref(current_gst.sink);

return _temp_frame;

}

My problem is that the function

current_gst.sample = gst_app_sink_pull_sample(current_gst.appsink);

takes too long to execute (but only the first time is called upon the start of the program). I thought about using gst_app_sink_try_pull_sample(), but then my code fails to continue and finds null references.

Does anyone know how I can reduce the time it takes ? Or how I could implement gst_app_sink_try_pull_sample() without encountering null ref ?

2

gst_app_sink_pull_sample() will wait until a sample is available to be pulled. The try variabt will not and can return NULL if there is no sample available.

The time taken to retrieve a sample is therefore the time it takes to do whatever processing has been configured in the pipeline.

It sounds like there are some pipeline setup costs that you are paying when bringing up the pipeline that would need to be investigated and quantified.

1 Like
3

I see, my current pipeline is the following:

pipeline_str = “rtspsrc location=” + source_video + " ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw,format=BGR ! queue ! appsink name=stream sync=true";

I also tried the try variant, and as you said it started to return NULL values and it was even worst.

Do you think there are changes I could do to the pipeline to solve the appsink issue ?

4

rtspsrc by default has a 2 second latency built in. You can change this using the latency property.

5

I’ve just tried changing the default latency value, but the problem persists in the exact same way.