Problem with getting samples from pipeline on demand

Application Development
1

Hello,

I would like to receive samples from pipeline on demand using frame-stepping but having some problem with it.

The pipeline:

"filesrc ! qtdemux ! h264parse ! decodebin ! gldownload ! videoconvert ! appsink name=sink

I initialize it and set up to paused state. I also subscribe on “new-samples” events at appsink.

    appsink = gst_bin_get_by_name(GST_BIN(pipeline), "sink");
    g_signal_connect(appsink, "new-sample", G_CALLBACK(on_new_sample), NULL);
    g_object_set(appsink, "emit-signals", TRUE, "sync", FALSE, NULL);

    gst_element_set_state(pipeline, GST_STATE_PAUSED);

Then I am sending “new step” event:

    
    gst_element_send_event(
        appsink,
        gst_event_new_step(
             GST_FORMAT_DEFAULT,
             1,
             ABS(1.0),
             TRUE,
             FALSE
         )
     );

and waiting for “new-sample” to be emmited and processed by “on_new_sample” callback. But this doesn’t happen.
What is the correct way of managing this?

2

I know I’m not really answering your question as to why it doesn’t work, but: if you want to just get one frame out at a time, it’s easier not to use the new-sample signal and instead just use gst_app_sink_pull_sample() instead whenever you want the next frame.

3

Thank you, but i think this function doesn’t do what I mean. This function works on playing pipeline. I want to obtain frames in the row, right from the first one, so I thought after initialization I need to set up pipeline to paused state and then on demand I trigger it using “gst_event_new_ste” to do one step and get the next frame.

4

There’s also pull_preroll() to pull a preroll sample in PAUSED state.

Why does it matter for you if the pipeline is PAUSED or PLAYING?

Are there other elements/sinks in the full pipeline?

5

My pipeline is the following:
filesrc ! qtdemux ! h264parse ! decodebin ! gldownload ! videoconvert ! appsink name=sink

I want to have a function “nextFrame()” that gets the next consecutive frame from the file on demand. So when I first call “nextFrame()” I get the first frame of the file, when I call it second time I get the second frame and etc. There is no particular periodicity for calling this function, so between consecutive calls of “nextFrame()” there is undefined time gap.

If the pipeline is in the PLAYING state, as I understand, there will be a flow of frames continuously going on. And by calling “gst_app_sink_pull_sample()” I will only get the frame that is currently passing though the pipeline, not the consecutive of the previously called “nextFrame()” result.

6

That’s not how it works in this case. Data flow is “blocking” by default, so if a sink doesn’t consume or process the data, the upstream elements are blocked from processing more data as well.

However, there’s a queue inside appsink, which is unlimited by default and which you want to set to max-buffers=1 (or some small number, the limit doesn’t really matter, basically the amount of frames to pre-cache). That will ensure that upstream (filesrc/demuxer/decoders) will get blocked once the appsink-internal queue runs full.

If you don’t pull out data it will run full and block. When you make space by pulling out a frame it will unblock and allow upstream to decode another frame and then block again.

In PLAYING state a clock is running and time advances conceptually, but here the appsink is not looking at the clock or timestamps anyway because you set it to sync=false (which makes sense for your use case).

7

Yeah, I misunderstood the way pipeline works. Thank you! Now I am able to get the frames on demand.