How to implement a sparse subtitle stream so mpegtsmux doesn't wait around trying to sync

General Discussion
1

I have live video/audio, and occasionally there might be a subtitle to go along with it.

The documentation says that sparse streams are useful for sparse streams like subtitles, but the following test case never gets started nor keeps going unless you regularly push buffers, which may not exist at all. I’ve also tried to push a gap event every time the videotestsrc outputs a buffer unless the subtitle pts+duration is already higher but I’m not quite managing to make that work properly. Also if I understand correctly that isn’t supposed to be for live sources in the first place. I feel like there must be something I’m overlooking, probably related to GstAggregator.

#include <gst/gst.h>
#include <gst/app/app.h>

// this is how it seems to be done in subprojects/gst-plugins-good/tests/check/elements/splitmuxsrc.c
GstPadProbeReturn app_src_stream_start_probe (GstPad *pad, GstPadProbeInfo *info, gpointer user_data) {
    GstEvent *event = gst_pad_probe_info_get_event(info);

    if (event->type != GST_EVENT_STREAM_START) {
        return GST_PAD_PROBE_OK;
    }

    GstStreamFlags flags = 0;

    gst_event_parse_stream_flags(event, &flags);

    if ((flags & GST_STREAM_FLAG_SPARSE) == 0) {
        if (!gst_event_is_writable(event)) {
            event = gst_event_make_writable(event);

            if (!event) {
                return GST_PAD_PROBE_OK;
            }
        }

        gst_event_set_stream_flags(event, (flags | GST_STREAM_FLAG_SPARSE));
        GST_PAD_PROBE_INFO_DATA(info) = event;

        gst_pad_remove_probe(pad, GST_PAD_PROBE_INFO_ID(info));
    }

    return GST_PAD_PROBE_REMOVE;
}

int main(int argc, char *argv[]) {
   GError *error = NULL;
   GstElement *pipeline, *appsrc;
   GstPad *srcpad;

   gst_init (&argc, &argv);

   pipeline = gst_parse_launch ("videotestsrc is-live=true ! video/x-raw,width=1920,height=1080,framerate=25/1 ! x264enc bitrate=500 ! h264parse ! queue ! mpegtsmux emit-signals=true name=mux alignment=7 ! filesink location=test.ts     appsrc name=appsrc is-live=true min-latency=0 max-latency=2000000000 caps=subpicture/x-dvb ! queue ! mux.", &error);

   /* Check for errors */
   if (error != NULL) {
       g_printerr ("Error creating pipeline: %s\n", error->message);
       return -1;
   }

   appsrc = gst_bin_get_by_name (GST_BIN (pipeline), "appsrc");
   srcpad = gst_element_get_static_pad (appsrc, "src");
   guint id = gst_pad_add_probe (srcpad, GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM, app_src_stream_start_probe, NULL, NULL);

   gst_object_unref (srcpad);
   gst_object_unref (appsrc);

   gst_element_set_state (pipeline, GST_STATE_PLAYING);

   /* Wait until error or EOS */
   GstBus* bus = gst_element_get_bus (pipeline);
   GstMessage *msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);

   /* Free resources */
   if (msg != NULL)
       gst_message_unref (msg);
   gst_object_unref (bus);
   gst_element_set_state (pipeline, GST_STATE_NULL);
   gst_object_unref (pipeline);

   return 0;
}

Edit: I tested in 1.22.5 and current main branch 1.23.0 development.

2

Solved with gap events after all, thanks to the experts at Centricular. :slight_smile:

1 Like