Associating additional data with a frame - webrtc

Application Development
3

Hi,

If you decide to use Unregistered User Data SEI messages, the injection has to be done manually with an element or a pad probe after the parser. We are working on adding support in the parser to do it using the attached VideoSEIUserDataUnregisteredMeta of the incoming buffers so that it’s easier to inject SEI messages simply by attaching this meta to a buffer.

Here is a quick example from memory on how to do it, not sure it compiles though :slight_smile:

GstPadProbeReturn example_inject_sei_cb (GstPad* pad, GstPadProbeInfo* info)
{
    GstBuffer* buffer;
    GstBuffer* new_buffer;
    GstH264SEIMessage sei_msg;
    GstH264UserDataUnregistered* udu;
    GstMemory* sei_memory;
    GArray* sei_data;

    buffer = GST_PAD_PROBE_INFO_BUFFER(info);

    memset(&sei_msg, 0, sizeof(GstH264SEIMessage));
    sei_msg.payloadType = GST_H264_SEI_USER_DATA_UNREGISTERED;
    udu = &sei_msg.payload.user_data_unregistered;
    udu->uuid = /* UUID for your custom messages */ 
    udu->data = /* Serialized JSON data or any other format */
    udu->size = /* Size of the data */

    sei_data = g_array_new(FALSE, FALSE, sizeof(sei_msg));
    g_array_append_vals(sei_data, &sei_msg, 1);
    sei_memory = gst_h264_create_sei_memory_avc(4, sei_data);
    g_array_unref(sei_data);

    new_buffer = gst_h264_parser_insert_sei_avc(m_parser, 4, buffer, sei_memory);
    if (new_buffer != NULL)
    {
        info->data = new_buffer;
        info->size = gst_buffer_get_size(new_buffer);
        gst_buffer_unref(buffer);
    }

    return GST_PAD_PROBE_OK;
}
3 Likes