Unable to set Custom Metadata in Buffer from Python

Hi,

I’m unsure if this is a GStreamer issue or a GObject issue.

I’m trying to set custom metadata to buffers so that I can read them down in the pipeline in a different element. I’m using the buffer.add_custom_meta + custom_meta.get_structure construct to get and set my metadata into the structure, but when I get + read the structure in another element the data is not there. I suspect the get_structure call returns a copy of the underlying structure because if I get and set the structure using ctypes then the data is correctly set.

Here is a small example of what I’m doing.

Upstream:

custom_meta = buffer.add_custom_meta("custom_meta_name_here")
meta_structure = custom_meta.get_structure()
meta_structure.set_value("value_name", 0)

Downstream:

custom_meta = buffer.get_custom_meta("custom_meta_name_here")
meta_structure = custom_meta.get_structure()
value = meta_structure.get_value("value_name") # or get_int, get_<type>, doesn't matter, the data isn't there

If in the upstream element I call gst_custom_meta_get_structure, and gst_structure_set_value using ctypes, then the downstream element is able to read the data from the struct.

I’m guessing I’m receiving a copy of the structure in python as hash(meta_structure) and the address of the pointer returned by gst_custom_meta_get_structure are different.

Thanks.

2 Likes

Hi, I also tried this approach and I can reproduce this behavior. When writing the metadata I am never able to read them back with the python interface.

This is a BaseTransform implementation to write metadata and a python script with a probe on the fakesink to read the written metadata.

I cannot read the metadata neither from the BaseTransform itself, just after writing them, nor from the probe in a downstream element.

add_custom_meta.py:

import gi
import random

gi.require_version('Gst', '1.0')
gi.require_version('GstBase', '1.0')
gi.require_version('GstVideo', '1.0')

from gi.repository import Gst, GObject, GstBase, GstVideo

import numpy as np

Gst.init(None)
FIXED_CAPS = Gst.Caps.from_string('video/x-raw,format=I420,width=[1,2147483647],height=[1,2147483647]')

class ExampleTransform(GstBase.BaseTransform):
    __gstmetadata__ = ('Example','Transform', 'Example', 'example')
    __gsttemplates__ = (Gst.PadTemplate.new("src", Gst.PadDirection.SRC, Gst.PadPresence.ALWAYS, FIXED_CAPS),
                        Gst.PadTemplate.new("sink", Gst.PadDirection.SINK, Gst.PadPresence.ALWAYS, FIXED_CAPS))

    def do_transform_ip(self, buffer):
        try:
            # Step 1: write metadata
            custom_meta = buffer.add_custom_meta("my_custom_meta")
            meta_structure = custom_meta.get_structure()
            value = random.randint(0, 1000)
            meta_structure.set_value("value_name", value)


            # Step 2: try to read the just written metadata
            custom_meta2 = buffer.get_custom_meta("my_custom_meta")
            meta_structure2 = custom_meta2.get_structure()
            value2 = meta_structure2.get_value("value_name")

            # Structure 2 has a different address, it is a copy. The read value is None
            print(f"add_custom_meta: meta address  {hash(custom_meta)}")
            print(f"add_custom_meta: meta2 address {hash(custom_meta2)}")
            print(f"add_custom_meta: meta struct address  {hash(meta_structure)}")
            print(f"add_custom_meta: meta2 struct address {hash(meta_structure2)}")
            print(f"add_custom_meta: written value: {value} - read value {value2}")

            return Gst.FlowReturn.OK

        except Gst.MapError as e:
            Gst.error("Mapping error: %s" % e)
            return Gst.FlowReturn.ERROR

GObject.type_register(ExampleTransform)
__gstelementfactory__ = ("add_custom_meta", Gst.Rank.NONE, ExampleTransform)

custom_meta_bug_main.py:

import gi

gi.require_version("Gst", "1.0")
gi.require_version("GstApp", "1.0")
gi.require_version("GstVideo", "1.0")
from gi.repository import Gst, GstApp, GObject, GLib, GstVideo

Gst.init(None)

def transform_func(*args):
    print("in transform")
    return True

meta_info = Gst.meta_register_custom("my_custom_meta", ["tag"], transform_func, None)

def read_custom_meta_callback(pad, info):
    buffer = info.get_buffer()

    custom_meta = buffer.get_custom_meta("my_custom_meta")
    meta_structure = custom_meta.get_structure()
    value = meta_structure.get_value("value_name")

    print("--")
    print(f"probe callback: meta address  {hash(custom_meta)}")
    print(f"probe callback: meta struct address  {hash(meta_structure)}")
    print(f"probe callback: read value {value}")

    print("=============================")

    return Gst.PadProbeReturn.OK


def main():
    pipeline = Gst.parse_launch("videotestsrc ! video/x-raw,format=I420,width=1920,height=1080,framerate=30/1 ! add_custom_meta ! fakesink name=meta_reader")

    src_element = pipeline.get_by_name('meta_reader')
    srcpad = src_element.get_static_pad('sink')
    _ = srcpad.add_probe(Gst.PadProbeType.BUFFER, read_custom_meta_callback)

    mainloop = GLib.MainLoop()

    pipeline.set_state(Gst.State.PLAYING)
    mainloop.run()

if __name__ == "__main__":
    main()

And here is a sample of the output for an execution of the main script:

=============================
add_custom_meta: meta address  139861247934640
add_custom_meta: meta2 address 139861247934640
add_custom_meta: meta struct address  139861248072480
add_custom_meta: meta2 struct address 139861248072784
add_custom_meta: written value: 309 - read value None
--
probe callback: meta address  139861247934640
probe callback: meta struct address  139861248072480
probe callback: read value None
=============================
add_custom_meta: meta address  139861247934688
add_custom_meta: meta2 address 139861247934688
add_custom_meta: meta struct address  139861248072480
add_custom_meta: meta2 struct address 139861248072784
add_custom_meta: written value: 940 - read value None
--
probe callback: meta address  139861247934688
probe callback: meta struct address  139861248072480
probe callback: read value None
=============================
add_custom_meta: meta address  139861247934736
add_custom_meta: meta2 address 139861247934736
add_custom_meta: meta struct address  139861248072480
add_custom_meta: meta2 struct address 139861248072784
add_custom_meta: written value: 253 - read value None
--
probe callback: meta address  139861247934736
probe callback: meta struct address  139861248072480
probe callback: read value None
=============================

@ivan94fi thanks for your feedback!

Can someone from the GStreamer team point me in the right direction? As I said I suspect it could be a GObject bug but I’d like to exclude the possibility of a GStreamer bug first, an then posting an issue in the GObject repo.

Thanks.

@spbaecchi I’ve been struggling with the same thing for a while now. You said that when you used ctypes for get_structure and set_value, it worked for you?

Could you please share the code for that if you can?

is there any solution for this, because I’m also trying and nothing work with me
?

Hi,

I am also struggling with the same issue. Does anyone have a solution/recommendation?

There are a couple of issues about this in gitlab, e.g. [REGRESSION] pygobject 3.13 now copies the GstStructure when getting them from a GstCaps, making it impossible to properly modify structures from caps in place. (#76) · Issues · GStreamer / gstreamer · GitLab

This is a regression from pygobject 3.13 (more than 10 years ago) and so far nobody solved this, and it’s also not clear how to solve this. The best solution for this right now would be to not use Python for such things, at least not for the parts that have to modify buffers/buffer lists/events/caps/queries/structures (instead of creating a modified copy) but to use C or Rust instead.