Sink plugin for writing metadata

Hi,
I am currently writing a plugin in Rust that receives (video) metadata and writes it to file. I’ve based it on gst_base::BaseSink and implemented the functions start(), stop() and render(). The intention is to collect the metadata (VideoRegionOfInterest) and write it to file once the pipeline reaches eos and calls stop(). I am able to run the plugin in the pipeline and can verify that the functions above are called. However, I don’t seem to be able to retrieve the metdata from the gst::Buffer even though it has been added in previous plugins. If I use a video filter plugin, the metadata is available. Here are the snippets:

When using the sink plugin, no metadata available:

fn render(&self, buffer: &gst::Buffer) -> Result<gst::FlowSuccess, gst::FlowError> {                                                                                                                   
        event!(Level::INFO, "Render");                                                               
        let mut state = self.state.lock().unwrap();                                                  
        match *state {                                                                               
            State::Stopped => {                                                                      
                event!(Level::ERROR, "State was not started");                                       
                return Err(gst::FlowError::Error);                                                   
            },                                                                                       
            State::Started(ref mut value) => {                                                       
                let roi_metas = buffer.iter_meta::<VideoRegionOfInterestMeta>();                                                                                                                           
                event!(Level::INFO, "Meta count {}", roi_metas.count()); //<- always 0              
...
}

when using this Video Filter plugin, the metadata is available

/// Implementation of gst::VideoFilter virtual methods
impl VideoFilterImpl for AdvPrint {
    fn transform_frame_ip(                                                                           
        &self,                                    
        frame: &mut gst_video::VideoFrameRef<&mut gst::BufferRef>,
    ) -> Result<gst::FlowSuccess, gst::FlowError> {
        let width = frame.width() as usize;       
        let height = frame.height() as usize;     
        let stride = frame.plane_stride()[0];                                                                                                                                                              

        println!(                                 
            "ADVPRINT width: {}, height: {} stride: {}",
            width, height, stride
        );                                        
        let roi_metas = frame.buffer().iter_meta::<VideoRegionOfInterestMeta>(); // metadata available

What could be the issue? Is there a better approach to accomplish what I am trying to do? Thanks for any input.

PS: I cannot provide an running example at the moment.

That would be useful to have. Otherwise you’ll have to trace through the code yourself to see where the meta gets lost and why.

Theoretically what you describe should work just fine, especially if a filter at the same place as the sink does receive the metas.

Hi slomo,

I did find a solution to this by trial and error. After setting the caps in advtestresult to

            let caps = gst_video::VideoCapsBuilder::new()
                .format(gst_video::VideoFormat::Rgbx)
                .features(["memory:CUDAMemory"])

the metadata was reaching the plugin as expected. These caps are not set for the videofilter plugin (which does receive the metadata). But for a sink there might be different rules, which I don’t know of.

gst-launch-1.0 
advinput location=head_pose/data_one.json ! 
decodebin ! 
videoconvert ! 
cudaupload  ! 
'video/x-raw(memory:CUDAMemory), format=RGBx' ! 
advmetadata location=data_one.json ! 
advtrtagegender ! 
advtrtheadpose ! 
cudadownload ! 
advtestresult location=result_one.json

If you happen to know good gstreamer material to read, please let me know. Thanks.