I’m developing a plugin that does a collection of processes involving sending frame buffers to external process via 0MQ, waiting, then collecting the resultant buffers from 0MQ to send to the source pad. The external process takes on the order of 100s of milliseconds to complete. I’ve baked in a configurable fallback delay such that a buffer is sent to source pad exactly 1s (default delay) after the input buffer was received on the sink. This works fine as part of a single pipeline, however it becomes more tricky when I’m trying to sync it back up with other sources.
For example, with a live source (SRT, or otherwise) tee’d to my plugin and another path then combined back using compositor the output always freezes when the plugin starts outputting. I don’t want to alter the PTS of the buffer, I want to sync it back to the analogous frame that was tee’d off a second earlier. I’ve tried using latency settings to no avail.
The implementation of the plugin is as follows: sink chain first copies the buffer to 0MQ then moves it into a crossthread channel. A separate thread reads from the channel as available and waits until the time the buffer arrived to the sink + the configured delay. At the end of the time, the thread attempts to find the same PTS buffer from the receiving 0MQ or simply copies the original buffer to the output source pad.
Is there some obvious hurdle I’m overlooking or plugin implementation detail I’ve likely skipped, or can this be handled as part of a pipeline change?
