Hi all!
I’m working on adding support of GStreamer plugin format for LSP Plugins. Some of these plugins are extensively using MIDI for input and output. I searched documentation for MIDI support and found only this article.
I need to know the current state of MIDI in GStreamer and how I can deal with it in plugins derived from GstAudioFilter class.
The upstream GStreamer repository can be found here: GStreamer / gstreamer · GitLab fwiw.
Have you already looked at existing plugins such as the fluidsynth plugin or the wildmidi one (plus the GstNonstreamAudioDecoder base class)?
I’ve just looked against fluidsynth plugin.
As I understand, each MIDI event is associated with a GstBuffer. Since each MIDI event should be timestamped, I, being experienced in real-time audio processing, consider it to be an ‘doubtful but OK’ solution because in real time audio we operate on a contiguous buffer and avoid memory allocations when processing data. As I see, one MIDI event which consists of 1 to 3 bytes is packed int GstBuffer, timestamped and then transferred via the GstPad.
But how can I mix multiple GstPads, where the first one is an audio pad, and the second one is a MIDI pad?
The Plugin writers Guide is pretty poor and explains development only of plugins which have single sink, single source and the same number of channels at the input and at the output.
For plugins which come from audio production, this is not true and now I’m thinking on how to adopt to all this stuff.