Flutter gtreamer integration

General Discussion
1

Hey, I am trying to write a flutter integration with gstreamer.
The options to render “native” stuff in flutter are

  1. Pass a pixel buffer pointer (from the RAM) to flutter which will then do the magic and create an opengl texture out of it. This is a convenience helper AFAICT.
  2. draw using OpenGL yourself.
  3. There is the new Impller engine which uses vulkan.

I would love to hear what you think would be the best path.
This is what I think (respectively):

  1. totally possible although IDRK how to get a videoframe pixels though this is probably doable.
  2. I think this would be ideal, but only if gstreamer already stores decoded videos on the GPU, otherwise it would be the same as 1. (seems like gtk4 integration does this)
  3. The Impeller is not fully cross-platform yet so I ditched that.
2

After many researches here are my conclusions

  1. pixel buffers are super inefficient (nevertheless easy)
  2. I couldn’t make opengl to work on flutter (also haven’t seen anyone else doing that for anything that needs context sharing with flutter) it should be possible, though I came to conclusion that it would be much better to use platform specific API’s (d3d / metal…).
  3. on Windows the backend uses ANGLE->d3d11 which makes things a bit more complicated and requires some (not idiomatic) hacks in order to avoid GPU race conditions.
    FWIW I haven’t tried using gst-opengl video sink although someone said it should work with ANGLE.