How should you determine which language to write a GStreamer application in?

What factors should go into deciding what language to write a new GStreamer application in? Tim Müller suggested at the GStreamer conference that if you don’t have a preference of which language to use, “use Rust instead of C perhaps.”

Obviously, the competency and understanding of those working on/maintaining the application can affect preference, but are there any GStreamer-specific drawbacks in each language? I feel like given the fact that GStreamer is written in C that C (or even C++) are good options. Given Tim’s comments, clearly Rust is a good option. But are there any drawbacks to writing a GStreamer in something like Python or Java?

Are there any features of GStreamer that are available in some languages but not others? Is there a risk of lack of future support of any one language over another? Are there other considerations to make?

Lots of things to consider of course - what kind of timeline you’re looking at, what language other components are that you need to use or integrate with, what you and your team are familiar with, etc.

The recommendation to use Rust was mostly meant in lieu of C/C++, for new developments where that’s an option and where there isn’t a legacy C/C++ code base to integrate with. The reason we often recommend Rust here for new developments is that that’s the direction in which GStreamer and the community itself are moving, but also because it avoids many of the usual footguns you have in C/C++ both with GObject and multi-threaded programming. That’s not to say that there isn’t a learning curve, but in our experience so far people have found it to be worth it in the end.

Of course the general idea is still that people should be able to write GStreamer applications in any programming language they like and for which bindings exist.

In practice some are better-maintained and more widely-used than others however, and another question is also how easy is it to find support for problems you have with some code in a particular language.

That’s why for compiled languages the recommendation is usually Rust, or C/C++ on top of the GStreamer C API.

For scripting language python is most widely used, but it has some limitations or at least warts when it comes to lower level things like buffer/memory management, and it’s also suboptimal when hooking into streaming threads (e.g. via pad probes) because of the GIL. There’s a huge community and many examples around the python bindings though, so it should be fairly easy to get support for it…

That’s not to say other bindings are bad in any way, it’s just that they will vary in how up to date they are, how well maintained, and how popular.

2 Likes

We chose to use Python because we were already very familiar with it, almost everything else in our code base is Python, and it’s a nice language to work in. Our use case is quite an extensive live video mixing application handling 4K video at 25fps, and we haven’t hit any insurmountable performance issues. tpm definitely has a point here, though:

it’s also suboptimal when hooking into streaming threads (e.g. via pad probes) because of the GIL

We had to ensure we don’t take too much of a performance hit there. But other than that, Python just glues everything together and the heavy lifting is done by GStreamer in native code. If you are fluent in Python, I would definitely recommend starting off with it, and see what (if any) performance issues you run into.