Qt6 QML GL Sink Troubleshooting

[SeeRich]

@ystreet00, hoping I can get some input from you since it seems you are the original author.

First, the warning I am encountering:

qtsink gstqml6glsink.cc:367:gst_qml6_gl_sink_change_state:<sink> error: Could not initialize window system

I am trying to utilize GstGLQt6VideoItem inside of a mixed QML/Qt Widgets application.

I can create a minimal reproducible example if that is requested.

My application begins by creating a QApplication and setting the Qt Quick graphics API to OpenGL (i.e. QQuickWindow::setGraphicsApi(QSGRendererInterface::OpenGL))

I then instantiate a widget which contains a QQuickWidget. Here is the rough logic for the GStreamer + QQuickWidget integration:

VideoWidget::VideoWidget(QWidget* parent)
    : QWidget(parent), _impl(std::make_unique<impl>(this))
{
    setLayout(new QVBoxLayout(this));

    // Construct quick widget
    auto qw = new QQuickWidget(this);
    _impl->_quick_widget = qw;
    qw->setResizeMode(QQuickWidget::SizeRootObjectToView);

    connect(qw, &QQuickWidget::statusChanged, this, &VideoWidget::quickWidgetStatusChanged);
    connect(qw, &QQuickWidget::sceneGraphError, this, &VideoWidget::sceneGraphError);
    qw->resize(300, 300);
    qw->setResizeMode(QQuickWidget::SizeRootObjectToView);

    // Build the GStreamer pipeline
    std::stringstream ss;
    // clang-format off
    auto desktopDir = std::filesystem::path(QStandardPaths::writableLocation(QStandardPaths::DesktopLocation).toStdString());
    auto filesrcPath = desktopDir / "test.mp4";
    ss << fmt::format("filesrc location={} ! qtdemux", filesrcPath.string());
    ss << " ! " << "queue";
    ss << " ! " << "decodebin";
    ss << " ! " << "queue";
    ss << " ! " << "videoconvert";
    ss << " ! " << "queue";
    ss << " ! " << "glupload";
    ss << " ! " << "qml6glsink name=sink";

    LOG_INFO("Pipeline: {}", ss.str());
    // clang-format on
    _impl->_pipeline = gst_parse_launch(ss.str().c_str(), NULL);
    if(!_impl->_pipeline) {
        LOG_ERROR("VideoWidget: failed to parse Gstreamer Pipeline");
        return;
    }
    auto sink = gst_bin_get_by_name(GST_BIN(_impl->_pipeline), "sink");

    qw->setSource(QUrl("qrc:qml/Video.qml"));

    // Find and set the videoItem on the sink
    auto rootObject = qw->rootObject();
    if(rootObject) {
        QQuickItem* videoItem = rootObject->findChild<QQuickItem*>("videoItem");
        g_assert(videoItem);
        g_object_set(sink, "widget", videoItem, NULL);
    }

    QTimer::singleShot(1000, this, [this]() {
        LOG_INFO("VideoWidget: set playing");
        gst_element_set_state(_impl->_pipeline, GST_STATE_PLAYING);
    });

    // Add quick widget to this widgets layout
    layout()->addWidget(qw);
}

When the oneshot timer attempts to set the pipeline state to playing, I receive the warning above and the video fails to play.

[ystreet00]

qtsink gstqml6glsink.cc:367:gst_qml6_gl_sink_change_state:<sink> error: Could not initialize window system

This error means that your environment or your build does not support same configuration between Qt and libgstgl. That being the window systems (X11/Wayland/Cocoa/etc) or GL API (OpenGL or OpenGL ES) or EGL vs GLX vs CGL vs etc. You would need to provide more information about how you configured the build, what OS you are trying to run on, how libgstgl is configured, etc.

[SeeRich]

Sorry for the delay in getting back to you @ystreet00. I finally got around to testing this. I believe the error was due to the QQuickView widget not being properly parented because I was able to fix this issue without changing my GStreamer configuration/build.

However, now I have an additional issue I am facing. When I try to render two videos beside each other (horizontally), one of the videos (always the right-hand side) is continuously flashing green.

Here is an example screenshot:

I created a minimal reproducible example of this issue in hopes that you may be able to spot what I am doing wrong. The code is pretty simple, the complexity in the README mainly comes from having to compile GStreamer on Windows 11 since the prepackaged binaries don’t include the Qt QML plugin.

Example code: GitHub - SeeRich/qmlgstdemo

The example will build two binaries:

1. good.exe: this demonstrates that using two separate widgets doesn’t have any issues.
2. bad.exe: Rendering two GstGLQt6VideoItem in the same QML scene seems to have the problems mentioned above.