Using dmabuf with qml6glsink

Application Development
1

Hello,
I have a Qt 6 application utilizing qml6glsink.
It works perfectly on eglfs. I am using h/w decoder (vah264dec) and works perfectly, while CPU utilization is super low.

When running the same app (it runs on basically Intel NUC like device with ubuntu core) under wayland I have a high CPU usage with some frame drops (on again 4k 30 fps)

If I understand correctly this is due to dmabuf not being used, and having overhead from that.
From what I see in gstreamer 1.26 release notes

Qt5/Qt6 QML GL sinks now support direct DMABuf import from hardware decoders

Am I not doing something to utilize that? gst-inspect on qmlglsink shows that it only accepts ` video/x-raw(memory:GLMemory)

void Player::createVideoBin() {
    VLOG(1) << Q_FUNC_INFO;

    if ((m_videoBin = gst_bin_new("videobin")) == nullptr) {
        LOG(ERROR) << "Unable to create videobin";
        return;
    }

    m_queuePostDecodeBin = gst_element_factory_make("queue", nullptr);
    m_qmlSink = gst_element_factory_make(getQmlSinkName(), nullptr);
    if (m_qmlSink == nullptr) {
        LOG(ERROR) << "Unable to create qmlglsink";
        return;
    }

    g_object_set(G_OBJECT(m_qmlSink), "show-preroll-frame", TRUE, "sync", TRUE, NULL);

    if (m_output) {
        g_object_set(G_OBJECT(m_qmlSink), "widget", m_output, NULL);
        m_needToSetOutputItem = false;
        VLOG(1) << "Set m_qmlSink widget during createVideoBin";
    }

    m_glUpload = gst_element_factory_make("gluploadelement", nullptr);
    gst_bin_add_many(GST_BIN(m_videoBin), m_queuePostDecodeBin, m_glUpload, m_qmlSink, NULL);
    gst_element_link_many(m_queuePostDecodeBin, m_glUpload, m_qmlSink, NULL);

    GstPad *videopad = gst_element_get_static_pad(m_queuePostDecodeBin, "sink");
    gst_element_add_pad(m_videoBin, gst_ghost_pad_new("sink", videopad));
    gst_object_unref(videopad);

    gst_bin_add(GST_BIN(m_pipeline), m_videoBin);
}
2 
Qt5/Qt6 QML GL sinks now support direct DMABuf import from hardware decoders

This means that video/x-raw(memory:GLMemory),texture-target=external-oes are supported as input to qml(6)glsink. Of course this is only supported with OpenGL ES as external-oes textures are only defined there.

As such, for dma-buf to work coherently and efficiently, you need to ensure that the conversion from the dma-buf to a GL texture takes a fast path, either directly into a 2D texture, or an external-oes texture. Anything else involves a user-side copy of the data.

3

is it glupload or gluploadelement?

4

It’s glupload. Just copy pasting the parts of code and failed at editing :slight_smile:

5

Thanks! I rebuilt Qt with -opengl es2 and it immediately became perfect 30 fps with very low CPU usage.

1 Like