Shared context failing to fill info

Application Development
1

I’m trying to make gstreamer use the opengl context created by Unity and can’t get it to work.

https://gstreamer.freedesktop.org/documentation/additional/design/opengl.html?gi-language=c provides an example to share a context with gstreamer so I’ve created:

  UNITY_INTERFACE_EXPORT void UNITY_INTERFACE_API UnityPluginLoad(IUnityInterfaces* unityInterfacesPtr) {
    g_unityContext = eglGetCurrentContext();
    g_unityDisplay = eglGetCurrentDisplay();

    ...
  }

And a bus call

static gboolean
sync_bus_call(GstBus *bus, GstMessage *msg, gpointer data)
{
  switch (GST_MESSAGE_TYPE(msg)) {
    case GST_MESSAGE_NEED_CONTEXT: {
      const gchar *context_type;
      GstContext *context = NULL;

      gst_message_parse_context_type(msg, &context_type);
      Logger::logFormat("got need context %s\n", context_type);

      if (g_strcmp0(context_type, "gst.gl.app_context") == 0) {
        context = gst_context_new("gst.gl.app_context", TRUE);
        GstStructure *s = gst_context_writable_structure(context);
        gst_structure_set(s, "context", GST_TYPE_GL_CONTEXT, g_gstContext, NULL);

        // Assign to the element that requested it
        gst_element_set_context(GST_ELEMENT(msg->src), context);

        gst_context_unref(context);
      }
      break;
      }
    default:
      break;
    }

    return FALSE; // let other messages pass
}

This fails and gives

wrapped context could not retrieve config. The application may be missing a call to gst_gl_context_fill_info() or the specific platform implemention is not implemented for retrieving the config from a wrapped OpenGL context.

So I need to call gst_gl_context_fill_infosure I can do that…

void InitGStreamerInstance(GStreamerInstance *instance) {
  ...

  if(!g_gstContext) {
    g_gstDisplayEGL = gst_gl_display_egl_new_with_egl_display(g_unityDisplay);
    g_gstDisplay = GST_GL_DISPLAY(g_gstDisplayEGL);

    g_gstContext = gst_gl_context_new_wrapped(
        g_gstDisplay,
        (guintptr)g_unityContext,
        GST_GL_PLATFORM_EGL,
        GST_GL_API_GLES2
    );

    GError *error = nullptr;

    Logger::log("Filling info...");
    if (!gst_gl_context_fill_info(g_gstContext, &error)) {
        Logger::logFormat("gst_gl_context_fill_info failed: %s", error ? error->message : "unknown error");
        if (error) g_error_free(error);
    }
  }
}

Which gives

Filling info…
gst_gl_context_fill_info failed: unknown error

Which couldn’t be any more helpful. Clearly I’m doing something wrong here. What?

2

gst_gl_context_fill_info must be called with the relevant GL context active in the current thread. GStreamer will not activate or deactivate the GL context for you and you must ensure this yourself. You must also notify GStreamer that you have activated the GL context manually yourself by calling gst_gl_context_activate(g_gstContext, TRUE); before calling gst_gl_context_fill_info. If there was another GL context active already, then it is very likely that you must return the thread to the previous state once you are done.

The ‘unknown error’ you are receiving would have also resulted in a critical that included something like: context->priv->active_thread == g_thread_self () indicating that you have not activated the wrapped GL context correctly.

The Qt plugins are the most complete example for this whole process and can be seen here: subprojects/gst-plugins-good/ext/qt/gstqtglutility.cc · main · GStreamer / gstreamer · GitLab

Depending on what platform you are developing for, the warning:

wrapped context could not retrieve config. The application may be missing a call to gst_gl_context_fill_info() or the specific platform implemention is not implemented for retrieving the config from a wrapped OpenGL context.

May be expected as GStreamer may not have the relevant implementation for that platform. Currently only GLX and EGL have supported implementations to avoid that warning.

3

Thanks, this sample did help somewhat and I got past that issue.

I think I’m missing something else vital here though. I’m trying to run this in Unity which is straying from being GStreamer related but I’ve gotten my current context with

EGLDisplay g_unityDisplay = eglGetCurrentDisplay();

I then create a GstGLDisplayEGL which I need to share it so…

GstGLDisplayEGL *display = gst_gl_display_egl_new_with_egl_display(g_unityDisplay);

This one line here is a problem. If my pipeline contains a gl element like glcolorconvert or gldownload it now hard crashes my app on pipeline initialization. The logs end on

[GStreamer] No value transform to serialize field ‘gl-allocation-params’ of type ‘GstGLAllocationParams’[GStreamer] Expected field ‘gl-min-free-queue-size’ in structure: GstBufferPoolConfig, caps=(GstCaps)“video/x-raw(memory:GLMemory),\ format=(string)RGBA,\ width=(int)320,\ height=(int)240,\ interlace-mode=(string)progressive,\ multiview-mode=(string)mono,\ multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,\ pixel-aspect-ratio=(fraction)1/1,\ framerate=(fraction)30/1,\ texture-target=(string)2D,\ colorimetry=(string)2:1:16:4”, size=(uint)0, min-buffers=(uint)0, max-buffers=(uint)0, allocator=(GstAllocator)“NULL”, params=(GstAllocationParams)“GstAllocationParams,\ flags=(GstMemoryFlags)0,\ align=(guint64)0,\ prefix=(guint64)0,\ padding=(guint64)0;”, options=(string)< GstBufferPoolOptionVideoMeta >, gl-allocation-params=(GstGLAllocationParams)NULL;

But I don’t know what this is telling me? To be clear, the ONLY thing I’ve done is create the display. You can think of the function being

void init() {
    EGLDisplay g_unityDisplay = eglGetCurrentDisplay();
    GstGLDisplayEGL *display = gst_gl_display_egl_new_with_egl_display(g_unityDisplay);
    return;
}

That is enough to hard crash with the log above.

Moving the call around changes behavior but none of it is correct. Take this for example

  GstGLDisplayEGL *display = gst_gl_display_egl_new();

  glGenTextures(1, &instance->oesTextureId);
  glBindTexture(GL_TEXTURE_EXTERNAL_OES, instance->oesTextureId);
  glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
  glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
  glBindTexture(GL_TEXTURE_EXTERNAL_OES, 0);

  glGenTextures(1, &instance->textureId);
  glBindTexture(GL_TEXTURE_2D, instance->textureId);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
  glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, instance->width, instance->height, 
               0, GL_RGBA, GL_UNSIGNED_BYTE, nullptr);

This usually prints something like OESTextureId is 167 and textureId is 0.

Commenting out the display prints that OESTextureId is 167 and textureId is 168. I would think this means something inside gst-plugins-base/gst-libs/gst/gl/egl/gstgldisplay_egl.c at master · GStreamer/gst-plugins-base · GitHub is altering the active context in some way but nothing jumps out at me. I’m not sure where to go here.

For a full overview I’m trying to zero copy a GPU texture in Unity on a Quest 3. The path I’m going down I think should be

  • Share the Unity context with GStreamer so textures can be shared
  • Decode with the hardware decoder and push frames to an appsink
  • Do some GPU copy with a method I haven’t discovered yet to copy from the texture ID of the GstGLMemory sample into the Unity created texture which should be valid as they’re shared

The only other path I’ve found that can do this on Android was using glimagesink push to ANativeWindow backed by a SurfaceTexture and calling UpdateTexImage every frame but this resulted in A LOT of lost frames and horribly choppy movement. I don’t think there’s a good way to solve that which is why I’ve moved to trying the appsink path instead.

Horribly confused about caps
4

I’ve been experimenting more

void GLInit() {
  g_unityDisplay = eglGetCurrentDisplay();
  g_unityContext = eglGetCurrentContext();

  Logger::logFormat("Unity context %p", g_unityContext);
  Logger::logFormat("Unity display %p", g_unityDisplay);

  Logger::log("Before display");
  GstGLDisplayEGL *display = gst_gl_display_egl_new_with_egl_display(g_unityDisplay);

  Logger::log("Before shared context");
  sharedContext = gst_gl_context_new_wrapped(
    GST_GL_DISPLAY_CAST(display),
    (guintptr)g_unityContext,
    GST_GL_PLATFORM_EGL,
    GST_GL_API_GLES2
  );

  Logger::log("Before activate");
  gst_gl_context_activate(sharedContext, TRUE);
  GError *error;
  Logger::log("After activate");
  if (!gst_gl_context_fill_info (sharedContext, &error)) {
    Logger::log("Failed to retrieve context info");
    gst_gl_context_activate (sharedContext, FALSE);
    return;
  }

  Logger::log("Before false activate");
  gst_gl_context_activate (sharedContext, FALSE);

  Logger::log("GL has been init successfully.");

  initialized = true;
}

This prints

Unity context 0xb400007c3074a980
Unity display 0xb400007c5ed298c0
Before display
Before shared context
Before activate
After activate
[GStreamer] GL_VERSION: OpenGL ES 3.2 V@0819.0.2 (GIT@651341e91d, I8d46b3bbb7, 1754290135) (Date:08/04/25)
[GStreamer] GL_SHADING_LANGUAGE_VERSION: OpenGL ES GLSL ES 3.20
[GStreamer] GL_VENDOR: Qualcomm
[GStreamer] GL_RENDERER: Adreno (TM) 740
[GStreamer] egl config not available. ID is 0
Before false activate
GL has been init successfully.

I’m somewhat concerned about egl config not available. ID is 0

Later on I still get

[GStreamer] wrapped context could not retrieve config. The application may be missing a call to gst_gl_context_fill_info() or the specific platform implemention is not implemented for retrieving the config from a wrapped OpenGL context.

Which I don’t think is correct and it looks like GStreamer still makes its own context ignoring mine

[GStreamer] Bound OpenGL|ES
[GStreamer] config set: 12970367456180244512, 1
[GStreamer] gl context created: 12970367452537140864
[GStreamer] created context
[GStreamer] available GL APIs: gles2
[GStreamer] GL_VERSION: OpenGL ES 3.2 V@0819.0.2 (GIT@651341e91d, I8d46b3bbb7, 1754290135) (Date:08/04/25)
[GStreamer] GL_SHADING_LANGUAGE_VERSION: OpenGL ES GLSL ES 3.20
[GStreamer] GL_VENDOR: Qualcomm
[GStreamer] GL_RENDERER: Adreno (TM) 740
[GStreamer] Enabling GL context debugging
[GStreamer] gl thread running
[GStreamer] gl thread created

Why configID = 0 when call eglQueryContext after Unity 2018.3(include) - Unity Engine - Unity Discussions makes it sound like eglQueryContext can intentionally return 0 for a config and the path down gst_gl_context_egl_fill_info returns true in this case

5

Never mind it IS shared. Some key takeaways from this.

[GStreamer] wrapped context could not retrieve config. The application may be missing a call to gst_gl_context_fill_info() or the specific platform implemention is not implemented for retrieving the config from a wrapped OpenGL context.

This is normal for Unity. Newer versions won’t return a config and this is fine.

[GStreamer] Bound OpenGL|ES
[GStreamer] config set: 12970367456180244512, 1
[GStreamer] gl context created: 12970367452537140864
[GStreamer] created context
[GStreamer] available GL APIs: gles2
[GStreamer] GL_VERSION: OpenGL ES 3.2 V@0819.0.2 (GIT@651341e91d, I8d46b3bbb7, 1754290135) (Date:08/04/25)
[GStreamer] GL_SHADING_LANGUAGE_VERSION: OpenGL ES GLSL ES 3.20
[GStreamer] GL_VENDOR: Qualcomm
[GStreamer] GL_RENDERER: Adreno (TM) 740
[GStreamer] Enabling GL context debugging
[GStreamer] gl thread running
[GStreamer] gl thread created

Yes it might look like it’s making a new config and doing who knows what but this is normal too.

To share a config with Unity:

Add https://gstreamer.freedesktop.org/documentation/additional/design/opengl.html?gi-language=c#sharing-opengl-context-with-gstreamer to your code

Use a function similar to this to setup the required GL

void GLInit() {
  g_unityDisplay = eglGetCurrentDisplay();
  g_unityContext = eglGetCurrentContext();

  Logger::log("Before display");
  GstGLDisplayEGL *display = gst_gl_display_egl_new_with_egl_display(g_unityDisplay);

  sharedContext = gst_gl_context_new_wrapped(
    GST_GL_DISPLAY_CAST(display),
    (guintptr)g_unityContext,
    GST_GL_PLATFORM_EGL,
    GST_GL_API_GLES2
  );

  gst_gl_context_activate(sharedContext, TRUE);
  GError *error;
  if (!gst_gl_context_fill_info (sharedContext, &error)) {
    gst_gl_context_activate (sharedContext, FALSE);
    return;
  }

  gst_gl_context_activate (sharedContext, FALSE);
}

Attach your sync bus call to your pipeline with

gst_bus_set_sync_handler(bus, (GstBusSyncHandler)sync_bus_call, NULL, NULL);

And you should be good to go.