Shared context failing to fill info

Application Development
3

Thanks, this sample did help somewhat and I got past that issue.

I think I’m missing something else vital here though. I’m trying to run this in Unity which is straying from being GStreamer related but I’ve gotten my current context with

EGLDisplay g_unityDisplay = eglGetCurrentDisplay();

I then create a GstGLDisplayEGL which I need to share it so…

GstGLDisplayEGL *display = gst_gl_display_egl_new_with_egl_display(g_unityDisplay);

This one line here is a problem. If my pipeline contains a gl element like glcolorconvert or gldownload it now hard crashes my app on pipeline initialization. The logs end on

[GStreamer] No value transform to serialize field ‘gl-allocation-params’ of type ‘GstGLAllocationParams’[GStreamer] Expected field ‘gl-min-free-queue-size’ in structure: GstBufferPoolConfig, caps=(GstCaps)“video/x-raw(memory:GLMemory),\ format=(string)RGBA,\ width=(int)320,\ height=(int)240,\ interlace-mode=(string)progressive,\ multiview-mode=(string)mono,\ multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,\ pixel-aspect-ratio=(fraction)1/1,\ framerate=(fraction)30/1,\ texture-target=(string)2D,\ colorimetry=(string)2:1:16:4”, size=(uint)0, min-buffers=(uint)0, max-buffers=(uint)0, allocator=(GstAllocator)“NULL”, params=(GstAllocationParams)“GstAllocationParams,\ flags=(GstMemoryFlags)0,\ align=(guint64)0,\ prefix=(guint64)0,\ padding=(guint64)0;”, options=(string)< GstBufferPoolOptionVideoMeta >, gl-allocation-params=(GstGLAllocationParams)NULL;

But I don’t know what this is telling me? To be clear, the ONLY thing I’ve done is create the display. You can think of the function being

void init() {
    EGLDisplay g_unityDisplay = eglGetCurrentDisplay();
    GstGLDisplayEGL *display = gst_gl_display_egl_new_with_egl_display(g_unityDisplay);
    return;
}

That is enough to hard crash with the log above.

Moving the call around changes behavior but none of it is correct. Take this for example

  GstGLDisplayEGL *display = gst_gl_display_egl_new();

  glGenTextures(1, &instance->oesTextureId);
  glBindTexture(GL_TEXTURE_EXTERNAL_OES, instance->oesTextureId);
  glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
  glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
  glBindTexture(GL_TEXTURE_EXTERNAL_OES, 0);

  glGenTextures(1, &instance->textureId);
  glBindTexture(GL_TEXTURE_2D, instance->textureId);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
  glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, instance->width, instance->height, 
               0, GL_RGBA, GL_UNSIGNED_BYTE, nullptr);

This usually prints something like OESTextureId is 167 and textureId is 0.

Commenting out the display prints that OESTextureId is 167 and textureId is 168. I would think this means something inside gst-plugins-base/gst-libs/gst/gl/egl/gstgldisplay_egl.c at master · GStreamer/gst-plugins-base · GitHub is altering the active context in some way but nothing jumps out at me. I’m not sure where to go here.

For a full overview I’m trying to zero copy a GPU texture in Unity on a Quest 3. The path I’m going down I think should be

  • Share the Unity context with GStreamer so textures can be shared
  • Decode with the hardware decoder and push frames to an appsink
  • Do some GPU copy with a method I haven’t discovered yet to copy from the texture ID of the GstGLMemory sample into the Unity created texture which should be valid as they’re shared

The only other path I’ve found that can do this on Android was using glimagesink push to ANativeWindow backed by a SurfaceTexture and calling UpdateTexImage every frame but this resulted in A LOT of lost frames and horribly choppy movement. I don’t think there’s a good way to solve that which is why I’ve moved to trying the appsink path instead.

Horribly confused about caps