Handling glimagesink.client-draw on android

Application Development Embedded
1

Hi,

would it make sense to handle the client-draw signal of glimagesink on android?
If so, how would such a handler look like in the most simple case? That is, if the handler should simply draw the sample passed to the handler?

I’ve already experimented a bit: Added a client-draw handler to android tutorial-5. It is actually called and a valid sample is passed to it. Now, the handler seems to be responsible for drawing the sample, but how to do that?

I’ve found an example here, which handles client-draw, but I do not know if it is expected to run on android: doublecube

I’ve added the code of drawCallback from this example to my experimental client-draw handler, added “gstreamer-gl-1.0 gstreamer-gl-prototypes-1.0” to GSTREAMER_EXTRA_DEPS in Android.mk for that the required headers are found, but the linker isn’t really happy with that:

In function drawCallback': D:/Dev/Android/gstreamer/subprojects/gst-docs/examples/tutorials/android/android-tutorial-5/jni/tutorial-5.c:843: undefined reference to glEnable’
D:/Dev/Android/gstreamer/subprojects/gst-docs/examples/tutorials/android/android-tutorial-5/jni/tutorial-5.c:844: undefined reference to glBindTexture' D:/Dev/Android/gstreamer/subprojects/gst-docs/examples/tutorials/android/android-tutorial-5/jni/tutorial-5.c:845: undefined reference to glTexParameteri’
D:/Dev/Android/gstreamer/subprojects/gst-docs/examples/tutorials/android/android-tutorial-5/jni/tutorial-5.c:846: undefined reference to glTexParameteri' D:/Dev/Android/gstreamer/subprojects/gst-docs/examples/tutorials/android/android-tutorial-5/jni/tutorial-5.c:847: undefined reference to glTexParameteri’
D:/Dev/Android/gstreamer/subprojects/gst-docs/examples/tutorials/android/android-tutorial-5/jni/tutorial-5.c:848: undefined reference to glTexParameteri' D:/Dev/Android/gstreamer/subprojects/gst-docs/examples/tutorials/android/android-tutorial-5/jni/tutorial-5.c:849: undefined reference to glTexEnvi’

And so on. So I assume that a library is missing.

2

Adding the following to LOCAL_LDLIBS in Android.mk helped with the linker problems:

-lGLESv3

The sample doublecube was probably not made for OpenGL ES, so some functions are not available.

In the meantime, I’ve managed to clear the background using glClearColor. I change the red value every time my client-draw handler is called - that seems to work.

Now, I have to figure out how to draw the sample.

3

You can either use the -lGLESv2 approach to have some gl functions directly available and linkable, or you can access a list of GL functions from the GstGLContext.

That example is built for legacy desktop OpenGL which is not in any way compatible with OpenGL ES used on Android. You need to write your own texture drawing pass inside client-draw. An example of this using more modern GL APIs (also compatible with OpenGL ES) is available from: subprojects/gst-plugins-base/tests/examples/gl/sdl/sdlshare.c · main · GStreamer / gstreamer · GitLab

4

@ystreet00
Thanks for your comment - it was very helpful.
I’ve merged the code from the example you gave into my application. I now have a spinning triangle filled with colors (red at the upper edge, blue at the lower left and green at the lower-right edge), but the video output is still missing.

From what I get, there seems to be a target mismatch:

gl->BindTexture(GL_TEXTURE_2D, texture);

causes an GL_INVALID_OPERATION error.

Corresponding Logcat entry:

…/gst-libs/gst/gl/gstgldebug.c:307:_gst_gl_debug_callback: high: GL error from API id:4, GL_INVALID_OPERATION in glBindTexture(target mismatch)

GL documentation states:

GL_INVALID_OPERATION is generated if texture was previously created with a target that doesn’t match that of target.

So I checked the target of the texture:

GstBuffer *buf = gst_sample_get_buffer (sample);
GstMemory *mem = gst_buffer_peek_memory(buf, 0);
if (gst_is_gl_memory(mem)) {
GstGLMemory *glMem = GST_GL_MEMORY_CAST(mem);
GstGLTextureTarget target = gst_gl_memory_get_texture_target(glMem);
guint targetGL = gst_gl_texture_target_to_gl(target);
gchar *targetString = gst_gl_texture_target_to_string(target);
GST_INFO(“targetGL: %d, targetString: %s”,targetGL, targetString);
}

That gave me:

drawCallback targetGL: 36197, targetString: external-oes

So, my first approach was to change the target passed to gl->BindTexture:

gl->BindTexture(GL_TEXTURE_EXTERNAL_OES, texture);

That way, I got rid of the GL_INVALID_OPERATION error, but I still do not see any video.

Maybe the shader used in the example is not compatible with GL_TEXTURE_EXTERNAL_OES?!

5

Ok, I’ve modified the fragment shader, and now it works!

static const gchar *texture_frag = "\

#extension GL_OES_EGL_image_external : enable\n
#ifdef GL_ES\n
precision mediump float;\n
#endif\n
varying vec2 v_texcoord;\n
uniform samplerExternalOES tex;\n
void main()\n
{\n
gl_FragColor = texture2D(tex, v_texcoord);\n
}";

6

Note that the following code in the example raised a SEGFAULT:

gl->ClearDepth(1.0); // Enables Clearing Of The Depth Buffer

I’ve replaced it by:

gl->ClearDepthf(1);

7

adv - your comment regarding replacing gl->ClearDepth(1.0); with gl->ClearDepthf(1); was epic!! It saved my day! Thank you!