Subtitle Overlays and Hardware-Accelerated Playback, how to get it working?

Hello

I just realized that I have a problem when I want to render Subtitle on my decoded frame.

The problem is described here:Subtitle Overlays and Hardware-Accelerated Playback

Roughly summarized:
If I use the Android HW decoder, the decoded frame is not in memory and the GST plugins cannot draw on the framebuffer.

The linked text describes how it “could” work best.
But unfortunately not how I can implement it with the current status.

According to my decodebin3 element I have the following caps:
video/x-raw(memory:GLMemory), format=(string)RGBA, texture-target=(string)external-oes

If I have understood this correctly, then the problem is the “external-oes”
I have now tried to load the data from the GLMemory with gldownload and then convert it to a texture-target=2d with videoconvert.
That wouldn’t be very performant, but it’s probably the only way at the moment, right?
Apart from that, it doesn’t seem to work either.

I have tried the following:
Decodebin3 → gldownloac → capsfilter → videoconvert → …
Decodebin3 → gldownloac → videoconvert → capsfilter → …

The error message I get:
../gst/gstpad. c:5820:pre_eventfunc_check:<amcvideodec-c2exynosh264decoder1:sink> caps video/x-h264, stream-format=(string)byte-stream, pixel-aspect-ratio=(fraction)1/1, width=(int)1280, height=(int)720, framerate=(fraction)50/1, coded-picture-structure=(string)frame, chroma-format=(string)4: 2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, colorimetry=(string)bt709, parsed=(boolean)true, alignment=(string)au, profile=(string)high, level=(string)4 not accepted

If I use libav as decoder, then it works. But I would like to use amc.

Does anyone have an idea how I can implement this with GStreamer 1.24.7?

external-oes means the memory can be consumed, but not written to by GPU. That means, compositing external-oes with the overlay composition (texture-2d) should not be a problem. If there is any issues, then its probably due to a bug. The error you have posted seems different, its seems like the parser and the decoder failed to negotiate. The message is not precise enough to conclude on why though.

It is not possible to directly download an external-eos texture. You would need to convert to a 2D texture using glcolorconvert and then download.

1 Like

That was the missing link.
After insert glcolorconvert everythink works like expected :+1: