Hardware decoding with h.265/VP9 on Meta Quest 2 fail but h.264 works (Android)

We’re working with GStreamer on the Meta Quest 2 and have successfully set up a hardware video decoding pipeline using h.264 and amcviddec-omxqcomvideodecoderavc.

h.264 is fine for us for the time being, but we wanted to test with h.265/HEVC and VP9 as well to see if there is any improvement, and whilst we can get that to work fine outside the Meta Quest 2, on the device itself we can’t get the pipeline up due to various errors.

For h.265/HEVC, there are a bunch of warnings being spammed, and nothing displays:

11-20 10:54:53.137  7824  7986 W GStreamer: ../gst/videoparsers/gsth265parse.c:gst_h265_parse_handle_frame:1307: broken/invalid nal Type: 1 Slice_TRAIL_R, Size: 150182 will be dropped

For VP9, it fails to parse the ‘superframe’:

11-20 11:07:39.311  9029  9186 E GStreamer: ../gst-libs/gst/codecparsers/gstvp9parser.c:gst_vp9_parser_parse_superframe_info:896: Failed to parse superframe

According to this StackOverflow post the first error could mean the SPS is not being sent any more after the stream starts, and adding config-interval to h265parse might help, but this seems to not have any effect - I’m also not sure why this would be necessary since my desktop does not need this option to pick up the stream at a later point in time when using the same codec?

VP9 works on my desktop as well, albeit with garbled colours despite the colour formats presumable being correct, but it comes through, so this might be an encoder problem instead, I guess.

I wanted to put the question out there anyway to see if these are GStreamer issues we should be reporting upstream, device quirks, or perhaps there is something we’re doing wrong on our end.

h.265/HEVC works with GStreamer 1.24 for me now.

I haven’t tested VP9 yet.

1 Like

Hi NoTuxNoBux

how do you deploy the GStreamer on the Quest 2. With Unity?

Did you tried h.265/HEVC with alpha channel?

Cause, I am looking for a solution to stream Unreal Engine or Unity content with alpha channel to an Oculus Quest3 app. So the Quests passthrough is not sent back and forth over the quest link.