We’re working with GStreamer on the Meta Quest 2 and have successfully set up a hardware video decoding pipeline using h.264 and amcviddec-omxqcomvideodecoderavc.
h.264 is fine for us for the time being, but we wanted to test with h.265/HEVC and VP9 as well to see if there is any improvement, and whilst we can get that to work fine outside the Meta Quest 2, on the device itself we can’t get the pipeline up due to various errors.
For h.265/HEVC, there are a bunch of warnings being spammed, and nothing displays:
11-20 10:54:53.137 7824 7986 W GStreamer: ../gst/videoparsers/gsth265parse.c:gst_h265_parse_handle_frame:1307: broken/invalid nal Type: 1 Slice_TRAIL_R, Size: 150182 will be dropped
For VP9, it fails to parse the ‘superframe’:
11-20 11:07:39.311 9029 9186 E GStreamer: ../gst-libs/gst/codecparsers/gstvp9parser.c:gst_vp9_parser_parse_superframe_info:896: Failed to parse superframe
According to this StackOverflow post the first error could mean the SPS is not being sent any more after the stream starts, and adding config-interval to h265parse might help, but this seems to not have any effect - I’m also not sure why this would be necessary since my desktop does not need this option to pick up the stream at a later point in time when using the same codec?
VP9 works on my desktop as well, albeit with garbled colours despite the colour formats presumable being correct, but it comes through, so this might be an encoder problem instead, I guess.
I wanted to put the question out there anyway to see if these are GStreamer issues we should be reporting upstream, device quirks, or perhaps there is something we’re doing wrong on our end.