Cea708overlay stuttering/stopping

Application Development
1

Hello,

I’m working on displaying closed captions on a video. The problem is that the video stutters constantly, even if there are no closed captions to display, and even gets stuck for seconds at a time when it has to display text. Other worth-mentioning behavior is that the text that is displayed seems to not be correct and repeats a lot of characters.

This is done in an app getting live ts, decoding video and audio. The app(C++/Qt5) runs in a custom linux based system made with yocto (x86 intel), so everything is cross-compiled and using vaapi to decode. The video content of the ts comes from an ATSC1 signal and the video is encoded in MPEG2.

Since the behavior is the same using the app and launching gstreamer from the terminal, here it is a simple example of what’s not working (target machine):
gst-launch-1.0 filesrc location=file.ts ! tsdemux ! queue ! mpegvideoparse ! vaapimpeg2dec ! cea708overlay ! glimagesink

It seems to me that the problem comes from the interaction between vaapidecode and glimagesink and maybe not so much from cea708overlay. But I really don’t know, and I would appreciate any tips on what may be happening.

Many thanks,

(I’ve debugged cea708overlay plugin code to get some sort of explanation. I’m not a rust expert, but it seems like it is receiving a lot of repeated buffers containing repeated “VideoCaptionMeta”, this explains why a lot of repeated text is displayed. And it may be an indication that the problem does come from the decoder.)

2

If you are having performance issues, then performance profiling is always the go-to for discovering why. E.g. you may be missing a queue between the decoder and glimagesink which would decouple rendering from decoding.

Receiving multiple GstVideoCaptionMeta on a single buffer may be a problem in the stream or the decoder. If it’s one for each frame or field then that is fine and expected.

Each GstVideoCaptionMeta may result in a rerender of the captions which can happen every frame. It’s also possible that the caption stream contains the stuttering itself.