Hi, What’s the purpose of the “parsed” field in GstCaps?
The parsed field (sometimes also framed, or stream-format=xyz) is used to indicate the the data is parsed in a way that makes sense for the format in question, e.g. for an MP3 audio stream that would be that the buffers are mp3 audio frames, and properly timestamped and such.
The purpose is to distinguish between unparsed/unframed data, such as e.g. the output of filesrc location=foo.mp3 ! typefind ! .... which may be audio/mpeg,mpegversion=1,layer=3 but would not be parsed, since it’s just chunks of 4kB and won’t be timestamped, whereas once it’s gone through an mpegaudioparse it will be framed and timestamped.
This is mostly used for formats which can exist inside a container and also ‘raw’ in a file.
Many elements will need or want input data properly timestamped, e.g. decoders or muxers, so the parsed field makes sure that e.g. filesrc location=foo.mp3 ! mpg123audiodec or filesrc location=foo.mp3 ! matroskamux won’t work.
Thank you for your answer! It means that all parsers MUST include “parsed=true” field in their source pad templates and all elements (decoders/muxers) MUST require this field (<=> “parsed=true” present in the sink pad template). Correct?
Generally speaking that’s right, yes.
Muxers usually need framed and properly timestamped input, so MUST for muxers sink pad templates, but it depends on the format in question. e.g. Vorbis audio doesn’t exist in unframed form, so there’s no need for such a field in Vorbis caps.
Decoders could theoretically accept unframed input, and in the past they did, which then led to slightly different seek behaviour/bugs depending on which e.g. mp3 decoder was being used, but nowadays we pretty much always require parsers in front of decoders and centralise seek logic in the parser where needed. Only exception is jpeg decoders where for backwards compatibility and because jpegparse only got a rank recently we had the decoder parse the input as well if needed.
I suspect that the same logic applies to demuxers, meaning that a stream should be marked as parsed after it passed a demuxer.
In the following pipeline
filesrc location=/tmp/video-h264.mkv ! matroskademux ! rtph264pay …
the payload element could require that the input data is parsed and the caps negotiation would fail if this information is not provided. Do you agree?
That is already the case here because the payloader requires stream-format={avc,bytestream},alignment={nal,au} as input, and if it was not parsed then the caps would be just video/x-h264, stream-format=(string)byte-stream without any alignment field, which would not be accepted by the payloader.
It sounds like you have a very specific problem that you’re trying to solve with this - what is that?
I’m just a curious person :). Thanks a lot for your time Tim!
1 Like