new to gstreamer, is there a method hook into when a new fmp4 fragment is ready? i was hoping to use this event to update my hls playlist with a .part file, which will refer to a single moov inside of the fmp4 fragment which will contain multiple movs. so splitmuxsink creates larger segments and then my part files are fragments inside each segment which the server takes care of extracting and returning
Check the examples in the repository.
In short, a whole fragment is always output in one go as a single buffer list. If the first buffer of the list has the HEADER and DISCONT flags set then it’s the “initialization segment” (moov etc). In that case the fragment header (moof) will be the second buffer of the list, otherwise the first (it will have the HEADER flag set but not DISCONT). All following buffers in the list are the actual data of the fragment.
1 Like
i have read the source of the cmafhlsink but i will give it another read as well as that example as i didnt quite understand how when i use cmafmux with basehlssink it ends up in one file with the moov and all the moof + mdat vs with cmafhlssink it ends up with init and m4s segments.
i am trying to create a long playlist (24 hours to start) for archival purposes so if i can have large segments (15m to 1 hour) and small fragments (6s to 15s) and use .part features of llhls i thought i could work on keeping the playlist file small. i changed hlssink3 to allow for continuing the playlist by inserting a discontinuity whenever the stream is restarted if the playlist is already there for this purpose.
i tried this before with regular hls but without incremental playlist updates and part files it didnt really work, the segments are either too long to see recent history or too short & result in a large playlist. so I figured if i can get an event whenever a new moof + mdat is available i can update the playlist with a part pointing with either a byterange part to that file or simply a custom endpoint that would extract the moov + the nth moof + mdat and return it as a single fragment.
thank you for your guidance, i should be able to get it to work from here.
in case anyone else ever needs to do this; I ended up doing it the “wrong” way, introducing a new signal in cmafmux which would indicate when a fragment was completed, and what the byte ranges were for that fragment. then I hooked into that signal in order to know when the fragment was done.
I would strongly recommend to simply get the buffer from the buffer list. its relatively straightforward and there is a good example of how to do it in the hlscmafsink element. I used the signal as I was still getting comfortable with gstreamer and rust at that time.
then I would update the playlist manifest with a #EXT-X-PART annotation using the byte-range syntax.
this approach is not 100% compliant with the hls spec and I needed to make some additional compromises to get it to work the way I wanted.
first, I needed to emit my first segment early & short. otherwise, the playlist would not be playable until the first segment was completed, as the playlist requires at least one full valid segment to be considered well-formed.
also, I needed to modify the underlying library to support the #EXT-X-PART annotation, as well as allow for segments which are not complete to be omitted from the produced playlist. I am not sure if this is spec compliant or not.
next, I had to ensure when inserting a discontinuity after a restart, that any partial fragments occurring before the discontinuity were removed and translated into a regular segment.
I made four mistakes in my implementation.
The first was, I did not properly implement support for the init.mp4 header fragment like the hlscmaf element does. I simply included it in every full segment. This meant I didn’t need to worry about removing the related boxes from each segment and I did not have to handle when those headers changed. however it would have been nice to do this properly to reduce the metadata overhead.
The second was, I did not implement PART fragment independence incorrectly. each part happened to start with a key-frame but this was coincidental. so all parts were marked as INDEPENDENT. I think it would have been better to handle this correctly.
The third was, I did not handle chunks at all. I think this would have been nice because chunks would potentially provide a smaller resolution.
The fourth is I did not have a custom server for serving delta playlist updates, although this might help with some of the issues I describe below.
finally, in terms of how well this worked for very long streams, I was ultimately not very happy with the performance. the trade-off between playlist manifest length, segment duration, part duration and load times is just awkward. I ended up doing 7 day long playlists with this approach. there is a good reason no one does it like this.
what works really well about this approach is the long file segments with small parts within the segments being byterange-able. this makes it really easy to cache the entire segment at the edge for historical stuff and then retrieve slices of it quickly via the byte-ranges.
what doesn’t work well is that there is not an obvious way to retrieve PARTs from historical segments in a playlist manifest without including them all. but I would only like to retrieve them in the playlist when seeking to that particular segment; & only use them in that context.
this would require changes to the player, which I did not want to make during this leg of the implementation.
however, with partial delta playlist updates, it may be possible to make this work by making using of #EXT-X-SKIP events without outright breaking hls compliance. if we skip all segments except the first, the ones before and after discontinuities, have PROGRAM-DATE-TIME headers, and then only return n segments with parts around the requested start segment, I believe it might work…
I still have more experimentation to do on this. I would need proper benchmarks to come to any real conclusions. however, it seems my dream of very long hls playlists that still have fast seeking are currently out of reach.