Duration of fmp4 in cmafmux

Application Development
1

Hello!

I am trying to write correct metadata for fragmented mp4 files. We are using the cmafmux and taking inspiration from the HLS live example in gst-plugin-rs repo.

I get the duration / end_time of a fragment by going:

// Calculate the end time at this point. The duration of the segment header is set
// to the whole duration of this segment.
let end_time = start_time + first.duration().ok_or(gst::FlowError::Eos)?;
tracing::info!("segment header duration: {}", first.duration().unwrap());

Just as I see in the HLS live example. The start time / end times I get in this way line up. With that I mean that the end time I calculate for a fragment matches the start time I get for the next item. All peachy.

But when I compare the duration I get with what gst-discover-1.0 or ffprobe gives me it does not match.

Example:

I have generated the item item-00000000, the print above gives:

segment header duration: 0:00:05.943495333

I create self contained files by concatenating the header with the fragment.

The tool gst-disoverer-1.0 gives:

$ gst-discoverer-1.0 test_create_items_fmp4/self-contained.0 

Properties:
  Duration: 0:00:06.002000000
  Seekable: yes
  Live: no
  container #0: Quicktime
    video #1: H.264 (Constrained Baseline Profile)
      Stream ID: f2cd67b38fa926e925ee0999d7183ec9e733e6462b227301919c369fcd67e282/001
      Width: 320
      Height: 180
      Depth: 24
      Frame rate: 30/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 0
      Max bitrate: 0

And ffprobe:

$ ffprobe -hide_banner  test_create_items_fmp4/self-contained.0 
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test_create_items_fmp4/self-contained.0':
  Metadata:
    major_brand     : cmf2
    minor_version   : 0
    compatible_brands: iso6cmfccfsd
    creation_time   : 2023-12-04T06:21:04.000000Z
  Duration: 00:00:06.00, start: 0.082333, bitrate: 1379 kb/s
  Stream #0:0[0x1](und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, smpte170m, progressive), 320x180, 1373 kb/s, SAR 1:1 DAR 16:9, 30.29 fps, 30 tbr, 3k tbn (default)
    Metadata:
      creation_time   : 2023-12-04T06:21:04.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]

Using the pbutils API calls does not match either. Am I doing some kind of thinko? Or is this expected in some way?

I construct the cmafmux like:

let cmafmux = gst::ElementFactory::make_with_name("cmafmux", None)?;
cmafmux.set_property("fragment-duration", item_duration_ns);
cmafmux.set_property_from_str("header-update-mode", "update");
cmafmux.set_property("write-mehd", true);

Many thanks for reading!

2

Can you provide a small stand-alone example that prints the duration like this and creates such files? That would help understanding what goes on there :slight_smile:

Also for looking at the MP4 files in detail, MP4Box.js - JavaScript MP4 Reader/Fragmenter is a very useful tool.

3

So, let’s try!

I use the hls-live example in gst-plugins-rs as a base, and change:

diff --git a/mux/fmp4/examples/hls_live.rs b/mux/fmp4/examples/hls_live.rs
index 4f58a6a8..5aa18d49 100644
--- a/mux/fmp4/examples/hls_live.rs
+++ b/mux/fmp4/examples/hls_live.rs
@@ -166,7 +166,7 @@ fn trim_segments(state: &mut StreamState) {
             let mut path = state.path.clone();
             path.push(segment.path);
             println!("Removing {}", path.display());
-            std::fs::remove_file(path).expect("Failed to remove old segment");
+            //std::fs::remove_file(path).expect("Failed to remove old segment");
         } else {
             break;
         }
@@ -319,6 +319,7 @@ fn setup_appsink(appsink: &gst_app::AppSink, name: &str, path: &Path, is_video:
                 }
 
                 let duration = first.duration().unwrap();
+                println!("duration of item: {}", duration.useconds());
 
                 let mut file = std::fs::File::create(&path).expect("failed to open fragment");
                 for buffer in &*buffer_list {

And I run:

$ cargo run --example hls_live
    Skipping git submodule `https://github.com/gtk-rs/gir` due to update strategy in .gitmodules
    Skipping git submodule `https://github.com/gtk-rs/gir-files` due to update strategy in .gitmodules
    Skipping git submodule `https://gitlab.freedesktop.org/gstreamer/gir-files-rs.git` due to update strategy in .gitmodules
    Skipping git submodule `https://github.com/gtk-rs/gir` due to update strategy in .gitmodules
    Skipping git submodule `https://github.com/gtk-rs/gir-files` due to update strategy in .gitmodules
    Skipping git submodule `https://github.com/gtk-rs/gir` due to update strategy in .gitmodules
    Skipping git submodule `https://github.com/gtk-rs/gir-files` due to update strategy in .gitmodules
   Compiling gst-plugin-fmp4 v0.12.0-alpha.1 (/home/jonasdn/sandbox/gst-plugins-rs/mux/fmp4)
    Finished dev [optimized + debuginfo] target(s) in 2.13s
     Running `/home/jonasdn/sandbox/gst-plugins-rs/target/debug/examples/hls_live`
Writing master manifest to hls_live_stream/manifest.m3u8
writing header to hls_live_stream/audio_0/init.cmfi
writing header to hls_live_stream/audio_1/init.cmfi
duration of item: 2484535
duration of item: 2484535
wrote segment with date time 2023-12-04 10:26:44.152603974 UTC to hls_live_stream/audio_0/segment_0.cmfa
writing manifest to hls_live_stream/audio_0/manifest.m3u8
wrote segment with date time 2023-12-04 10:26:44.152590127 UTC to hls_live_stream/audio_1/segment_0.cmfa
writing manifest to hls_live_stream/audio_1/manifest.m3u8
writing header to hls_live_stream/video_0/init.cmfi
duration of item: 2500000
wrote segment with date time 2023-12-04 10:26:44.157011014 UTC to hls_live_stream/video_0/segment_0.cmfv
writing manifest to hls_live_stream/video_0/manifest.m3u8
duration of item: 2484535
duration of item: 2484535
wrote segment with date time 2023-12-04 10:26:46.637125274 UTC to hls_live_stream/audio_1/segment_1.cmfa
writing manifest to hls_live_stream/audio_1/manifest.m3u8
wrote segment with date time 2023-12-04 10:26:46.637139121 UTC to hls_live_stream/audio_0/segment_1.cmfa
writing manifest to hls_live_stream/audio_0/manifest.m3u8
duration of item: 2500000
wrote segment with date time 2023-12-04 10:26:46.657011014 UTC to hls_live_stream/video_0/segment_1.cmfv
writing manifest to hls_live_stream/video_0/manifest.m3u8
^C

I then check some:

hls_live_stream/video_0 (main*) $ cat init.cmfi segment_0.cmfv > 1.mp4
hls_live_stream/video_0 (main*) $  gst-discoverer-1.0 1.mp4
Analyzing file:///home/jonasdn/sandbox/gst-plugins-rs/mux/fmp4/hls_live_stream/video_0/1.mp4
Done discovering file:///home/jonasdn/sandbox/gst-plugins-rs/mux/fmp4/hls_live_stream/video_0/1.mp4

Properties:
  Duration: 0:00:02.504000000
  Seekable: yes
  Live: no
  container #0: Quicktime
    video #1: H.264 (Main Profile)
      Stream ID: 0df551223fda082d81a036381df7ec2b40689b3a1eaf268b2caf9f52ad92138b/001
      Width: 1280
      Height: 720
      Depth: 24
      Frame rate: 30/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 0
      Max bitrate: 0

2.504 != 2.484535

4

Thanks! That will need a bit of time to analyze, not sure when I’ll get to that but maybe someone else has time before that :slight_smile:

1 Like
5

Hello Jonas! :slight_smile:

I’ve been working with getting correct durations with cmafmux as well, it has worked pretty well the past few months but in some very very very rare case we still get wrong durations.

Some tips:

  1. Don’t use ffprobe for durations, it only provides an estimate.
  2. cmafmux has an estimation on what to set the timescale to. This does not always work. There’s a pad property on the sink pads where you can configure it yourself.

I haven’t tried gst-discover for durations. We use mp4info and look at “duration_with_fragments” and “timescale” to get an accurate fraction. Our function tests validate that the diff between fragments are within 0.1ms which works.

1 Like
6

Thank you Johan!

The timescale helped us!

1 Like