Help with raw h264 file conversion

Hi guys.
I’m new to Gstreamer and I’m trying to use it to “build” an mp4 file from a raw h264 file.

Long story short, I’m working on a bare metal video recorder using an NXP microcontroller and an UVC webcam. I’ve managed the low-level part of the project, now I’m stuck on what was supposed to be the easy part - taking the raw h264 file and creating a valid mp4 file.

I’ve used ffmpeg to do that, however it only works if i specify the codec. Which basically reencodes the data. Which, as I’m sure you’re aware, takes a long time and is completely unnecessary. Using “codec copy” I only get about 1 second of video, despite my best efforts.
Gstreamer, on the other hand, correctly plays the file using this command:

launch-1.0 filesrc location=[FILE LOCATION] ! video/x-h264 ! h264parse ! avdec_h264 ! videoconvert ! videorate ! video/x-raw,framerate=30/1 ! autovideosink

I haven’t managed to get it to output an .mp4 file though. I’d be most grateful for any pointer.
Thank you!

find the reference for your problem

gst-launch-1.0 filesrc location=stream.h264 ! video/x-h264 ! h264parse ! mp4mux ! filesink location=video.mp4

The output is

Use Windows high-resolution clock, precision: 1 ms
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
ERROR: from element /GstPipeline:pipeline0/GstMP4Mux:mp4mux0: Could not multiplex stream.
Additional debug info:
../gst/isomp4/gstqtmux.c(5410): gst_qt_mux_add_buffer (): /GstPipeline:pipeline0/GstMP4Mux:mp4mux0:
Buffer has no PTS.
New clock: GstSystemClock
Execution ended after 0:00:00.006311000
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstH264Parse:h264parse0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbaseparse.c(3694): gst_base_parse_loop (): /GstPipeline:pipeline0/GstH264Parse:h264parse0:
streaming stopped, reason error (-5)
Freeing pipeline ...

The same file works with ffmpeg, but i do get a couple of corrupt frames (to be expected). If its any help:

ffmpeg -framerate 30 -i stream.h264 -c:v h264_nvenc outputFFMPEG.mp4 -y

ffmpeg version Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
  libavutil      58. 17.100 / 58. 17.100
  libavcodec     60. 23.100 / 60. 23.100
  libavformat    60. 10.101 / 60. 10.101
  libavdevice    60.  2.101 / 60.  2.101
  libavfilter     9. 11.100 /  9. 11.100
  libswscale      7.  3.100 /  7.  3.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
Input #0, h264, from 'stream.h264':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1920x1080, 30 fps, 1200k tbr, 1200k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
[h264 @ 0000023a2b61a480] concealing 4768 DC, 4768 AC, 4768 MV errors in P frame
Output #0, mp4, to 'outputFFMPEG.mp4':
    encoder         : Lavf60.10.101
  Stream #0:0: Video: h264 (Main) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080, q=2-31, 2000 kb/s, 30 fps, 15360 tbn
      encoder         : Lavc60.23.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: N/A
[vist#0:0/h264 @ 0000023a2bdc9840] corrupt decoded framete=N/A speed=N/A
[h264 @ 0000023a2bdb9c40] concealing 6163 DC, 6163 AC, 6163 MV errors in P frames speed=6.62x
[vist#0:0/h264 @ 0000023a2bdc9840] corrupt decoded frame
[h264 @ 0000023a2bb2f600] concealing 2519 DC, 2519 AC, 2519 MV errors in P frames speed=13.8x
[vist#0:0/h264 @ 0000023a2bdc9840] corrupt decoded frame
[h264 @ 0000023a2c0f6e00] error while decoding MB 80 16, bytestream -584.5kbits/s speed=16.8x
[h264 @ 0000023a2c0f6e00] concealing 6209 DC, 6209 AC, 6209 MV errors in P frame
[vist#0:0/h264 @ 0000023a2bdc9840] corrupt decoded frame
[out#0/mp4 @ 0000023a2b62ffc0] video:33637kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.141131%
frame= 3940 fps=506 q=23.0 Lsize=   33684kB time=00:02:11.20 bitrate=2103.2kbits/s speed=16.8x

try this one

gst-launch-1.0 filesrc location=/tmp/test.h264 ! video/x-h264 ! h264parse ! avdec_h264 ! videoconvert ! videorate ! video/x-raw,framerate=30/1 ! autovideosink

as I stated in my original question, the videosink works perfectly. the actual saving to a file fails hilariously. :))

mp4mux only seems to support some specific settings for H264, so check if your stream complies:

  stream-format: { (string)avc, (string)avc3 }
      alignment: au
          width: [ 16, 2147483647 ]
         height: [ 16, 2147483647 ]

You can also try decoding and re-encoding to check if the problem can be the “type” of h264:
gst-launch-1.0 filesrc location=stream.h264 ! video/x-h264 ! h264parse ! avdec_h264 ! x264enc ! mp4mux ! filesink location=video.mp4

Moreover from the error message: ERROR: from element /GstPipeline:pipeline0/GstMP4Mux:mp4mux0: Could not multiplex stream. Additional debug info: ../gst/isomp4/gstqtmux.c(5410): gst_qt_mux_add_buffer (): /GstPipeline:pipeline0/GstMP4Mux:mp4mux0: Buffer has no PTS.
I would try checking if the timestamping is the problem (Buffer has no PTS).
Maybe some elements have a do-timestamp property you can set to TRUE?
Or try adding a clocksync.

Also, I have tried googling and found this: android - GStreamer mp4mux gives "Buffer has no PTS" error using custom appsrc - Stack Overflow

So it might be that the h264parse is messing your timestamps up

Thank you for the hints.

I did come up across this thread [my camera is also built with a sonix chip, just like that one].But I barely know anything video-related. I’m more of a “solder that here”, “put that ic there”, “uuuuh! registers!” kinda guy.

Anyway, I will try your suggestions, thank you again for taking the time. And as a post scriptum, would it help if I posted a link to a quick 5 minute raw video captured with my cam?

That thread also mentions h264parse breaking the stream.

So it would seem likely that this camera produces h264 stream with some content/settings that h264parse cant handle correctly.

My best suggestion is to remove it, however, you will need to check if mp4mux can accept the h264 stream you provide.

Otherwise you will be stuck with trans-coding…

The other option is to see if you have any possibilities to tweak the h264 encoder settings on the camera to make things more compatible.

If by RAW you mean h264 then yeah, maybe somebody will be curious enough to try playing it and help see what’s the issue :stuck_out_tongue_winking_eye:

You were dead right. I ended up using ffmpeg, mostly by trial and [an excruciating amount of] error, and ended up with the following command:

ffmpeg -r 30 -err_detect ignore_err -i test.raw -copyinkf -c:v copy output.mp4 -r 30

the relevant part was the -r 30, which basically means “F*** timestamps, dear Watson, you go ahead and treat everything as if it’s all 30 fps.”. Interestingly enough, -framerate or -fps switches were as useful as holy water is - didn’t do anything bad, but didn’t really work either.
I’m sure there’s some homologous setting in gstreamer.

I posted this answer here, even though it’s a gstreamer forum, because some poor soul might end up finding this in the future.

Thank you for all your help.

1 Like