Hi guys.
I’m new to Gstreamer and I’m trying to use it to “build” an mp4 file from a raw h264 file.
Long story short, I’m working on a bare metal video recorder using an NXP microcontroller and an UVC webcam. I’ve managed the low-level part of the project, now I’m stuck on what was supposed to be the easy part - taking the raw h264 file and creating a valid mp4 file.
I’ve used ffmpeg to do that, however it only works if i specify the codec. Which basically reencodes the data. Which, as I’m sure you’re aware, takes a long time and is completely unnecessary. Using “codec copy” I only get about 1 second of video, despite my best efforts.
Gstreamer, on the other hand, correctly plays the file using this command:
launch-1.0 filesrc location=[FILE LOCATION] ! video/x-h264 ! h264parse ! avdec_h264 ! videoconvert ! videorate ! video/x-raw,framerate=30/1 ! autovideosink
I haven’t managed to get it to output an .mp4 file though. I’d be most grateful for any pointer.
Thank you!
find the reference for your problem
gst-launch-1.0 filesrc location=stream.h264 ! video/x-h264 ! h264parse ! mp4mux ! filesink location=video.mp4
The output is
Use Windows high-resolution clock, precision: 1 ms
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
ERROR: from element /GstPipeline:pipeline0/GstMP4Mux:mp4mux0: Could not multiplex stream.
Additional debug info:
../gst/isomp4/gstqtmux.c(5410): gst_qt_mux_add_buffer (): /GstPipeline:pipeline0/GstMP4Mux:mp4mux0:
Buffer has no PTS.
New clock: GstSystemClock
Execution ended after 0:00:00.006311000
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstH264Parse:h264parse0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbaseparse.c(3694): gst_base_parse_loop (): /GstPipeline:pipeline0/GstH264Parse:h264parse0:
streaming stopped, reason error (-5)
Freeing pipeline ...
The same file works with ffmpeg, but i do get a couple of corrupt frames (to be expected). If its any help:
ffmpeg -framerate 30 -i stream.h264 -c:v h264_nvenc outputFFMPEG.mp4 -y
ffmpeg version 2023-08-28-git-b5273c619d-essentials_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
libavutil 58. 17.100 / 58. 17.100
libavcodec 60. 23.100 / 60. 23.100
libavformat 60. 10.101 / 60. 10.101
libavdevice 60. 2.101 / 60. 2.101
libavfilter 9. 11.100 / 9. 11.100
libswscale 7. 3.100 / 7. 3.100
libswresample 4. 11.100 / 4. 11.100
libpostproc 57. 2.100 / 57. 2.100
Input #0, h264, from 'stream.h264':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1920x1080, 30 fps, 1200k tbr, 1200k tbn
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
[h264 @ 0000023a2b61a480] concealing 4768 DC, 4768 AC, 4768 MV errors in P frame
Output #0, mp4, to 'outputFFMPEG.mp4':
Metadata:
encoder : Lavf60.10.101
Stream #0:0: Video: h264 (Main) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080, q=2-31, 2000 kb/s, 30 fps, 15360 tbn
Metadata:
encoder : Lavc60.23.100 h264_nvenc
Side data:
cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: N/A
[vist#0:0/h264 @ 0000023a2bdc9840] corrupt decoded framete=N/A speed=N/A
[h264 @ 0000023a2bdb9c40] concealing 6163 DC, 6163 AC, 6163 MV errors in P frames speed=6.62x
[vist#0:0/h264 @ 0000023a2bdc9840] corrupt decoded frame
[h264 @ 0000023a2bb2f600] concealing 2519 DC, 2519 AC, 2519 MV errors in P frames speed=13.8x
[vist#0:0/h264 @ 0000023a2bdc9840] corrupt decoded frame
[h264 @ 0000023a2c0f6e00] error while decoding MB 80 16, bytestream -584.5kbits/s speed=16.8x
[h264 @ 0000023a2c0f6e00] concealing 6209 DC, 6209 AC, 6209 MV errors in P frame
[vist#0:0/h264 @ 0000023a2bdc9840] corrupt decoded frame
[out#0/mp4 @ 0000023a2b62ffc0] video:33637kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.141131%
frame= 3940 fps=506 q=23.0 Lsize= 33684kB time=00:02:11.20 bitrate=2103.2kbits/s speed=16.8x
try this one
gst-launch-1.0 filesrc location=/tmp/test.h264 ! video/x-h264 ! h264parse ! avdec_h264 ! videoconvert ! videorate ! video/x-raw,framerate=30/1 ! autovideosink
as I stated in my original question, the videosink works perfectly. the actual saving to a file fails hilariously. :))
mp4mux only seems to support some specific settings for H264, so check if your stream complies:
video/x-h264:
stream-format: { (string)avc, (string)avc3 }
alignment: au
width: [ 16, 2147483647 ]
height: [ 16, 2147483647 ]
You can also try decoding and re-encoding to check if the problem can be the “type” of h264:
gst-launch-1.0 filesrc location=stream.h264 ! video/x-h264 ! h264parse ! avdec_h264 ! x264enc ! mp4mux ! filesink location=video.mp4
Moreover from the error message: ERROR: from element /GstPipeline:pipeline0/GstMP4Mux:mp4mux0: Could not multiplex stream. Additional debug info: ../gst/isomp4/gstqtmux.c(5410): gst_qt_mux_add_buffer (): /GstPipeline:pipeline0/GstMP4Mux:mp4mux0: Buffer has no PTS.
I would try checking if the timestamping is the problem (Buffer has no PTS).
Maybe some elements have a do-timestamp property you can set to TRUE?
Or try adding a clocksync.
Also, I have tried googling and found this: android - GStreamer mp4mux gives "Buffer has no PTS" error using custom appsrc - Stack Overflow
So it might be that the h264parse is messing your timestamps up
Thank you for the hints.
I did come up across this thread [my camera is also built with a sonix chip, just like that one].But I barely know anything video-related. I’m more of a “solder that here”, “put that ic there”, “uuuuh! registers!” kinda guy.
Anyway, I will try your suggestions, thank you again for taking the time. And as a post scriptum, would it help if I posted a link to a quick 5 minute raw video captured with my cam?
That thread also mentions h264parse breaking the stream.
So it would seem likely that this camera produces h264 stream with some content/settings that h264parse cant handle correctly.
My best suggestion is to remove it, however, you will need to check if mp4mux can accept the h264 stream you provide.
Otherwise you will be stuck with trans-coding…
The other option is to see if you have any possibilities to tweak the h264 encoder settings on the camera to make things more compatible.
If by RAW you mean h264 then yeah, maybe somebody will be curious enough to try playing it and help see what’s the issue 
You were dead right. I ended up using ffmpeg, mostly by trial and [an excruciating amount of] error, and ended up with the following command:
ffmpeg -r 30 -err_detect ignore_err -i test.raw -copyinkf -c:v copy output.mp4 -r 30
the relevant part was the -r 30, which basically means “F*** timestamps, dear Watson, you go ahead and treat everything as if it’s all 30 fps.”. Interestingly enough, -framerate or -fps switches were as useful as holy water is - didn’t do anything bad, but didn’t really work either.
I’m sure there’s some homologous setting in gstreamer.
I posted this answer here, even though it’s a gstreamer forum, because some poor soul might end up finding this in the future.
Thank you for all your help.
1 Like