I’m trying to concatenate multiple mp4 files. My pipeline seems to work ok until I reach a certain number of files. I can’t remember how many, but I used ChatGPT and Claude to help identify where the issue may be and it was suggested to add queues between the demux and the concat. That did help, but only a little.
With the files I’m testing (All approx 10MB in size), it appears that 35 files is the point at which my pipeline basically halts, but it works fine with 34 files. Here is the last few lines of output from GStreamer:
Redistribute latency...98.4 %)
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...99.2 %)
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...99.0 %)
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...
0:00:51.3 / 0:00:51.4 (99.9 %)
It sits here forever. With 34 files, this pipeline finishes almost instantly. The 35th file opens and plays fine, so I don’t think it’s corrupt. I’ve ran this with debug level 3 and I don’t know that I see anything obvious (see below):
Use Windows high-resolution clock, precision: 1 ms
Setting pipeline to PAUSED ...
0:00:00.123267700 8164 30556 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux34> atom has bogus size 18446744073709551615
0:00:00.123293700 8164 33456 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux33> atom has bogus size 18446744073709551615
0:00:00.123490600 8164 484 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux32> atom has bogus size 18446744073709551615
0:00:00.123858400 8164 30140 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux31> atom has bogus size 18446744073709551615
0:00:00.123869100 8164 9528 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux30> atom has bogus size 18446744073709551615
0:00:00.123977400 8164 30176 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux29> atom has bogus size 18446744073709551615
0:00:00.124119400 8164 26792 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux28> atom has bogus size 18446744073709551615
0:00:00.124288400 8164 7656 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux27> atom has bogus size 18446744073709551615
0:00:00.124483100 8164 11800 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux26> atom has bogus size 18446744073709551615
0:00:00.124633300 8164 28540 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux25> atom has bogus size 18446744073709551615
0:00:00.124809400 8164 27000 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux24> atom has bogus size 18446744073709551615
0:00:00.124893800 8164 30448 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux23> atom has bogus size 18446744073709551615
0:00:00.125072200 8164 7012 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux22> atom has bogus size 18446744073709551615
0:00:00.125246600 8164 8640 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux21> atom has bogus size 18446744073709551615
0:00:00.125415400 8164 6300 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux20> atom has bogus size 18446744073709551615
0:00:00.125546700 8164 34176 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux19> atom has bogus size 18446744073709551615
0:00:00.125855600 8164 27084 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux17> atom has bogus size 18446744073709551615
0:00:00.125857900 8164 25212 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux18> atom has bogus size 18446744073709551615
0:00:00.126022000 8164 34192 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux16> atom has bogus size 18446744073709551615
0:00:00.126178000 8164 33616 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux15> atom has bogus size 18446744073709551615
0:00:00.126355000 8164 17128 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux14> atom has bogus size 18446744073709551615
0:00:00.126513600 8164 25152 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux13> atom has bogus size 18446744073709551615
0:00:00.126690200 8164 28776 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux12> atom has bogus size 18446744073709551615
0:00:00.126855300 8164 20760 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux11> atom has bogus size 18446744073709551615
0:00:00.127010000 8164 1268 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux10> atom has bogus size 18446744073709551615
0:00:00.127163000 8164 29552 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux9> atom has bogus size 18446744073709551615
0:00:00.127340500 8164 20972 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux8> atom has bogus size 18446744073709551615
0:00:00.127498000 8164 32336 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux7> atom has bogus size 18446744073709551615
0:00:00.127628300 8164 17560 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux6> atom has bogus size 18446744073709551615
0:00:00.127807400 8164 3480 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux5> atom has bogus size 18446744073709551615
0:00:00.127958800 8164 30688 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux4> atom has bogus size 18446744073709551615
0:00:00.128159100 8164 24216 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux3> atom has bogus size 18446744073709551615
0:00:00.128327400 8164 13128 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux2> atom has bogus size 18446744073709551615
Pipeline is PREROLLING ...
0:00:00.128444600 8164 22132 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux1> atom has bogus size 18446744073709551615
0:00:00.128631800 8164 21888 WARN qtdemux qtdemux.c:560:gst_qtdemux_pull_atom:<demux0> atom has bogus size 18446744073709551615
Redistribute latency...
0:00:00.346235200 8164 21452 FIXME basesink gstbasesink.c:3399:gst_base_sink_default_event:<filesink0> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
Redistribute latency...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
0:00:00.363595800 8164 21452 FIXME aggregator gstaggregator.c:1543:gst_aggregator_loop:<mux> Subclass should call gst_aggregator_selected_samples() from its aggregate implementation.
Here is the pipeline:
gst-launch-1.0 concat name=vconcat ! queue ! h264timestamper ! h264parse ! mp4mux name=mux ! filesink location="./test.mp4" ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\7D3F1AE6-E9AA-4E89-939F-DED7E3381040.mp4" ! qtdemux name=demux0 demux0.video_0 ! queue ! h264parse ! vconcat. demux0.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\400EE83E-606A-4A88-BF2A-2133B5FF72B3.mp4" ! qtdemux name=demux1 demux1.video_0 ! queue ! h264parse ! vconcat. demux1.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\00139D31-47A7-4C92-B893-BF355951584F.mp4" ! qtdemux name=demux2 demux2.video_0 ! queue ! h264parse ! vconcat. demux2.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\7C179DA0-2C05-43DB-9A77-50D8F78834DA.mp4" ! qtdemux name=demux3 demux3.video_0 ! queue ! h264parse ! vconcat. demux3.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\907FBD4D-A492-4924-9737-811204689512.mp4" ! qtdemux name=demux4 demux4.video_0 ! queue ! h264parse ! vconcat. demux4.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\8B26ECBF-AE01-4F7F-B8D3-FC466F13199E.mp4" ! qtdemux name=demux5 demux5.video_0 ! queue ! h264parse ! vconcat. demux5.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\41F9ACCF-596E-45AC-9330-065B6C90084C.mp4" ! qtdemux name=demux6 demux6.video_0 ! queue ! h264parse ! vconcat. demux6.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\7ED9E488-4E21-4A30-86AB-669CB1178EC4.mp4" ! qtdemux name=demux7 demux7.video_0 ! queue ! h264parse ! vconcat. demux7.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\994EB291-2FCE-457B-94A6-50085539A8C5.mp4" ! qtdemux name=demux8 demux8.video_0 ! queue ! h264parse ! vconcat. demux8.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\A61C311D-75A7-44EC-97EB-12FAF40EB9DB.mp4" ! qtdemux name=demux9 demux9.video_0 ! queue ! h264parse ! vconcat. demux9.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\48D2A0F3-4B65-423B-8F2C-9EDCCE67E3B5.mp4" ! qtdemux name=demux10 demux10.video_0 ! queue ! h264parse ! vconcat. demux10.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\4EB18333-1C66-4DD0-B249-9ED5D12A085E.mp4" ! qtdemux name=demux11 demux11.video_0 ! queue ! h264parse ! vconcat. demux11.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\FDCE6E2D-A41D-4F37-8ECC-A19EB7AF607F.mp4" ! qtdemux name=demux12 demux12.video_0 ! queue ! h264parse ! vconcat. demux12.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\773ADA79-9A4F-4101-AD2C-FD5E40736C2C.mp4" ! qtdemux name=demux13 demux13.video_0 ! queue ! h264parse ! vconcat. demux13.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\BD9FC25F-065B-492F-BBA7-77ACC3757F9D.mp4" ! qtdemux name=demux14 demux14.video_0 ! queue ! h264parse ! vconcat. demux14.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\6409DE3C-5C44-4078-9B36-A8817052CB19.mp4" ! qtdemux name=demux15 demux15.video_0 ! queue ! h264parse ! vconcat. demux15.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\6D5730D0-2660-4D6F-A041-E5B3AC70EA15.mp4" ! qtdemux name=demux16 demux16.video_0 ! queue ! h264parse ! vconcat. demux16.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\D6120AD0-78F6-46FE-901F-2AD0BD57BB71.mp4" ! qtdemux name=demux17 demux17.video_0 ! queue ! h264parse ! vconcat. demux17.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\C771463C-8FC5-4B4C-AF3C-C838E8283061.mp4" ! qtdemux name=demux18 demux18.video_0 ! queue ! h264parse ! vconcat. demux18.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\297092E1-C02F-4677-AD33-8154A389D735.mp4" ! qtdemux name=demux19 demux19.video_0 ! queue ! h264parse ! vconcat. demux19.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\662EF1E3-19C4-4B1B-B077-86695D844F6D.mp4" ! qtdemux name=demux20 demux20.video_0 ! queue ! h264parse ! vconcat. demux20.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\166B76ED-0E65-422B-818A-23B2F7C275D0\\5C789314-FB20-4ADC-9A2B-17DCF0FC4751.mp4" ! qtdemux name=demux21 demux21.video_0 ! queue ! h264parse ! vconcat. demux21.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\F16CFD67-9A2C-463C-A148-E5B683172064\\2A0CBE87-CA87-498F-ACF2-E8A07DEDEB48.mp4" ! qtdemux name=demux22 demux22.video_0 ! queue ! h264parse ! vconcat. demux22.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\F16CFD67-9A2C-463C-A148-E5B683172064\\81294F2F-6226-44DB-A6F8-8C5B5055EE7F.mp4" ! qtdemux name=demux23 demux23.video_0 ! queue ! h264parse ! vconcat. demux23.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\F16CFD67-9A2C-463C-A148-E5B683172064\\DC3CBD43-6EA0-4F51-899D-2369BD77ACFB.mp4" ! qtdemux name=demux24 demux24.video_0 ! queue ! h264parse ! vconcat. demux24.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\F16CFD67-9A2C-463C-A148-E5B683172064\\8664DC3B-C46F-483D-B046-6CDA9FE6F7E9.mp4" ! qtdemux name=demux25 demux25.video_0 ! queue ! h264parse ! vconcat. demux25.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\F16CFD67-9A2C-463C-A148-E5B683172064\\85FF1004-C85A-4527-974E-0737538B9AB2.mp4" ! qtdemux name=demux26 demux26.video_0 ! queue ! h264parse ! vconcat. demux26.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\F16CFD67-9A2C-463C-A148-E5B683172064\\47A99EC0-7DAD-433C-BC53-30B9FCF4E705.mp4" ! qtdemux name=demux27 demux27.video_0 ! queue ! h264parse ! vconcat. demux27.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\F16CFD67-9A2C-463C-A148-E5B683172064\\B6F83649-02DD-4610-94C5-AB64EDDB4B62.mp4" ! qtdemux name=demux28 demux28.video_0 ! queue ! h264parse ! vconcat. demux28.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\F16CFD67-9A2C-463C-A148-E5B683172064\\AF0F18D7-2971-4C49-9B04-E6768E632158.mp4" ! qtdemux name=demux29 demux29.video_0 ! queue ! h264parse ! vconcat. demux29.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\F16CFD67-9A2C-463C-A148-E5B683172064\\9731D449-7DD2-4599-9897-5CDC51A463E1.mp4" ! qtdemux name=demux30 demux30.video_0 ! queue ! h264parse ! vconcat. demux30.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\F16CFD67-9A2C-463C-A148-E5B683172064\\577E38FF-4546-4DA0-BF5C-404EC7B21114.mp4" ! qtdemux name=demux31 demux31.video_0 ! queue ! h264parse ! vconcat. demux31.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\F16CFD67-9A2C-463C-A148-E5B683172064\\81C933FD-F9E4-4D2A-BB31-0FE349373D08.mp4" ! qtdemux name=demux32 demux32.video_0 ! queue ! h264parse ! vconcat. demux32.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\F16CFD67-9A2C-463C-A148-E5B683172064\\8A4D9595-C7C5-4908-A150-0BA62F432A80.mp4" ! qtdemux name=demux33 demux33.video_0 ! queue ! h264parse ! vconcat. demux33.audio_0 ! queue ! aacparse ! aconcat. ^
filesrc location="D:\\VideoRecording\\Recordings\\F16CFD67-9A2C-463C-A148-E5B683172064\\CA7D0F5E-7FC6-4130-A86F-B9A852EBF611.mp4" ! qtdemux name=demux34 demux34.video_0 ! queue ! h264parse ! vconcat. demux34.audio_0 ! queue ! aacparse ! aconcat. ^
concat name=aconcat ! queue ! aacparse ! mux.
Am I using queues incorrectly? I’ve never felt completely comfortable understanding where to use queues and how to use various queue options, so I’m leaning towards that being my problem.