Problems when dealing with mpeg-ts and H26X encoding with more than one slice

Plugin and library development
1

Hello peeps and dweebs!

I am by no means an expert on encoding so this is me reaching out for some help, I have talked a bit with @ndufresne and @thaytan at the GStreamer conference about this issue.

We are getting video out of an AXIS Camera (P3818-PLVE) in a matter sort of like this:

video_caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,payload=(int)96"
gst-launch-1.0 -v rtspsrc location="$uri"  latency=40                       \
    protocols=tcp name=src ntp-sync=true buffer-mode=synced                 \
    ntp-time-source=clock-time                                              \
    mpegtsmux name=payloader alignment=7                                    \
    src.                                                                    \
    ! "$video_caps"                                                         \
    ! queue                                                                 \
    ! rtph265depay                                                          \
    ! h265parse config-interval=-1                                          \
    ! payloader.                                                            \
    payloader. ! srtsink uri=srt://$to:5000 wait-for-connection=false

And this works fine for us on most AXIS Camera models. But this one. This one uses two imaging chips to produce a really really wide image out of multiple inputs. And produces encoded video with 2 slices. All this is according to AXIS.

The resulting video from this is not pretty, it is split and green like in the attached image in VLC:

vlcsnap-2023-09-29-09h15m15s311
vlcsnap-2023-09-29-09h15m15s3111920×960 59.4 KB

So, is there something going on here in the h26xparse elements? And/or could something be fishy with the coded data?

I would like to attach samples, but I do not really know how or where. I gave some to @thaytan via USB as well.

2

Can you upload a short sample of such a stream, e.g. the output of mpegtsmux?

Does it only play back strangely in VLC, or also in e.g. ffplay and GStreamer-based applications?

3

Using gst-play I got an image for a short while and then a splat:

$ GST_DEBUG=*:3 gst-play-1.0 srt.mkv 
Press 'k' to see a list of keyboard shortcuts.
Now playing /home/jonasdn/sandbox/spiideo-gstreamer/srt-analyzer/srt.mkv
0:00:00.007789212 1026029 0x563738305970 WARN                 basesrc gstbasesrc.c:3693:gst_base_src_start_complete:<source> pad not activated yet
0:00:00.007914962 1026029 0x563738305970 WARN                 basesrc gstbasesrc.c:3693:gst_base_src_start_complete:<source> pad not activated yet
0:00:00.016765971 1026029 0x7fd37c000d40 WARN           matroskademux matroska-demux.c:5192:gst_matroska_demux_parse_contents_seekentry:<matroskademux0> Too large SeekPosition 18446744073709551615
0:00:00.016772539 1026029 0x7fd37c000d40 WARN           matroskademux matroska-demux.c:5213:gst_matroska_demux_parse_contents_seekentry:<matroskademux0> Incomplete seekhead entry (0x1549a966/18446744073709551615)
0:00:00.016774736 1026029 0x7fd37c000d40 WARN           matroskademux matroska-demux.c:5192:gst_matroska_demux_parse_contents_seekentry:<matroskademux0> Too large SeekPosition 18446744073709551615
0:00:00.016776730 1026029 0x7fd37c000d40 WARN           matroskademux matroska-demux.c:5213:gst_matroska_demux_parse_contents_seekentry:<matroskademux0> Incomplete seekhead entry (0x1654ae6b/18446744073709551615)
0:00:00.016778553 1026029 0x7fd37c000d40 WARN           matroskademux matroska-demux.c:5192:gst_matroska_demux_parse_contents_seekentry:<matroskademux0> Too large SeekPosition 18446744073709551615
0:00:00.016780367 1026029 0x7fd37c000d40 WARN           matroskademux matroska-demux.c:5213:gst_matroska_demux_parse_contents_seekentry:<matroskademux0> Incomplete seekhead entry (0x1043a770/18446744073709551615)
0:00:00.016783102 1026029 0x7fd37c000d40 WARN           matroskademux matroska-demux.c:5192:gst_matroska_demux_parse_contents_seekentry:<matroskademux0> Too large SeekPosition 18446744073709551615
0:00:00.016784941 1026029 0x7fd37c000d40 WARN           matroskademux matroska-demux.c:5213:gst_matroska_demux_parse_contents_seekentry:<matroskademux0> Incomplete seekhead entry (0x1c53bb6b/18446744073709551615)
0:00:00.016786759 1026029 0x7fd37c000d40 WARN           matroskademux matroska-demux.c:5192:gst_matroska_demux_parse_contents_seekentry:<matroskademux0> Too large SeekPosition 18446744073709551615
0:00:00.016790548 1026029 0x7fd37c000d40 WARN           matroskademux matroska-demux.c:5213:gst_matroska_demux_parse_contents_seekentry:<matroskademux0> Incomplete seekhead entry (0x1254c367/18446744073709551615)
0:00:00.016817154 1026029 0x7fd37c000d40 WARN      matroskareadcommon matroska-read-common.c:1997:gst_matroska_read_common_parse_info:<matroskademux0:sink> Invalid duration 0.000000
Redistribute latency...
0:00:00.025258818 1026029 0x7fd37c000d40 FIXME           videodecoder gstvideodecoder.c:1199:gst_video_decoder_drain_out:<libde265dec0> Sub-class should implement drain()
0:00:00.025287307 1026029 0x7fd37c000d40 WARN                 default libde265-dec.c:583:gst_libde265_dec_set_format:<libde265dec0> warning: Unsupported extra data version 1, decoding may fail
WARNING Unsupported extra data version 1, decoding may fail
WARNING debug information: ../ext/libde265/libde265-dec.c(583): gst_libde265_dec_set_format (): /GstPlayBin:playbin/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstLibde265Dec:libde265dec0
0:00:00.030516167 1026029 0x7fd37c000d40 FIXME           videodecoder gstvideodecoder.c:1199:gst_video_decoder_drain_out:<libde265dec0> Sub-class should implement drain()
Redistribute latency...
Redistribute latency...     
0:00:01.6 / 0:00:12.5       
Reached end of play list.

Uploaded a gdp in this issue: Problems when dealing with mpeg-ts and H26X encoding with more than one slice (#3009) · Issues · GStreamer / gstreamer · GitLab

4

Also, worth nothing I guess, that when we send this out using RTP we do not see this issue, then all is well. :slight_smile: So something with the alignment-thingie and the clearing of headers of the depay and/or the marker mpeg-ts groks?

5

If you use avdec_h265 instead of libde265, does it work better? And I was thinking of trying the MPEG-TS stream that directly comes out of the mpegtsmux, without first going over SRT. Or alternatively might be worth trying a different container format (like Matroska) at that point too.

It would be useful to get the stream captured from the earliest point possible before something might’ve broken it (if it’s broken and not just a problem on the receiver side).

6

Ok!

So it looks like the same (green split) if I capture the mpeg-ts stream before sending out srt. And it looks fine (no green split) if I use matroska instead.

I do not have avdec_h265 atm, do you know if I can get it to Fedora in an easy way?

7

I attached the matroska- and ts-files in the issue: Problems when dealing with mpeg-ts and H26X encoding with more than one slice (#3009) · Issues · GStreamer / gstreamer · GitLab

8

A colleague tried with avdec_h265 same green split.

9

Ok, sounds like something goes wrong when muxing then (or when converting to byte-stream in h265parse for the muxing).

Let’s close this here, there’s quite clearly a bug somewhere and not you doing something wrong. Discussion can continue in the gitlab issue :slight_smile: