Timing problem with x264enc

Application Development
1

Hi,

I wrote a code to receive from many udp video sources and switch between them at any moment.

Those sources has to be created dynamically so I used interpipe plugin in order to not handle timestamps/probes to align the new created bin to the pipeline time.

The interpipesink bin

udpsrc, rtpjitterbuffer, rtpmp2tdepay, tsparse, tsdemux, queue, h264parse, avdec_h264, queue, videoconvert, interpipesink

The interpipesrc pipeline:

interpipesrc is-live=1 allow-renegotiation=true stream-sync=0 format=3 ! videoconvert ! queue ! x264enc name=encoder key-int-max=30 speed-preset=1 tune=4 bitrate=15000 ! h264parse config-interval=1 ! rtph264pay ! udpsink sync=1

I skipped some parameters to simplify, if needed I can write down the actual pipelines.

After the play command either the bin and the pipeline starts from time 0 so all works well.

When I create a new receiving bin ad I change the listen-to proprierty of interpipesrc, the video stops for few seconds, because, I think, x264enc skips very fast many buffers.

If I add a tee and a viewer branch the problem go away.

The interpipesrc pipeline with tee:

interpipesrc is-live=1 allow-renegotiation=true stream-sync=0 format=3 ! videoconvert ! queue ! tee name=mytee mytee. ! queue ! videoconvert ! autovideosink sync=0 mytee. ! queue ! x264enc name=encoder key-int-max=30 speed-preset=1 tune=4 bitrate=15000 ! h264parse config-interval=1 ! rtph264pay ! udpsink sync=1

Any idea about this problem?

Thanks

2

interpipe is a third party GStreamer plugin (not an upstream one) which is known to have problematic timestamp handling. I suspect your use case can be made to work somehow, but I can’t help with that myself.

There are two other options you could consider:

  1. use intervideosink and intervideosrc. Those only work for raw video frames, but that’s what you’re passing through the interpipe here anyway, so that’s definitely an option for you. You may want to force a framerate on the intervideosrc with a capsfilter after it. These element have the advantage that they’re really easy to use and the intervideosrc will generate blank frames or repeat frames if there’s no input, and it will handle different framerate input fine as well. It also outputs a consistently timestamped stream starting from 0, no matter what the input timestamps are. The downside is that the frames may not be passed through 1:1, but it’s more like pointing a camera at a projector screen, with intervideosrc being the camera and intervideosink being the projector.
  2. use the inter elements from the Rust plugins set, which have been inspired by interpipe. These elements have some restrictions around clocks/base time of the producer/consumer pipelines (see example code in documentation), but those shouldn’t be a problem for you, especially if the producer pipeline is live.
3

Hi Tim,

your reply is gold! I completely missed the intervideo plugin.

I will give it a try asap.

Thank you.