Memory Issue on Simple Pipeline

Application Development
1

Hello,

I am facing a memory issue with a simple GStreamer pipeline that I cannot resolve:

 pipeline_description = ( 
            "flvmux name=mux ! filesink location=out.flv "
            "appsrc name=overlay_src format=time is-live=true do-timestamp=true caps=video/x-raw,width=1280,height=720,framerate=1/1,format=NV12 ! videoconvert ! cmp. "
            "appsrc name=video_src format=time is-live=true do-timestamp=true caps=video/x-raw,width=864,height=576,framerate=60/1,format=NV12 ! videoconvert ! cmp. "
            "compositor name=cmp background=black max-threads=2 ! videorate ! video/x-raw,framerate=60/1 ! videoconvert ! x264enc speed-preset=ultrafast bitrate=4500 tune=zerolatency ! h264parse ! mux. " 
            "appsrc name=audio_src format=time is-live=true do-timestamp=true caps=audio/x-raw,rate=48000,channels=2,format=S16LE,layout=interleaved !  audioconvert ! faac bitrate=128000 ! aacparse !  mux. "
        
        )

When I add an appsrc overlay, the RAM usage increases exponentially and keeps growing until the system eventually fails.

However, when I use only two appsrc sources (video and audio), the RAM usage remains stable. I observed the same issue when using imagefreeze, which led me to test with an appsrc instead.

Do you have any insights as to what might be causing this behavior? Additionally, I’ve noticed that whenever I add a queue (regardless of its position in the pipeline), the video framerate drops to 47 FPS instead of 60, even when using videorate.

Thank you for your help!

2

I’ve been able to run various tests and it appears that the problem lies with the h264 encoder and the addition of an image.

When I use “autovideosink”, I have no problem, but on any computer, the behavior is the same with this pipeline:

gst-launch-1.0 -v filesrc location=overlay.png ! pngdec ! videoconvert ! imagefreeze ! videorate ! video/x-raw,framerate=60/1 ! x264enc speed-preset=fast ! h264parse ! flvmux name=mux ! filesink location=overlay.flv

  • High CPU load
  • RAM rising without stopping on four-core CPUs (I haven’t noticed it on an 8-core).

Perhaps the problem is that my system doesn’t have enough cores? What’s strange is that with ffmpeg, I haven’t encountered this problem.

3

Can you check if setting threads=4 on x264enc improves the situation? Or any other number > 0.

4

About RAM usage, maybe you’re hitting some leaks… Try the leaks tracer, leaks

5

Thank you for your response.

I investigated the issue, and it turns out that this happens on a machine where the processor’s memory is overloaded—or at least, that’s my assumption. I’m not entirely sure how the imagefreeze plugin behaves, but from what I understand, the CPU has to handle a 1280x720 RGBA image and convert it for the encoder. That’s a significant amount of data to process while maintaining 60 FPS.

What I’ve noticed is that when I switch to 30 FPS, the issue disappears. This suggests that the CPU can’t process the data fast enough, causing it to spill into memory. However, since it never catches up, the backlog keeps growing indefinitely. I could limit buffer size and drop frames, but that causes the video to malfunction, eventually freezing.

I also noticed that when I add other video sources while requesting 60 FPS, the actual frame rate drops to 43 FPS. I assume GStreamer recognizes that the CPU can’t handle that much data and adjusts accordingly. When I set it to 30 FPS, everything runs smoothly—at least on this machine.

On a more powerful CPU, 60 FPS is achievable, which suggests that this particular processor isn’t fast enough to handle the data stream efficiently. The machine in question is an Athlon 5370 with 2MB of cache. Interestingly, if the image or video resolution is below 960x720, the CPU can maintain 60 FPS without issue.

6

did you tried to replace flvmux with matroskamux? do you have the same leak?

also a pipeline without any queue? try to add some queue between source and encoder and sink