Severe artifacts when decoding multiple rtp/h.264 streams on Windows PC


In my application we capture and record 4 RTP/H.264 streams carrying video @ 25Hz. This is done on Windows x86 PC which has 10 i9-cores and 32GB of memory. This PC is also installed with RTX 3070 GPU.

Our decoding code relies on OpenCV with gstreamer backend. we use cv::VideoCapture and pass to it
the gstreamer-pipe as a string for OpenCV to set up the pipe.

This works fine when we decode and record 1 or 2 streams, but when we go beyond to 3 or 4 streams we encounter image artifacts and frame drops. When working with 3-4 streams CPU load seems very high, close to 100%.

Seems that GStreamer struggles and fails to decode due to CPU load in my case.
BTW, same code/application on Linux PC manages the load and I don’t see artifacts.

Any idea/direction for a solution here will be welcomed.
Thank you.

Can you run in a profiler to see where all the CPU usage comes from, especially in comparison to the same on Linux?

What resolution are those H.264 streams, and what decoders are being used?

You could check if there’s a way to increase the default kernel-side UDP receive buffer size in Windows (basically the Windows-equivalent to something like sysctl -w net.core.wmem_max=2097152 on Linux), but if you can’t decode fast enough that’s probably not going to help much either.