The title pretty much says it all. Displays can often be set to a choice of different resolutions and not all of them result in square pixels. I do want to use force-aspect-ratio=true because I want correct video proportions, with black stripes if necessary.
Is there a standard way of doing this?
It seems to me that what I want is a video element which passes everything straight through except that when it is asked (from downstream) for the pixel aspect ratio, it gets the pixel aspect ratio from upstream and multiplies it by an appropriate adjustment before passing it on.
I don’t know if exactly what you are looking for exists but I would look at the videoscale element which can scale a video frame based on the downstream caps.
You can add a capsfilter just after it to set the desired parameters.
Check here for some parameters.
I did try videoscale and caps filters but I didn’t find anything that could produce the required effect. I know that I can get arbitrary proportions from videoscale if I turn off force-aspect-ratio in the video sink. But that’s not what I want. I do want to enforce aspect ratio, I just want to enforce it to a slightly different value. If I specify a caps filter with a pixel aspect ratio different from what the original video states, then it doesn’t link.
For instance this plays ok, because the caps filter pixel-aspect-ratio does not attempt to change anything:
gst-launch-1.0 filesrc location=25TONGT.MPG ! decodebin ! videoconvert ! videoscale ! video/x-raw,pixel-aspect-ratio=(fraction)1/1 ! ximagesink
But as soon as I try to change the pixel-aspect-ratio, it fails:
gst-launch-1.0 filesrc location=25TONGT.MPG ! decodebin ! videoconvert ! videoscale ! video/x-raw,pixel-aspect-ratio=(fraction)11/10 ! ximagesink
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
Redistribute latency…
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0: Delayed linking failed.
Additional debug info:
./grammar.y(506): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0:
failed delayed linking some pad of GstDecodeBin named decodebin0 to some pad of GstVideoConvert named videoconvert0
Redistribute latency…
Can you try
videoscale add-borders=false
It looks like the documentation may be messed up for the videoscale element. Do you also NOT see this as a property?
More can be found in the source.
gst-launch-1.0 filesrc location=25TONGT.MPG ! decodebin ! videoconvert ! videoscale add-borders=false ! video/x-raw,pixel-aspect-ratio=(fraction)11/10 ! ximagesink
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
Redistribute latency…
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0: Delayed linking failed.
Additional debug info:
./grammar.y(506): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0:
failed delayed linking some pad of GstDecodeBin named decodebin0 to some pad of GstVideoConvert named videoconvert0
I also tried putting the caps filter before the videoscale instead of after, same failure.
I have found an example on this page:
— https://gstreamer.freedesktop.org/documentation/ximagesink/index.html?gi-language=c
gst-launch-1.0 -v videotestsrc ! video/x-raw, pixel-aspect-ratio=(fraction)4/3 ! videoscale ! ximagesink
which works as given, but when I use a file source it doesn’t:
gst-launch-1.0 filesrc location=25TONGT.MPG ! decodebin ! videoconvert ! video/x-raw,pixel-aspect-ratio=(fraction)4/3 ! videoscale ! ximagesink
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
Redistribute latency…
Redistribute latency…
ERROR: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMpegPSDemux:mpegpsdemux0: Internal data stream error.
Additional debug info:
gstmpegdemux.c(2954): gst_ps_demux_loop (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMpegPSDemux:mpegpsdemux0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn’t want to preroll.
I suspect that videotestsrc is capable of producing images with whatever aspect ratio you ask for, whereas of course a video file is not.
You may try something like:
gst-launch-1.0 filesrc location=~/Videos/Big_Buck_Bunny_1080_10s_30MB.mp4 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert ! videoscale ! video/x-raw,pixel-aspect-ratio=2/1 ! ximagesink pixel-aspect-ratio=2/1 force-aspect-ratio=1 -v
And if you don’t want black stripes, set the final width and height such as:
gst-launch-1.0 filesrc location=~/Videos/Big_Buck_Bunny_1080_10s_30MB.mp4 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert ! videoscale ! video/x-raw,width=960,height=1080,pixel-aspect-ratio=2/1 ! ximagesink pixel-aspect-ratio=2/1 -v
Thank you for this, it works a treat on Linux. However I distribute software for Windows, Mac and Linux so I need a solution on Windows & Mac also. I haven’t checked it out on the Mac yet but on Windows I have been using autovideosink, which doesn’t have any pixel-aspect-ratio property. I find in fact it resolves to d3d11videosink… but that doesn’t have any pixel-aspect-ratio property either. So, do you (or anyone) have any ideas for Windows & Mac?
Incidentally I have some videos which have non-square pixels and which correctly provide a non-unity pixel-aspect-ratio, and all video sinks seem able to handle that correctly without needing any help.
I’ve now looked at the Mac. I use caopengllayersink in my app, which has no pixel-aspect-ratio property (and doesn’t work if I merely try to change aspect ratio with a caps filter after the videoscale).
Using glimagesink on the command line, it does have pixel-aspect-ratio but jams up at 8.9% without displaying anything if I try the above approach of specifying the new aspect ratio in both the caps filter and in the pixel-aspect-ratio property of the videosink. (actually it sometimes works! but usually not)
To be specific:
Windows: this plays the video but without any change to aspect ratio. d3d11videosink has no pixel-aspect-ratio property.
gst-launch-1.0.exe filesrc location=25TONGT.MPG ! decodebin ! videoconvert ! videoscale ! video/x-raw,pixel-aspect-ratio=1/2 ! d3d11videosink force-aspect-ratio=1 -v
Mac: this occasionally plays the video with aspect ratio adjustment but usually jams up as described above:
gst-launch-1.0 filesrc location=25TONGT.MPG ! decodebin ! videoconvert ! videoscale ! video/x-raw,pixel-aspect-ratio=1/2 ! glimagesink pixel-aspect-ratio=1/2 force-aspect-ratio=1 -v
It would be great if there was some way of telling the videosink that the video’s pixel aspect ratio is different from what the video itself says. Because all of these video sinks work correctly when the video itself has a non-unity pixel aspect ratio.
Not very clean, though you may cheat with capssetter element such as:
gst-launch-1.0 filesrc location=~/Videos/Big_Buck_Bunny_1080_10s_30MB.mp4 ! qtdemux ! capssetter caps="video/x-h264,pixel-aspect-ratio=1/2" ! h264parse ! avdec_h264 ! videoconvert ! videoscale ! video/x-raw,pixel-aspect-ratio=1/1 ! autovideosink
That’s brilliant, thank you. I had not heard of the capssetter element. Your pipeline works for me on Windows, Mac, and Linux. Also I couldn’t see the need for a caps filter between videoscale and videosink - after all I don’t need one for displaying videos with non-unity pixel aspect ratio - so I removed that and it still works.
However it’s not yet a solution for me. My app uses a pipeline like this:
filesrc ! decodebin ! videoconvert ! videoscale ! imagesink
which gives me four places where I could insert the capssetter. Unfortunately none of them work. I tried moving the capssetter to all possible locations in the line which you provided and I find it has to go either between demux and h264parse, or between h264parse and avdec_h264. But I cannot specify these elements in my pipeline - I have to work with arbitrary videos which is why I use decodebin.
So it seems to me that I need to find a way of inserting a capssetter into the internals of decodebin. I would be very grateful for a suggestion of how to do that.
This would be sub-optimal in many cases, but you could try:
gst-launch-1.0 filesrc location=~/Videos/Big_Buck_Bunny_1080_10s_30MB.mp4 ! decodebin ! autovideoconvert ! video/x-raw ! capssetter caps="video/x-raw,pixel-aspect-ratio=1/2" ! videoscale ! video/x-raw,pixel-aspect-ratio=1/1 ! autovideoconvert ! autovideosink -v
The bad point is that if you have an accelerated decoder outputting into non-system memory (such as NVMM, GL, D3D11) and videosink handles that memory, then the double autovideoconvert would involve copying to and from system memory (video/x-raw). This could be better handled programmatically.
EDIT: This is weird. Forcing 1/2 results in double conversion in my case, resulting in 1/4 width. I tried to pass 1/sqrt(2) as 3/5 or 5/7 but a converter failed and it didn’t display.
The failure may be related to my videoconverter (nvvidconv) having some scaling capabilities. It works fine with:
gst-launch-1.0 filesrc location=~/Videos/Big_Buck_Bunny_1080_10s_30MB.mp4 ! decodebin ! capssetter caps="video/x-raw(memory:NVMM),pixel-aspect-ratio=1/2" ! autovideoconvert ! "video/x-raw(memory:NVMM),pixel-aspect-ratio=1/1" ! autovideoconvert ! autovideosink -v
Feel free to play with this, though I’d advise a programmatic way.
Someone more skilled may provide better advice.
This looks a bit scary to me. I like the way your previous solution with capssetter works - it’s just a question of putting the capssetter in the right place, which happens to be somewhere inside decodebin. So I might read up about decodebin, see if I can find a way.
Does this work for you?
gst-launch-1.0 filesrc location=25TONGT.MPG ! parsebin ! capssetter caps=video/x-h264,pixel-aspect-ratio=1/2 ! decodebin3 ! videoconvertscale ! capsfilter caps=video/x-raw,pixel-aspect-ratio=1/1 ! autovideosink
For my own understanding, are you using capsetter here to essentially trick the videoscale element to convert all buffers to a 1/1 aspect ratio because the buffer caps (1/2) don’t match the downstream caps filter (1/1)?
Your line above plays the video but with its original aspect ratio, the capssetter has no effect. Maybe the decodebin is setting the pixel-aspect-ratio back to that specified by the video itself.
All I’m trying to do is compensate for non-square video display pixels - I want a square video to display as a square!
I’ve been looking into this again and realised I was being stupid, using a capssetter with video/x-h264 in places where the video has been decoded so it should be x-raw. So I now have an answer and it can be done. Bear in mind that to use this, you have to determine the pixel-aspect-ratio of the video itself and divide that by the pixel-aspect-ratio of the display. In my experience the vast majority of videos have a pixel-aspect-ratio of 1/1, but not all of them. And a big thank you to Honey_Patouceul for pointing me in the right direction.
This pipeline works on Linux (ximagesink), Windows (d3d11videosink) but not on Mac (autovideosink=glimagesink):
gst-launch-1.0 filesrc location=MyVideo.mp4 ! decodebin ! capssetter caps=“video/x-raw,pixel-aspect-ratio=2/1” ! videoconvert ! videoscale ! videosink
However on Mac you can do it this way, because glimagesink has a pixel-aspect-ratio property which is documented as being the pixel-aspect-ratio of the device (not of the video), so:
gst-launch-1.0 filesrc location=MyVideo.mp4 ! decodebin ! videoconvert ! videoscale ! glimagesink pixel-aspect-ratio=2/1
And if you are working programatically on Mac then you can also use caopengllayersink with the capssetter pipeline as above.