Queue not handling caps queries correctly?

Queue seems to forward upstream caps queries, which is making hard for me to create a dynamic pipeline, since it always answers caps queries with caps that are not yet relevant.

Which means, under slow encoding, on a pipeline with dynamic relinking, when you relink, the elements will query upstream for caps, and receive much later caps which are wrong.

Example

Element A → queue → probe → (dynamic link element B/C)

  1. Caps event is pushed by A at 1s →
  2. Queue SINK receives caps event at 1s →
  3. A pushes another caps event →
  4. Queue SRC probe receives first caps event at 2s->
  5. probe relinks queue src to B
  6. B does a caps query →
  7. Queue forwards query and receives the second caps → error = no common format.

How can we do a dynamic pipeline after a queue if the queried caps are always wrong and will not be aligned to the caps we have inside the probe?

In your scenario, the problem seems to be in both A and B/C’s caps handling. You have to give a more concrete example to let us find a solution here.

It’s also not clear to me why the upstream caps query from B would break something. Why does B even do an upstream caps query, what’s the goal there?

Up to the point if having the second caps event on the source pad of the queue, everything should function correctly. At that point your probe would see the second caps event and could decide to dynamically relink to C if that’s required for the new caps.

And generally, the code handling the dynamic aspects of the pipeline might have to adjust caps negotiation depending on what the overall goal / knowledge of the code is. It’s not guaranteed that the default handling will always give ideal or even working results in dynamic scenarios, and because of timing aspects that might also be impossible without outside hints.

Thank you for the response.

The upstream query is not my own action, it is done automatically by gst_pad_link → gst_pad_link_check_compatible_unlocked.

So i have a probe that checks for caps events in the queue src, and relinks accordingly. When i receive caps video/x-raw for example and link to a compositor pad, that pad queries caps upstream to the queue. However my demuxer before the queue is fast and has already changed caps to video/x-vp8 so the link fails, even though i am linking for video/x-raw, i am still in the probe for the first caps, and the second caps have not reached the queue src.

To work around this i have made the probe also intercept caps queries and answer with the latest queue src caps instead of forward, but it seems to me like this should be the default behavior, since forwarding upstream queries will break when the queue is shifted.

Perhaps my mistake is that i have a queue that at least for a moment reads faster that it writes, but I would ideally want to create an application that does not have very strict timing constraints in order to work.

I think in this case you want to use gst_pad_link_full() without GST_PAD_LINK_CHECK_CAPS. That would avoid the caps queries and everything involved with it. That way you would defer any actual negotiation failure to a later time when the CAPS event is handled.

Also in your case you want to make sure that the caps query at the “switch” point is correct regardless of your current problem. For upstream and downstream queries you probably want to return all possible caps. E.g. when the pad is linked to compositor, a downstream caps query on the queue would only return video/x-raw and not video/x-vp8 (it should return both at least), which is obviously not correct because your switching logic has an effect on the supported caps.

Thank you, gst_pad_link_full seems to work, no query is sent at all.

For the second point, is it correct to return all possible caps on a query? As far as I am aware, demuxers use fixed caps so I don’t want to communicate that “i can handle all these caps”, since i can only handle one format at any given time.

Reference :

gst_pad_use_fixed_caps() is used on the source pad with fixed caps. As long as the pad is not negotiated, the default CAPS query will return the caps presented in the padtemplate. As soon as the pad is negotiated, the CAPS query will return the negotiated caps (and nothing else

.

Thinking a bit about this, theoretically, the only way for a dynamic link after a queue to work without workarounds, is for the upstream elements to happen to still be using the same caps from the moment they send the caps event to the queue sink until it is consumed by the queue src. I realize now there are valid workarounds, just putting this out there in case anyone else faces the same issue.

That’s what I would do in your case for upstream caps queries, yes.