Issues getting whipclientsink to even stream (from inside a container)

General Discussion
1

I’m working on a project which streams the pulseaudio output inside a Docker container via WHIP; the container builds the rust plugins from source to do this.

Previously, I’d been using whipsink (deprecated) to do this, but it has recently stopped working (for mystery reasons - networking? Cloudflare Stream issues? Plugins changes?). I’ve moved to whipclientsink - recommended - and invoke it like so:

gst-launch-1.0 pulsesrc ! audioconvert ! opusenc ! rtpopuspay ! "application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2" ! whip.sink_0 whipclientsink name=whip signaller::whip-endpoint="$WHIP_ENDPOINT" &

which gives me the error

WARNING: erroneous pipeline: could not link rtpopuspay0 to whip, whip can't handle caps application/x-rtp, media=(string)audio, encoding-name=(string)OPUS, payload=(int)96, clock-rate=(int)48000, encoding-params=(string)2

I believe those caps should be supported, but the documentation is poor in this regard. Could this be down to how the rust plugins were compiled? I’ve included the relevant build steps from my Dockerfile in the details element below.

Dockerfile build steps 

RUN apt-get update && apt-get install -y \
  libgstreamer1.0-dev \
  libgstreamer-plugins-base1.0-dev \
  libgstreamer-plugins-bad1.0-dev \
  git \
  curl \
  pkg-config \
  libssl-dev \
  build-essential \
  libglib2.0-dev \
  && rm -rf /var/lib/apt/lists/*
RUN curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain nightly -y
ENV PATH="/root/.cargo/bin:${PATH}"
RUN cargo install cargo-c
RUN cd /root && \ 
  git clone https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs.git
RUN cd /root/gst-plugins-rs && \
  cargo cbuild -p gst-plugin-webrtc  --release --prefix=/usr

I’ve spent many hours bashing my head into the same limited set of problems, and am bewildered, and frustrated by the limited documentation available. Any insight would be appreciated.

2

The primary differences between whipsink and whipclientsink can be checked with gst-inspect and looking at the pad templates.

whipclientsink accepts raw or encoded streams. RTP payloading will be taken care of by the element.

  SINK template: 'audio_%u'
    Availability: On request
    Capabilities:
      audio/x-raw
      audio/x-opus
    Type: GstWebRTCSinkPad

  SINK template: 'video_%u'
    Availability: On request
    Capabilities:
      video/x-raw
      video/x-raw(memory:CUDAMemory)
      video/x-raw(memory:GLMemory)
      video/x-raw(memory:NVMM)
      video/x-raw(memory:D3D11Memory)
      video/x-vp8
      video/x-h264
      video/x-vp9
      video/x-h265
      video/x-av1
    Type: GstWebRTCSinkPad

whipsink accepted RTP payloaded streams.

Pad Templates:
  SINK template: 'sink_%u'
    Availability: On request
    Capabilities:
      application/x-rtp

Removing the RTP payloading when using whipclientsink should work.

3

Thanks for the pointer. Just going opusenc ! "audio/x-opus" ! whipclientsink... connects something together, and gets rid of the error, so I can mark this part as solved.

However, Cloudflare insists that no connection has been made. When I’ve tried using simple-whip-client (also based around GStreamer), I get 400 errors from Cloudflare’s WHIP endpoint to suggest that ICE connection has failed. So back to the drawing board - thanks for the reminder about gst-inspect.