You may have a look to: Stream frame opencv via rtsp using gstreamer c++ - #6 by Honey_Patouceul for an example of RTSP streaming from OpenCV application.
For color, VideoWriter would expect BGR only, so assuming you’re reading RGBA frames from sl, you would convert into BGR instead of RGB with:
image_BGR = cv2.cvtColor(image_ocv_out, cv2.COLOR_RGBA2BGR)
If you want to use NVENC, as writer pipeline you may try:
rtph264_writer = cv2.VideoWriter('appsrc ! video/x-raw,format=BGR ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc insert-sps-pps=1 insert-vui=1 idrinterval=30 ! queue ! h264parse ! rtph264pay ! udpsink host=127.0.0.1 port=5004', cv2.CAP_GSTREAMER, 0, float(fps), (w,h))