Add custom/input events (rs)

General Discussion
1

I have this code creating new custom upstream event in python, can you please help me to translate that in rust ?

I know I can add the event via rust this way:

// Send the event
        let event = gst::event::Navigation::new(custom_structure);
        self.pipeline().send_event(event);

But I can’t create the custom structure as worked in python:

def add_devices(self): 
    if not self.set_devices: print("add_devices") 
       self.set_devices = True 
       custom_structure = Gst.Structure.new_empty("VirtualDevicesReady") 
       value = GObject.Value(GObject.ValueArray) 
       value.set_value([ "/dev/input/event6", 
           "/dev/input/event7", 
           "/dev/input/event8", ]) 
       
        custom_structure.set_value("paths", value) event = Gst.Event.new_custom(Gst.EventType.CUSTOM_UPSTREAM, custom_structure) 
       
       self.pipe.send_event(event)
2

You would do something like (untested)

let mut paths = glib::ValueArray::new();
paths.append(glib::Value::from("/dev/input/event6");
paths.append(glib::Value::from("/dev/input/event7");
paths.append(glib::Value::from("/dev/input/event8");

let s = gst::Structure::builder("VirtualDevicesReady")
    .field("paths", paths)
    .build();

let event = gst::event::CustomUpstream::new(s);

self.pipeline().send_event(event);

With glib: Improve `ValueArray` API, add tests and assertions for invalid … by sdroege · Pull Request #1411 · gtk-rs/gtk-rs-core · GitHub the construction of the value array can also be written a bit easier

let paths = glib::ValueArray::new(["/dev/input/event6", "/dev/input/event7", "/dev/input/event8"]);
3

awesome stuff!!

I change it a bit to this to make it compile:

//
            let mut paths = glib::ValueArray::new(/* u32 */);
            let paths = glib::ValueArray::new(["/dev/input/event6", "/dev/input/event7", "/dev/input/event8"]);

            let s = gst::Structure::builder("VirtualDevicesReady")
                .field("paths", paths)
                .build();

            let event = gst::event::CustomUpstream::new(s);

            self.pipeline().send_event(event);
//

Just one error ;

error[E0277]: `*mut ValueArray` cannot be sent between threads safely
    --> net/webrtc/examples/webrtc-send.rs:284:33
     |
284  |                 .field("paths", paths)
     |                  -----          ^^^^^ `*mut ValueArray` cannot be sent between threads safely
     |                  |
     |                  required by a bound introduced by this call
     |
     = help: within `ValueArray`, the trait `std::marker::Send` is not implemented for `*mut ValueArray`, which is required by `ValueArray: std::marker::Send`
note: required because it appears within the type `PhantomData<*mut ValueArray>`
    --> /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/core/src/marker.rs:740:12
note: required because it appears within the type `Boxed<GValueArray, ValueArray>`

BTW: Im adding this function to =

4

Ah I see, yes that won’t work because glib::ValueArray can contain also non-threadsafe types but gst::Structure always has to contain thread-safe types. That would need a bit of unsafe code to promise to the compiler that your array only contains thread-safe types :slight_smile:

Is there a reason why you use glib::ValueArray (which is deprecated since a decade) instead of gst::Array?

In any case, this here would work

             let s = gst::Structure::builder("VirtualDevicesReady")
-                .field("paths", paths)
+                .field("paths", unsafe { glib::SendValue::unsafe_from(glib::Value::from(paths).into_raw()) }) // Safe because all values contained in the array are `Send`
                 .build();
5

thanks for that.
I’m actually not sure why I need to use the glib::ValueArray , I assumed I needed that based on the inital python sample code.

Following your advice I changed the function to use gst:Array instead and its now compiling and it seems nice and simple

async fn add_devices(&self) {

            println!("add_devices");
            //self.set_devices = true;

            let paths = Array::new(["/dev/input/event6", "/dev/input/event7", "/dev/input/event8"]);

            let s = gst::Structure::builder("VirtualDevicesReady")
                .field("paths", paths)
                .build();

            let event = gst::event::CustomUpstream::new(s);

            self.pipeline().send_event(event);
    }

Unfortunately I can’t see any mouse pointer in the stream as I do when I run the python code.

I know the /dev/input/event6-8 are holding the mouse/keyboard events as they are generated by a different API (GitHub - games-on-whales/inputtino: A virtual input library: supports mouse, keyboard, joypad, trackpad and more) , I still missing something to get it working with webrtcsink or there maybe better way to this in rust ?

6

I think I know now why it needs the glib:ValueArray . Im getting this error:
thread ‘’ panicked at gst-plugin-wayland-display/src/waylandsrc/imp.rs:200:22:
Should contain paths: ValueGetError { name: “paths”, error: WrongValueType(ValueTypeMismatchError { actual: GstValueArray, requested: GValueArray }) }
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

7

I manged to update the code from your previous example and it works using the old/deprecated libs:

 fn add_devices(&self) {

            println!("add_devices");
            //self.set_devices = true;

            let mut paths = glib::ValueArray::new(3);
            paths.append(&glib::Value::from("/dev/input/event6"));
            paths.append(&glib::Value::from("/dev/input/event7"));
            paths.append(&glib::Value::from("/dev/input/event8"));

            let s = gst::Structure::builder("VirtualDevicesReady")
                .field("paths", unsafe { glib::SendValue::unsafe_from(glib::Value::from(paths).into_raw()) }) 
                .build();

            let event = gst::event::CustomUpstream::new(s);

            self.pipeline().send_event(event);
    }

that add the inputs and I can see mouse/keyboard working via stream

now it would be really nice to replace those with gst::Array.

Thanks!!

1 Like