GST-Plugin-Scanner only searches for x86 python

I am currently facing an issue during a multi-arch build with docker. The images are build with buildx for arm64 and amd64 (x86_64). I get an issue with loading my Python plugins and discovered that gst-plugin-scanner is looking at the wrong location.

** (gst-plugin-scanner:61): CRITICAL **: 07:12:03.675: Couldn't g_module_open libpython. Reason: /usr/lib/x86_64-linux-gnu/libpython3.10.so: cannot open shared object file: No such file or direct
ory

It should be

/usr/lib/aarch64-linux-gnu/libpython3.10.so

Is there any ENV variable I can set?