TL;DR: We’re trying to automate the building of Windows MSIs using cerbero in a github action while trying to leverage the standard
windows-latestimage rather than a self-hosted real machine. The work is over here in case anyone is interested in doing the same thing or already has. If you have interest in doing this, feel free to have a look; my efforts are on the1.24-lldc-btg-testingbranch. Our current issue is that x264 seems to generate bad files in configure and then fail at compile time, but the logs and source directory artifact are not illuminating the problem, and I could use some help if anyone has seen the issue.
We also maintain a fork on github of cerbero, and I have been adding a few small patches as I try to push this effort forward, but I’m hitting a lot of odd snags that we don’t see when trying to build in our command line environment or using a self-hosted runner. I would be happy to go into some of those details, but the current hiccup is that the x264 library fails to compile, like this:
Running command ['C:/mingw64/bin/make.EXE', 'V=1'] in C:/a/cerbero-action/cerbero-action/cerbero/cerbero-build/sources/msvc_x86_64/x264-20191217-2245
cat common/opencl/x264-cl.h common/opencl/bidir.cl common/opencl/downscale.cl common/opencl/weightp.cl common/opencl/intra.cl common/opencl/subpel.cl common/opencl/motionsearch.cl | ./tools/cltostr.sh common/oclobj.h
dependency file generation...
x86_64-w64-mingw32-gcc.exe: error: macro name missing after '-D'
x86_64-w64-mingw32-gcc.exe: fatal error: no input files
compilation terminated.
make.EXE: *** [Makefile:314: .depend] Error 1
The log doesn’t give any extra information beyond that and the source directory saved off from the failure doesn’t show any -D... args that are missing names. For example, the config.mak shows all CFLAGS args are fully formed.
Out of the other forks on github, I’ve seen that AMDGlobal has a github action defined but it’s unclear if it’s working at all for them since there’s no actions history posted (I tried replicating it; it did not work for me but it had a similar initial error as my first attempt).
Much of what I’ve dealt with is msys2 configuration. The default windows-latest base image that Microsoft/GitHub puts out already contains msys2 at the usual c:\msys64 location but it’s not on the PATH. Trying to directly use it by adding a step to expose that to the GITHUB_PATH has the odd effect of seeing things like cerbero-uninstalled “succeed” without doing anything – no errors logged, etc. It just quits. My action deletes that and uses the github action for installing MSYS2. This causes it to install in a non-standard location, but it’s visible from the path, so I modified cerbero to identify the parent directory of msys2.exe when it’s trying to find the MSYS installation.
Another gotcha was that the image by default contains Python 3.9, but has readily available newer versions. Cerbero only enforces 3.7 even though it uses a glob feature introduced in 3.10, which causes it to error out. I patched that in our local branch as well, and per the actions/setup-python and windows-latest design, using that action allows one to upgrade python “in place” to >=3.10 without causing some shadow/extra installation of python to exist in the image.
Another python-related thing is that the expectation that there will be a Scripts directory doesn’t seem to get met in the windows runner, which causes the attempt to delete that directory to fail after trying to move everything into bin. Patch here. I’ve tried looking up the commits related to that and I know there’s some obscure convention for why Scripts vs. bin, but what I have not found is why cerbro is having this hard expectation that by the time it arrives at that line of code that there will in-fact be a Scripts/... directory present on the system. It leaves me wondering if some other script/code generation was supposed to happen but was skipped, leading to a lot of this other mess. I see this issue by @nirbheek; I suspect it is related, but it is also clear that investigation is still in progress by @amyspark.
Similarly, the version of Perl in windows-latest appears to cause problems that can be mitigated by amending the actions/setup-msys2 to install perl.
The cerbero-defined version of wavpack also fails to build because of some interaction between ninja, cmake, that produces a bad $-escape error message (issue here). Bumping the version of wavpack to 5.7 solves that but I don’t see why from the change lists between versions. Credit to @jawilson for this patch.
All of these changes got me to where I’m at now, where x264 fails on an apparent file generation problem likely during the configure step but with no apparent build artifact pointing directly at the issue.
Assuming we get this figured out, I do plan to upstream any necessary changes to cerbero, but I’m holding off until the dust settles enough to where pointing the action at the upstream gitlab cerbero repository works.
Thank you for taking the time to read this.