PROJECT: Restructure Autobuild

[ann0see]

Currently the auto build process is fragile, error prone and messy. @pljones wrote an idea to re-structure the autobuild as follows:

---

1. Pull the repo to somewhere subsequent tasks can access it

For Each executable format - meaning Win-32, ARM64-Android version xyz, Linux-64, whatever: platform+binary combination - one at a time
2) Clang format check
3) Build the binary
End For

For Each packaging target (i.e. we bundle multiple binaries for one installer / deployment)
5) Build the deployable artifact
End For

6. if it's a release, push the deployables to where they should live
   The above is just to see if running the builds in series rather than parallel cuts failures
   But it also helps if we want to use multiple deployment schemes -- like supporting DEB and RPM on Linux, we should only need to build the binaries once, if we're doing static builds. And it should ease things even if we're not - so long as resolving the dependecies isn't too hard.

---

What are the problems at the moment?
How should a clean build process look like? Which features does it need (doesn't it need)? What do we need to change?

The goal of this discussion is to find design errors and get a todo list for how a new process needs to look like.

CC: @hoffie @nefarius2001 @pljones @henkdegroot (not sure who was also involved except me.)

[ann0see]

Comment by hoffie:
It's too many layers, too many abstractions in places where we thought we might need them but now don't. Versions and directories are hardcoded in multiple places. Jobs run with full-permission GITHUB_SECRETs

[hoffie]

FYI I'm working on bigger refactorings of the whole Autobuild logic. I'm not changing anything fundamentally (this can come later, if required), I'm just aiming at making any extensions simpler and more readable. I'm dropping lots of unused abstractions and wrappers. There's going to be one script per platform which calls different bash/powershell functions for the different stages. This saves loads of copy-pasted boilerplate code.

If anyone plans similar changes right now, we should probably coordinate.

[hoffie]

Just a short update: Lots of the prerequisites have already been submitted or even merged:

• Build: Parallelize compilation on Android, Mac (CodeQL) and Debian #2444
• Build: Use jom to parallelize Windows builds #2445
• Autobuild: Clean up workflow description #2470
• Autobuild: Cleanup analyze_git_references.py #2471
• Android: Use C++17, update to liboboe 1.6.1, improve oboe source handling #2472
• Autobuild: Stop installing unneeded debian packages (Linux/Android) #2473
• Autobuild: Reorganize macOS/iOS build scripts #2476
• Android: Use aqtinstall for Qt installation #2497
• Autobuild: Reduce Qt download size & bump aqtinstall to 2.0.6 #2498
• Autobuild: Don't cache pip #2499

The largest part (diff-wise) is going to be submitted soon as well.
Preview for Windows: https://github.com/jamulussoftware/jamulus/compare/master...hoffie:autobuild-merge-scripts?expand=1

After that, my priorities for 3.9.0 are:

• Provide Debian/Ubuntu ARM builds #1452
• Code: Add Qt6 compatibility #2299
• Native Apple M1 releases (arm64) #2357

[hoffie]

Now tracked here: #2503

[ann0see]

Currently the autobuild is also wasteful:

There's no need to e.g. build JACK for windows on every PR commit. I'd prefer a simple compile check for each commit including CodeQL. Only build like for a release on beta, rc and release tags. This means:

1. Do not build JACK on every PR (maybe on merge to master)
2. Do not build legacy mac on every PR
3. Maybe even android could be skipped (but that's worth a discussion)

Do not use multiple scripts to set up the build environment:

I suggested looking into docker, but we might need to update it every time then.

[hoffie]

Opting to skip certain builds (i.e. legacy mac) risks PRs that break backwards compatibility without us noticing on PR merge. This would then come up during (pre-)release time, which might introduce delays.

I suggested looking into docker, but we might need to update it every time then.

A jamulus-build docker container would mainly help with making the build environment setup more explicit and static.
I'm hoping to get similar benefits by using actions/cache (#2277, https://github.com/hoffie/jamulus/tree/cache-autobuild-deps), which seems like less one-time and ongoing maintenance work.

[pljones]

Yeah, if we support a set of targets, unfortunately I think we have to build each target each time -- otherwise you risk:
a) the point of the build was the target you omit
b) something broke and you don't find it
... etc...

So I think we just need to break what we've got down into more manageable pieces.

[hoffie]

@ann0see #2526 (comment)

Worth a separate issue, but this should also be somehow restructured (e.g. build/gui and build/nox (better build/headless))

Not sure if you were suggesting this for resource usage reasons. Splitting the .deb building into two jobs might allow them to complete faster (is that necessary? I think, Linux might be the fastest build usually?). Overall, it will increase resource usage though as we'd need to spin up/use two VMs now, install dependencies two times, etc. I'm not against that separation, I just want to make sure that we do it for a specific reason. :)

[ann0see]

1. nefarius wanted the split (but I get your point, so probably dismiss this idea)
2. Nevertheless, build-gui and build-nox should not be in the root but in a build/ folder to be similar to Windows (and macOS?)

[ann0see]

Features I'd like to see:

Automated tests not only based on styling and compilation, but also on functionality

Audio check:

1. Spin up a server and a client
2. Send audio between server and client locally
3. Check if audio can be received

Chat check (or protocol check?)

1. Send chat messages and try to receive them. In theory this could later be done via API (but that would probably not be a real world test case)

Probably this kind of testing is complicated since it requires a VM with JACK and probably a bunch of testing code.

[hoffie]

That would be really cool, but certainly a different sub-project due to scope :)

[ann0see]

Yep.

[pljones]

Yep.

[ann0see]

Also related: #1921 (comment)

[jujudusud]

Don't know if it is the right place to talk about that but, there is a 64 bits version of Raspbian/Rapsberry Pi OS.
My thought is that we also have to provide a package for that version.

[hoffie]

#2526 adds the base for 32bit builds. We should check if those work on 64bit systems as well (I don't know whether they'd work CPU-wise similar to x86/amd64 and I don't know whether they'd work library-wise, i.e. if typical Raspberry distributions would include the necessary 32bit libraries).

@dcorson-ticino-com seems to be currently working on this based on the Chat. It seems that there are hurdles (armhf libs not available by default). It may be possible (like that: https://github.com/jamulussoftware/jamulus/pull/2526/files#diff-0b1aca041de3f0eba2ce24c52de0dec74b0ffe4fc32d3d8518737fcdf6925c2fR38-R40), but it may also be a hassle. Let's the what the further tests show).

If we really have to [1], adding aarch64 is probably possible as well and might be as simple as replacing armhf with aarch64.

[1] I'd still say it's "nice to have", not "we have to" ;))

[hoffie]

I'd still be interested if the 32bit builds would be sufficient (in order to save resources, @ann0see ;)), so it would be good if any testers could report back on that.

In addition, I've run a test build with aarch64 here: https://github.com/hoffie/jamulus/actions/runs/2012848366

[dcorson-ticino-com]

The ARM64 build installs and runs without a problem on my Odroid N2.