Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CURA-8415 Update installers and added self-hosted installer #16516

Merged
merged 37 commits into from
Aug 23, 2023

Conversation

jellespijker
Copy link
Member

@jellespijker jellespijker commented Aug 19, 2023

Description

The installer workflows have been seperated in OS specific workflows, all with the same inputs. They can be called with dispatch (UI) or from another workflow.

image

Please note that there are some invalid configurations possible, building the X64 won't work on the self-hosted Mac OS runner and vice-versa. We might want to consider dedicating another Macbook for X64, the same as and old Windows server for the msi and exe installers. Since this will bring down the overall build time from ~45 minutes to ~10

For getting the Mac binaries build with M1 support, an Macbook M1 has been converted to a self-hosted GH runner. See internal confluence documentation, regarding installed tooling, security practices and network logistics.

We support both msi and exe installers for Windows and pkg and dmg for Mac. These are now created in a single job, sequentially. Parallelizing the creation and signing these between runners/jobs had a negligible gain but introduced a lot of complexity. Since it is very likely that we will drop support for the exe and dmg I decided to keep this as is and not put any more effort in it. Also because single quick OS installers can be requested by executing the individual workflow.

If the "All Installer" workflow is run it will create the following artifacts:

  • Windows:
    • X64-msi
    • X64-exe
  • Linux-modern (glibc >= 2.35):
    • X64-AppImage
  • Linux-legacy (glibc < 2.35):
    • X64-AppImage
  • Macos
    • X64-dmg
    • X64-pkg
    • ARM64-dmg (self-hosted)
    • ARM64-pkg (self-hosted)
  • Chromebook
    • ARM64-AppImage (depends how bored I will get this weekend

Each OS run will also show a summary of al conan and pip managed dependencies.

The https://github.com/Ultimaker/conan-config now has branches in the form of runner/<runner.os>/<runner.arch> to allow for runner specific overrides in the conan-config without polluting the master branch for all (community-)devs and other runners. These are a subset of the master branch config and will be appended on.

The MacOS builds should be backward compatible, due to the simple usage of MacOS 12 which somehow still results in unfindable coroutine headers as they are still located in experimental. This commit should fix that: Ultimaker/CuraEngine@3ac71c0

If we need to upgrade to MacOS 13 in order to use newer XCode (with full support for C++20/23) setting the following env variables should still ensure backward compatibility (atleast for the Python interp) #11841 (comment) . To minimize possible side-effects they're only set for the MacOS runners (see earlier comment for runner specific configurations)

cpython:ac_cv_func_mkfifoat=no
cpython:ac_cv_func_mknodat=no

Fixes #13135
Fixes #15833
Fixes #11841

An example run can be seen here: https://github.com/Ultimaker/Cura/actions/runs/5912156075

If we ever need to start from scratch,
temporarily remove `&& inputs.runner != 'self-hosted'`

Contributes to CURA-8415
Why should we build the installer twice?

Contributes to CURA-8415
# Conflicts:
#	.github/workflows/macos.yml
Contributes to CURA-8415
Contributes to CURA-8415
Contributes to CURA-8415
Zipping -> uploading -> downloading unzipping cost the same amount of time,
while it also increased the complexity.

Contributes to CURA-8415
@jellespijker jellespijker changed the title Cura 8415 self hosted runner CURA-8415 Aug 19, 2023
@jellespijker jellespijker changed the title CURA-8415 CURA-8415 Update installers and added self-hosted installer Aug 19, 2023
@casperlamboo casperlamboo self-assigned this Aug 23, 2023
packaging/MacOS/build_macos.py Outdated Show resolved Hide resolved
packaging/MacOS/build_macos.py Show resolved Hide resolved
@casperlamboo casperlamboo merged commit fce35b7 into main Aug 23, 2023
@rburema rburema deleted the CURA-8415_self_hosted_runner branch September 6, 2023 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants