[Discuss] CI Jobs for macOS, msvc and msys2

[lupyuen]

Is your feature request related to a problem? Please describe.

msvc build is broken. @xiaoxiang781216 has a suggestion:

• Enable One CI Job from each of macOS, msvc and msys2
• Which might prevent breakage of macOS / msvc / msys2 after merging
• If we don't have sufficient quota for macOS and Windows Jobs, we should reduce the Linux Arm32 Jobs

Describe the solution you'd like

I did some analysis of the msvc breakage:

• Our NuttX Dashboard shows that arm-13 is failing with the same error as msvc
• So actually we should enable more Linux Arm32 Builds (instead of Windows Builds)

But I shall still enable the Windows Builds and monitor the cost. Here's my computation:

• ASF has allocated a quota of 25 Full-Time Runners per week, we are using about 12 right now.
• Based on the Job Runtimes: https://github.com/NuttX/nuttx/actions/runs/11630100298/usage
• We estimate that msvc + msys2 will cost us 60 mins of GitHub Runner Minutes (Windows Runners cost twice as much as Linux Runners)
• So we might be OK to enable msvc + msys2 Builds
• But I will still shut them down if they overrun our quota

Sorry I can't enable macOS Builds right now:

• macOS Runners cost 10 times as much as Linux Runners. To enable One macOS Job, we need to disable 10 Linux Jobs! Which is not feasible.
• Our macOS Jobs are in a bad state right now, showing too many warnings. We need someone familiar with Intel Macs to clean up the macOS Jobs.
  https://github.com/NuttX/nuttx/actions/runs/11630100298/job/32388421934
  https://github.com/NuttX/nuttx/actions/runs/11630100298/job/32388422211

But can we still prevent breakage of Linux / macOS / msvc / msys2 Builds?

• Nope this is simply impossible. In the good old days: We were using far too many GitHub Runners. This is not sustainable, we don't have the budget to run all the CI Checks we used to.
• So we should expect some breakage to happen. We have to be prepared to backtrack and figure out which PR broke the build.
• That's why we have tools like the NuttX Dashboard, to detect breakage earlier without depending on GitHub CI.
• Also we should show some love and respect to NuttX Devs: Previously they waited 2.5 hours for All CI Checks. Now they wait at most 1.5 hours, I think we should stick to this.

Describe alternatives you've considered

No response

Verification

• I have verified before submitting the report.

[lupyuen]

NuttX Dashboard shows these broken builds, enabling the arm-08, arm-10, arm-13 builds will help:

• libs/arch_libc:Add kasan protection for memset and other mm api #14570 (comment)
• userspace: Exclude nuttx/arch.h #14351 (comment)

[xiaoxiang781216]

@lupyuen the plan is good to enable msvc and msys2, but disable macOS since macOS consume a lot of quota.

[lupyuen]

Yep fully agree, macOS Builds will only run in our Mirror Repo. So they won't be charged to NuttX. Thanks!