[ci] compilation failing on macOS-latest GitHub Actions jobs using gcc

[jameslamb]

Description

All CI jobs on macOS started failing some time in the last 12 hours, with the following error.

[100%] Linking CXX shared library /Users/runner/work/LightGBM/LightGBM/lib_lightgbm.so
0  0x10a416ffa  __assert_rtn + 139
1  0x10a24a28d  mach_o::relocatable::Parser<x86_64>::parse(mach_o::relocatable::ParserOptions const&) + 4989
2  0x10a23af8f  mach_o::relocatable::Parser<x86_64>::parse(unsigned char const*, unsigned long long, char const*, long, ld::File::Ordinal, mach_o::relocatable::ParserOptions const&) + 207
3  0x10a2b19d4  ld::tool::InputFiles::makeFile(Options::FileInfo const&, bool) + 2036
4  0x10a2b4fa0  ___ZN2ld4tool10InputFilesC2ER7Options_block_invoke + 48
5  0x7ff81923d34a  _dispatch_client_callout2 + 8
6  0x7ff81924e8f5  _dispatch_apply_invoke + 213
7  0x7ff81923d317  _dispatch_client_callout + 8
8  0x7ff81924cc0c  _dispatch_root_queue_drain + 673
9  0x7ff81924d25c  _dispatch_worker_thread2 + 160
10  0x7ff8193f0f8a  _pthread_wqthread + 256
A linker snapshot was created at:
	/tmp/lib_lightgbm.so-2022-11-15-050227.ld-snapshot
ld: Assertion failed: (_file->_atomsArrayCount == computedAtomCount && "more atoms allocated than expected"), function parse, file macho_relocatable_file.cpp, line 2061.
collect2: error: ld returned 1 exit status
make[3]: *** [/Users/runner/work/LightGBM/LightGBM/lib_lightgbm.so] Error 1
make[2]: *** [CMakeFiles/_lightgbm.dir/all] Error 2
make[1]: *** [CMakeFiles/_lightgbm.dir/rule] Error 2
make: *** [_lightgbm] Error 2

This occurs after compiling all objects, at the stage where the linker creates lib_lightgbm.so.

Reproducible example

For example, on #5588, all Python jobs on macOS are failing (link)

And all the two R-package jobs on macOS using gcc.

Additional Comments

It seems like the issue might be "the version of ld included with XCode 14.0 breaks compatibility with gcc": iains/gcc-12-branch#6.

Other related discussions:

• https://github.com/orgs/Homebrew/discussions/3659
• GCC 12.2.0 Error open-source-parsers/jsoncpp#1434
• https://stackoverflow.com/a/73765819/3986677
• https://developer.apple.com/forums/thread/714565
• versions of XCode available on GitHub Actions images: (actions/runner-images/)
• how to work with multiple versions of XCode: https://hacknicity.medium.com/working-with-multiple-versions-of-xcode-e331c01aa6bc

[jameslamb]

Things that did not work so far:

• compiling with -O2 instead of -O3
• switching from brew install gcc to brew install 'gcc@11'

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.