Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CMake] Enable CMP0179 alongside CMP0156 for deduplication on LLD #116497

Merged
merged 1 commit into from
Nov 21, 2024

Conversation

tambry
Copy link
Contributor

@tambry tambry commented Nov 16, 2024

LLD has a bug regarding ordering of static link libraries in the ELF backend, which has been reported as #116669.
CMake 3.31.0 started properly deduplicating static libraries for LLD causing the following linking failure for libclang-cpp.so with -DLLVM_LINK_LLVM_DYLIB=ON:

ld.lld: error: undefined symbol: llvm::omp::getOpenMPClauseName(llvm::omp::Clause)
>>> referenced by OpenMPKinds.cpp
>>>               tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/OpenMPKinds.cpp.o:(clang::getOpenMPSimpleClauseTypeName(llvm::omp::Clause, unsigned int))
>>> referenced by SemaOpenMP.cpp
>>>               tools/clang/lib/Sema/CMakeFiles/obj.clangSema.dir/SemaOpenMP.cpp.o:(clang::SemaOpenMP::CheckOMPRequiresDecl(clang::SourceLocation, llvm::ArrayRef<clang::OMPClause*>))
>>> referenced by SemaOpenMP.cpp
>>>               tools/clang/lib/Sema/CMakeFiles/obj.clangSema.dir/SemaOpenMP.cpp.o:(clang::SemaOpenMP::CheckOMPRequiresDecl(clang::SourceLocation, llvm::ArrayRef<clang::OMPClause*>))
>>> referenced 166 more times

[tons more]

CMake 3.31 also introduced CMP0179, which builds on CMP0156 and makes the deduplication consistent across platforms.
By coincidence this works around the above LLD deficiency and is the fix that CMake 3.31.1 will implement.
However, the fix is to ignore CMP0156 unless CMP0179 is also enabled, i.e. no more deduplication.
So enable CMP0179 to keep the benefits of deduplication from CMP0156 on LLD and fix the build for CMake 3.31.0.

See: #116669
See: https://gitlab.kitware.com/cmake/cmake/-/issues/26447
Fixes: cb90d5b

@tambry tambry added the cmake Build system in general and CMake in particular label Nov 16, 2024
@tambry tambry self-assigned this Nov 16, 2024
LLD has a bug regarding ordering of static link libraries in the ELF backend, which has been reported as llvm#116669.
CMake 3.31.0 started properly deduplicating static libraries for LLD causing the following linking failure for libclang-cpp.so with -DLLVM_LINK_LLVM_DYLIB=ON:
```
ld.lld: error: undefined symbol: llvm::omp::getOpenMPClauseName(llvm::omp::Clause)
>>> referenced by OpenMPKinds.cpp
>>>               tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/OpenMPKinds.cpp.o:(clang::getOpenMPSimpleClauseTypeName(llvm::omp::Clause, unsigned int))
>>> referenced by SemaOpenMP.cpp
>>>               tools/clang/lib/Sema/CMakeFiles/obj.clangSema.dir/SemaOpenMP.cpp.o:(clang::SemaOpenMP::CheckOMPRequiresDecl(clang::SourceLocation, llvm::ArrayRef<clang::OMPClause*>))
>>> referenced by SemaOpenMP.cpp
>>>               tools/clang/lib/Sema/CMakeFiles/obj.clangSema.dir/SemaOpenMP.cpp.o:(clang::SemaOpenMP::CheckOMPRequiresDecl(clang::SourceLocation, llvm::ArrayRef<clang::OMPClause*>))
>>> referenced 166 more times

[tons more]
```

CMake 3.31 also introduced CMP0179, which builds on CMP0156 and makes the deduplication consistent across platforms.
By coincidence this works around the above LLD deficiency and is the fix that CMake 3.31.1 will implement.
However, the fix is to ignore CMP0156 unless CMP0179 is also enabled, i.e. no more deduplication.
So enable CMP0179 to keep the benefits of deduplication from CMP0156 on LLD and fix the build for CMake 3.31.0.

See: llvm#116669
See: https://gitlab.kitware.com/cmake/cmake/-/issues/26447
Fixes: cb90d5b
@tambry tambry changed the title [CMake] Enable CMP0179 when enabling CMP0156 to avoid LLD static library link order bug [CMake] Enable CMP0179 alongside CMP0156 for deduplication on LLD Nov 18, 2024
@tambry
Copy link
Contributor Author

tambry commented Nov 18, 2024

LLD bug has now been reported as #116669 and I've reworded the commit message to hopefully make the effect and intent of this change clearer. Please do have another look.

CMake side has also decided on their solution of forcing CMP0156 to OLD for ELF platforms unless CMP0179 is also NEW.
This PR counteracts that to bring us back in line with what we want since CMP0179 doesn't affect us in other ways.

Copy link
Contributor

@boomanaiden154 boomanaiden154 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable enough to me. Someone else should approve this too before landing though.

@tambry
Copy link
Contributor Author

tambry commented Nov 21, 2024

Gentle ping @compnerd for another look.

Copy link
Member

@compnerd compnerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that LLD has ignored the requirements for ELF semantics. Please test with BFD. I expect that we need to disable this policy for BFD and gold to successfully link. We can add an additional case for that if you want.

if(APPLE OR WIN32 OR LLVM_ENABLE_LLD)
  ...
endif()

However, if BFD and gold are able to link properly ignoring the symbol resolution semantics (and the output is the same), then this seems reasonable.

@etcwilde
Copy link
Contributor

I understand that LLD has ignored the requirements for ELF semantics. Please test with BFD. I expect that we need to disable this policy for BFD and gold to successfully link. We can add an additional case for that if you want.

if(APPLE OR WIN32 OR LLVM_ENABLE_LLD)
  ...
endif()

However, if BFD and gold are able to link properly ignoring the symbol resolution semantics (and the output is the same), then this seems reasonable.

CMake added linker detection in 3.29 specifically so that it could handle these cases.
The addition of CMAKE_LINKER_TYPE was part of this. The expectation from CMP0156 is that it only de-dupes the libraries on the link line for linkers that don't require it.
Something regressed in 3.31 for LLD. CMP0156 and CMP0179 should not affect BFD or gold, as they do require repetition of the libraries.

Copy link
Contributor

@etcwilde etcwilde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine to me. Thanks

Copy link
Member

@compnerd compnerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd missed the "based on linker capabilities" bit for CMP0156. In the case that it is LLD, taking the first occurrence seems fine. The documentation states that this only applies to Apple, Windows, and LLD.

@tambry
Copy link
Contributor Author

tambry commented Nov 21, 2024

Thanks!

Something regressed in 3.31 for LLD.

Not quite.
CMake wasn't capable of detecting LLD backends before so it didn't apply the deduplication to it. CMake 3.31 added such detection and thus could apply the deduplication. However the LLD backend turned out to be not conformant to its own documentation on this, i.e. it wasn't actually capable of linking when deduplicated as specified. This has been reported as #116669.
CMake 3.31.1 will add a workaround to disable the deduplication for LLD's ELF backend unless CMP0179 is also enabled, which changes the behaviour to be consistent across platforms and happens to work around this LLD bug.

@tambry tambry merged commit 505e049 into llvm:main Nov 21, 2024
60 of 61 checks passed
@tambry tambry deleted the cmp0179 branch November 21, 2024 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cmake Build system in general and CMake in particular
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants