Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable cross-language LTO for linux, windows and macos #806

Closed
3 tasks
NobodyXu opened this issue Feb 15, 2023 · 7 comments · Fixed by #817
Closed
3 tasks

Enable cross-language LTO for linux, windows and macos #806

NobodyXu opened this issue Feb 15, 2023 · 7 comments · Fixed by #817

Comments

@NobodyXu
Copy link
Member

NobodyXu commented Feb 15, 2023 •
edited
Loading

zstd already supports feature fat-lto and thin-lto to build libzstd with LTO.

All we need to do is to use clang and lld for all targets and any target uses cargo-zigbuild they already use clang.
#805 will switch to using rust-lld which will resolve the second todo.

I have also opened PR to add lto support to xz2 alexcrichton/xz2-rs#107 and bzip2 trifectatechfoundation/bzip2-rs#88

  • Linux
  • Macos
  • Windows
@NobodyXu NobodyXu mentioned this issue Feb 18, 2023
Merged
NobodyXu added a commit that referenced this issue Feb 19, 2023
@NobodyXu
Enable cross-lang-fat-lto on Linux (#817)
3f01076 
Fixed #806

 - Add new feature flag `cross-lang-fat-lto` and enable it on release for linux
 - Enable `-C linker-plugin-lto` for linux
 - Only use `-Z gcc-ld=lld` on non-windows targets when `cargo-zigbuild` is not enabled

Signed-off-by: Jiahao XU <[email protected]>
@NobodyXu NobodyXu changed the title Enable cross-language LTO Enable cross-language LTO for windows and macos Feb 19, 2023
@NobodyXu

This comment was marked as outdated.

Sign in to view
@NobodyXu NobodyXu reopened this Feb 19, 2023
@NobodyXu
Copy link
Member Author

NobodyXu commented Mar 3, 2023

Seems like default linker ld64 on MacOS can do LTO, so I might just try enabling it on release build.

@NobodyXu
Copy link
Member Author

NobodyXu commented Mar 5, 2023

Seems like default linker ld64 on MacOS can do LTO, so I might just try enabling it on release build.

I can't get this to work:

          ld: unknown option: -plugin-opt=O2
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

tried passing -flto but it seems that ld64 just does not support -plugin-opt

Tried lld but also failed due to the same error.

NobodyXu added a commit that referenced this issue Mar 10, 2023
@NobodyXu
Enable only feature cross-lang-fat-lto on MacOS
9752b7b 
while we cannot enable cross-lang-fat-lto on MacOS due to linker error
described [here], it turns out that enabling the feature
`cross-lang-fat-lto` does not break the build.

[here]: #806 (comment)

Signed-off-by: Jiahao XU <[email protected]>
@NobodyXu NobodyXu mentioned this issue Mar 10, 2023
Merged
@NobodyXu NobodyXu changed the title Enable cross-language LTO for windows and macos Enable cross-language LTO for linux, windows and macos Jun 2, 2023
@NobodyXu
Copy link
Member Author

NobodyXu commented Jun 2, 2023

#1113 temporarily disable linker-plugin-lto due to mismatch llvm version.

@NobodyXu
Copy link
Member Author

NobodyXu commented Jun 2, 2023

Since on linux we always uses cargo-zigbuild, so we need to wait for zig-cc to update to latest llvm.

@NobodyXu
Copy link
Member Author

NobodyXu commented Aug 4, 2023

zig v0.11.0 upgrades LLVM to 16, so now we are just waiting for cargo-zigbuild to update to zig v0.11.0 rust-cross/cargo-zigbuild#156

@NobodyXu
Copy link
Member Author

NobodyXu commented Aug 5, 2023

Since LLVM 17 is going to be released soon and cross-lang LTO is fragile due to these upgrades (old LLVM lld linker cannot parse new LLVM ir, possibly due to new IRs), I am going to close this as not planned.

@NobodyXu NobodyXu closed this as not planned Won't fix, can't repro, duplicate, stale Aug 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Enable cross-lang-fat-lto on Linux cargo-bins/cargo-binstall
1 participant
@NobodyXu