Skip to content

Fix autodiff builds on Apple CI #140137

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ZuseZ4 opened this issue Apr 22, 2025 · 2 comments
Open

Fix autodiff builds on Apple CI #140137

ZuseZ4 opened this issue Apr 22, 2025 · 2 comments
Labels
A-CI Area: Our Github Actions CI C-bug Category: This is a bug. E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue. F-autodiff `#![feature(autodiff)]` O-apple Operating system: Apple (macOS, iOS, tvOS, visionOS, watchOS) T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Comments

@ZuseZ4
Copy link
Member

ZuseZ4 commented Apr 22, 2025

Summary

We intend to enable autodiff next weekend.
Currently the dist-x86_64-linux and dist-aarch-linux runner succeeded, so we can ship autodiff in nightly for linux.
It would be nice, if std::autodiff would simultaneously become available for macos users.
For that, we'd want to fix the three apple dist builders, which all fail with the same error, of not finding the llvm dir, after building enzyme and llvm. This fails pretty late in the compilation pipeline, so I assume it's an easy fix, by just updating or adding the correct path to llvm.

The autodiff backend (which builds Enzyme) was originally introduced here: https://github.com/rust-lang/rust/pull/129176/files
The current implementations life at

// We want to link against registerEnzyme and in the future we want to use additional
and
.define("LLVM_DIR", builder.llvm_out(target));
.
Especially this commit fixed local MacOS builds, but as we can see that's not enough for CI yet.

You can test if your changes are successful through try builds (you'll need to ask a rustc member to run one for you).

To avoid having to guess the name of the right LLVM dir, you can probably download the logs of either of the 3 failed apple dist builds. I am not an expert here though, so I'd suggest to ping onur-ozkan who helped with previous autodiff bootstrap issues, or maybe jieyouxu (just add an @ before their name). Since it would be nice to have this as part of the first release and the fix should be quite small, I think it's fine if more than one person looks into it, so I think there's no need to claim it.

Command used

See https://github.com/rust-lang/rust/pull/140064/files

Expected behaviour

Build succeeds, similar to dist-x86_64-linux
#140064 (comment)

Actual behaviour

You can see the failing issue

failed try with plugins & llvm-enzyme enabled: dist-x86_64-apple (#140064 (comment))
failed try with plugins & llvm-enzyme enabled: dist-apple-various (#140064 (comment))
failed try with plugins & llvm-enzyme enabled: dist-aarch64-apple (#140064 (comment))

Bootstrap configuration (bootstrap.toml)

Varies a bit based on the runner, see #140064

Operating system

MacOS, different ones.

HEAD

HEAD

Additional context

Build Log

<log>

@ZuseZ4 ZuseZ4 added A-CI Area: Our Github Actions CI C-bug Category: This is a bug. E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue. F-autodiff `#![feature(autodiff)]` O-apple Operating system: Apple (macOS, iOS, tvOS, visionOS, watchOS) T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Apr 22, 2025
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Apr 22, 2025
@ZuseZ4 ZuseZ4 removed the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Apr 22, 2025
@ZuseZ4
Copy link
Member Author

ZuseZ4 commented Apr 22, 2025

I just found https://github.com/rust-lang/rustc-dev-guide/pull/1964/files#diff-2698d0add3d852fa6248afe666409fa1fd874e2226bfa58be18a9b815ed80b5a, and I think this vaguely matches the examples for when to reach out to them (sorry if it doesn't, please let me know). I hope someone might either know how paths or builds differ, or if there's a good way to reproduce this without having to re-run the slow try builds. Locally "it works for me".
@rustbot ping apple

@rustbot
Copy link
Collaborator

rustbot commented Apr 22, 2025

Hey Apple notification group! This issue or PR could use some Apple-specific
guidance. Could one of you weigh in? Thanks <3

(In case it's useful, here are some instructions for tackling these sorts of
issues).

cc @BlackHoleFox @hkratz @inflation @madsmtm @nvzqz @shepmaster @thomcc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-CI Area: Our Github Actions CI C-bug Category: This is a bug. E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue. F-autodiff `#![feature(autodiff)]` O-apple Operating system: Apple (macOS, iOS, tvOS, visionOS, watchOS) T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

No branches or pull requests

2 participants