Pick the max DWARF version when LTO'ing modules with different versions #136659

wesleywiser · 2025-02-06T21:10:22Z

Currently, when rustc compiles code with -Clto enabled that was built
with different choices for -Zdwarf-version, a warning will be
reported. It's very easy to observe this by compiling most anything (eg,
"hello world") and specifying -Clto -Zdwarf-version=5 since the
standard library is distributed with -Zdwarf-version=4.

This behavior isn't actually useful for a few reasons:

From observation, LLVM chooses to pick the highest DWARF version
anyway after issuing the warning.
Clang specifies that in this case, the max version should be picked
without a warning and as a general principle, we want to support
x-lang LTO with Clang which implies using the same module flag merge
behaviors.
Debuggers need to be able to handle a variety of versions within the
same debugging session as you can easily have some parts of a binary
(or some dynamic libraries within an application) all compiled with
different DWARF versions.

This commit changes the module flag merge behavior to match Clang and
use the highest version of DWARF. It also adds a test to ensure this
behavior is respected in the case of two crates being LTO'd together and
adds a test to ensure no warning is printed.

Fixes #130041 which fails due to these warnings being printed

cc #103057

rustbot · 2025-02-06T21:10:31Z

r? @Noratrieb

rustbot has assigned @Noratrieb.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

compiler/rustc_codegen_llvm/src/debuginfo/mod.rs

jieyouxu

Thanks, I think this behavior is strictly more sane.

jieyouxu · 2025-02-07T17:46:58Z

r? jieyouxu @bors r+ rollup

bors · 2025-02-07T17:47:02Z

📌 Commit 12fdab0 has been approved by jieyouxu

It is now in the queue for this repository.

…e_behavior, r=jieyouxu Pick the max DWARF version when LTO'ing modules with different versions Currently, when rustc compiles code with `-Clto` enabled that was built with different choices for `-Zdwarf-version`, a warning will be reported. It's very easy to observe this by compiling most anything (eg, "hello world") and specifying `-Clto -Zdwarf-version=5` since the standard library is distributed with `-Zdwarf-version=4`. This behavior isn't actually useful for a few reasons: - From observation, LLVM chooses to pick the highest DWARF version anyway after issuing the warning. - Clang specifies that in this case, the max version should be picked without a warning and as a general principle, we want to support x-lang LTO with Clang which implies using the same module flag merge behaviors. - Debuggers need to be able to handle a variety of versions within the same debugging session as you can easily have some parts of a binary (or some dynamic libraries within an application) all compiled with different DWARF versions. This commit changes the module flag merge behavior to match Clang and use the highest version of DWARF. It also adds a test to ensure this behavior is respected in the case of two crates being LTO'd together and adds a test to ensure no warning is printed. Fixes rust-lang#130041 which fails due to these warnings being printed cc rust-lang#103057

…iaskrgr Rollup of 7 pull requests Successful merges: - rust-lang#136640 (Debuginfo for function ZSTs should have alignment of 8 bits, not 1 bit) - rust-lang#136648 (Add a missing `//@ needs-symlink` to `tests/run-make/libs-through-symlinks`) - rust-lang#136651 (Label mismatched parameters at the def site for foreign functions) - rust-lang#136659 (Pick the max DWARF version when LTO'ing modules with different versions ) - rust-lang#136691 (Remove Linkage::Private and Linkage::Appending) - rust-lang#136692 (add module level doc for bootstrap:utils:exec) - rust-lang#136700 (i686-unknown-hurd-gnu: bump baseline CPU to Pentium 4) r? `@ghost` `@rustbot` modify labels: rollup

matthiaskrgr · 2025-02-08T02:56:09Z

@bors r-
#136712 (comment)

tests/assembly/dwarf-mixed-versions-lto.rs

Currently, when rustc compiles code with `-Clto` enabled that was built with different choices for `-Zdwarf-version`, a warning will be reported. It's very easy to observe this by compiling most anything (eg, "hello world") and specifying `-Clto -Zdwarf-version=5` since the standard library is distributed with `-Zdwarf-version=4`. This behavior isn't actually useful for a few reasons: - from observation, LLVM chooses to pick the highest DWARF version anyway after issuing the warning - Clang specifies that in this case, the max version should be picked without a warning and as a general principle, we want to support x-lang LTO with Clang which implies using the same module flag merge behaviors - Debuggers need to be able to handle a variety of versions withing the same debugging session as you can easily have some parts of a binary (or some dynamic libraries within an application) all compiled with different DWARF versions This commit changes the module flag merge behavior to match Clang and use the highest version of DWARF. It also adds a test to ensure this behavior is respected in the case of two crates being LTO'd together and updates the test added in the previous commit to ensure no warning is printed.

jieyouxu

Thanks

jieyouxu · 2025-02-08T16:38:33Z

@bors r+ rollup=iffy

bors · 2025-02-08T16:38:35Z

📌 Commit bbc40e7 has been approved by jieyouxu

It is now in the queue for this repository.

…e_behavior, r=jieyouxu Pick the max DWARF version when LTO'ing modules with different versions Currently, when rustc compiles code with `-Clto` enabled that was built with different choices for `-Zdwarf-version`, a warning will be reported. It's very easy to observe this by compiling most anything (eg, "hello world") and specifying `-Clto -Zdwarf-version=5` since the standard library is distributed with `-Zdwarf-version=4`. This behavior isn't actually useful for a few reasons: - From observation, LLVM chooses to pick the highest DWARF version anyway after issuing the warning. - Clang specifies that in this case, the max version should be picked without a warning and as a general principle, we want to support x-lang LTO with Clang which implies using the same module flag merge behaviors. - Debuggers need to be able to handle a variety of versions within the same debugging session as you can easily have some parts of a binary (or some dynamic libraries within an application) all compiled with different DWARF versions. This commit changes the module flag merge behavior to match Clang and use the highest version of DWARF. It also adds a test to ensure this behavior is respected in the case of two crates being LTO'd together and adds a test to ensure no warning is printed. Fixes rust-lang#130041 which fails due to these warnings being printed cc rust-lang#103057

Rollup of 5 pull requests Successful merges: - rust-lang#134679 (Windows: remove readonly files) - rust-lang#136213 (Allow Rust to use a number of libc filesystem calls) - rust-lang#136530 (Implement `x perf` directly in bootstrap) - rust-lang#136601 (Detect (non-raw) borrows of null ZST pointers in CheckNull) - rust-lang#136659 (Pick the max DWARF version when LTO'ing modules with different versions ) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#136659 - wesleywiser:dwarf_version_lto_merge_behavior, r=jieyouxu Pick the max DWARF version when LTO'ing modules with different versions Currently, when rustc compiles code with `-Clto` enabled that was built with different choices for `-Zdwarf-version`, a warning will be reported. It's very easy to observe this by compiling most anything (eg, "hello world") and specifying `-Clto -Zdwarf-version=5` since the standard library is distributed with `-Zdwarf-version=4`. This behavior isn't actually useful for a few reasons: - From observation, LLVM chooses to pick the highest DWARF version anyway after issuing the warning. - Clang specifies that in this case, the max version should be picked without a warning and as a general principle, we want to support x-lang LTO with Clang which implies using the same module flag merge behaviors. - Debuggers need to be able to handle a variety of versions within the same debugging session as you can easily have some parts of a binary (or some dynamic libraries within an application) all compiled with different DWARF versions. This commit changes the module flag merge behavior to match Clang and use the highest version of DWARF. It also adds a test to ensure this behavior is respected in the case of two crates being LTO'd together and adds a test to ensure no warning is printed. Fixes rust-lang#130041 which fails due to these warnings being printed cc rust-lang#103057

Rollup of 5 pull requests Successful merges: - rust-lang#134679 (Windows: remove readonly files) - rust-lang#136213 (Allow Rust to use a number of libc filesystem calls) - rust-lang#136530 (Implement `x perf` directly in bootstrap) - rust-lang#136601 (Detect (non-raw) borrows of null ZST pointers in CheckNull) - rust-lang#136659 (Pick the max DWARF version when LTO'ing modules with different versions ) r? `@ghost` `@rustbot` modify labels: rollup

…n, r=petrochenkov Stabilize `-Zdwarf-version` as `-Cdwarf-version` I propose stabilizing `-Zdwarf-version` as `-Cdwarf-version`. This PR adds a new `-Cdwarf-version` flag, leaving the unstable `-Z` flag as is to ease the transition period. The `-Z` flag will be removed in the future. # `-Zdwarf-version` stabilization report ## What is the RFC for this feature and what changes have occurred to the user-facing design since the RFC was finalized? No RFC/MCP, this flag was added in rust-lang#98350 and was not deemed large enough to require additional process. The tracking issue for this feature is rust-lang#103057. ## What behavior are we committing to that has been controversial? Summarize the major arguments pro/con. None that has been extensively debated but there are a few questions that could have been chosen differently: 1. What should the flag name be? The current flag name is very specific to DWARF. Other debuginfo formats exist (msvc's CodeView format or https://en.wikipedia.org/wiki/Stabs) so we could have chosen to generalize the flag name (`-{C,Z} debuginfo-version=dwarf-5` for example). While this would extend cleanly to support formats other than DWARF, there are some downsides to this design. Neither CodeView nor Stabs have specification or format versions so it's not clear what values would be supported beyond `dwarf-{2,3,4,5}` or `codeview`. We would also need to take care to ensure the name does not lead users to think they can pick a format other than one supported by the target. For instance, what would `--target x86_64-pc-windows-msvc -Cdebuginfo-version=dwarf-5` do? 2. What is the behavior when flag is used on targets that do not support DWARF? Currently, passing `-{C,Z} dwarf-version` on targets like `*-windows-msvc` does not do anything. It may be preferable to emit a warning alerting the user that the flag has no effect on the target platform. Alternatively, we could emit an error but this could be annoying since it would require the use of target specific RUSTFLAGS to use the flag correctly (and there isn't a way to target "any platform that uses DWARF" using cfgs). 3. Does the precompiled standard library potentially using a different version of DWARF a problem? I don't believe this is an issue as debuggers (and other such tools) already must deal with the possibility that an application uses different DWARF versions across its statically or dynamically linked libraries. ## Are there extensions to this feature that remain unstable? How do we know that we are not accidentally committing to those. No extensions per se, although future DWARF versions could be considered as such. At present, we validate the requested DWARF version is between 2 and 5 (inclusive) so new DWARF versions will not automatically be supported until the validation logic is adjusted. ## Summarize the major parts of the implementation and provide links into the code (or to PRs) - Targets define their preferred or default DWARF version: https://github.com/rust-lang/rust/blob/34a5ea911c56e79bd451c63f04ea2f5023d7d1a3/compiler/rustc_target/src/spec/mod.rs#L2369 - We use the target default but this can be overriden by `-{C,Z} dwarf-version` https://github.com/rust-lang/rust/blob/34a5ea911c56e79bd451c63f04ea2f5023d7d1a3/compiler/rustc_session/src/session.rs#L738 - The flag is validated https://github.com/rust-lang/rust/blob/34a5ea911c56e79bd451c63f04ea2f5023d7d1a3/compiler/rustc_session/src/session.rs#L1253-L1258 - When debuginfo is generated, we tell LLVM to use the requested value or the target default https://github.com/rust-lang/rust/blob/34a5ea911c56e79bd451c63f04ea2f5023d7d1a3/compiler/rustc_codegen_llvm/src/debuginfo/mod.rs#L106 ## Summarize existing test coverage of this feature - Test that we actually generate the appropriate DWARF version - https://github.com/rust-lang/rust/blob/master/tests/assembly/dwarf5.rs - https://github.com/rust-lang/rust/blob/master/tests/assembly/dwarf4.rs - Test that LTO with different DWARF versions picks the highest version - https://github.com/rust-lang/rust/blob/master/tests/assembly/dwarf-mixed-versions-lto.rs - Test DWARF versions 2-5 are valid while 0, 1 and 6 report an error - https://github.com/rust-lang/rust/blob/master/tests/ui/debuginfo/dwarf-versions.rs - Ensure LLVM does not report a warning when LTO'ing different DWARF versions together - https://github.com/rust-lang/rust/blob/master/tests/ui/lto/dwarf-mixed-versions-lto.rs ## Has a call-for-testing period been conducted? If so, what feedback was received? No call-for-testing has been conducted but Rust for Linux has been using this flag without issue. ## What outstanding bugs in the issue tracker involve this feature? Are they stabilization-blocking? All reported bugs have been resolved. ## Summarize contributors to the feature by name for recognition and assuredness that people involved in the feature agree with stabilization - Initial implementation in rust-lang#98350 by `@pcwalton` - Stop emitting `.debug_pubnames` and `.debug_pubtypes` when using DWARF 5 in rust-lang#117962 by `@weihanglo.` - Refactoring & cleanups (rust-lang#135739), fix LLVM warning on LTO with different DWARF versions (rust-lang#136659) and argument validation (rust-lang#136746) by `@wesleywiser` ## What FIXMEs are still in the code for that feature and why is it ok to leave them there? No FIXMEs related to this feature. ## What static checks are done that are needed to prevent undefined behavior? This feature cannot cause undefined behavior. We ensure the DWARF version is one of the supported values [here](https://github.com/rust-lang/rust/blob/34a5ea911c56e79bd451c63f04ea2f5023d7d1a3/compiler/rustc_session/src/session.rs#L1255-L1257). ## In what way does this feature interact with the reference/specification, and are those edits prepared? No changes to reference/spec, unstable rustc docs are moved to the stable book as part of the stabilization PR. ## Does this feature introduce new expressions and can they produce temporaries? What are the lifetimes of those temporaries? No. ## What other unstable features may be exposed by this feature? `-Zembed-source` requires use of DWARF 5 extensions but has its own feature gate. ## What is tooling support like for this feature, w.r.t rustdoc, clippy, rust-analzyer, rustfmt, etc.? No support needed for rustdoc, clippy, rust-analyzer, rustfmt or rustup. Cargo could expose this as an option in build profiles but I would expect the decision as to what version should be used would be made for the entire crate graph at build time rather than by individual package authors. cc-rs has support for detecting the presence of `-{C,Z} dwarf-version` in `RUSTFLAGS` and providing the corresponding flag to Clang/gcc (rust-lang/cc-rs#1395). --- Closes rust-lang#103057

… r=petrochenkov Stabilize `-Zdwarf-version` as `-Cdwarf-version` I propose stabilizing `-Zdwarf-version` as `-Cdwarf-version`. This PR adds a new `-Cdwarf-version` flag, leaving the unstable `-Z` flag as is to ease the transition period. The `-Z` flag will be removed in the future. # `-Zdwarf-version` stabilization report ## What is the RFC for this feature and what changes have occurred to the user-facing design since the RFC was finalized? No RFC/MCP, this flag was added in rust-lang#98350 and was not deemed large enough to require additional process. The tracking issue for this feature is rust-lang#103057. ## What behavior are we committing to that has been controversial? Summarize the major arguments pro/con. None that has been extensively debated but there are a few questions that could have been chosen differently: 1. What should the flag name be? The current flag name is very specific to DWARF. Other debuginfo formats exist (msvc's CodeView format or https://en.wikipedia.org/wiki/Stabs) so we could have chosen to generalize the flag name (`-{C,Z} debuginfo-version=dwarf-5` for example). While this would extend cleanly to support formats other than DWARF, there are some downsides to this design. Neither CodeView nor Stabs have specification or format versions so it's not clear what values would be supported beyond `dwarf-{2,3,4,5}` or `codeview`. We would also need to take care to ensure the name does not lead users to think they can pick a format other than one supported by the target. For instance, what would `--target x86_64-pc-windows-msvc -Cdebuginfo-version=dwarf-5` do? 2. What is the behavior when flag is used on targets that do not support DWARF? Currently, passing `-{C,Z} dwarf-version` on targets like `*-windows-msvc` does not do anything. It may be preferable to emit a warning alerting the user that the flag has no effect on the target platform. Alternatively, we could emit an error but this could be annoying since it would require the use of target specific RUSTFLAGS to use the flag correctly (and there isn't a way to target "any platform that uses DWARF" using cfgs). 3. Does the precompiled standard library potentially using a different version of DWARF a problem? I don't believe this is an issue as debuggers (and other such tools) already must deal with the possibility that an application uses different DWARF versions across its statically or dynamically linked libraries. ## Are there extensions to this feature that remain unstable? How do we know that we are not accidentally committing to those. No extensions per se, although future DWARF versions could be considered as such. At present, we validate the requested DWARF version is between 2 and 5 (inclusive) so new DWARF versions will not automatically be supported until the validation logic is adjusted. ## Summarize the major parts of the implementation and provide links into the code (or to PRs) - Targets define their preferred or default DWARF version: https://github.com/rust-lang/rust/blob/34a5ea911c56e79bd451c63f04ea2f5023d7d1a3/compiler/rustc_target/src/spec/mod.rs#L2369 - We use the target default but this can be overriden by `-{C,Z} dwarf-version` https://github.com/rust-lang/rust/blob/34a5ea911c56e79bd451c63f04ea2f5023d7d1a3/compiler/rustc_session/src/session.rs#L738 - The flag is validated https://github.com/rust-lang/rust/blob/34a5ea911c56e79bd451c63f04ea2f5023d7d1a3/compiler/rustc_session/src/session.rs#L1253-L1258 - When debuginfo is generated, we tell LLVM to use the requested value or the target default https://github.com/rust-lang/rust/blob/34a5ea911c56e79bd451c63f04ea2f5023d7d1a3/compiler/rustc_codegen_llvm/src/debuginfo/mod.rs#L106 ## Summarize existing test coverage of this feature - Test that we actually generate the appropriate DWARF version - https://github.com/rust-lang/rust/blob/master/tests/assembly/dwarf5.rs - https://github.com/rust-lang/rust/blob/master/tests/assembly/dwarf4.rs - Test that LTO with different DWARF versions picks the highest version - https://github.com/rust-lang/rust/blob/master/tests/assembly/dwarf-mixed-versions-lto.rs - Test DWARF versions 2-5 are valid while 0, 1 and 6 report an error - https://github.com/rust-lang/rust/blob/master/tests/ui/debuginfo/dwarf-versions.rs - Ensure LLVM does not report a warning when LTO'ing different DWARF versions together - https://github.com/rust-lang/rust/blob/master/tests/ui/lto/dwarf-mixed-versions-lto.rs ## Has a call-for-testing period been conducted? If so, what feedback was received? No call-for-testing has been conducted but Rust for Linux has been using this flag without issue. ## What outstanding bugs in the issue tracker involve this feature? Are they stabilization-blocking? All reported bugs have been resolved. ## Summarize contributors to the feature by name for recognition and assuredness that people involved in the feature agree with stabilization - Initial implementation in rust-lang#98350 by `@pcwalton` - Stop emitting `.debug_pubnames` and `.debug_pubtypes` when using DWARF 5 in rust-lang#117962 by `@weihanglo.` - Refactoring & cleanups (rust-lang#135739), fix LLVM warning on LTO with different DWARF versions (rust-lang#136659) and argument validation (rust-lang#136746) by `@wesleywiser` ## What FIXMEs are still in the code for that feature and why is it ok to leave them there? No FIXMEs related to this feature. ## What static checks are done that are needed to prevent undefined behavior? This feature cannot cause undefined behavior. We ensure the DWARF version is one of the supported values [here](https://github.com/rust-lang/rust/blob/34a5ea911c56e79bd451c63f04ea2f5023d7d1a3/compiler/rustc_session/src/session.rs#L1255-L1257). ## In what way does this feature interact with the reference/specification, and are those edits prepared? No changes to reference/spec, unstable rustc docs are moved to the stable book as part of the stabilization PR. ## Does this feature introduce new expressions and can they produce temporaries? What are the lifetimes of those temporaries? No. ## What other unstable features may be exposed by this feature? `-Zembed-source` requires use of DWARF 5 extensions but has its own feature gate. ## What is tooling support like for this feature, w.r.t rustdoc, clippy, rust-analzyer, rustfmt, etc.? No support needed for rustdoc, clippy, rust-analyzer, rustfmt or rustup. Cargo could expose this as an option in build profiles but I would expect the decision as to what version should be used would be made for the entire crate graph at build time rather than by individual package authors. cc-rs has support for detecting the presence of `-{C,Z} dwarf-version` in `RUSTFLAGS` and providing the corresponding flag to Clang/gcc (rust-lang/cc-rs#1395). --- Closes rust-lang#103057

rustbot assigned Noratrieb Feb 6, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Feb 6, 2025

jieyouxu reviewed Feb 7, 2025

View reviewed changes

compiler/rustc_codegen_llvm/src/debuginfo/mod.rs Show resolved Hide resolved

jieyouxu added the A-debuginfo Area: Debugging information in compiled programs (DWARF, PDB, etc.) label Feb 7, 2025

jieyouxu approved these changes Feb 7, 2025

View reviewed changes

rustbot assigned jieyouxu Feb 7, 2025

rustbot unassigned Noratrieb Feb 7, 2025

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 7, 2025

matthiaskrgr mentioned this pull request Feb 7, 2025

Rollup of 7 pull requests #136712

Closed

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 8, 2025

jieyouxu reviewed Feb 8, 2025

View reviewed changes

tests/assembly/dwarf-mixed-versions-lto.rs Outdated Show resolved Hide resolved

wesleywiser added 2 commits February 8, 2025 16:33

Add tests for -Zdwarf-version lto behavior

cd3dd5b

wesleywiser force-pushed the dwarf_version_lto_merge_behavior branch from 12fdab0 to bbc40e7 Compare February 8, 2025 16:33

jieyouxu approved these changes Feb 8, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Feb 8, 2025

Urgau mentioned this pull request Feb 8, 2025

Rollup of 6 pull requests #136753

Closed

Urgau mentioned this pull request Feb 8, 2025

Rollup of 5 pull requests #136754

Merged

bors merged commit 5ec56e5 into rust-lang:master Feb 9, 2025
6 checks passed

rustbot added this to the 1.86.0 milestone Feb 9, 2025

This was referenced Feb 12, 2025

Tracking issue for -Zdwarf-version #103057

Closed

Stabilize -Zdwarf-version as -Cdwarf-version #136926

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pick the max DWARF version when LTO'ing modules with different versions #136659

Pick the max DWARF version when LTO'ing modules with different versions #136659

wesleywiser commented Feb 6, 2025 •

edited

Loading

rustbot commented Feb 6, 2025

jieyouxu left a comment

jieyouxu commented Feb 7, 2025

bors commented Feb 7, 2025

matthiaskrgr commented Feb 8, 2025

jieyouxu left a comment

jieyouxu commented Feb 8, 2025

bors commented Feb 8, 2025

Pick the max DWARF version when LTO'ing modules with different versions #136659

Pick the max DWARF version when LTO'ing modules with different versions #136659

Conversation

wesleywiser commented Feb 6, 2025 • edited Loading

rustbot commented Feb 6, 2025

jieyouxu left a comment

Choose a reason for hiding this comment

jieyouxu commented Feb 7, 2025

bors commented Feb 7, 2025

matthiaskrgr commented Feb 8, 2025

jieyouxu left a comment

Choose a reason for hiding this comment

jieyouxu commented Feb 8, 2025

bors commented Feb 8, 2025

wesleywiser commented Feb 6, 2025 •

edited

Loading