Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mangle rustc_std_internal_symbols functions #127173

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

bjorn3
Copy link
Member

@bjorn3 bjorn3 commented Jun 30, 2024

This reduces the risk of issues when using a staticlib or rust dylib compiled with a different rustc version in a rust program. Currently this will either (in the case of staticlib) cause a linker error due to duplicate symbol definitions, or (in the case of rust dylibs) cause rustc_std_internal_symbols functions to be silently overridden. As rust gets more commonly used inside the implementation of libraries consumed with a C interface (like Spidermonkey, Ruby YJIT (curently has to do partial linking of all rust code to hide all symbols not part of the C api), the Rusticl OpenCL implementation in mesa) this is becoming much more of an issue. With this PR the only symbols remaining with an unmangled name are rust_eh_personality (LLVM doesn't allow renaming it) and __rust_no_alloc_shim_is_unstable.

Helps mitigate #104707

try-job: aarch64-gnu-debug
try-job: aarch64-apple
try-job: x86_64-apple-1
try-job: x86_64-mingw-1
try-job: i686-mingw-1
try-job: x86_64-msvc-1
try-job: i686-msvc-1
try-job: test-various
try-job: armhf-gnu

@rustbot
Copy link
Collaborator

rustbot commented Jun 30, 2024

r? @fee1-dead

rustbot has assigned @fee1-dead.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jun 30, 2024
@rustbot
Copy link
Collaborator

rustbot commented Jun 30, 2024

Some changes occurred in compiler/rustc_codegen_gcc

cc @antoyo, @GuillaumeGomez

@rust-log-analyzer

This comment has been minimized.

@bjorn3 bjorn3 force-pushed the mangle_rustc_std_internal_symbol branch from aea9a3c to db57b2a Compare June 30, 2024 16:50
@rust-log-analyzer

This comment has been minimized.

@bjorn3 bjorn3 force-pushed the mangle_rustc_std_internal_symbol branch from db57b2a to 4859689 Compare June 30, 2024 17:27
@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented Jun 30, 2024

☔ The latest upstream changes (presumably #127162) made this pull request unmergeable. Please resolve the merge conflicts.

@bjorn3 bjorn3 force-pushed the mangle_rustc_std_internal_symbol branch from 4859689 to 10bad43 Compare June 30, 2024 19:57
@rust-log-analyzer

This comment has been minimized.

@rustbot
Copy link
Collaborator

rustbot commented Jun 30, 2024

The Miri subtree was changed

cc @rust-lang/miri

@rust-log-analyzer

This comment has been minimized.

@fee1-dead
Copy link
Member

r? compiler

@bjorn3
Copy link
Member Author

bjorn3 commented Jul 1, 2024

Opened rust-lang/miri#3724 with some miri changes that can be made independently of this PR.

bors added a commit to rust-lang/miri that referenced this pull request Jul 2, 2024
Use the symbol_name query instead of trying to infer from the link_name attribute

This prevents the calculated name from going out of sync with exported_symbols. It also avoids having to special case the panic_impl lang item.

It also makes it easier to fix miri with rust-lang/rust#127173.
@bors
Copy link
Contributor

bors commented Jul 3, 2024

☔ The latest upstream changes (presumably #125507) made this pull request unmergeable. Please resolve the merge conflicts.

RalfJung pushed a commit to RalfJung/rust that referenced this pull request Jul 4, 2024
Use the symbol_name query instead of trying to infer from the link_name attribute

This prevents the calculated name from going out of sync with exported_symbols. It also avoids having to special case the panic_impl lang item.

It also makes it easier to fix miri with rust-lang#127173.
@bjorn3 bjorn3 force-pushed the mangle_rustc_std_internal_symbol branch from e1d353a to f8f4b88 Compare July 5, 2024 11:55
@rust-log-analyzer

This comment has been minimized.

@bjorn3 bjorn3 force-pushed the mangle_rustc_std_internal_symbol branch from f8f4b88 to 266c7c2 Compare July 5, 2024 12:56
@rust-log-analyzer

This comment has been minimized.

@bjorn3 bjorn3 force-pushed the mangle_rustc_std_internal_symbol branch from 266c7c2 to 9c91546 Compare July 5, 2024 13:35
@bjorn3
Copy link
Member Author

bjorn3 commented Feb 26, 2025

Changed the test to only run on Linux. It isn't reasonable to exhaustively list every unmangled symbol that the platform toolchain may add.

@rustbot ready

Copy link
Member

@jieyouxu jieyouxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two nits, then feel free to r= Wesley and me.

@jieyouxu
Copy link
Member

jieyouxu commented Mar 3, 2025

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 3, 2025
@bjorn3 bjorn3 force-pushed the mangle_rustc_std_internal_symbol branch from ded8d10 to e377765 Compare March 3, 2025 12:50
@bjorn3
Copy link
Member Author

bjorn3 commented Mar 3, 2025

@bors r=wesleywiser,jieyouxu

@bors
Copy link
Contributor

bors commented Mar 3, 2025

📌 Commit e377765 has been approved by wesleywiser,jieyouxu

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 3, 2025
@jieyouxu
Copy link
Member

jieyouxu commented Mar 3, 2025

@bors p=6 (threading between rollups)

@matthiaskrgr
Copy link
Member

@bors p=0 lets not do that, we already have around 70 PRs in queue and should make some progress to get that down a bit

bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 4, 2025
…bol, r=wesleywiser,jieyouxu

Mangle rustc_std_internal_symbols functions

This reduces the risk of issues when using a staticlib or rust dylib compiled with a different rustc version in a rust program. Currently this will either (in the case of staticlib) cause a linker error due to duplicate symbol definitions, or (in the case of rust dylibs) cause rustc_std_internal_symbols functions to be silently overridden. As rust gets more commonly used inside the implementation of libraries consumed with a C interface (like Spidermonkey, Ruby YJIT (curently has to do partial linking of all rust code to hide all symbols not part of the C api), the Rusticl OpenCL implementation in mesa) this is becoming much more of an issue. With this PR the only symbols remaining with an unmangled name are rust_eh_personality (LLVM doesn't allow renaming it) and `__rust_no_alloc_shim_is_unstable`.

Helps mitigate rust-lang#104707
@bors
Copy link
Contributor

bors commented Mar 4, 2025

⌛ Testing commit e377765 with merge 0d8f13a...

@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented Mar 4, 2025

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Mar 4, 2025
@bjorn3
Copy link
Member Author

bjorn3 commented Mar 4, 2025

I can't reproduce this locally on arm64 linux. Neither with nor without lld.

@jieyouxu
Copy link
Member

jieyouxu commented Mar 4, 2025

I can't reproduce this locally on arm64 linux. Neither with nor without lld.

I'm not 100% sure, but maybe this symbol comes from LLVM compiler-rt?

https://github.com/llvm/llvm-project/blob/794ba171a43baaf9de9739440ddb582500527031/compiler-rt/lib/builtins/cpu_model/aarch64.c#L59-L64

Though the symbol visibility of that is hidden so idk.

For staticlib we still keep checking symbols that don't contain rust as
substring.
@bjorn3
Copy link
Member Author

bjorn3 commented Mar 4, 2025

Though the symbol visibility of that is hidden so idk.

Looks like symbol.is_global() doesn't take symbol visibility into account, just linkage.

I've changed symbols_check to ignore symbols that don't have rust in the name, but keep symbols_check_archive check all symbols. I've also changed the test back to running on all targets.

@bjorn3 bjorn3 force-pushed the mangle_rustc_std_internal_symbol branch from e377765 to bf70cce Compare March 4, 2025 16:13
@jieyouxu
Copy link
Member

jieyouxu commented Mar 4, 2025

That seems reasonble. I'd like to see if there's any nasty surprises since the queue is currently backed up a lot, so

@bors try

bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 4, 2025
…bol, r=<try>

Mangle rustc_std_internal_symbols functions

This reduces the risk of issues when using a staticlib or rust dylib compiled with a different rustc version in a rust program. Currently this will either (in the case of staticlib) cause a linker error due to duplicate symbol definitions, or (in the case of rust dylibs) cause rustc_std_internal_symbols functions to be silently overridden. As rust gets more commonly used inside the implementation of libraries consumed with a C interface (like Spidermonkey, Ruby YJIT (curently has to do partial linking of all rust code to hide all symbols not part of the C api), the Rusticl OpenCL implementation in mesa) this is becoming much more of an issue. With this PR the only symbols remaining with an unmangled name are rust_eh_personality (LLVM doesn't allow renaming it) and `__rust_no_alloc_shim_is_unstable`.

Helps mitigate rust-lang#104707

try-job: aarch64-gnu-debug
try-job: aarch64-apple
try-job: x86_64-apple-1
try-job: x86_64-mingw-1
try-job: i686-mingw-1
try-job: x86_64-msvc-1
try-job: i686-msvc-1
try-job: test-various
try-job: armhf-gnu
@bors
Copy link
Contributor

bors commented Mar 4, 2025

⌛ Trying commit bf70cce with merge 6255ed0...

@rust-log-analyzer
Copy link
Collaborator

The job x86_64-mingw-1 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
Updating files:  98% (51242/52287)
Updating files:  99% (51765/52287)
Updating files: 100% (52287/52287)
Updating files: 100% (52287/52287), done.
branch 'try' set up to track 'origin/try'.
Switched to a new branch 'try'
[command]"C:\Program Files\Git\bin\git.exe" log -1 --format=%H
6255ed0489eeefaa2719bb0dd42337672839b069
##[group]Run src/ci/scripts/setup-environment.sh
src/ci/scripts/setup-environment.sh
---
file:.git/config remote.origin.url=https://github.com/rust-lang-ci/rust
file:.git/config remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
file:.git/config gc.auto=0
file:.git/config http.https://github.com/.extraheader=AUTHORIZATION: basic ***
file:.git/config branch.try.remote=origin
file:.git/config branch.try.merge=refs/heads/try
file:.git/config remote.upstream.fetch=+refs/heads/*:refs/remotes/upstream/*
file:.git/config submodule.library/backtrace.active=true
file:.git/config submodule.library/backtrace.url=https://github.com/rust-lang/backtrace-rs.git
file:.git/config submodule.library/stdarch.active=true
---
---- [run-make] tests\run-make\symbols-all-mangled stdout ----

error: rmake recipe failed to complete
status: exit code: 101
command: "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\test\\run-make\\symbols-all-mangled\\rmake.exe"
--- stderr -------------------------------

thread 'main' panicked at D:\a\rust\rust\tests\run-make\symbols-all-mangled\rmake.rs:43:9:
thread 'main' panicked at D:\a\rust\rust\tests\run-make\symbols-all-mangled\rmake.rs:43:9:
Unmangled symbol found: .refptr._RNvCsjWV00SMt6BG_7___rustc39___rust_alloc_error_handler_should_panic
------------------------------------------




failures:
    [run-make] tests\run-make\symbols-all-mangled

test result: FAILED. 276 passed; 1 failed; 103 ignored; 0 measured; 6 filtered out; finished in 210.54s

Some tests failed in compiletest suite=run-make mode=run-make host=x86_64-pc-windows-gnu target=x86_64-pc-windows-gnu
Build completed unsuccessfully in 2:30:36
make: *** [Makefile:126: ci-mingw-x] Error 1
  network time: Tue, 04 Mar 2025 19:37:14 GMT
##[error]Process completed with exit code 2.
Post job cleanup.
[command]"C:\Program Files\Git\bin\git.exe" version

@bors
Copy link
Contributor

bors commented Mar 4, 2025

💔 Test failed - checks-actions

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-attributes Area: Attributes (`#[…]`, `#![…]`) A-run-make Area: port run-make Makefiles to rmake.rs S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants