Stabilize target_feature_11 #116114

calebzulawski · 2023-09-24T03:20:06Z

Stabilization report

This is a redo of #99767. Most of this commit and report were copied from that PR. Thanks @LeSeulArtichaut!

Summary

Allows for safe functions to be marked with #[target_feature] attributes.

Functions marked with #[target_feature] are generally considered as unsafe functions: they are unsafe to call, cannot be assigned to safe function pointers, and don't implement the Fn* traits.

However, calling them from other #[target_feature] functions with a superset of features is safe.

// Demonstration function
#[target_feature(enable = "avx2")]
fn avx2() {}

fn foo() {
    // Calling `avx2` here is unsafe, as we must ensure
    // that AVX is available first.
    unsafe {
        avx2();
    }
}

#[target_feature(enable = "avx2")]
fn bar() {
    // Calling `avx2` here is safe.
    avx2();
}

Test cases

Tests for this feature can be found in src/test/ui/rfcs/rfc-2396-target_feature-11/.

Edge cases

Closures

target-feature 1.1: should closures inherit target-feature annotations? #73631

Closures defined inside functions marked with #[target_feature] inherit the target features of their parent function. They can still be assigned to safe function pointers and implement the appropriate Fn* traits.

#[target_feature(enable = "avx2")]
fn qux() {
    let my_closure = || avx2(); // this call to `avx2` is safe
    let f: fn() = my_closure;
}

This means that in order to call a function with #[target_feature], you must show that the target-feature is available while the function executes and for as long as whatever may escape from that function lives.

Fix #[inline(always)] on closures with target feature 1.1 #111836

Closures accept #[inline(always)], even within functions marked with #[target_feature]. Since these attributes conflict, #[inline(always)] wins out to maintain compatibility.

Special functions

The #[target_feature] attribute is forbidden from a variety of special functions, such as main, current and future lang items (e.g. #[start], #[panic_handler]), and default trait implementations.

Documentation

Reference: Document the target_feature_11 feature reference#1181

cc tracking issue #69098
cc @workingjubilee
r? @rust-lang/lang

eduardosm · 2023-09-25T19:52:25Z

#116078 has been merged and added a test with #![feature(target_feature_11)], so you'll have to rebase and remove the feature line.

calebzulawski · 2023-09-25T23:09:35Z

Thanks for the heads up!

I realized the stdarch submodule uses the feature as well. I'm not quite sure how to work around that. (cc @Amanieu)

Amanieu · 2023-09-29T14:57:44Z

I don't think any workaround is needed? stdarch doesn't specify its own features since it is directly included in libcore.

RalfJung · 2023-10-09T08:57:36Z

Do we make the issue around target features affecting ABI any worse with this? If I understand the RFC correctly, then this allows safe code to do something like

extern "C" fn no_target_feature(x: __m256) {}

#[target_feature(enable = "avx")]
extern "C" fn with_target_feature(x: __m256) {
  no_target_feature(x);
}

Calling with_target_feature will cause UB since no_target_feature is called in an ABI-incompatible way (the caller passes __m256 in an AVX register but the callee doesn't support AVX). Now there's not even any unsafe around to potentially catch this. That seems bad?

(See #116558 for the ABI issue itself.)

bjorn3 · 2023-10-09T09:03:24Z

One potential fix for that would be to deny using this feature for non-extern "Rust" functions.

est31 · 2023-10-09T15:53:48Z

@RalfJung is it indeed UB or is the caller aware of those target features?

In general when there is call of a function with ABI X (and I consider target features to be part of the ABI), then no matter what the ABI of the surrounding context is, the way X gets invoked should always be through that X ABI. That's a should and not a does because I don't know if it's actually implemented that way.

RalfJung · 2023-10-09T15:59:29Z

That's not how it works, and that's not how it could possibly work since no_target_feature might be invoked through a function pointer. The function type doesn't capture which target features are enabled (and most target features aren't even relevant).

So right now unfortunately the target features of the caller are used to compute the ABI. It's terrible but it's what we got. Also see #116558.

est31 · 2023-10-09T18:27:00Z

not how it could possibly work since no_target_feature might be invoked through a function pointer.

Right now on stable we require target_feature function pointers to be coerced to unsafe function pointers, and also to be unsafe themselves. This gives us a lot of design ability for the "safe" counterpart. We can for example require for safe function pointers that they have the entire target feature list attached to them, regardless of the caller context.

most target features aren't even relevant

IMO one can refine this later on to allow some fn pointer conversions safely while banning others. But it's generally in the domain of the "safe transmute" project.

right now unfortunately the target features of the caller are used to compute the ABI. It's terrible but it's what we got.

Yeah then the current target_feature 1.1 implementation is not sound and, I would think this is a clear case for a stabilization blocker.

RalfJung · 2023-10-09T18:34:05Z

Right now on stable we require target_feature function pointers to be coerced to unsafe function pointers, and also to be unsafe themselves. This gives us a lot of design ability for the "safe" counterpart. We can for example require for safe function pointers that they have the entire target feature list attached to them, regardless of the caller context.

I think I'd rather fix the ABI to not depend on target features, than equiv function pointers with target feature information. But anyway that's a discussion for #116558.

Note that this doesn't just affect #[target_feature], it also affects linking together crates that were compiled with different -C flags. I don't think we officially support that, but we also don't officially not support it, and in fact the distributed sysroot relies on it to work to some extent.

workingjubilee · 2023-10-09T18:49:32Z

Do we make the issue around target features affecting ABI any worse with this? If I understand the RFC correctly, then this allows safe code to do something like

I had been wondering about that, which is why I didn't immediately push for stabilizing this again. I don't believe anyone really knows since our ABI code is a mess.

calebzulawski · 2023-10-09T18:55:30Z

I support limiting it to extern "Rust". I think that actually improves the ABI situation. Right now, since unsafe is always required, these edge case invariants are easy to miss. If you can use a safe function (limited to the Rust ABI) you know you are using a "blessed" ABI; if you are required to use unsafe you know there are hidden invariants you need to watch out for.

tmandry · 2023-10-10T00:01:20Z

I'll restate here what I said in #116558 (comment): #[target_feature] should not affect the ABI of a function. That's especially important to make escaping closures coercible to function pointers; I don't think such a feature could ever be sound otherwise.

If we want to take advantage of #[target_feature] for ABI, it must be encoded in the types of function pointers somehow, e.g. extern "Rust+avx".

RalfJung · 2023-10-10T06:13:04Z

@calebzulawski

I support limiting it to extern "Rust".

Note that the reproducer for #116558 only uses #[target_feature] on extern "Rust" functions. So that's not enough. The issue comes about when a #[target_feature] function calls (or is called by) a function with a different set of target features.

est31 · 2023-10-10T11:20:10Z

@tmandry

I'll restate here what I said in #116558 (comment): #[target_feature] should not affect the ABI of a function.

You said this in the linked comment

I don't think #[target_feature] should ever affect the ABI of function pointers called from within them

to me that sounds like the opposite of what you were saying. Which one do you mean?

calebzulawski · 2023-10-10T12:49:32Z

@calebzulawski

I support limiting it to extern "Rust".

Note that the reproducer for #116558 only uses #[target_feature] on extern "Rust" functions. So that's not enough. The issue comes about when a #[target_feature] function calls (or is called by) a function with a different set of target features.

I see, you're right. So it sounds to me like this is less of a TF 1.1 problem, and more like it is impossible to use even existing #[target_feature] beyond extern "Rust". I guess there is no action to be taken for TF1.1 specifically?

RalfJung · 2023-10-10T13:21:18Z

The issue isn't with TF1.1, but I feel like allowing more things in safe code here would be a mistake given the pitfalls we currently have.

bors · 2023-10-16T17:18:36Z

☔ The latest upstream changes (presumably #116550) made this pull request unmergeable. Please resolve the merge conflicts.

tmandry · 2023-10-17T21:39:47Z

to me that sounds like the opposite of what you were saying. Which one do you mean?

Both :). Those are different statements but I don't think they are opposites.

RalfJung · 2023-11-11T18:00:24Z

rust-lang/lang-team#235 suggests a design to resolve the ABI issues, which would then hopefully unblock target-feature 1.1.

apiraino · 2024-12-12T11:38:20Z

Visited during T-compiler triage on Zulip.

@calebzulawski when you have a chance could you update the existing summary comment to include the new information from T-lang meeting? Thanks

@rustbot author

apiraino · 2024-12-12T11:59:11Z

Superseded by #134090

Stabilize target_feature_11 # Stabilization report This is an updated version of rust-lang#116114, which is itself a redo of rust-lang#99767. Most of this commit and report were copied from those PRs. Thanks `@LeSeulArtichaut` and `@calebzulawski!` ## Summary Allows for safe functions to be marked with `#[target_feature]` attributes. Functions marked with `#[target_feature]` are generally considered as unsafe functions: they are unsafe to call, cannot *generally* be assigned to safe function pointers, and don't implement the `Fn*` traits. However, calling them from other `#[target_feature]` functions with a superset of features is safe. ```rust // Demonstration function #[target_feature(enable = "avx2")] fn avx2() {} fn foo() { // Calling `avx2` here is unsafe, as we must ensure // that AVX is available first. unsafe { avx2(); } } #[target_feature(enable = "avx2")] fn bar() { // Calling `avx2` here is safe. avx2(); } ``` Moreover, once rust-lang#135504 is merged, they can be converted to safe function pointers in a context in which calling them is safe: ```rust // Demonstration function #[target_feature(enable = "avx2")] fn avx2() {} fn foo() -> fn() { // Converting `avx2` to fn() is a compilation error here. avx2 } #[target_feature(enable = "avx2")] fn bar() -> fn() { // `avx2` coerces to fn() here avx2 } ``` See the section "Closures" below for justification of this behaviour. ## Test cases Tests for this feature can be found in [`tests/ui/target_feature/`](https://github.com/rust-lang/rust/tree/f6cb952dc115fd1311b02b694933e31d8dc8b002/tests/ui/target-feature). ## Edge cases ### Closures * [target-feature 1.1: should closures inherit target-feature annotations? rust-lang#73631](rust-lang#73631) Closures defined inside functions marked with #[target_feature] inherit the target features of their parent function. They can still be assigned to safe function pointers and implement the appropriate `Fn*` traits. ```rust #[target_feature(enable = "avx2")] fn qux() { let my_closure = || avx2(); // this call to `avx2` is safe let f: fn() = my_closure; } ``` This means that in order to call a function with #[target_feature], you must guarantee that the target-feature is available while the function, any closures defined inside it, as well as any safe function pointers obtained from target-feature functions inside it, execute. This is usually ensured because target features are assumed to never disappear, and: - on any unsafe call to a `#[target_feature]` function, presence of the target feature is guaranteed by the programmer through the safety requirements of the unsafe call. - on any safe call, this is guaranteed recursively by the caller. If you work in an environment where target features can be disabled, it is your responsibility to ensure that no code inside a target feature function (including inside a closure) runs after this (until the feature is enabled again). **Note:** this has an effect on existing code, as nowadays closures do not inherit features from the enclosing function, and thus this strengthens a safety requirement. It was originally proposed in rust-lang#73631 to solve this by adding a new type of UB: “taking a target feature away from your process after having run code that uses that target feature is UB” . This was motivated by userspace code already assuming in a few places that CPU features never disappear from a program during execution (see i.e. https://github.com/rust-lang/stdarch/blob/2e29bdf90832931ea499755bb4ad7a6b0809295a/crates/std_detect/src/detect/arch/x86.rs); however, concerns were raised in the context of the Linux kernel; thus, we propose to relax that requirement to "causing the set of usable features to be reduced is unsafe; when doing so, the programmer is required to ensure that no closures or safe fn pointers that use removed features are still in scope". * [Fix #[inline(always)] on closures with target feature 1.1 rust-lang#111836](rust-lang#111836) Closures accept `#[inline(always)]`, even within functions marked with `#[target_feature]`. Since these attributes conflict, `#[inline(always)]` wins out to maintain compatibility. ### ABI concerns * [The extern "C" ABI of SIMD vector types depends on target features rust-lang#116558](rust-lang#116558) The ABI of some types can change when compiling a function with different target features. This could have introduced unsoundness with target_feature_11, but recent fixes (rust-lang#133102, rust-lang#132173) either make those situations invalid or make the ABI no longer dependent on features. Thus, those issues should no longer occur. ### Special functions The `#[target_feature]` attribute is forbidden from a variety of special functions, such as main, current and future lang items (e.g. `#[start]`, `#[panic_handler]`), safe default trait implementations and safe trait methods. This was not disallowed at the time of the first stabilization PR for target_features_11, and resulted in the following issues/PRs: * [`#[target_feature]` is allowed on `main` rust-lang#108645](rust-lang#108645) * [`#[target_feature]` is allowed on default implementations rust-lang#108646](rust-lang#108646) * [#[target_feature] is allowed on #[panic_handler] with target_feature 1.1 rust-lang#109411](rust-lang#109411) * [Prevent using `#[target_feature]` on lang item functions rust-lang#115910](rust-lang#115910) ## Documentation * Reference: [Document the `target_feature_11` feature reference#1181](rust-lang/reference#1181) --- cc tracking issue rust-lang#69098 cc `@workingjubilee` cc `@RalfJung` r? `@rust-lang/lang`

rustbot assigned nikomatsakis Sep 24, 2023

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Sep 24, 2023

calebzulawski mentioned this pull request Sep 24, 2023

Tracking issue for target_feature 1.1 RFC #69098

Open

10 tasks

rakshith-ravi mentioned this pull request Sep 24, 2023

Possibility of new fns const? RustCrypto/password-hashes#461

Closed

Stabilize target_feature_11

f0748b2

calebzulawski force-pushed the stabilize-tf11 branch from 10c6f3a to f0748b2 Compare September 25, 2023 23:06

Dylan-DPC added S-blocked Status: Blocked on something else such as an RFC or other implementation work. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 11, 2023

veluca93 mentioned this pull request Dec 9, 2024

Stabilize target_feature_11 #134090

Open

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-blocked Status: Blocked on something else such as an RFC or other implementation work. labels Dec 12, 2024

apiraino closed this Dec 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stabilize target_feature_11 #116114

Stabilize target_feature_11 #116114

calebzulawski commented Sep 24, 2023

eduardosm commented Sep 25, 2023

calebzulawski commented Sep 25, 2023

Amanieu commented Sep 29, 2023

RalfJung commented Oct 9, 2023 •

edited

Loading

bjorn3 commented Oct 9, 2023

est31 commented Oct 9, 2023

RalfJung commented Oct 9, 2023 •

edited

Loading

est31 commented Oct 9, 2023

RalfJung commented Oct 9, 2023 •

edited

Loading

workingjubilee commented Oct 9, 2023

calebzulawski commented Oct 9, 2023

tmandry commented Oct 10, 2023

RalfJung commented Oct 10, 2023

est31 commented Oct 10, 2023

calebzulawski commented Oct 10, 2023 •

edited

Loading

RalfJung commented Oct 10, 2023

bors commented Oct 16, 2023

tmandry commented Oct 17, 2023

RalfJung commented Nov 11, 2023

apiraino commented Dec 12, 2024

apiraino commented Dec 12, 2024

Stabilize target_feature_11 #116114

Stabilize target_feature_11 #116114

Conversation

calebzulawski commented Sep 24, 2023

Stabilization report

Summary

Test cases

Edge cases

Closures

Special functions

Documentation

eduardosm commented Sep 25, 2023

calebzulawski commented Sep 25, 2023

Amanieu commented Sep 29, 2023

RalfJung commented Oct 9, 2023 • edited Loading

bjorn3 commented Oct 9, 2023

est31 commented Oct 9, 2023

RalfJung commented Oct 9, 2023 • edited Loading

est31 commented Oct 9, 2023

RalfJung commented Oct 9, 2023 • edited Loading

workingjubilee commented Oct 9, 2023

calebzulawski commented Oct 9, 2023

tmandry commented Oct 10, 2023

RalfJung commented Oct 10, 2023

est31 commented Oct 10, 2023

calebzulawski commented Oct 10, 2023 • edited Loading

RalfJung commented Oct 10, 2023

bors commented Oct 16, 2023

tmandry commented Oct 17, 2023

RalfJung commented Nov 11, 2023

apiraino commented Dec 12, 2024

apiraino commented Dec 12, 2024

RalfJung commented Oct 9, 2023 •

edited

Loading

RalfJung commented Oct 9, 2023 •

edited

Loading

RalfJung commented Oct 9, 2023 •

edited

Loading

calebzulawski commented Oct 10, 2023 •

edited

Loading