Experimental feature gate for super let

[m-ou-se]

This adds an experimental feature gate, #![feature(super_let)], for the super let experiment.

Tracking issue: #139076

Liaison: @nikomatsakis

Description

There's a rough (inaccurate) description here: https://blog.m-ou.se/super-let/

In short, super let allows you to define something that lives long enough to be borrowed by the tail expression of the block. For example:

let a = {
    super let b = temp();
    &b
};

Here, b is extended to live as long as a, similar to how in let a = &temp();, the temporary will be extended to live as long as a.

Properties

During the temporary lifetimes work we did last year, we explored the properties of "super let" and concluded that the fundamental property should be that these two are always equivalent in any context:

1. & $expr
2. { super let a = & $expr; a }

And, additionally, that these are equivalent in any context when $expr is a temporary (aka rvalue):

1. & $expr
2. { super let a = $expr; & a }

This makes it possible to give a name to a temporary without affecting how temporary lifetimes work, such that a macro can transparently use a block in its expansion, without that having any effect on the outside.

Implementing pin!() correctly

With super let, we can properly implement the pin!() macro without hacks: ✨

pub macro pin($value:expr $(,)?) {
    {
        super let mut pinned = $value;
        unsafe { $crate::pin::Pin::new_unchecked(&mut pinned) }
    }
}

This is important, as there is currently no way to express it without hacks in Rust 2021 and before (see hacky definition), and no way to express it at all in Rust 2024 (see issue).

Fixing format_args!()

This will also allow us to express format_args!() in a way where one can assign the result to a variable, fixing a long standing issue:

let f = format_args!("Hello {name}!"); // error today, but accepted in the future! (after separate FCP)

Experiment

The precise definition of super let, what happens for super let x; (without initializer), and whether to accept super let _ = _ else { .. } are still open questions, to be answered by the experiment.

Furthermore, once we have a more complete understanding of the feature, we might be able to come up with a better syntax. (Which could be just a different keywords, or an entirely different way of naming temporaries that doesn't involve a block and a (super) let statement.)

[nikomatsakis]

@rustbot labels +I-lang-nominated

I'm happy to serve as champion/liaison for this, it's something @m-ou-se and I have extensively discussed in the past; but if others on @rust-lang/lang are interested, that could work too.

[traviscross]

Exciting. First of all , thanks to @m-ou-se for putting this up.

Given that the primary equivalence includes a reference on the RHS of the super let,

                &$expr
---------------------------
{ super let a = &$expr; a }

and that consequently this behavior must involve extending the lifetime of temporaries created in this RHS, rather than just extending the "lifetime" (note scare quotes) of the named place itself, I find myself wondering whether it wouldn't be better to express this as a new type of reference constructor, &super $expr / &super mut $expr, rather than as a new type of let statement. Then we'd write, e.g.,

let a = {
    let b = &super temp();
    b
};

and,

pub macro pin($value:expr $(,)?) {
    {
        let pinned = &super mut { $value }; // <-- Force a move.
        unsafe { $crate::pin::Pin::new_unchecked(pinned) }
    }
}

and we've have the equivalence:

                &$expr
---------------------------
{ let a = &super $expr; a }

This feels to me like less new syntax, and for that reason, appealingly, would resolve two of the three open questions:

what happens for super let x; (without initializer), and whether to accept super let _ = _ else { .. }

What this gives up is being able to directly name the temporary rather than naming a reference to it, but -- I don't know -- this point felt more compelling to me in earlier drafts before we realized the primary equivalence is the one where super let is used with a reference on the RHS anyway.

Maybe this is actually a benefit regardless. We're talking about extending lifetimes, and references are what have lifetimes. By ensuring there's always a reference, we can talk about what's happening, e.g. with extending temporary lifetimes, in terms of the lifetime of that reference. That model feels a bit better to me at the moment, and seems maybe simpler to explain.

(It's been awhile since we talked about this. I recall there were reasons we thought that doing an expression position syntax might be challenging to reason about. I'm not sure offhand whether framing this as a reference constructor, rather than as the other things I recall us exploring, might help at all with those. It's something to explore.)

I'm curious of course to hear what people think.

[m-ou-se]

I have considered that, but that would actually not solve #138718

This is the pin definition you suggest, which we'd use with a &super expression:

pub macro pin($value:expr $(,)?) {
    {
        let pinned = &super mut { $value }; // <-- Force a move.
        unsafe { $crate::pin::Pin::new_unchecked(pinned) }
    }
}

But how would that behave in

select(std::pin::pin!(foo(&mut temp())));

?

We must force a move, but we must also make sure that temp() lives long enough. In your expansion, you use an extra {} for that move, but that { $value } causes (non-extended) temporaries in $value to be dropped, as of Rust 2024.

This is exactly what caused #138596

We must make sure that the temporaries in $value are not dropped at the ; of the let. (Even when not lifetime-extended!) In my opinion, that makes that statement special, not the expression. That statement effectively does not have its own scope. That's why I've been talking about 'super' (let) statements, rather than 'super' expressions.

I agree it's a bit weird, but that's why we need to experiment.

[m-ou-se]

Basically, the second equivalence,

                  & $rvalue
    ------------------------------
    { super let a = $rvalue; & a }

can be used to force a move (lvalue-to-rvalue conversion).

In Rust 2021 and before, you'd be able to trigger that with this eqiuvalence:

      & $rvalue
    -------------  ?
    & { $rvalue }

But that equivalence is gone in Rust 2024, as temporaries are dropped at the }. (Causing #138596.)

One could attempt to use the identity function:

              & $rvalue
    ---------------------  ?
    & identity( $rvalue )

But that's also not equivalent, because of temporary lifetime extension that might apply to temporaries in $rvalue.

I think it makes sense to use a let statement to force a move.

[m-ou-se]

Anyway, I'm not necessarily opposed to &super or something like that, if we can find a good solution.

The point of this experiment is to have a syntax, so we can play around with lots of test cases. Then we can come up with the actual syntax afterwards, once we form a better understanding of all the cases where we'd use this.

[traviscross]

Yes, agree it's weird (kind of a cursed problem) and that working all this out is what we gain from the experiment.

But how would that behave in

select(std::pin::pin!(foo(&mut temp())));

?

Probably I was expecting let x = &super mut $expr, and similarly super let x = &mut $expr, to extend the temporary and all temporaries in $expr, including non-extended ones, such that the reference could then appear in the tail expression of the block. Under that rule, the above would work, but indeed probably that rule would do "too much", particularly when $expr contains blocks itself.

Anyway, I'm entirely in favor of experimentation here under any syntax. Probably I just hope we can find some elegant way to avoid if at all possible having to add another kind of let. But maybe in the end it will work out best. Hopefully this will all come into a bit sharper focus as we work out how to write down all the rules exactly for what gets extended when.

[m-ou-se]

It should affect non-extended temporaries, but it should not affect non-extended temporaries in blocks:

select(pin!({temp().f()})); // should drop temp() at the `}`
select(pin!(temp().f())); // should drop temp() at the `;`

[traviscross]

I don't want to run this back through CI, but in follow-up work, let's please remember to add a test that confirms we get this error when the code is cfged out. That's correctly done here (it's apparent in the code and I tested locally), but we should always want a test for this.

[traviscross]

@bors r+ rollup

[bors]

📌 Commit 14e6a96 has been approved by traviscross

It is now in the queue for this repository.

[traviscross]

Thanks again to @m-ou-se for putting this forward. We can I think call this lang experiment accepted.

(We didn't quite get to this one in the lang meeting today, but we circulated it async without hearing objections, and this is rather clearly necessary work. The feature gate is of course a two-day door. And both Niko and I are willing to champion it (we'll flip a coin or something).)