Remove ability to disable some target features #116584

GuentherVIII · 2023-10-09T22:46:10Z

The documentation of TargetOptions.features claims that these features will always be passed to LLVM, but the logic implemented in #83084 allows them to be disabled with -C target-feature. Change the implementation to match the documentation.

Disabling x87 on x86 changes the way functions return floats. The second commit makes x87 mandatory on the existing i686 targets. Disabling floating point usage now requires a new target. #116344 does not fully get fixed by this (softfloat targets still have a problem with +x87), but it helps a lot.

In preparation for #115919 sse and sse2 are also made mandatory.

See https://rust-lang.zulipchat.com/#narrow/stream/136281-t-opsem/topic/Float.20ABI.20hell/near/394473707 and #116344.

Lang team: see nomination comment below.

@rustbot label: +A-abi +O-x86 +T-opsem +T-compiler

Change the features enabled by various target specifications from defaults to mandatory features. This will be used to prevent -C target-feature=-x87 and -C target-feature=-sse from changing the ABI on x86.

These features enable registers that are or will be used in the ABI. Instead of only being implied by the target cpu, they are now explicitly enabled so that they cannot be disabled by -C target-feature.

rustbot · 2023-10-09T22:46:18Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @cjgillot (or someone else) soon.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

@rustbot author: the review is finished, PR author should check the comments and take action accordingly
@rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

rustbot · 2023-10-09T22:46:21Z

These commits modify compiler targets.
(See the Target Tier Policy.)

workingjubilee · 2023-10-09T23:15:32Z

?
That should've worked.

@rustbot label: +A-abi +O-x86 +T-opsem

cjgillot · 2023-10-10T10:55:48Z

I'm not familiar enough with this code to review.
r? compiler

petrochenkov · 2023-10-12T11:39:20Z

cc @nagisa as the author of #83084.

petrochenkov · 2023-10-12T11:44:47Z

Change the implementation to match the documentation.

This alone is not a reason to make the change, that comment is from 2014 and it's not a public documentation, exactly.
Maybe it's the comment that needs to be changed.

Why does it make sense to have non-removable features for targets?

If yes, then are all features currently specified in target specs supposed to be non-removable?
Perhaps it makes sense to have two fields in target specs, for non-removable features, and for features that are just enabled by default?

petrochenkov · 2023-10-12T11:51:00Z

I'm not sure whether it needs a full MCP, but I'd like to see the "disabling x87 on x86 changes the way functions return floats" sentence to be elaborated into a more detailed proposal.
(I'll read the linked threads in the meantime.)
@rustbot author

workingjubilee · 2023-10-12T20:02:46Z

Why does it make sense to have non-removable features for targets?

Because some can change the call ABI, and this makes changing the features incompatible with the prescribed call ABI, especially for extern "C".

If yes, then are all features currently specified in target specs supposed to be non-removable?

Only ones that change ABI must be considered immutable. Specifically, we need to know what the "baseline" ABI is supposed to be for the target and for that knowledge to not disappear. If LLVM supported different ways of handling this problem, we could adopt them, but afaik it does not.

chorman0773 · 2023-10-13T13:49:59Z

Only ones that change ABI must be considered immutable

This seems like it would imply that avx, avx512f, and avx10 cannot be enabled.

workingjubilee · 2023-10-13T20:34:56Z

It doesn't, because I was talking about the features of the target CPU, not the features of the target ISA.

RalfJung · 2023-10-14T08:42:52Z

With this PR, what happens when someone does -C target-features=-x87? Does it just silently get ignored? I think it'd be prudent to at least emit a warning saying that the target spec features are overwriting the -C flag.

I'm not sure whether it needs a full MCP, but I'd like to see the "disabling x87 on x86 changes the way functions return floats" sentence to be elaborated into a more detailed proposal.

This is one half of resolving #116344, so that issue should hopefully provide the necessary context.

But yeah I agree this needs some form of process, t-compiler MCP at least.

nagisa · 2023-10-15T22:09:20Z

Why is the feature stability mechanism insufficient for this? If you don't want to make it possible to disable them (at which point they are ABI-critical and it doesn’t make sense to make it possible to enable a disabled feature either, since it would also change the ABI) just don’t make the feature stably controllable via -Ctarget-feature in the first place?

x87 in particular is not in the X86_ALLOWED_FEATURES set, so in that respect closing down the loophole where you could pass in features that aren’t in this list would also fix the immediate issue (and possibly others)

It doesn't, because I was talking about the features of the target CPU, not the features of the target ISA.

I have trouble understanding what this sentence means. Neither LLVM nor Rust has a concept of target ISA features, as far as I know? One way or another everything comes down to a single set of flags, whether they are configured implicitly (e.g. through target-cpu metadata) or explicitly… What makes avx512 features "ISA" features and not "CPU" ones?

nagisa · 2023-10-15T22:17:38Z

compiler/rustc_codegen_llvm/src/llvm_util.rs

-    // the host target are overridden by `-Ctarget-cpu=*`. On the other hand, what about when both
-    // `--target` and `-Ctarget-cpu=*` are specified? Both then imply some target features and both
-    // flags are specified by the user on the CLI. It isn't as clear-cut which order of precedence
-    // should be taken in cases like these.


I don’t believe this change addresses this FIXME at all. It still isn’t at all clear to me, even if we agreed that we don’t want to allow some features to be configurable outside of --target specifications, that this specific way to build up the feature set is the absolutely the correct one.

For example:

rustc -Ctarget-cpu=native --target=x86_64-unknown-linux-gnu

pretty clearly shows that the intent is for the -Ctarget-cpu=native to take precedence over whatever x86_64-unknown-linux-gnu means (including it not having any of the fancy AVX extensions.)

On the other hand

rustc -Ctarget-cpu=pentium4 --target=x86_64v4-unknown-linux-gnu

is quite a bit more ambiguous: does the user want something that would run on a pentium4? Or something that would run on an x86_64v4 class hardware, which pentium4 is not? Should the ordering of flags be taken into the account (unusual for rustc, but standard behaviour in gcc/clang style CLIs.) Perhaps it would be better to just error outright?

petrochenkov · 2023-10-16T11:09:11Z

I'll try to play a devil's advocate here.

-Ctarget-features is an inherently unsafe feature that relies on users understanding what they are doing.
What if by trying to make it safe you remove support for its main use cases, is it still worth doing then?

ABI differences turn into unsafety only if functions with one set of features are called as functions with another incompatible set of features.
If the user avoids such calls then everything is good and creating a new target (which is a major nuisance) is not necessary.

Maybe if we are trying to improve safety, it should be done in a more fine-grained way too.
For example, by including the feature bitset into function pointer types (internally for a start, without a surface syntax).
Then to convert function pointer types the user will have to perform an unsafe operation.

RalfJung · 2023-10-16T11:38:39Z

-Ctarget-features is inherently unsafe feature

I wasn't aware of that, but rustc -C help indeed says so. It does that in a completely unhelpful way though since it doesn't tell me what the safety requirements are.

-Ctarget-cpu fundamentally has the same problem and has no such comment.

workingjubilee · 2023-10-16T21:31:36Z

@petrochenkov: -Ctarget-features is an inherently unsafe feature that relies on users understanding what they are doing. What if by trying to make it safe you remove support for its main use cases, is it still worth doing then?

The problem is that, as Ralf highlighted, very few people actually understand the safety requirements. The majority of users only know that they can "solve a problem" by enabling/disabling features, they don't necessarily understand the larger ramifications of that. Users can and do set incompatible sets of features that fail to even pass LLVM's isel. I would even argue that very few compiler engineers understand the safety requirements, given the number of ABI bugs we've had along these lines.

@nagisa: x87 in particular is not in the X86_ALLOWED_FEATURES set, so in that respect closing down the loophole where you could pass in features that aren’t in this list would also fix the immediate issue (and possibly others)

That could work for x87, at least, but I'm not sure it works for sse and sse2.

I have trouble understanding what this sentence means. Neither LLVM nor Rust has a concept of target ISA features, as far as I know? One way or another everything comes down to a single set of flags, whether they are configured implicitly (e.g. through target-cpu metadata) or explicitly… What makes avx512 features "ISA" features and not "CPU" ones?

In the model I was implicitly referencing, a given CPU implements a specific instance of an ISA, with certain ISA extensions, and -Ctarget-cpu implicitly designates a subset of the ISA as the target by targeting that CPU. Our targets are effectively defined in terms of specific CPU instances we want code to be able to run on.

RalfJung · 2023-11-04T11:26:08Z

For the one use case of disabling x87 I know of, kernel code which wants to avoid saving the user mode x87 registers, linking in code that uses the x87 is a problem even if no function returning a float is ever called. In the Linux kernel, for example, calling any function working with floats is bad without calling kernel_fpu_begin() before. Which suggests that calling functions with target_feature(x87) from code without it should be unsafe?

Yes, that should either be a separate target or such calls should be disallowed. (Not just unsafe, they are always UB so allowing them makes little sense even in unsafe code.) Usually, "softfloat" is a separate target, so that would be a good pattern for x86 to follow as well.

workingjubilee · 2023-11-04T23:53:55Z

That's already what x86_64-unknown-none does.

Please refer to the target spec:

rust/compiler/rustc_target/src/spec/x86_64_unknown_none.rs

Lines 21 to 23 in 4b85902

    
           features: 
        
               "-mmx,-sse,-sse2,-sse3,-ssse3,-sse4.1,-sse4.2,-3dnow,-3dnowa,-avx,-avx2,+soft-float" 
        
                   .into(),

workingjubilee · 2023-11-05T00:20:40Z

If we actually want to support this configuration being set for any given x86_64-?-? target, we can't really use the xmm registers for anything, as we have to at least assume that these features may be enabled or disabled without reference to the target. This prevents a basic optimization, like actually passing vector registers as registers, from being implemented, without it being effectively instantly UB to compile. Because, yes, ever calling that code is incorrect, even if it passes or receives no such values!

And if it is unsupported, and we're okay with it being UB to enable, then why are we allowing people to set an option that would be effectively guaranteed to make all linkage to the Rust core library virtually instantly undefined behavior, no ifs, ands, or buts?

workingjubilee · 2023-11-05T00:32:33Z

Setting -Ctarget-feature=-sse2,-sse as a compiler flag, as a user, then thinking that would take care of it, and then linking to libcore, will pull in... who cares about floats?... slice APIs that have been precompiled to use the target features. These will thrash the xmm register state, which means that a kernel cannot assume that the xmm registers have been preserved, even if it does "only" integer operations. In order to catch this, it will have to manually disable and reenable the vector registers each time, because the only way to protect itself from the Rust libcore will be to make sure SSE instructions trap, which is itself the kind of heavyweight operation you would probably prefer not to do with a context switch.

Or, more likely, they recompile libcore for the kernel anyways, thus reinventing x86_64-unknown-none, which already exists.

Muon · 2023-11-05T01:09:28Z

Sounds like it's likely some people have lurking bugs because of this. Is there any way to prevent linking against code compiled with a different ABI? Is any metadata about the target embedded in the object files?

workingjubilee · 2023-11-05T01:12:26Z

And... people keep rediscovering this property, again and again, after thrashing with the compiler and their setups for HOURS. The fact that these combinations of flags are de facto unviable has bitten learned academics who specialize in verification of instruction sets: https://alastairreid.github.io/verifying-vectorized-code2/

Apologies if anyone thinks I should have stated this earlier! I refrained, because I could probably go on like this for what might well be the entire rest of the actual year, until 2024 January 1. Thus, simply stating "no one actually understands this" seemed enough.

workingjubilee · 2023-11-05T01:21:14Z

Sounds like it's likely some people have lurking bugs because of this. Is there any way to prevent linking against code compiled with a different ABI? Is any metadata about the target embedded in the object files?

The answer to that is a familiar, joyous refrain:
🎵 It depends on the target! 🎵

RalfJung · 2023-11-05T09:58:58Z

Is there any way to prevent linking against code compiled with a different ABI?

Yes. Make sure that all code built for the same target is ABI-compatible, no matter the target features. Reject any code or target features where we can't make sure this is the case. Linking files with different targets is already "obviously" wrong. (I don't know if we actually prevent it on a technical level though.)

That's exactly what this PR does.

So, looks like we should get this to be decided by some team. But is it t-compiler or t-lang? Or both?

RalfJung · 2023-11-05T10:19:10Z

Given that stabilizing target features seems to be a t-lang thing, I am going to assume the same applies here.

@rust-lang/lang, we'd like to remove the ability to disable certain target features. The broad goal we are going for here is that code built for the same target is ABI-compatible no matter the target features. This is an easy rule to remember and a principle that it seems reasonable to enforce. This principle is currently violated in several ways (for x86, that's tracked in #116344 and #116558). This PR is one part of achieving that, this pre-RFC is another part, and then one final piece is needed to reject +x87/+sse on x86_64-unknown-none (or to reject float-taking/float-returning-functions) as that would similarly change the ABI.

This particular PR fixes #116344 for our tier 1 targets. The issue is that disabling the "x87" feature on an x86 (32bit or 64bit) target changes the ABI, so linking two object files that were compiled with different target features can lead to UB in entirely safe code. In particular, calling any float-returning function from the standard library when x87 is disabled will break (caller and callee disagree about where the return value should go), and if we want to fix #115567 by returning floats in SSE registers for "Rust" functions, then similar ABI incompatibilities will arise when sse is disabled.

Disabling these target features also does not guarantee code that does not use SSE/x87 registers, since the standard library was built with those registers available. SSE registers are used even in code that doesn't involve any floats. So if the goal is to e.g. not save SSE registers in a context switch, then disabling the SSE target feature does not suffice, you also need to rebuild the standard library.

In other words, -sse/-x87 is (a) unsuited to generate code that doesn't use SSE/x87 registers, and (b) a major footgun due to ABI incompatibilities. We have the x86_64-unknown-none target for cases like that, which guarantees that all code, including the standard library, is built without use of SSE/x87 registers. (Many other architectures have similar softfloat targets. This PR here is for x86 only, but we should do a thorough audit of all stabilized target features and keep this concern in mind when stabilizing target features in the future.)

EDIT: I have created a design meeting proposal for the wider problem space here, in case you think this needs more fundamental discussion.

RalfJung · 2023-11-05T16:34:07Z

Implementation-wise, will this PR also reject enabling +softfloat? That also seems necessary, since enabling that feature also seems to change the float ABI.

RalfJung · 2023-11-06T07:29:04Z

Implementation-wise, will this PR also reject enabling +softfloat? That also seems necessary, since enabling that feature also seems to change the float ABI.

I think we'll need a list of forbidden features for each target (complementing the list of required features), then we can use that to reject +softfloat on hardfloat targets. (And we can later use that to reject +x87 on softfloat targets.)

RalfJung · 2023-11-06T07:35:20Z

Hm, it still seems fairly fragile that each x86 target has to list those required/forbidden features though. Maybe we should use a different approach where features in the features list are marked based on whether they are "float-ABI-affecting" and then we reject changing those features? (We also have SIMD-ABI-affecting features but there's a different proposal for how to handle them so that we can still support toggling those features.)

The tricky part with this approach is figuring out whether -Ctarget-cpu changed any of the "bad" features... (or whether they are implied by other explicitly specified features). We might have to just compute the feature set for the baseline target and the feature set the user configured and then compare that the two are identical for all float-ABI-affecting features.

RalfJung · 2023-11-06T07:51:46Z

compiler/rustc_codegen_llvm/src/llvm_util.rs

+            .split(',')
+            .filter(|v| !v.is_empty() && backend_feature_name(v).is_some() && v.starts_with("+"))
+            .map(String::from),
+    );


So this will silently ignore -Ctarget-features=-x87? That doesn't seem great, we shouldn't just ignore that. We should hard error or at least warn about ignoring this flag.

I think we'll want to compare that in the final feature set, certain features (for x86: x87, soft-float; for other targets we need to still determine this list) have the same state as without applying the user config. This needs to take into account -Ctarget-cpu and features implying other features.

GuentherVIII · 2023-11-06T23:58:51Z

Comments in rustc led me to believe that this minimal change could unblock #115919, but the code implementing the problematic design is almost entirely in LLVM. Target, target-cpu and target-feature together specify the CPU features the generated code will use, and the ABI follows from that. rustc could work around that to try to make the ABI follow only from target alone, but that requires at minimum a complete list of all ABI-affecting features in all targets, and maybe more. A clean solution would have the complete logic in rustc instead.

That's a bunch of code for a small reduction in unsoundness. To trigger the problem, you have to link an object compiled with different target-feature-flags into the program. I think "link an object" is the important part, not "different target-feature-flags". The object could also have been compiled with a different "--target" flag, or a different rustc version, or a pascal compiler, or it could have been downloaded from the internet. Making linking objects into the program safer would be nice in general, but there are far too many ways in which problematic objects can be produced.

That said, giving an error message for some known-problematic arguments to -C target-feature might be an efficient way to improve the situation?

workingjubilee · 2023-11-07T00:37:43Z

@GuentherVIII I want to

acknowledge you are correct that the problem is linkage.
- The complication is that we ship a precompiled libstd, including libcore, and link that in to every single Rust program built for a tier 1 or tier 2 target, unless exceptional measures (from the perspective of "typical or even fairly advanced Rust compiler user") are taken. So we are creating a lot of possible unsound linkages just on our own.
Disagree on the idea that the problem is necessarily LLVM. The idea that rustc reasons about target features the way LLVM does is largely illusory, caused by an accidental conflation that we have probably also-accidentally disposed of.
- We have several "canonizing" maps of "Rust target feature" to "LLVM target feature" already, for various reasons. This is partly because LLVM sees fit to change what a target feature means at their leisure, whereas Rust wants to expose stable interfaces, because we have made these features something you can set and unset in source code, thus subject to Rust stability expectations, at least somewhat.
- We have adopted a framing that means "a target feature is something you can put on a single function" due to accepting and implementing the target_feature / cfg_target_feature / cfg_feature_enabled RFC, which means the (several) target features in LLVM that only work if you annotate an entire LLVM module with them, thus affecting all functions, and often only work if the entire program has them set, are not actually coherent if treated as Rust target features.
- Thus, we have largely backed ourselves into a corner of accidentally having to redefine what a "target feature" even means, in terms of Rust, instead of being able to simply rely on LLVM to do the work.

And for what we have decided on so far, we can pretty much decide if things are being used correctly in a fairly likely way for most cases and emit errors in the known-problem cases, yes. It's true that it may be a "best effort" in reality, but that's to be expected, and it describes the "99%" case, even amongst the set of "advanced" users.

It's not necessarily about sound or unsound, it's about user experience, so that we aren't sitting smugly on an answer we already have and letting people waste hundreds of hours on trying to resolve cryptic issues.

It also is about the questions already raised re: actually being able to use optimizations in the compiler without exploding things in the face of users. We reserve the right to change the Rust ABI between versions of the compiler without notice, but that means that if setting -Ctarget-feature affects the Rust ABI for a given feature, then the soundness of setting that flag for a given feature and then linking libcore is determined by the Rust compiler version, and subject to change without notice. Thus we really already know an answer to whether it's sound or unsound.

RalfJung · 2023-11-07T07:16:53Z

That's a bunch of code for a small reduction in unsoundness.

That's describing a lot of rustc. ;)

To trigger the problem, you have to link an object compiled with different target-feature-flags into the program.

As Jubilee said: this happens every time someone uses -Ctarget-feature or -Ctarget-cpu, since libcore and libstd are still built with the default target features.

that requires at minimum a complete list of all ABI-affecting features in all targets

Correct. But that doesn't seem so bad? Such features are fairly rare, in particular if we assume we can exclude repr(simd) types from consideration because they will hopefully use a different mechanism.

We can start with the tier 1 targets, and then slowly work our way through the rest.

bors · 2023-11-08T21:29:54Z

☔ The latest upstream changes (presumably #117716) made this pull request unmergeable. Please resolve the merge conflicts.

traviscross · 2023-12-13T08:11:37Z

This question is scheduled for a T-lang design meeting on 2023-12-20:

rust-lang/lang-team#235

traviscross · 2023-12-21T05:07:20Z

@rustbot labels -I-lang-nominated

We discussed this question in the T-lang design meeting on 2023-12-20. The consensus was generally in favor of fixing this, and we were interested in seeing more work in this direction. While we considered various ways that this could be fixed, we were supportive of finding the most minimal, simplest way to fix this as the first step, assuming that such an approach proves to be feasible. We'll need to see at least the results of a crater run and further analysis to confirm that feasibility.

We appreciate @RalfJung and all the others who have been investing time to push this forward.

Dylan-DPC · 2024-02-16T06:30:34Z

@GuentherVIII any updates on this? thanks

Dylan-DPC · 2024-04-08T12:37:06Z

Closing this as inactive. Feel free to reöpen this pr or create a new pr if you get the time to work on this. Thanks

Günther Brammer added 2 commits October 9, 2023 02:19

Override -C target-feature with features implied by --target

f4a7b5b

Change the features enabled by various target specifications from defaults to mandatory features. This will be used to prevent -C target-feature=-x87 and -C target-feature=-sse from changing the ABI on x86.

Make x87, sse and sse2 mandatory on targets enabling them

66625a9

These features enable registers that are or will be used in the ABI. Instead of only being implied by the target cpu, they are now explicitly enabled so that they cannot be disabled by -C target-feature.

rustbot assigned cjgillot Oct 9, 2023

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Oct 9, 2023

rustbot added A-ABI Area: Concerning the application binary interface (ABI) O-x86 T-opsem Relevant to the opsem team labels Oct 9, 2023

workingjubilee mentioned this pull request Oct 9, 2023

Rustbot whiffs labels if they're in the opening PR message rust-lang/triagebot#1731

Open

rustbot assigned petrochenkov and unassigned cjgillot Oct 10, 2023

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 12, 2023

petrochenkov mentioned this pull request Oct 13, 2023

Return f32 and f64 in XMM0 instead of FP0 on i686 #115919

Closed

nagisa reviewed Oct 15, 2023

View reviewed changes

RalfJung added T-lang Relevant to the language team, which will review and decide on the PR/issue. I-lang-nominated Nominated for discussion during a lang team meeting. and removed T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 5, 2023

RalfJung reviewed Nov 6, 2023

View reviewed changes

RalfJung mentioned this pull request Nov 11, 2023

Design meeting: resolve ABI issues around target-feature rust-lang/lang-team#235

Closed

rustbot removed the I-lang-nominated Nominated for discussion during a lang team meeting. label Dec 21, 2023

Dylan-DPC closed this Apr 8, 2024

Dylan-DPC added S-inactive Status: Inactive and waiting on the author. This is often applied to closed PRs. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 8, 2024

beetrees mentioned this pull request Apr 23, 2024

Ensure floats are returned losslessly by the Rust ABI on 32-bit x86 #123351

Merged

Remove ability to disable some target features #116584

Remove ability to disable some target features #116584

Conversation

GuentherVIII commented Oct 9, 2023 • edited by RalfJung Loading

rustbot commented Oct 9, 2023

rustbot commented Oct 9, 2023

workingjubilee commented Oct 9, 2023

cjgillot commented Oct 10, 2023

petrochenkov commented Oct 12, 2023

petrochenkov commented Oct 12, 2023

petrochenkov commented Oct 12, 2023

workingjubilee commented Oct 12, 2023

chorman0773 commented Oct 13, 2023

workingjubilee commented Oct 13, 2023

RalfJung commented Oct 14, 2023 • edited Loading

nagisa commented Oct 15, 2023

nagisa Oct 15, 2023 • edited Loading

Choose a reason for hiding this comment

petrochenkov commented Oct 16, 2023 • edited Loading

RalfJung commented Oct 16, 2023

workingjubilee commented Oct 16, 2023 • edited Loading

RalfJung commented Nov 4, 2023

workingjubilee commented Nov 4, 2023 • edited Loading

workingjubilee commented Nov 5, 2023

workingjubilee commented Nov 5, 2023

Muon commented Nov 5, 2023

workingjubilee commented Nov 5, 2023

workingjubilee commented Nov 5, 2023 • edited Loading

RalfJung commented Nov 5, 2023 • edited Loading

RalfJung commented Nov 5, 2023 • edited Loading

RalfJung commented Nov 5, 2023

RalfJung commented Nov 6, 2023

RalfJung commented Nov 6, 2023 • edited Loading

RalfJung Nov 6, 2023 • edited Loading

Choose a reason for hiding this comment

GuentherVIII commented Nov 6, 2023

workingjubilee commented Nov 7, 2023 • edited Loading

RalfJung commented Nov 7, 2023 • edited Loading

bors commented Nov 8, 2023

traviscross commented Dec 13, 2023

traviscross commented Dec 21, 2023

Dylan-DPC commented Feb 16, 2024

Dylan-DPC commented Apr 8, 2024

GuentherVIII commented Oct 9, 2023 •

edited by RalfJung

Loading

RalfJung commented Oct 14, 2023 •

edited

Loading

nagisa Oct 15, 2023 •

edited

Loading

petrochenkov commented Oct 16, 2023 •

edited

Loading

workingjubilee commented Oct 16, 2023 •

edited

Loading

workingjubilee commented Nov 4, 2023 •

edited

Loading

workingjubilee commented Nov 5, 2023 •

edited

Loading

RalfJung commented Nov 5, 2023 •

edited

Loading

RalfJung commented Nov 5, 2023 •

edited

Loading

RalfJung commented Nov 6, 2023 •

edited

Loading

RalfJung Nov 6, 2023 •

edited

Loading

workingjubilee commented Nov 7, 2023 •

edited

Loading

RalfJung commented Nov 7, 2023 •

edited

Loading