Support s390x z13 vector ABI

[taiki-e]

cc #130869

This resolves the following fixmes:

• rust/compiler/rustc_target/src/abi/call/s390x.rs

  Lines 1 to 2 in 58420a0

  	// FIXME: The assumes we're using the non-vector ABI, i.e., compiling
  	// for a pre-z13 machine or using -mno-vx.

• rust/compiler/rustc_target/src/spec/targets/s390x_unknown_linux_gnu.rs

  Lines 9 to 11 in 58420a0

  	// FIXME: The ABI implementation in abi/call/s390x.rs is for now hard-coded to assume the no-vector
  	// ABI. Pass the -vector feature string to LLVM to respect this assumption.
  	base.features = "-vector".into();

Refs: Section 1.2.3 "Parameter Passing" and section 1.2.5 "Return Values" in ELF Application Binary Interface s390x Supplement, Version 1.6.1 (lzsabi_s390x.pdf in https://github.com/IBM/s390x-abi/releases/tag/v1.6.1)

This PR extends #127731 #132173 (merged) 's ABI check to handle cases where vector target feature is disabled.
If we do not do ABI check, we run into the ABI problems as described in #116558 and #130869 (comment), and the problem of the compiler generating strange code (#131586 (comment)).

cc @uweigand

@rustbot label +O-SystemZ +A-ABI

[rustbot]

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[jieyouxu]

r? compiler

[uweigand]

In addition to the problem mentioned inline, we should also verify that building with soft-float works as expected. This option implicitly disables any use of floating-point or vector registers, and therefore also disables use of the vector facility. I believe the LLVM back-end should handle that logic correctly, but it would be good to have the test check.

[taiki-e]

The current implementation of this PR ignores ZST (zero sized type) in a structure, is this incorrect? Or is the current implementation where ZST is specifically ignored correct?

From the ABI doc mentioned in PR description:

‹vector_arg›: A vector_arg has one of the following types:

• Any vector type whose size is 16 bytes or less.
• A structure equivalent to such a vector type, where “equivalent” has the same meaning as for double_or_float.

And double_or_float is:

‹double_or_float›: A double_or_float is one of the following:

• A float or _Decimal32.
• A double or _Decimal64.
• A structure equivalent to one of the above. A structure is equivalent to a type 𝑇 if and only if it has exactly one member, which is either of type 𝑇 itself or a structure equivalent to type 𝑇.

Looking at the existing code, is the handling of ZST platform dependent? (#125854)

rust/compiler/rustc_target/src/callconv/s390x.rs

Lines 45 to 54 in 730ecfe

	if arg.is_ignore() {
	// s390x-unknown-linux-{gnu,musl,uclibc} doesn't ignore ZSTs.
	if cx.target_spec().os == "linux"
	&& matches!(&*cx.target_spec().env, "gnu" | "musl" | "uclibc")
	&& arg.layout.is_zst()
	{
	arg.make_indirect_from_ignore();
	}
	return;
	}

[uweigand]

This is unfortunately a bit more complicated due to historical reasons. For the purposes of deciding whether an aggregate is a "structure equivalent to a float or vector", zero-sized members in C are not ignored. The presence of a zero-sized bitfield, nested struct, or nested array member does make the aggregate not equivalent, i.e. it will be passed in GPRs and not in FPRs/VRs.

However, for C++ types, there are some zero-sized members that are ignored; this applies in particular to zero-sized base classes as well as C++20 empty data members (marked with [[no_unique_address]]). This is not currently documented in the ABI (but then again, the ABI doesn't document anything about C++ ...).

The question is, what does this now mean for Rust? My understanding is that Rust here attempts to be ABI-compatible with C only (not C++), and therefore I think it makes sense that Rust should be treated like C, i.e. zero-sized members are not ignored. Current Rust code seems to correctly implement this for FP types via is_single_fp_element, so I think this PR should do the same for vector types.

As to the second question, what if the whole argument is zero-sized? Both the strict reading of the ABI document and the actual GCC/clang implementation agree that this is passed via implicit pointer (which arguably doesn't make much sense, but we cannot change this anymore). So the current Rust code looks correct to me.

[taiki-e]

Thanks. I fixed this PR's implementation by adding&using is_single_vector_element based on is_single_fp_element.

rust/tests/assembly/s390x-vector-abi.rs

Lines 251 to 265 in 55d960e

	// CHECK-LABEL: vector_wrapper_with_zst_arg_small:
	// CHECK: .cfi_startproc
	// CHECK-NOT: vlgvg
	// CHECK-NEXT: br %r14
	#[cfg_attr(no_vector, target_feature(enable = "vector"))]
	#[no_mangle]
	unsafe extern "C" fn vector_wrapper_with_zst_arg_small(x: WrapperWithZst<i8x8>) -> i64 {
	unsafe { *(&x as *const WrapperWithZst<i8x8> as *const i64) }
	}
	// CHECK-LABEL: vector_wrapper_with_zst_arg:
	// CHECK: lg %r2, 0(%r2)
	// CHECK-NEXT: br %r14
	#[cfg_attr(no_vector, target_feature(enable = "vector"))]
	#[no_mangle]
	unsafe extern "C" fn vector_wrapper_with_zst_arg(x: WrapperWithZst<i8x16>) -> i64 {

[uweigand]

There's one more quirk I think we need to handle here: for single-element vector aggregates, the aggregate must have no padding, i.e. the aggregate size must match the vector size. (This is different for FP aggregates, where e.g. an 8-byte aligned struct with a single float element is passed like a double.)

[taiki-e]

Ah, good catch. Older versions of this PR had a check for that, but I accidentally removed it when I rewrote it based on is_single_fp_element.

Re-added size check and added a test for this case:

rust/tests/assembly/s390x-vector-abi.rs

Lines 265 to 271 in 1bd1416

	// https://github.com/rust-lang/rust/pull/131586#discussion_r1837071121
	// CHECK-LABEL: vector_wrapper_padding_arg:
	// CHECK: lg %r2, 0(%r2)
	// CHECK-NEXT: br %r14
	#[cfg_attr(no_vector, target_feature(enable = "vector"))]
	#[no_mangle]
	unsafe extern "C" fn vector_wrapper_padding_arg(x: WrapperAlign16<i8x8>) -> i64 {

[uweigand]

Thanks! This now looks fully correct from an s390x ABI perspective.

[bors]

☔ The latest upstream changes (presumably #133006) made this pull request unmergeable. Please resolve the merge conflicts.

[compiler-errors]

@bors r=compiler-errors,uweigand

[bors]

📌 Commit 7652e34 has been approved by compiler-errors,uweigand

It is now in the queue for this repository.

[RalfJung]

This PR extends #127731 #132173 (merged) 's ABI check to handle cases where vector target feature is disabled.
If we do not do ABI check, we run into the ABI problems as described in #116558 and #130869 (comment), and the problem of the compiler generating strange code (#131586 (comment)).

Is this an outdated part of the description? This PR does not seem to touch that logic at all.

[taiki-e]

@RalfJung That part is indeed outdated since it is handled by #132842 that opened after this PR and merged before this PR.¹

(As for the ABI check, only the addition of the test was eventually remained in this PR.)

Footnotes

1. To be more precise, that PR was merged after this PR was approved by the target maintainer: https://github.com/rust-lang/rust/pull/131586#issuecomment-2475414732 ↩