Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update SDK to preview 6 #104696

Merged
merged 8 commits into from
Jul 24, 2024
Merged

Update SDK to preview 6 #104696

merged 8 commits into from
Jul 24, 2024

Conversation

am11
Copy link
Member

@am11 am11 commented Jul 10, 2024

Related: dotnet/arcade#14925
Related #104737

@dotnet-issue-labeler dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Jul 10, 2024
@dotnet-policy-service dotnet-policy-service bot added the community-contribution Indicates that the PR has been added by a community member label Jul 10, 2024
@am11 am11 added area-Infrastructure and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Jul 10, 2024
Copy link
Contributor

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

@am11 am11 closed this Jul 10, 2024
@am11 am11 reopened this Jul 10, 2024
@am11 am11 marked this pull request as ready for review July 11, 2024 06:43
Fix `error NU1903: Package 'System.Text.Json' 8.0.0 has a known high severity vulnerability`
@akoeplinger
Copy link
Member

akoeplinger commented Jul 11, 2024

Given the prevalence of dotnet/dnceng#3304 (not yet fixed) and dotnet/dnceng#3100 (fixed in P7) I wonder if we should postpone the P6 upgrade and upgrade straight to P7

@am11
Copy link
Member Author

am11 commented Jul 11, 2024

Is it due to preview 6? I saw that dotnet/sdk repo is using preview 5 and yet it's also failing with the same error. Normally when we have a tracking / known issue, we do not block the progression.

@ViktorHofer
Copy link
Member

I discussed this with @mmitche offline and our consensus was that repos strongly affected by the NuGet restore issue should manually update to a P7 SDK (ideally to the one that we use inside the VMR).

Runtime definitely meets the bar but I would recommend to do that in a follow-up PR.

@am11
Copy link
Member Author

am11 commented Jul 12, 2024

#104765 is tracking inlinearray test failure. (behavior changed in #103612, we don't use live-builds everywhere..)

@am11
Copy link
Member Author

am11 commented Jul 12, 2024

@ilonatommy TBH that merge was really unnecessary. It took several hours to get it to good state.. :(

Merging main where there is no conflict is not needed because anyone can merge to main within 3-5 hours window when CI is running in PR, so it's a never-ending chase!

@ilonatommy
Copy link
Member

ilonatommy commented Jul 12, 2024

Merging was blocked by merge conflict coming from this one: #104751, without it we would not be able to close this PR.

@am11
Copy link
Member Author

am11 commented Jul 12, 2024

Merging was blocked by merge conflict coming from this one: #104751, without it we would not be able to close this PR.

There was no merge conflict. If two PRs are changing the same line with same delta, git is smart enough to mark it no-diff. i.e. 8a4a2cf (#104696) has no delta.

@ilonatommy
Copy link
Member

Oh, I did not know that, sorry. In this situation we can use /ba-g to unblock the merge and do it before CI finishes, as it was already green before and the merge change is non-breaking. Right?

@ViktorHofer
Copy link
Member

Let's wait until CI finishes. This isn't on any critical path.

@am11
Copy link
Member Author

am11 commented Jul 16, 2024

@ViktorHofer if there is nothing else, please get this in this week. You probably know this, it is not really non-critical either. Among other things, we should be testing builds with new SDK to find out issues in outerloop etc. The sooner we do that the faster we will know of issues, resolve and stabilize.

Copy link
Member

@ViktorHofer ViktorHofer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm inclined to hold off on this until we figured out the root cause that forced us to revert the SDK upgrade in Arcade: dotnet/arcade#14933, cc @mmitche

@ericstj
Copy link
Member

ericstj commented Jul 23, 2024

@mmitche can you let us know if there's anything blocking this update? You mentioned there:

Potentially unblocks some symbol publishing timeouts in build promotion

Did that turn out to be fixed by undoing the SDK update, or not?

@mmitche
Copy link
Member

mmitche commented Jul 23, 2024

@mmitche can you let us know if there's anything blocking this update? You mentioned there:

Potentially unblocks some symbol publishing timeouts in build promotion

Did that turn out to be fixed by undoing the SDK update, or not?

This was in arcade's usage of p6 in the publishing process. This would not be affected by upgrading here. In fact, I think it would be largely good to get a bit more dogfooding as we didn't get a chance to dig into that failure.

@ericstj
Copy link
Member

ericstj commented Jul 24, 2024

Wow, the elusive green PR validation build 😮 🟢

Are we OK merging this one @ViktorHofer?

@ViktorHofer ViktorHofer merged commit 4a8aca8 into dotnet:main Jul 24, 2024
163 checks passed
@ViktorHofer
Copy link
Member

Let's try things out :shipit:

@am11 am11 deleted the feature/update-sdk branch July 24, 2024 19:07
directhex pushed a commit that referenced this pull request Jul 26, 2024
* Update SDK to preview 6

* Update Shared.csproj

Fix `error NU1903: Package 'System.Text.Json' 8.0.0 has a known high severity vulnerability`

* Fix with existing version.

---------

Co-authored-by: Ilona Tomkowicz <[email protected]>
Co-authored-by: Viktor Hofer <[email protected]>
ilonatommy added a commit that referenced this pull request Aug 1, 2024
…05110)

* Remove support for older LLVM versions, and re-order linker flags

We have generally tried to support linking multiple versions of LLVM
within our git tree. Every new LLVM version moves symbols around between
libraries, and as a result, every new version of LLVM requires different
linker flags to build. The command line tool `llvm-config` should tell
you the exact flags you need, but it is a problem for us when
cross-compiling to rely on this, and as a result, we transcribe
the result of various llvm-config outputs directly into Mono's
CMakeLists.txt.

In an effort to support multiple versions of LLVM, flags common between
all supported versions were kept in one place, then the version-specific
flags appended afterwards. And this has worked fine for years. However:

1. Whilst we only link with `lld`, it is common for contributors and
   source-build to link with `gold`, `bfd`, or some other GNU-flavoured
   linker, where library order is essential
2. The list of common libraries to link has remained unchanged for
   years, but the symbol intra-dependencies may have changed a long time
   ago, so common symbol order cannot be assumed to remain valid between
   LLVM versions

This has resulted in a long-standing problem for people using e.g.
Debian or Ubuntu or GitHub CodeSpaces, rather than always building
with one of our dockerfiles representing our "real" build environment,
when targeting platforms which use Mono and link LLVM.

* Bumping clang and llvm - make docs less ambiguous. (#105401)

* Bump main to RC1 (#105338)

* Update SDK to preview 6 (#104696)

* Update SDK to preview 6

* Update Shared.csproj

Fix `error NU1903: Package 'System.Text.Json' 8.0.0 has a known high severity vulnerability`

* Fix with existing version.

---------

Co-authored-by: Ilona Tomkowicz <[email protected]>
Co-authored-by: Viktor Hofer <[email protected]>

* Change DefaultMaximumErrorResponseLength to KB from Byte (#105396)

* Change DefaultMaximumErrorResponseLength to KB from Byte

* Handle overflow

* Review feedback

* Fix warning for MakeGenericType annotation mismatch (#104921)

Fixes warning code when a generic type whose type parameters have DAM
annotations is used with MakeGenericType, over a type that doesn't
have matching annotations.

The code IL2070 used to mention the 'this' argument. Instead it should
have been IL2071 which mentions the generic argument as the cause of
the mismatch. Similar for MakeGenericMethod with IL2090 and IL2091.

* Set GCStressIncompatible on GenericContext tests (#104686)

Co-authored-by: Vladimir Sadov <[email protected]>

* Add runtime config parameter to force ijwhost to load assemblies in an isolated context (#105337)

* Add support for isolated load context in LoadInMemoryAssemblyInContext by passing -1 as loadContext
* Have ijwhost check a runtime config parameter to determine if it should run in an isolated load context
* Added test for ijwhost isolated load context runtime config option

* [RISC-V] Fix passing float and uint arguments in VM (#105021)

* Add tests

* Fix passing float and uint arguments in VM

* Change test lib name so it doesn't clash with managed DLL on Windows

* Fix platform analyzer attribute order for MacCatalyst (#105409)

We need to make sure the attribute for MacCatalyst comes _after_ the iOS one due to how MacCatalyst is a superset of iOS: https://learn.microsoft.com/en-us/dotnet/standard/analyzers/platform-compat-analyzer#platform-inclusion

This caused an error in aspnetcore in the latest dependency flow because the analyzer thought AesGcm is _only_ supported on MacCatalyst:
> error CA1416: (NETCORE_ENGINEERING_TELEMETRY=Build) This call site is reachable on all platforms. 'AesGcm.Decrypt(ReadOnlySpan<byte>, ReadOnlySpan<byte>, ReadOnlySpan<byte>, Span<byte>, ReadOnlySpan<byte>)' is only supported on: 'maccatalyst' 13.0 and later.

* Use correct `ExceptionArgument` value in  `System.IO.Pipelines` (#105418)

* Remove zlib from requirements script and instruction files (#105419)

* Remove zlib from requirements instructions

* Remove zlib from native requirements installation script

* Revert "Remove zlib from requirements script and instruction files (#105419)" (#105449)

This reverts commit 3ec6286.

* Ensure that WaitForPendingFinalizers has seen the expected Full GC count (#105289)

* Ensure that WaitForPendingFinalizers has seen the expected Full GC

* NativeAOT and some renames

* a testcase

* make the test not unsafe and make OuterLoop

* Use unsigned math when comparing collection ticks

* cast the diff to int when comparing gc ticks

* Migrate to zlib-ng, part 3: Remove zlib and zlib-intel source code and license mentions (second attempt) (#105371)

* Remove zlib/
* Remove zlib-intel/
* Remove third party notice
* Remove patches
* Remove version txts
* Remove cgmanifest.json entries
* Remove installer third party notice
* Update docs
---------
Co-authored-by: Jan Kotas <[email protected]>

* [browser] Trigger relink on `EmccMaximumHeapSize` change (#105027)

* Edit test + trigger relink.

* Remove logging to speed up the test + decrease loop runs to prevent "Browser has been disconnected" error.

* Feedback - properties are not bool-only anymore.

* Fix: workload needed when heap size set.

---------

Co-authored-by: Larry Ewing <[email protected]>

* Delete erroneous Socket test (#105448)

* Pull the python dependency from the EmsdkVersion where possible (#105437)

* Use `BinaryPrimitives` more in the ILCompiler (#105404)

* JIT: use VNVisitReachingVNs in IsVNNeverNegative (#105197)

* Set xunit env var to not print output for passing tests (#105392)

* Set xunit env var to not print output for passing tests

Fixes #103445

* Update xunit.console.targets

* Update xunit.console.targets

* Some more automated C# modernization in corelib (#105151)

* Fix IDE0056 on corelib (indexing can be simplified)

* Fix IDE (null check can be simplified)

* Fix IDE0078 (use pattern matching)

* Fix IDE0019 (use pattern matching)

* Fix IDE0066 (use switch expression)

* Fix IDE0250 (struct can be made readonly)

* Fix nullability warning and address PR feedback

* Address PR feedback and revert a downlevel change

* Wrap any `?? throw new`s that go beyond 120 characters

* Fix ECMA 355 Partition download links (#105454)

On https://github.com/dotnet/runtime/blob/main/docs/project/dotnet-standards.md the Partition with Notes download links using HTTP protocol fail to download in Chrome:

>Mixed Content: The site at 'https://github.com/' was loaded over a secure connection, but the file at 'https://download.microsoft.com/download/7/3/3/733AD403-90B2-4064-A81E-01035A7FE13C/MS%20Partition%20I.pdf' was redirected through an insecure connection. This file should be served over HTTPS. See https://blog.chromium.org/2020/02/protecting-users-from-insecure.html for more details.

(It might be reasonble for somebody else to followup fixing all domains on all pages with regex `\(http://([.a-z0-9-]+)` replacing with `(https://$1`, but I didn't test each blog site supports HTTPS.)

* Add swiftcall signature check for `mono_class_try_get_swift_error_class` (#105408)

* Add signature check for swiftcall

* Handle null values for swift_error_ptr

* Enable NativeAOT runtime tests on MacCatalyst (#102882)

This PR updates the CLRTest.Execute.Bash.targets file to set the apple run command for MacCatalyst. The command apple just-run used on Apple mobile is not permitted, and apple test requires the a test runner. Additionally, it is necessary to locate Info.plist in the Contents/ directory and the binary in Contents/MacOS/ within the bundle.

---------

Co-authored-by: Ivan Povazan <[email protected]>

* Resolving an antigen failure (#105260)

* Resolving an antigen failure

* Fix method accessibility so xunit doesn't complain

* Support field access on GetType() of T constrained to be Enum (#105351)

Adds trimming support for instance.GetType().GetFields(), where
instance is a variable of type `T` that is constrained to System.Enum.

This includes a change to have ILLink's TypeProxy track a
TypeReference instead of TypeDefinition, which was necessary to allow
TypeProxy to represent a generic parameter.

Note that this only supports the specific case where `GetType()` is
called on a variable of type `T` that is constrained to `Enum`. A
variable of type `Enum` is not supported, so the following will still
warn:


```csharp
static void M(Enum v) {
    v.GetType().GetFields();
}
```

* Update mono to support shuffle for constant inputs (#105299)

* Support mono creating xconst in a few more places

* Update mono to support shuffle for constant inputs

* Ensure that arm64 also accelerates shuffle for non-constant inputs

* Ensure OP_XZERO and OP_XONES are recognized as being constant

* Ensure shuffle creates a correct instruction when the fsig doesn't match the necessary parameter count

* Ensure that getting the index for floating-point shuffle is possible

* Ensure the right class handle is passed down to LLVM so overload resolution can function

* Make sure we update the original xconst if we mutate it

* Return a new constant and instead of mutating the existing one

* Insert relevant xcast nodes

* Add some asserts around the ecount

* Ensure we get the right element type

* Ensure we don't create nodes unnecessarily for create_elementwise

* Ensure that create_elementwise still works for other vector sizes

* Ensure indentation of switch cases is correct for Mono

* Make TooDeepJsonDocument test more consistent across platforms (#105445)

* Make TooDeepJsonDocument test more consistent across platforms

Run the test on a thread with as consistent a stack size as possible so that we don't inadvertently succeed due to having a really large stack.

* Disable test on mono

* Update the TypeLib embedding and add comments on API use (#105416)

There is an undocumented semantic of Win32 Resource APIs.
The missing semantic is that all resource type/name
strings are transparently converted to uppercase
when calling any of the Win32 Resource APIs.

We don't want to apply this undocumented semantic to the reader/writer API
so we document it instead. We are avoiding applying the behavior
since ReadyToRun scenarios are designed to be a byte for byte copy
of the resource, including name as it was written by other tooling.

* Update docs for ByRefLike with generics for work in .NET 10 (#103318)

Co-authored-by: Jan Kotas <[email protected]>

* Fix double printing in StressLog and simplify stresslog macros (#105420)

* Add support for nested types in the `corelib.h` parser for rooting descriptors (#105432)

* Fixed for assertion failure due to not checking if we are processing Eph samples (#105164)

* [PERF] Use correct python executable on windows in venv (#105451)

* Fix up Fuzzlyn CI scripts for new hardware intrinsics support (#105470)

1) Strip out the extensions in the seed name when using it for file/directory names, since the list of extensions is quite long
2) Limit the number of unreduced/uncategorized example seeds we show

* Try to re-enable DeepEquals_TooDeepJsonDocument_ThrowsInsufficientExecutionStackException test on mono (#105509)

* zlib-ng: avoid suppressing WD4242 and WD4244 (#105433)

WD4242 and WD4244 are compiler warnings that should not be suppressed because the warn about possible loss of data.

WD4242 shows up in zlib-ng/arch/*/slide_hash*.c files and comes from the arguments passed to the slide_hash_chain method.
WD4244 happens in Windows when building in Debug configuration, in various zlib-ng/deflate*.c files, and comes from the arguments passed to the check_match method.

Fixed by:

- Adding asserts to verify the values are below the maximum allowed for their type.
- Casting them the proper type before passing them as arguments to their methods.
- Removing the WD suppressions, which unfortunately also propagated to other unrelated cmake files.
- Fixed a similar loss of data error in an unrelated mono file where the warning suppression was propagated due to this.

* Delete outdated comments (#105519)

* Arm64/Sve: Add FFR register liveness tracking (#105348)

* Add tracking of FFR register

somewhat workable

code cleanup

Remove FFR

Add all the GetFfr*

wip

Work with MskCns() model

Use physReg approach

Remove commented prototypes

working

Remove bunch of unnecessary code

Remove SpecialImport from GetFFR/SetFFR/LoadFirstFaulting

some more code cleanup

some fixup

* Change condition for PhysReg

* jit format

* review feedback

* unspill for LoadVectorFirstFaulting as well

* Use the right opReg

* skip spilling tracking

* review feedback

* Use non-existent REG_FFR

* Do not reload from FFR for GetFfr()

* review feedback

* Make just GrabTemp

* fix build and formatting

* missed another build failure for arm

* Fix throwing exception when calling RunClassConstructor on a generic type with a static constructor (#105513)

* Fix throwing exception when calling RunClassConstructor on a generic type with a static constructor

#99183 seems to have done away with assuming that a generic type's static constructor was
always "initialized". As a result, if you call RunClassConstructor on it, the runtime would throw an exception.

Fixes #103891

* Apply suggestions from code review

---------

Co-authored-by: Jan Kotas <[email protected]>

* [RISC-V][LoongArch64] New passing info for floating-point structs (#103945)

* Replace StructFloatFieldInfoFlags with FpStructInRegistersInfo which carries also exact field sizes and offsets

* Replace StructFloatFieldInfoFlags with FpStruct::Flags in profiler

* Remove FpStructInRegistersInfo::FromOldFlags()

* Fix duplicating types in HandleInlineArray

* Remove signedness from FpStruct::IntKind because most probably we won't need it

* Remove old StructFloatFieldInfoFlags calculating routine

* Typo in TARGET_LOONGARCH64

* Remove m_returnedFpFieldOffsets from ArgIterator

* Add missing ENREGISTERED_PARAMTYPE_MAXSIZE condition to C# version of FpStruct info calculation

* Rename RISCV64PassStructInRegister to match settled casing for RiscV in class names

* Update hardcoded flags for float and double in ArgIteratorTemplate::ComputeReturnFlags()

This fixes JIT/HardwareIntrinsics/General/Vector* tests.

* Fix build on other platforms

* Update LoongArch to use FpStructInRegistersInfo

* Remove unused old flag masks

* LoongArch64 typo

Co-authored-by: Qiao Pengcheng <[email protected]>

* Missing FpStruct namespace

Co-authored-by: Qiao Pengcheng <[email protected]>

* Missing FpStruct namespace

Co-authored-by: Qiao Pengcheng <[email protected]>

* Missing FpStruct namespace

Co-authored-by: Qiao Pengcheng <[email protected]>

* Use FpStruct namespace everywhere in JIT

* JIT review

* Update StructFloatFieldInfoFlags description

* Revert to hitherto instruction set order as it's not the point of this PR

* Unify get{LoongArch,RiscV}64PassFpStructInRegistersInfo JIT interfaces

* Use JIT_TO_EE_TRANSITION instead of _LEAF because MethodTable::GetFpStructInRegistersInfo may throw

* Remove FpStruct::IntKind, we should have similar info in ClassLayout in JIT

* Change JIT interface to return a struct similar to CORINFO_SWIFT_LOWERING to facilitate code unification in the future

* Change JIT to use new Swift-like getFpStructLowering

* Cache CORINFO_FPSTRUCT_LOWERING

* Update LoongArch classifier to use CORINFO_FPSTRUCT_LOWERING

* Update StructFloatInfoFlags doc comment on C#

* Move StructFloatFieldInfoFlags and FpStructInRegistersInfo out of the JIT interface

* Merge LoongArch and RISC-V AOT calculation of FpStructInRegistersInfo because they were identical. Move it to Common\Internal/Runtime because it's no longer exposed in JIT interface.

* Don't zero-initialize CORINFO_FPSTRUCT_LOWERING

* Add note for CORINFO_FPSTRUCT_LOWERING::loweredElements type

---------

Co-authored-by: Qiao Pengcheng <[email protected]>

* Ensure we don't reuse temps when calling fgMorphArgs on LIR nodes (#105508)

* Ensure constant evaluation of shifts on xarch broadcast the operand to the correct size (#105487)

* Ensure constant evaluation of shifts on xarch broadcast the operand to the correct size

* Ensure we don't try to execute AVX2 code on unsupported platforms

* Use ConcurrentDictionary in runtimecounters test (#105520)

* Use ConcurrentDictionary in runtimecounters test

Fixes #105443

* Fix build break

* Fix issue 98506 - Excessive exceptions generated in StackTraceSymbols (#105530)

* Fix issue 98506 - Excessive exceptions generated in StackTraceSymbols

* Code review feedback

* Clean up some usages of LowLevelList<T> (#105407)

* Fix ShuffleThunk cache heap (#105480)

There was a problem with using heap from the related LoaderAllocator for
shuffle thunk cache heap. I have tested it again and it seems that the
issue is gone.

So I am removing the workaround, making the cache use LoaderAllocator
local heap.

Close #55697

* [browser] Fix computing destination sub path and publish extension target path in Wasm SDK (#105458)

* Bump flags to LLVM 19, not 16

---------

Co-authored-by: Larry Ewing <[email protected]>
Co-authored-by: Alexander Köplinger <[email protected]>
Co-authored-by: Ilona Tomkowicz <[email protected]>
Co-authored-by: Carlos Sánchez López <[email protected]>
Co-authored-by: Adeel Mujahid <[email protected]>
Co-authored-by: Viktor Hofer <[email protected]>
Co-authored-by: Ahmet Ibrahim Aksoy <[email protected]>
Co-authored-by: Sven Boemer <[email protected]>
Co-authored-by: Steve Pfister <[email protected]>
Co-authored-by: Vladimir Sadov <[email protected]>
Co-authored-by: Mike Oliphant <[email protected]>
Co-authored-by: Tomasz Sowiński <[email protected]>
Co-authored-by: xtqqczze <[email protected]>
Co-authored-by: Jan Kotas <[email protected]>
Co-authored-by: Stephen Toub <[email protected]>
Co-authored-by: Paulus Pärssinen <[email protected]>
Co-authored-by: Egor Bogatov <[email protected]>
Co-authored-by: Carl Walsh <[email protected]>
Co-authored-by: Milos Kotlar <[email protected]>
Co-authored-by: Ivan Povazan <[email protected]>
Co-authored-by: Tanner Gooding <[email protected]>
Co-authored-by: Aaron Robinson <[email protected]>
Co-authored-by: Jeremy Koritzinsky <[email protected]>
Co-authored-by: Mukund Raghav Sharma (Moko) <[email protected]>
Co-authored-by: Cameron Aavik <[email protected]>
Co-authored-by: Jakob Botsch Nielsen <[email protected]>
Co-authored-by: Kunal Pathak <[email protected]>
Co-authored-by: Qiao Pengcheng <[email protected]>
Co-authored-by: Mike McLaughlin <[email protected]>
Co-authored-by: Huo Yaoyuan <[email protected]>
Co-authored-by: Jan Vorlicek <[email protected]>
Co-authored-by: Marek Fišera <[email protected]>
agocke added a commit to agocke/runtime that referenced this pull request Aug 1, 2024
@jakobbotsch
Copy link
Member

I discussed this with @mmitche offline and our consensus was that repos strongly affected by the NuGet restore issue should manually update to a P7 SDK (ideally to the one that we use inside the VMR).

Runtime definitely meets the bar but I would recommend to do that in a follow-up PR.

What is the process for doing this, and when can we do that? The issue highlighted by @akoeplinger above is super disruptive.

@mmitche
Copy link
Member

mmitche commented Aug 13, 2024

Update the repo's global.json with the new SDK version. That said, this will get updated today in Arcade as part of the normal release process, then flow out to repos afterward.

@github-actions github-actions bot locked and limited conversation to collaborators Sep 14, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-Infrastructure community-contribution Indicates that the PR has been added by a community member
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

7 participants