Native AOT doesn't include new core lib APIs provided by NuGet packages

[cfbao]

Description

Native AOT builds don't include new APIs in core .NET libraries that are provided by NuGet packages.

Reproduction Steps

• Create a native AOT project targeting net8.
• Make it reference .NET 9 preview packages e.g. System.Text.Json & System.Threading.Channels.
• Use new APIs added in .NET 9, e.g. JsonSerializerOptions.IndentCharacter & Channel.CreateUnboundedPrioritized

Minimal repro repo: https://github.com/cfbao/dotnet-native-aot-bug

Expected behavior

Published native executables can run successfully.

Actual behavior

Exception thrown at runtime, e.g.

Unhandled Exception: System.MissingMethodException: Method not found: 'System.Threading.Channels.Channel`1<!!0> System.Threading.Channels.Channel.CreateUnboundedPrioritized()'.
   at Internal.Runtime.TypeLoaderExceptionHelper.CreateMissingMethodException(ExceptionStringID, String) + 0x40
   at Internal.Runtime.CompilerHelpers.ThrowHelpers.ThrowMissingMethodException(ExceptionStringID, String) + 0x9
   at Program.<<Main>$>d__0.MoveNext() + 0x15
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine&) + 0x3f
   at Program.<Main>$(String[]) + 0x34
   at Program.<Main>(String[] args) + 0x9
   at Chenfeng.NativeAot.ChannelPreview!<BaseAddress>+0xb46d0

Regression?

No response

Known Workarounds

Updating the project to target .NET 9 resolves this issue, but this may not be viable for all projects.

Configuration

Reproduced with

• project targeting .NET 8
• built with both .NET 8 and .NET 9 SDK
• Windows 10 x64

Unclear if specific to the OS.

Other information

This issue only happens with native AOT.
A self-contained and fully trimmed (but not native AOT compiled) build doesn't have this same issue, and can use the new APIs at runtime successfully.

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

Cc @sbomer FYI - I believe this is caused by ILCompiler package carrying its own copy of inbox assemblies and replacing everything with them. We run the usual .NET SDK assembly resolution that comes up with everything correctly, and then run ILCompiler targets logic that replaces every assembly that has a copy in the ILCompiler package with the ILCompiler version.

Bugs like this are the reason why we'd like to switch PublishAot to using the NativeAOT runtime pack (because then we get out of the business of replacing pieces of runtime pack with assemblies that are in the ILCompiler package). But this is blocked on SDK for reasons described in dotnet/sdk#37872 (comment) and numerous emails with the SDK team.

[jkotas]

Duplicate of #109872 . The fix being backported to .NET 8 and .NET 9