FileNotFoundException for Microsoft.Testing.Platform.dll using EnableMSTestRunner with netstandard2.0

[jeremy-visionaid]

Describe the bug

Microsoft.Testing.Platform.dll, etc. are not being copied to the output directory via _CopyFilesMarkedCopyLocal when targeting netstandard2.0

Steps To Reproduce

Add multi-targeting to a test project like so:

  <PropertyGroup>
    <TargetFrameworks>netstandard2.0;net8.0</TargetFrameworks>
    <OutputType>Exe</OutputType>

    <IsPackable>false</IsPackable>
    <IsTestProject>true</IsTestProject>
    <EnableMSTestRunner>true</EnableMSTestRunner>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.11.1" />
    <PackageReference Include="MSTest" Version="3.6.2" />
  </ItemGroup>

Expected behavior

Tests can be run via Test Explorer and/or running the test binary

Actual behavior

Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Testing.Platform, Version=1.4.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
   at TestingPlatformEntryPoint.Main(String[] args)
   at TestingPlatformEntryPoint.<Main>(String[] args)

Additional context

It appears to work OK for net6.0 and net8.0...

MSTest v3.6.2 (UTC 30/10/2024) [win-x64 - .NET 8.0.10]
...

[Evangelink]

@jeremy-visionaid I confirm the behavior and I was going to close as by-design because I thought console app weren't supporting netstandard2.0 but it appears it does so this seems to be a bug on our side.

Out of curiosity, may I ask what's your "goal" in testing as netstandard? I have never seen such kind of test.

[thomhurst]

@jeremy-visionaid I confirm the behavior and I was going to close as by-design because I thought console app weren't supporting netstandard2.0 but it appears it does so this seems to be a bug on our side.

Out of curiosity, may I ask what's your "goal" in testing as netstandard? I have never seen such kind of test.

@Evangelink are you sure? Net standard isn't a runtime, it's more like an interface that both net framework and net core implement. So it shouldn't be possible to test or run a console app because you wouldn't have an actual runtime. So by design actually makes sense.

I made a basic console app and switched the target framework to netstandard2.0 and it indeed doesn't run (as expected.)

[jeremy-visionaid]

Ha, of course you're both right. I'm somewhat surprised at the actual behaviour there given that it shouldn't be runnable.

The background is that I was making some contributions to an upstream library, and was tired and copied the TFMs from a library to a test project to ensure it was being tested for all the combinations of dependencies that were being built (There were inconsistent TFMs for the projects across the wider solution, so it was non-trivial to reason which TFMs would be resolved for the dependencies). I was thinking that it would have been nice to just set the same ones for the test project, but didn't quite think through the consequences. Whether it's reasonable/valid or not, libraries/NuGets could do something like <TargetFrameworks>netstandard2.0;net48;net8.0</TargetFrameworks> and have code that behaves differently for all the TFMs. Tests targeting netstandard2.0 show up in the Test Explorer, but do not run (EnableMSTestRunner is something of an aside).

I have changes lined up for the upstream to target more modern TFMs and make them consistent, so the underlying reason for doing it won't exist for very long, but the behaviour for MSTest was nonetheless curious/surprising. Whether anything should be done about it is another matter!

[Evangelink]

@Evangelink are you sure? Net standard isn't a runtime, it's more like an interface that both net framework and net core implement. So it shouldn't be possible to test or run a console app because you wouldn't have an actual runtime. So by design actually makes sense.

That's what I thought! Interestingly, dotnet run/dotnet exec don't work but VS can run a netstandard app, I am curious how they do it. @MarcoRossignoli any idea?

[MarcoRossignoli]

That's what I thought! Interestingly, dotnet run/dotnet exec don't work but VS can run a netstandard app, I am curious how they do it. @MarcoRossignoli any idea?****

I don't know actually, with a breakpoint and process explorer we can check, I suppose it's using a dotnet muxer and passing the target dll like dotnet exec lib.dll the main is in the metadata and so I suppose it will work using the "chosen" dotnet version.

[nohwnd]

It works because the dll you are producing is internally an executable (it has entrypoint and everything), and cmd uses CreateProcessW to run it. It does not care about the extension, but most likely checks the "magic" header in the file, I cannot find evidence for this, and I am not so willing to look at the source code of cmd if I could even find it :D but if you rename the file to .fll it still works, and I don't see registry query for extension handling, but I do see query for image handling).

Once the process is started, .NET Framework CLR is bootstrapped by windows (because it knows about .NET Framework apps), and the execution is then handed over to .NET Framework, and the program runs as normal.

(We are using absolutely the same in our integration testst in testfx, where we produce DLLs, not exe for .NET framework targets, and we hand them over to cmd to execute them.)

In VS it is the same, there the debug host is also calling cmd.

[nohwnd]

For .NET technically it can also work, the netstandard2.0 built dll is not internally special, but netstandard does not produce runtimeconfg.json during build, because it is not meant to run.

dotnet exec  netstandard2.0\ConsoleApp1.dll
A fatal error was encountered. The library 'hostpolicy.dll' required to execute the application 
was not found in 'S:\w\.NETConf2024\code\ConsoleApp1\ConsoleApp1\bin\Debug\netstandard2.0\'.
Failed to run as a self-contained app.
  - The application was run as a self-contained app because 
     '...\Debug\netstandard2.0\ConsoleApp1.runtimeconfig.json' was not found.
  - If this should be a framework-dependent app,
     add the '...\Debug\netstandard2.0\ConsoleApp1.runtimeconfig.json' file and specify the appropriate framework.

But if you produce one, e.g. by building net9.0 TFM and taking that, or by simply writing it, to tell the app which runtime it should choose, the netstandard2.0 dll will run as well via dotnet.

dotnet exec --runtimeconfig net9.0\ConsoleApp1.runtimeconfig.json  netstandard2.0\ConsoleApp1.dll
Hello, World!
.NET 9.0.0-rc.1.24431.7

We use this in testhost in vstest to be able to run as the desired TFM, and as fallback for newer TFMs, but we do not use it to run netstandard2.0 apps, and I don't think we should start doing that.