Support tests sharding natively

[ThomasFerroAgicap]

Is your feature request related to a problem? Please describe.

We are trying to reduce the time of our CI by splitting the execution between multiple runners.

Describe the solution you'd like

Similar to what we can do in Playwright, I think that implementing sharding in the dotnet test CLI would greatly benefit many projects. One possible API could be based on the previous example:

# The first runner could execute this, running the first fifth of the tests in MyTests.dll
dotnet test "MyTests.dll" --shard 1/5
# Run the second fifth of the tests in MyTests.dll
dotnet test "MyTests.dll" --shard 2/5
# ...
dotnet test "MyTests.dll" --shard 3/5
dotnet test "MyTests.dll" --shard 4/5
dotnet test "MyTests.dll" --shard 5/5

Additional context

We already tried these two solutions that does not completely satisfy us:

Work around the issue by splitting our tests in multiple projects

We can have multiple test projects and configure our CI to split the projects to run between multiple runners. This is not ideal as it forces an architecture upon us, multiplying the projects count for no other purpose than the tests execution.

Keep a single test project and split the tests to run manually

We also tried to implement our own "sharding" by:

1. Listing all of the tests with the dotnet test --list-tests command;
2. Split the result in shards in a bash script;
3. Have one CI runner per shard.

The return of the dotnet test --list-tests command forces us to manipulate it multiple times, removing the headers that cannot be muted and manage the parameterized tests that appears multiple times. Those manipulations are tedious and seem hacky, compared to a native solution provided by the SDK.

[nohwnd]

@Evangelink do you want to consider something like this for the new testing platform, it won't be added to vstest since we are not adding new features there. Or this won't fit into our plan any time soon?

[Evangelink]

I think the concept is interesting but I don't know if it can really be a platform feature as it mainly depends upon the framework (e.g. are they discovering all tests before running, is there some guarantee of "order"...)

[Evangelink]

I cannot move the issue because it's a different org so I recreated it microsoft/testfx#4068.