Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluate VS Test Platform as a better replacement for test discovery and execution #83

Open
wants to merge 15 commits into
base: master
Choose a base branch
from

Conversation

janaka
Copy link
Collaborator

@janaka janaka commented May 24, 2018

The purpose of this PR is to evaluate VS Test Platform in more depth, understand if it solves the below problems, and hence if we should move over.

Currently we use the dotnet test and dotnet vstest for test execution and generate TRX files which we parse for test discovery. This approach has several limitations.

  • Doesn't support multiple test projects per workspace.
  • Doesn't scale well to 100s of tests. We theorise performance will suffer.
  • Differing output formats between test frameworks complicates code and testing; has resulted in more bugs; and inability to support some functionality consistently across frameworks.

VS Test Platform is the underlying system, which has been open sourced, that dotnet vstest commands and Visual Studio test explorer is build upon. My understanding is that it has a superset of functionality compared to the dotnet cli and designed to enable scalable and performant test discovery and execution tools.

Note that VS Test Platform project itself doesn't have any client code, it's a backend + remote API. Hence this is also building on the great work over at https://github.com/gfrancischini/vscode-vstest-framework

Evaluation tasks are here

If any are missing please comment below.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant