Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable running perftests via Test Explorer in Visual Studio #3279

Merged
merged 5 commits into from
Dec 25, 2024

Conversation

nickshulman
Copy link
Contributor

This change makes it so the default value of "RunPerfTests" is true. This means that if you run the test in Visual Studio using either Test Explorer or ReSharper's Unit Test Explorer, the test will actually get run. This change does not affect TestRunner (or SkylineTester) because TestRunner always explicitly sets RunPerfTests to false if it has not been specified on the command line.

Also, this change makes the default for "RunSmallMoleculeTestVersions" true as well.

What this does mean is that if you tell Visual Studio to run all of the tests, the perf tests will get run in addition to all of the other tests. My understanding is that nobody really ever tells Visual Studio to run all the tests anymore (probably because it's faster to run all the tests in parallel via SkylineTester).

This change makes it much easier to debug a single perf test, or even to run a single perf test with code coverage via ReSharper.

This change makes it so the default value of "RunPerfTests" is true.
This means that if you run the test in Visual Studio using either Test Explorer or ReSharper's Unit Test Explorer, the test will actually get run.
This change does not affect TestRunner (or SkylineTester) because TestRunner always explicitly sets RunPerfTests to false if it has not been specified on the command line.

Also, this change makes the default for "RunSmallMoleculeTestVersions" true as well.
@chambm
Copy link
Member

chambm commented Dec 13, 2024

I usually have TestRunner as my startup project and just change its debug commandline (which always has perftests=on) to test=whatever. Then I can F5 or Ctrl-F5 for easy running/debugging, which you can't do with Visual Studio testing AFAIK unless your editor focus is in the test method.

With the old way of doing code coverage on TC using vstest, your change here would have caused perftests to run under code coverage which would have been bad. But the current way uses TestRunner with perftests=off, so it won't affect that.

I wonder how this would affect the "continuous coverage" mode which was accidentally enabled on one of my VS sessions recently. It was pretty unusable even without perf tests though. :)

@bspratt
Copy link
Member

bspratt commented Dec 13, 2024

I'd be in favor of this, but as one who uses TestRunner/SkylineTester pretty much exclusively I'll defer to others.

@nickshulman nickshulman merged commit 4f7e262 into master Dec 25, 2024
11 checks passed
@nickshulman nickshulman deleted the Skyline/work/20241213_RunPerfTestsDefault branch December 25, 2024 04:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants