tests: allow user to select tests via command line args #1211
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Concept ACK I still think it's an interesting idea for the future to look into existing test frameworks, which typically support running a subset of tests. (But that's no reason not to merge this, of course.) |
| @@ -28,7 +29,8 @@ | |||
|
|
|||
| #define CONDITIONAL_TEST(cnt, nam) if (COUNT < (cnt)) { printf("Skipping %s (iteration count too low)\n", nam); } else | |||
|
|
|||
| static int COUNT = 64; | |||
| static const int DEFAULT_COUNT = 64; | |||
| static int COUNT = DEFAULT_COUNT; | |||
There was a problem hiding this comment.
MSVC doesn't like that line apparently because DEFAULT_COUNT is not a constant, which may be true by a strict reading of the standard...
Maybe try #defining DEFAULT_COUNT instead, or set COUNT = DEFAULT_COUNT; only at the beginning of main().
(By the way, this shows that CI on MSVC will timeout after 60 min in case of failure instead of just aborting. This will be fixed by e433034)
`./tests a` results in the COUNT being set to 64 instead of erroring out as before. This is fixed in the next commit.
jonasnick
force-pushed
the
test-flags
branch
from
c4ffc97
to
3e404f2
Compare
|
Concept ACK |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces the ability for users to select specific tests to run via command line arguments. The aim is to eliminate the need to comment out parts of the
tests.cfile in order to speed up testing. The implementation uses similar command line flags as the benchmarks, for example:The approach taken in this PR may not be the most straightforward, but it ensures backwards compatibility. The shell script located at https://gist.github.com/jonasnick/5e37248e3fa5911cd41e1da2f5f4e395 can be used to test the changes introduced in this PR. I'd be happy to add more flags if needed.