Add command-line flags for more control over test cases to run and how they're handled

Add command-line flags for more control over test cases to run and how they're handled #1794