command/pr, command/list: List individual affected tests

[YakDriver]

Proposal

Add an option to automatically discover individual tests (rather than all tests in affected files) affected by a PR. (NOTE: This would only be practical for changes to _test.go files.)

Description

For broad PRs and PRs targetting files with many tests, the resulting number of tests to be run grows exponentially. It would be super-handy if an option allowed looking deeper at the actual changes to find specific individual tests affected rather than test families.

Current behavior (82 tests to be run)

% tctest list 194782
Discovering tests for pr #194782 (https://github.com/terraform-providers/terraform-provider-aws/pull/194782)...
    TestAccAWSInstance

Running make testacc TESTARGS='-run=TestAccAWSInstance' results in 82 tests run.

Proposed behavior (1 test to be run)

Perhaps use -i for individual test discovery.

% tctest list -i 194782
Discovering tests for pr #194782 (https://github.com/terraform-providers/terraform-provider-aws/pull/194782)...
    TestAccAWSInstance_RootBlockDevice_KmsKeyArn

Running make testacc TESTARGS='-run=TestAccAWSInstance_RootBlockDevice_KmsKeyArn' results in 1 test run.

Design thoughts

Basic process:

1. Go through each changed line in the PR
2. Given a changed line in a _test.go file:
   • Scan back until func declaration
   • If func is TestAcc, add to the list of individual tests
   • If func is anything else (e.g., testAcc...(), find...()), initially, revert to current behavior and add test family to list (this could be analyzed with some sort of magic to identify individual tests but that may not be feasible)
3. Given a changed line in a file other than _test.go file, revert to current behavior and add corresponding test family(ies) to list