Hierarchy view vs. data-driven tests

[csoltenborn]

Google Test Adapter currently considers improved support for the hierarchy view. In particular, we want to improve on the visual representation of parameterized tests (the gtest equivalent of data-driven tests). The GTA issue csoltenborn/GoogleTestAdapter#210 has an example screenshot of what we have in mind.

And that's pretty much already the question: What are benefits/drawbacks of handling such tests via the hierarchy view vs. via explicit data-driven tests? Here are my thoughts - please correct me where I'm wrong and add the stuff I've been missing. And of course, it would be nice if that information would become part of the according docs :-)

Hierarchy view

• (+) all tests (with all values) are displayed after discovery
• (+) each test (for a particular value) can be executed alone
• (-) not available before VS2017

Data-driven tests

• (+) available for VS2012-2015
• (-) after discovery, only the "base test" itself is shown in the test explorer - no way to already see the particular values or even to execute the test for a particular value
• (-) values have to be saved in property bags to be displayed in test explorer
  • How do we do this? The above link only says that is has to be done, but not how
• (-) even after all tests are executed (and thus all values can be seen in the test explorer), it's not possible to execute a single test
  • is this true? Technically, a TestCase contains all information needed to create command line params for gtest which result in that test be executed. In the case of data-driven tests, this would be more difficult, because part of the information would be stored within the property bag as described above, but it would still be feasible for us... but does the test framework/test explorer allow this?

[frboyer]

Also, if values in a data driven test come from an external file and we want to list them in the Test Explorer (as they are listed by "--gtest_list_tests"), currently there does not seem to be a way to force a "refresh" of the tests list (force to rerun the test discovery without recompiling). It's possible to close the solution and reopen it, but it's a bit cumbersome.

[mayankbansal018]

@PBoraMSFT , @pvlakshm , can you please respond to this.

[singhsarab]

Adding values to property bag.
It can done by registering TestProperty and then Setting the property on the test case object. References from MSTest adapter:
https://github.com/Microsoft/testfx/blob/e50e1f784e82a5f29fd8cdbf0d91147d33880d45/src/Adapter/MSTest.CoreAdapter/Constants.cs#L33
https://github.com/Microsoft/testfx/blob/3197e7bb9f7fd4219c7855be575fb5def205aff7/src/Adapter/MSTest.CoreAdapter/ObjectModel/UnitTestElement.cs#L106

Executing single test.
It is possible to select and run single test, depends on the implementation of the adapter.
If the TestCase has some unique information added while discovery, same can be used later for execution.

Adding @ManishJayaswal @kendrahavens as it is an IDE related query.

[kendrahavens]

Hey, I'm sorry it took a bit to get this routed to us. Is there still interest in this issue with the current options?

If you are interested in what behavior would fit well with the Test Explorer, I can provide an example of how xUnit data-driven tests are currently displayed. It's similar to your example screenshot above.

Currently, the GTest adapter handles it as demonstrated below. You are able to see the input parameters and you can run only a single parameterized test without running the others:

[Evangelink]

@csoltenborn Is this issue still relevant? As far as I can tell from the description, it seems everything is covered.

[csoltenborn]

Not really, partly because I'm not actively working on GTA at the moment... feel free to close - in the worst case, I will reopen (or ask for it).