Support DataSourceAttribute in .Net Core.

[magol]

Description

We have following code

[TestMethod]
[DeploymentItem("MyTests.Configuration.xml")]
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", "|DataDirectory|\\MyTests.Configuration.xml", "MyTestCase", DataAccessMethod.Sequential)]
public void GetFiles_OnRootDirectory_CallsApiCorrectly()
{
    var foo = TestContext.DataRow["MyFooData"]);
}

with following xml file

<?xml version="1.0" encoding="utf-8" ?>
<Rows>
  <MyTestCase>
    <MyFooData>true</MyFooData>
  </MyTestCase>
  <MyTestCase>
    <MyFooData>false</MyFooData>
  </MyTestCase>

  <OtherTestCase>
    <MyFooData>false</MyFooData>
  </OtherTestCase>
</Rows>

Is TestContext.DataRow expected to come to MsTest v2, or is it any other way to do the same thing?

Steps to reproduce

1. Install .NET Core 2.0.0-preview2-006497
2. Target the test project to ,NET Core 2.0
3. Use MSTest.TestAdapter version 1.2.0-beta
4. Try to build

Expected behavior

The compile past

Actual behavior

I get following build error

error CS1061: 'TestContext' does not contain a definition for 'DataRow' and no extension method 'DataRow' accepting a first argument of type 'TestContext' could be found (are you missing a using directive or an assembly reference?)

Environment

.NET Core 2.0.0-preview2-006497
Visual Studio 15.3.0 preview 7.0
Windows 10

AB#2303200

[AbhitejJohn]

@magol: Apologies for the delay. The .Net Core version of the adapter does not support DataSource yet. I'll update the title and mark this as a request for adding in that support.
As a workaround, you could update to the latest beta and use DynamicData instead.

[magol]

@AbhitejJohn Thanks for the answer. What is you time framme to add this feature? When can I expert to be able to use this?

[AbhitejJohn]

@magol: Don't have one yet. Put this temporarily in S123 where it would most likely be addressed.

[nvianhd]

I hope MS still support DataRow on Dot Net Core

[AbhitejJohn]

@nvianhd: It still does and much more!

[magol]

@AbhitejJohn What is the status on this? The S123 milestone have past due by about 1 month.

[AbhitejJohn]

@magol : Sorry, we had a few other things that came up and this took the back seat. Tagging @sudiptadmsft and @pvlakshm to see when we can bring this in.

[omkarmore83]

I am waiting on this too.. @AbhitejJohn any ETA?

[rhyous]

Trying to move my Unit Tests to .NET Core 2.0. Of course, I have many row tests.
I'll have a look at Dynamic data. But the problem is all the existing row tests use DataSourceAttribute.

[alex182]

this would be great to have working

[magol]

Any progress on this?

[AbhitejJohn]

Sorry, I haven't been working on the framework for a while now. Tagging @cltshivash @pvlakshm .

[alenjalex-zz]

I am also waiting on this.

[sirkris]

More than a year since the initial post and still waiting. This is unacceptable, as it makes it impossible to make use of data-driven unit tests in .NET Core. Please fix this! I'm gonna have to hack in a really sloppy workaround for my test project to work in the meantime....

[cilerler]

As @AbhitejJohn already mentioned above DynamicData is way to go at this point. Here is an example for CSV reading

private static string[] SplitCsv(string input)
{
    var csvSplit = new Regex("(?:^|,)(\"(?:[^\"]+|\"\")*\"|[^,]*)", RegexOptions.Compiled);
    var list = new List<string>();
    foreach (Match match in csvSplit.Matches(input))
    {
        string value = match.Value;
        if (value.Length == 0)
        {
            list.Add(string.Empty);
        }

        list.Add(value.TrimStart(','));
    }
    return list.ToArray();
}

private static IEnumerable<string[]> GetData()
{
    IEnumerable<string> rows = System.IO.File.ReadAllLines(@"Resources\NameAddressCityStateZip.csv").Skip(1);
    foreach (string row in rows)
    {
        yield return SplitCsv(row);
    }
}

[TestMethod]
[DynamicData(nameof(GetData), DynamicDataSourceType.Method)]
//x [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", @"Resources\NameAddressCityStateZip.csv", "NameAddressCityStateZip#csv", DataAccessMethod.Sequential)]
public void TestMethod1(string input, string expected)
{
    // Arrange
    //x string input = _testContext.Properties["Data"].ToString(); //x _testContext.DataRow["Data"].ToString();
    //x string expected = _testContext.Properties["Expected"].ToString(); //x _testContext.DataRow["Expected"].ToString();
    var parser = _serviceProvider.GetService<Parser>();

    // Act
    string actual = parser.MultiParser(input, ModeType.NameAddressCityStateZipCountry).ToString();

    // Assert
    Assert.AreEqual(expected, actual);
}

[rhyous]

@cilerler It is NOT that easy.

Is the DynamicDataAttribute available in other .NET platforms yet?

We have a bunch of code that compiles to multiple .Net versions. The code and unit tests are shared between platforms. So if I change the unit tests to use DynamicData, I break them for .Net Framework. So basically, I cannot share unit test code.

At best I could add a compiler directive to all my unit tests so in .NET Framework it uses the old code and in .NetCore it uses the new code. Actually, I already did this in my open source projects, only the unit tests are simply not run in .NetCore as it doesn't support the DataSourceAttribute.

Not to mention that the GetData() method you have above 1) doesn't take into account Xml row tests or Excel row tests, and 2) is far too naive to be used by product code. We need to get this working for Csv, Xml, and Excel. I have a CsvParser, but I have to figure out what Xml parser to use or write my own, figure out what Excel parser to use or write my own, etc. Now I have to bring in either new code, which requires approval and sprint time, etc, or new open source projects, which means a security team involved to vet the open source project and paperwork and process.

So again, if it were that easy, we would have done it already.

[sirkris]

@rhyous is spot-on. Relying on DynamicData instead of just implementing this (is there some pressing reason not to?) is not a viable long-term solution to this problem, in my opinion.

[cilerler]

@rhyous

1. The very basic example I provided above is for demonstration purpose only to let others see how easy it is to implement it. I literally didn't expect no one to use it for production.
2. As you already mentioned above you have compiler directives which means it should not break your code base. (you can't expect everyone having a need to support multiple frameworks, many developers are developing fresh new projects and the method I presented may help them to continue with the latest stuff)
3. The situation is pretty simple writing your own parser VS waiting for years to get it from Microsoft. (Well, I usually prefer to write my own at this point in order to be able to use the latest stuff)

Therefore please take it into the consideration that it could be very easy workaround where things are not complicated as it in your projects.

[rhyous]

Wow. Let's ignore that unhelpful response and talk about implementation of a single solution in the right place, which is in this project, instead having customers implement many one-off solutions.

It would be better to implement this once than to have many users implement their own hacks. It looks like this feature is tagged with up-for-grabs so anyone can implement it. That is greateness of open source.

So if DynamicDataAttribute is the future, why not implement DataSourceAttribute in this project as a child of DynamicDataAttribute.

We could probably write an attribute that inherits from DynamicDataAttribute (except the class is sealed, but that is easy to change, it is an open source project after all) and simply have child class that acts exactly like the DynamicDataAttribute but has the signature of the DataSourceAttribute and we enhance the DataSourceAttribute to populate the GetData needs.

Assuming the posters here decide to fork testfx, implement this ourselves, and do a pull request. We could use the source for DataSourceAttribute and associated classes. Can someone at Microsoft comment on if that code can be made available?

[sebainones]

I'm facing the same situation. I was trying to move my Tests to .Net Core but for me is strictly necessary to use DataRow as my information comes only from the DB. So, without this, I wouldn't migrate to .Net Core.
@AbhitejJohn It's been more than a year since the post. Is there any ETA or at least do you have on your product backlog.
I want to know if it makes sense to wait or if I should be considering other options.
Thanks

[Justin-Lloyd]

+1 on the above. Unit testing does not look particularly well supported by Microsoft.

[hansmbakker]

DataSourceAttribute / DataRow would be required for https://github.com/Microsoft/Recognizers-Text to move to .Net Core for tests.

[AbhitejJohn]

@AbhitejJohn It's been more than a year since the post. Is there any ETA or at least do you have on your product backlog.

@sebainones : I haven't been getting much time since I've moved off of working on this area. @cltshivash should be able to help answer if this is on the immediate backlog. What @cilerler has added seems to be good start to unblock folks. It does create an API diff between .NetCore and .NetFramework but the DynamicData schematics was what we were hoping we could converge on.

[fhucko]

I have opened links presented here in comments, and they are useless or such voodoo that I do not understand it.
Is there a clean replace solution? I guess no. I will just write my own solution, like rest of the world.

[gburghardt]

I found a solution I am happy with using the CsvHelper NuGet package and the [DynamicData] attribute.

My answer on StackOverflow to What is the replacement for TestContext.DataRow[“MyColumnName”] shows all the details. A brief summary is here.

The CSV File (Example.csv)

A,B,IsLessThanZero
1,2,FALSE
3,-5,TRUE

Important: Make sure this CSV file is included in your test project and "Copy To Output Directory" is set to "Always" in the properties for the CSV file in Solution Explorer.

Data Transfer Object Used By CsvHelper

public class AdditionData
{
    public int A { get; set; }
    public int B { get; set; }
    public bool IsLessThanZero { get; set; }
}

The Test Class

[TestClass]
public class ExampleTests
{
    // HINT: Look in {Your Test Project Folder}\bin\{Configuration}\netcore3.1\FolderYourCsvFileIsIn for the CSV file.
    //       Change this path to work with your test project folder structure.
    private static readonly string DataFilePath = Path.GetDirectoryName(typeof(ExampleTests).Assembly.Location) + @"\FolderYourCsvFileIsIn\Example.csv";

    [TestMethod]
    [DynamicData(nameof(GetData), DynamicDataSourceType.Method)]
    public void AddingTwoNumbers(AdditionData data)
    {
        bool isLessThanZero = data.A + data.B < 0;

        Assert.AreEqual(data.IsLessThanZero, isLessThanZero);
    }

    private static IEnumerable<object[]> GetData()
    {
        using var stream = new StreamReader(DataFilePath);
        using var reader = new CsvReader(stream, new CsvConfiguration(CultureInfo.CurrentCulture));
        var rows = reader.GetRecords<AdditionData>();

        foreach (var row in rows)
        {
            yield return new object[] { row };
        }
    }
}

After building your solution, you will see a single test named "AddingTwoNumbers" in Test Explorer. Running this single test runs all variants defined in your CSV file.

[fforjan]

If anyone is here, if you are using XML, this is what we have attempted on our side :

[AttributeUsage(AttributeTargets.Method)]
    public class XMLDataSourceAttribute : Attribute, ITestDataSource
    {
        private readonly string path;

        public XMLDataSourceAttribute (string path)
        {
            this.path = path;
        }

        public IEnumerable<object[]> GetData(MethodInfo methodInfo)
        {       
             var parameters = methodInfo.GetParameters(); 

            var type = parameters[0].ParameterType;  // we assume there is only one parameter
            var constructor = type.GetConstructors().First(); // we assume there is only constructor with the right signature

            var content = XElement.Load(path);
            foreach(var line in content.Elements())
            {
                yield return new[] { constructor.Invoke(new[] { line }) };
            }
        }

        public string GetDisplayName(MethodInfo methodInfo, object[] data)
        {
            return data[0].ToString();
        }            
    }

    public class Data
    {
        public Data(XElement element)
        {
            this.From = element.Element("From").Value;
            this.To = element.Element("To").Value;
            this.Tooltip = element.Element("Tooltip").Value;
            this.BecauseMessage = element.Element("BecauseMessage").Value;
            this.HasError = Convert.ToBoolean(element.Element("HasError").Value);
        }

        public string From { get; private set; }
        public string To { get; private set; }
        public bool HasError { get; private set; }
        public string Tooltip { get; private set; }
        public string BecauseMessage { get; private set; }

        public override string ToString()
        {
            return string.Format(CultureInfo.InvariantCulture, "From '{0}' To '{1}'", this.From, this.To);
        }
    }

and we switched the attribute from

 [DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", @"|DataDirectory|\TestData.xml", "TestData",
            DataAccessMethod.Sequential)]
public void TestMethod() 

to

[XMLDataSource(@"TestData.xml")]
public void TestMethod(Data data)  

[sirkris]

Did the attempt succeed?

…

On Mon, Apr 18, 2022 at 6:37 AM Frederic Forjan ***@***.***> wrote: If anyone is here, if you are using XML, this is what we have attempted on our side : [AttributeUsage(AttributeTargets.Method)] public class CurrentDataRowAttribute : Attribute, ITestDataSource { private readonly string path; public CurrentDataRowAttribute(string path) { this.path = path; } public IEnumerable<object[]> GetData(MethodInfo methodInfo) { var content = XElement.Load(path); foreach(var line in content.Elements()) { yield return new[] { new Data(line) }; } } public string GetDisplayName(MethodInfo methodInfo, object[] data) { return ((Data)data[0]).ToString(); } } public class Data { public Data(XElement element) { this.From = element.Element("From").Value; this.To = element.Element("To").Value; this.Tooltip = element.Element("Tooltip").Value; this.BecauseMessage = element.Element("BecauseMessage").Value; this.HasError = Convert.ToBoolean(element.Element("HasError").Value); } public string From { get; private set; } public string To { get; private set; } public bool HasError { get; private set; } public string Tooltip { get; private set; } public string BecauseMessage { get; private set; } public override string ToString() { return string.Format(CultureInfo.InvariantCulture, "From '{0}' To '{1}'", this.From, this.To); } } and we switched the attribute from [DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", @"|DataDirectory|\TestData.xml", "TestData", DataAccessMethod.Sequential)] public void TestMethod() to [CurrentDataRow(@"TestData.xml")] public void TestMethod(Data data) — Reply to this email directly, view it on GitHub <#233 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFTJZJE2TWZPJW6P24TDEDVFVQR5ANCNFSM4DVWQD5A> . You are receiving this because you commented.Message ID: ***@***.***>

[rhyous]

@fforjan That is cool. I haven't fixed all my Xml ones. I have a project, github.com/rhyous/unittesting (also there is a NuGet package) that does Json, I haven't implement XML yet.

https://github.com/rhyous/UnitTesting

I kept meaning to get Xml done, but I ended up converting my Xml to json instead.

[fforjan]

@rhyous just curiositry, we have lot of comment into our XML. How did you solve the comment in JSON ?

[fforjan]

@sirkris we have done couple (3 test so far...) of migration and are going to discuss in our team. As @rhyous raised, the other option would be to move to JSON but we are not keen on that due to XML Comments

[rhyous]

Unlike Xml, Json doesn't support comments. They would be considered a syntax error.
You can put them in the json as a property or you can't have them.

{
  "Prop1": "value1",
  "Prop1Comment":"Some comment",
}

However, you don't have to deserialize the comment. The object you serialize with can exclude them and they simply won't be serialized.

public class MyObject
{
    public string Prop1 { get; set; }
}

However, I don't usually do that. My only comment is usually a TestName and Message property the is deserialized. The TestName should show up as the test name in VS. The message I pass into the Assert.

{
  "TestName": "Some Test 1",
  ...
  "Message":"The expectation was . . .",
}

[Evangelink]

Will move forward by closing this task as won't fix as exploratoration are showing there is no good abstraction level that would simplify a good chunk of complexity from users while still providing a good flexibility and strong typing.

We will revisit our view if we get some compelling stories.