Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix DataRow issues related to serialization/deserialization #4078

Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/Adapter/MSTest.TestAdapter/Constants.cs
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@
internal static readonly TestProperty CssProjectStructureProperty = TestProperty.Register("CssProjectStructure", CssProjectStructureLabel, typeof(string), TestPropertyAttributes.Hidden, typeof(TestCase));

internal static readonly TestProperty TestClassNameProperty = TestProperty.Register("MSTestDiscoverer.TestClassName", TestClassNameLabel, typeof(string), TestPropertyAttributes.Hidden, typeof(TestCase));
internal static readonly TestProperty TestCaseIndexProperty = TestProperty.Register("TestCaseIndexProperty", TestCaseIndexLabel, typeof(int), TestPropertyAttributes.Hidden, typeof(TestCase))

Check failure on line 68 in src/Adapter/MSTest.TestAdapter/Constants.cs

View check run for this annotation

Azure Pipelines / microsoft.testfx (Build Linux Release)

src/Adapter/MSTest.TestAdapter/Constants.cs#L68

src/Adapter/MSTest.TestAdapter/Constants.cs(68,195): error CS1002: (NETCORE_ENGINEERING_TELEMETRY=Build) ; expected

Check failure on line 68 in src/Adapter/MSTest.TestAdapter/Constants.cs

View check run for this annotation

Azure Pipelines / microsoft.testfx (Build Linux Release)

src/Adapter/MSTest.TestAdapter/Constants.cs#L68

src/Adapter/MSTest.TestAdapter/Constants.cs(68,195): error CS1002: (NETCORE_ENGINEERING_TELEMETRY=Build) ; expected

Check failure on line 68 in src/Adapter/MSTest.TestAdapter/Constants.cs

View check run for this annotation

Azure Pipelines / microsoft.testfx (Build Linux Release)

src/Adapter/MSTest.TestAdapter/Constants.cs#L68

src/Adapter/MSTest.TestAdapter/Constants.cs(68,195): error CS1002: (NETCORE_ENGINEERING_TELEMETRY=Build) ; expected

Check failure on line 68 in src/Adapter/MSTest.TestAdapter/Constants.cs

View check run for this annotation

Azure Pipelines / microsoft.testfx (Build Linux Release)

src/Adapter/MSTest.TestAdapter/Constants.cs#L68

src/Adapter/MSTest.TestAdapter/Constants.cs(68,195): error CS1002: (NETCORE_ENGINEERING_TELEMETRY=Build) ; expected

Check failure on line 68 in src/Adapter/MSTest.TestAdapter/Constants.cs

View check run for this annotation

Azure Pipelines / microsoft.testfx (Build Linux Release)

src/Adapter/MSTest.TestAdapter/Constants.cs#L68

src/Adapter/MSTest.TestAdapter/Constants.cs(68,195): error CS1002: (NETCORE_ENGINEERING_TELEMETRY=Build) ; expected

Check failure on line 68 in src/Adapter/MSTest.TestAdapter/Constants.cs

View check run for this annotation

Azure Pipelines / microsoft.testfx (Build Linux Debug)

src/Adapter/MSTest.TestAdapter/Constants.cs#L68

src/Adapter/MSTest.TestAdapter/Constants.cs(68,195): error CS1002: (NETCORE_ENGINEERING_TELEMETRY=Build) ; expected

Check failure on line 68 in src/Adapter/MSTest.TestAdapter/Constants.cs

View check run for this annotation

Azure Pipelines / microsoft.testfx (Build Linux Debug)

src/Adapter/MSTest.TestAdapter/Constants.cs#L68

src/Adapter/MSTest.TestAdapter/Constants.cs(68,195): error CS1002: (NETCORE_ENGINEERING_TELEMETRY=Build) ; expected

Check failure on line 68 in src/Adapter/MSTest.TestAdapter/Constants.cs

View check run for this annotation

Azure Pipelines / microsoft.testfx (Build Linux Debug)

src/Adapter/MSTest.TestAdapter/Constants.cs#L68

src/Adapter/MSTest.TestAdapter/Constants.cs(68,195): error CS1002: (NETCORE_ENGINEERING_TELEMETRY=Build) ; expected

Check failure on line 68 in src/Adapter/MSTest.TestAdapter/Constants.cs

View check run for this annotation

Azure Pipelines / microsoft.testfx (Build Linux Debug)

src/Adapter/MSTest.TestAdapter/Constants.cs#L68

src/Adapter/MSTest.TestAdapter/Constants.cs(68,195): error CS1002: (NETCORE_ENGINEERING_TELEMETRY=Build) ; expected

Check failure on line 68 in src/Adapter/MSTest.TestAdapter/Constants.cs

View check run for this annotation

Azure Pipelines / microsoft.testfx (Build Linux Debug)

src/Adapter/MSTest.TestAdapter/Constants.cs#L68

src/Adapter/MSTest.TestAdapter/Constants.cs(68,195): error CS1002: (NETCORE_ENGINEERING_TELEMETRY=Build) ; expected

internal static readonly TestProperty DeclaringClassNameProperty = TestProperty.Register("MSTestDiscoverer.DeclaringClassName", DeclaringClassNameLabel, typeof(string), TestPropertyAttributes.Hidden, typeof(TestCase));

Expand Down Expand Up @@ -131,6 +132,7 @@
/// These Property names should not be localized.
/// </summary>
private const string TestClassNameLabel = "ClassName";
private const string TestCaseIndexLabel = "TestCaseIndex";
private const string DeclaringClassNameLabel = "DeclaringClassName";
private const string IsAsyncLabel = "IsAsync";
private const string TestCategoryLabel = "TestCategory";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -460,6 +460,7 @@ private static bool ProcessITestDataSourceTests(UnitTestElement test, Reflection
try
{
discoveredTest.TestMethod.SerializedData = DataSerializationHelper.Serialize(d);
discoveredTest.TestMethod.ActualData = d;
discoveredTest.TestMethod.DataType = DynamicDataType.ITestDataSource;
}
catch (SerializationException ex)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ internal UnitTestResult[] RunTestMethod()
{
if (_test.DataType == DynamicDataType.ITestDataSource)
{
object?[]? data = DataSerializationHelper.Deserialize(_test.SerializedData);
object?[]? data = _test.ActualData ?? DataSerializationHelper.Deserialize(_test.SerializedData);
TestResult[] testResults = ExecuteTestWithDataSource(null, data);
results.AddRange(testResults);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,9 +76,11 @@ internal static UnitTestElement ToUnitTestElement(this TestCase testCase, string
Constants.TestIdGenerationStrategyProperty,
(int)TestIdGenerationStrategy.FullyQualified);

int? testCaseIndex = testCase.GetPropertyValue(Constants.TestCaseIndexProperty) as int?;

TestMethod testMethod = testCase.ContainsManagedMethodAndType()
? new(testCase.GetManagedType(), testCase.GetManagedMethod(), testCase.GetHierarchy()!, name, testClassName!, source, isAsync, testCase.DisplayName, testIdGenerationStrategy)
: new(name, testClassName!, source, isAsync, testCase.DisplayName, testIdGenerationStrategy);
? new(testCase.GetManagedType(), testCase.GetManagedMethod(), testCase.GetHierarchy()!, name, testClassName!, source, isAsync, testCase.DisplayName, testIdGenerationStrategy, testCaseIndex)
: new(name, testClassName!, source, isAsync, testCase.DisplayName, testIdGenerationStrategy, testCaseIndex);
var dataType = (DynamicDataType)testCase.GetPropertyValue(Constants.TestDynamicDataTypeProperty, (int)DynamicDataType.None);
if (dataType != DynamicDataType.None)
{
Expand Down
36 changes: 33 additions & 3 deletions src/Adapter/MSTest.TestAdapter/ObjectModel/TestMethod.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System.Collections.Concurrent;
using System.Collections.ObjectModel;
using System.Diagnostics.CodeAnalysis;

Expand Down Expand Up @@ -28,20 +29,23 @@ public sealed class TestMethod : ITestMethod
private string? _declaringClassFullName;
private string? _declaringAssemblyName;

private static readonly ConcurrentDictionary<int, object?[]> DataDictionary = new();
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notes:

  • This is bad. The dictionary in theory can grow up indefinitely.
  • This PR is mostly to open a discussion about the possibilities to improve the situation for some parameterized tests broken scenarios.

@Evangelink thoughts? I feel like some of the parameterized test issues could be fixable by trying to move forward in this direction without the need for breaking changes?

private static int s_currentTestIndex;

public TestMethod(string name, string fullClassName, string assemblyName, bool isAsync)
: this(null, null, null, name, fullClassName, assemblyName, isAsync, null, TestIdGenerationStrategy.FullyQualified)
{
}

internal TestMethod(string name, string fullClassName, string assemblyName, bool isAsync, string? displayName,
TestIdGenerationStrategy testIdGenerationStrategy)
: this(null, null, null, name, fullClassName, assemblyName, isAsync, displayName, testIdGenerationStrategy)
TestIdGenerationStrategy testIdGenerationStrategy, int? testCaseIndex = null)
: this(null, null, null, name, fullClassName, assemblyName, isAsync, displayName, testIdGenerationStrategy, testCaseIndex)
{
}

internal TestMethod(string? managedTypeName, string? managedMethodName, string?[]? hierarchyValues, string name,
string fullClassName, string assemblyName, bool isAsync, string? displayName,
TestIdGenerationStrategy testIdGenerationStrategy)
TestIdGenerationStrategy testIdGenerationStrategy, int? testCaseIndex = null)
{
Guard.NotNullOrWhiteSpace(assemblyName);

Expand All @@ -64,6 +68,7 @@ internal TestMethod(string? managedTypeName, string? managedMethodName, string?[
ManagedTypeName = managedTypeName;
ManagedMethodName = managedMethodName;
TestIdGenerationStrategy = testIdGenerationStrategy;
Index = testCaseIndex ?? Interlocked.Increment(ref s_currentTestIndex);
}

/// <inheritdoc />
Expand All @@ -72,6 +77,8 @@ internal TestMethod(string? managedTypeName, string? managedMethodName, string?[
/// <inheritdoc />
public string FullClassName { get; }

internal int Index { get; }

/// <summary>
/// Gets or sets the declaring assembly full name. This will be used while getting navigation data.
/// This will be null if AssemblyName is same as DeclaringAssemblyName.
Expand Down Expand Up @@ -137,6 +144,29 @@ public string? DeclaringClassFullName
/// </summary>
internal string?[]? SerializedData { get; set; }

private object?[]? _actualData;

[DisallowNull]
internal object?[]? ActualData
{
get
{
if (_actualData is not null)
{
return _actualData;
}

DataDictionary.TryGetValue(Index, out _actualData);
return _actualData;
}

set
{
_actualData = value;
DataDictionary.TryAdd(Index, value);
}
}

/// <summary>
/// Gets or sets the test group set during discovery.
/// </summary>
Expand Down
2 changes: 2 additions & 0 deletions src/Adapter/MSTest.TestAdapter/ObjectModel/UnitTestElement.cs
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,8 @@ internal TestCase ToTestCase()
testCase.SetPropertyValue(Constants.TestClassNameProperty, TestMethod.FullClassName);
}

testCase.SetPropertyValue(Constants.TestCaseIndexProperty, TestMethod.Index);

IReadOnlyCollection<string?> hierarchy = TestMethod.Hierarchy;
if (hierarchy is { Count: > 0 })
{
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using Microsoft.Testing.Platform.Acceptance.IntegrationTests;
using Microsoft.Testing.Platform.Acceptance.IntegrationTests.Helpers;
using Microsoft.Testing.Platform.Helpers;

namespace MSTest.Acceptance.IntegrationTests;

[TestGroup]
public class DataRowTests : AcceptanceTestBase
{
private const string SourceCode = """
#file DataSourceTests.csproj
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>$TargetFramework$</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<OutputType>Exe</OutputType>
<LangVersion>preview</LangVersion>
<EnableMSTestRunner>true</EnableMSTestRunner>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="$MicrosoftNETTestSdkVersion$" />
<PackageReference Include="MSTest.TestAdapter" Version="$MSTestVersion$" />
<PackageReference Include="MSTest.TestFramework" Version="$MSTestVersion$" />
</ItemGroup>
</Project>

#file MyTestClass.cs
using Microsoft.VisualStudio.TestTools.UnitTesting;

[TestClass]
public class MyTestClass
{
[DataTestMethod]
[DataRow((byte)0, new object[] { (byte)0 })]
[DataRow((short)0, new object[] { (short)0 })]
[DataRow((long)0, new object[] { (long)0 })]
public void CheckNestedInputTypes(object org, object nested)
{
Assert.AreEqual(org.GetType(), (((object[])nested)[0].GetType()));
}
}
""";

private readonly AcceptanceFixture _acceptanceFixture;

public DataRowTests(ITestExecutionContext testExecutionContext, AcceptanceFixture acceptanceFixture)
: base(testExecutionContext) => _acceptanceFixture = acceptanceFixture;

public async Task TestDataRowNumericalInArrayDoesNotLoseOriginalType()
{
using TestAsset generator = await TestAsset.GenerateAssetAsync(
"DataRowTests",
SourceCode
.PatchCodeWithReplace("$MSTestVersion$", MSTestVersion)
.PatchCodeWithReplace("$MicrosoftNETTestSdkVersion$", MicrosoftNETTestSdkVersion)
.PatchCodeWithReplace("$TargetFramework$", TargetFrameworks.NetCurrent.Arguments),
addPublicFeeds: true);

await DotnetCli.RunAsync(
$"build {generator.TargetAssetPath} -c Release",
_acceptanceFixture.NuGetGlobalPackagesFolder.Path,
retryCount: 0);

var testHost = TestHost.LocateFrom(generator.TargetAssetPath, "DataSourceTests", TargetFrameworks.NetCurrent.Arguments);

TestHostResult result = await testHost.ExecuteAsync();
result.AssertExitCodeIs(ExitCodes.Success);
result.AssertOutputContainsSummary(failed: 0, passed: 3, skipped: 0);
}
}
Loading