TestContext.WriteLine() interleaves logs

[AbhishekMarda]

Describe the bug

I saw existing an bug that was closed : #386
This is similar, but in a slightly different context.

Essentially, when using TestContext.WriteLine() in a base class, it seems to not be thread-safe and interleaves logs from tests that exist in the inherited classes.

I found this when running in Azure Pipelines as part of a VSTest@2 task:

    - task: VSTest@3
      inputs:
        testSelector: 'testAssemblies'
        testAssemblyVer2: |
          **\testbin\**\*Tests.dll
        runSettingsFile: $(REPOROOT)\test\test.runsettings
        runInParallel: true
        publishTestResults: true
        testFilterCriteria: ${{ parameters.test_filter }}

Steps To Reproduce

In the CSproj, the versions are:

    <PackageReference Include="MSTest.TestAdapter" Version="2.2.10" />
    <PackageReference Include="MSTest.TestFramework" Version="2.2.10" />

The runsettings file:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
	<!-- MSTest adapter -->
	<MSTest>
		<CaptureTraceOutput>True</CaptureTraceOutput>
		<Parallelize>
			<Workers>4</Workers>
		</Parallelize>
	</MSTest>
</RunSettings>

Let there be a TestBase.cs

[TestClass]
public abstract class TestBase 
{
  public TestContext TestContext { get; set; }

  [TestInitialize]
  public void Init()
  { // do something, but just mentioning that all inherited classes have the same init method, in case that has an effect
  }
 
  protected LogMessage(string message)
  {
    TestContext.WriteLine($"{TestContext.TestName} | {message}");
  }
}

Say there's two classes

[TestClass]
public class MyTestClass1 : TestBase
{
  [TestMethod]
  public void Method1()
  {
     for(int i=0; i<10; i++)
     {
        LogMessage("Hello Method1!");
        Thread.Sleep( (new Random()).Next(1000,10000) );
     }
  }
}

[TestClass]
public class MyTestClass2 : TestBase
{
  [TestMethod]
  public void Method2()
  {
     for(int i=0; i<10; i++)
     {
        LogMessage("Hello Method2!");
        Thread.Sleep( (new Random()).Next(1000,10000) );
     }
  }
}

Expected behavior

For each class method, the output in Azure DevOps should be associated with its own method. I
For eg, for Method1:

Method1 | Hello Method1
Method1 | Hello Method1
...

and similar for Method2.

Actual behavior

Logs for Method1 will look as follows:

Method1 | Hello Method1
Method1 | Hello Method1
Method2 | Hello Method2

And something similar for Method2, where logs will be dispersed all across the different tests. The number of logs will still be 10+10=20, but one logfile may have 15 logs and the other may have 5, and which test's log they end up in is completely random.

Additional context

The TestContext is being set correctly, since the method name is being captured correctly and is a separate instance in each class instance. However, it seems that WriteLine() is using the same stream or is not thread safe?
The logger being used is trx as part of the VSTest task.

[Evangelink]

Hi @AbhishekMarda,

When bumping to latest version (3.2.2) of MSTest, the error is no longer showing up as described in the linked ticket. I have tested in trx and in VS.

Could you please bump and confirm this is good?

[AbhishekMarda]

@Evangelink that does seem to fix it, thanks. Seems like I was using an old version, although I'm curious as to why this was added as the default when setting up with the latest Visual Studio version. Maybe it's a nuget source configuration.

[Evangelink]

Cool thanks for confirming.

although I'm curious as to why this was added as the default when setting up with the latest Visual Studio version

I'm not sure to understand the question here. Do you mean, why is creating a new MSTest project from VS doesn't give you latest version? If so, this is sadly a limitation of the current templating system. For netfx, there is a strict version inserted into VS and because VS still allows to create projects before net462 (although no longer supported) we cannot bump MSTest version because it wouldn't work for old TFMs. As for net/netcore, the setup of the template comes from the selected SDK, so if you target net8.0, net7.0... you would end up with different templates (different version of MSTest and also different properties set).

I find it confusing but sadly this is working as is for everyone.