Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Universal Test X86 Release Fabric tests consistently fail in CI #14114

Open
Yajur-Grover opened this issue Nov 14, 2024 · 2 comments
Open

Universal Test X86 Release Fabric tests consistently fail in CI #14114

Yajur-Grover opened this issue Nov 14, 2024 · 2 comments

Comments

@Yajur-Grover
Copy link
Contributor

Problem Description

The Universal Test X86 Release Fabric tests recently are failing more often and regularly than usual CI test issues. When they fail, they fail with the following error:

Failed Future2Test.Futureint_Then_DestroyLambdaAfterInvoke<Future2Test> [Futureint_Then_DestroyLambdaAfterInvoke] [2 ms]
 Error Message:
  Expected: [ isDestroyed ] is true
 Actual: it is false
   Line: 234
 Stack Trace:
 at futureTest.cpp:234 in D:\a\_work\1\s\vnext\Mso.UnitTests\future\futureTest.cpp:line 234

Currently, they're being resolved by just re-running the test until it passes.

Steps To Reproduce

  • Open a PR
  • Wait for test to fail on CI

Expected Results

No response

CLI version

npx @react-native-community/cli -v

Environment

npx @react-native-community/cli info

Community Modules

Test should pass on the first attempt.

Target Platform Version

None

Target Device(s)

No response

Visual Studio Version

None

Build Configuration

None

Snack, code example, screenshot, or link to a repository

No response

@microsoft-github-policy-service microsoft-github-policy-service bot added the Needs: Triage 🔍 New issue that needs to be reviewed by the issue management team (label applied by bot) label Nov 14, 2024
@chrisglein
Copy link
Member

This started failing in the same timeline as the VS update. There was an agent update.
Failure rate on this is really high, so retries are... painful.
New agents are installing VS on a fresh image. The default versions we get may not match what folks are getting locally with non-fresh installs. We potentially would need to update to lock to an older version of VS if that's what's happening here.
This might result in an update to the dependency script.

@Yajur-Grover Can you confirm the VS version that's running in CI when this fails? We should cross reference with what people have locally to see if there's a version diff there.

@chrisglein chrisglein added Area: Test Infrastructure and removed Needs: Triage 🔍 New issue that needs to be reviewed by the issue management team (label applied by bot) labels Nov 14, 2024
@chrisglein chrisglein added this to the Next milestone Nov 14, 2024
@Yajur-Grover
Copy link
Contributor Author

Yajur-Grover commented Nov 14, 2024

From this link, it looks like CI is running VS 17.12.0: https://dev.azure.com/ms/react-native-windows/_build/results?buildId=617414&view=logs&j=fecd28b7-c47d-51f4-7907-96a9c6766fbe&t=0b144516-1c93-531b-c20c-0712414f829f&l=46

Relevant log from CI:

VSTest version 17.12.0 (x64)
vstest.console.exe "D:\a\_work\1\s\vnext\target\x86\Release\Microsoft.ReactNative.Cxx.UnitTests\Microsoft.ReactNative.Cxx.UnitTests.exe"

I'm running 17.11.5 locally.

edit: 17.12.0 was released on 11/12/24, which is in line with when the test started failing as well: https://learn.microsoft.com/en-us/visualstudio/releases/2022/release-notes#17.12.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

2 participants