Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ignore] Dummry issue to draft UTR message #8547

Open
agraebe opened this issue Oct 4, 2024 · 0 comments
Open

[ignore] Dummry issue to draft UTR message #8547

agraebe opened this issue Oct 4, 2024 · 0 comments
Assignees

Comments

@agraebe
Copy link

agraebe commented Oct 4, 2024

Context

We started the UTR workstream by recapping what we know about the customer needs. We documented the initial product decision in the same document.

Content Requirements

The key factors for the GitHub comment are:

  • users only care about failed test runs
  • success and skipped runs only need a count
  • failed test runs should include a list of all test runs with the following details: test name, failure type, link to logs

Because we are building a flaky test solution, I suggest we also annotate failed test runs associated with flaky tests.

Draft proposal

48 tests were run on e7d148a. 22 failed. 2 skipped. 24 passed. View Full Report ↗︎

First 5 failed test runs:

Test name Failure reason Flaky Logs
Check switches through onboarding flow for existing users who already have uploads A test exceeded the timeout of 40000 ms, so you should increase the timeout value or optimize the test to run faster. Logs
Check has different onboarding states for different repos A test exceeded the timeout of 10000 ms, so you should increase the timeout value or optimize the test to run faster. Logs
Check switches through onboarding flow for existing users who already have uploads A test exceeded the timeout of 40000 ms, so you should increase the timeout value or optimize the test to run faster. ⚠️ Logs
Check switches through onboarding flow for existing users who already have uploads A test exceeded the timeout of 40000 ms, so you should increase the timeout value or optimize the test to run faster. Logs
Check has different onboarding states for different repos A test exceeded the timeout of 10000 ms, so you should increase the timeout value or optimize the test to run faster. ⚠️ Logs

... and 17 more

@agraebe agraebe self-assigned this Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant