Skip to content
This repository has been archived by the owner on Feb 18, 2021. It is now read-only.

List of flaky tests #1122

Open
brbrr opened this issue Apr 6, 2018 · 0 comments
Open

List of flaky tests #1122

brbrr opened this issue Apr 6, 2018 · 0 comments

Comments

@brbrr
Copy link
Contributor

brbrr commented Apr 6, 2018

We define a "flaky" test result as a test that exhibits both a passing and a failing result with the same code.
https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html

This is an ongoing list of flaky tests, If you saw one - please update this list. Make sure to include host and a viewport.

  1. [WPCOM] Switching Themes: (desktop) Switching Themes
  2. [WPCOM] Sign Up (desktop, en) Sign up for a site on a premium paid plan through main flow
  3. [WPCOM] Sign Up (mobile, en) Basic sign up for a free site
  4. [WPCOM] Sign Up (mobile, en) Sign up for a site on a business paid plan w/ domain name coming in via /create as business flow1. [WPCOM] Editor: Pages (mobile) Public Pages
  5. [WPCOM] Editor: Posts (desktop) Edit a Post
  6. [WPCOM] Editor: Pages (desktop) Password Protected Pages
  7. [WPCOM] Notifications: (desktop)
  8. [WPCOM] Editor: Posts (desktop) Basic Public Post
  9. [WPCOM] Invites: (mobile) Inviting New User as an Contributor, then change them to Author

We need to get rid of as many flaky tests as possible to make sure folks are confident in our test results. This also will decrease Flow Patrol distraction, and unnecessarily context switches.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant