Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix unit test flakiness (createAttendanceForBooking) #765

Open
ikusteu opened this issue May 30, 2023 · 0 comments
Open

Fix unit test flakiness (createAttendanceForBooking) #765

ikusteu opened this issue May 30, 2023 · 0 comments

Comments

@ikusteu
Copy link
Collaborator

ikusteu commented May 30, 2023

I've worked on optimising (unit) test runs in terms of speed of the entire CI workflow, reducing load (more taking into account the load/performance) on firebase emulators, all with the goal of reducing flaky tests.
I've made significant progress and now, our tests are passing, they're quite fast (thanks to Vitest) and are given a lot of time, in terms of timeout, when a large number of tests are making requests against firebase emulators.
However, every now and then, a flaky test comes along. Therefore, I've created a unit test matrix, allowing us to run 36 unit tests as CI workflows and, in effect, helping us pinpoint the flaky tests.

Currently, only one flaky test is found, the frequency of failures is 3/36 (8.3%). I'm pretty sure that the test is not flaky per se, but the load on firebase emulators slows down the response, and, as the test probably runs multiple requests + polls for assertions (using waitFor test helper), the assertions get timed out. I don't think we should extend the default assertion timeout, nor test timeout for all tests (as they're already at 8000ms/15000ms respectively), but rather increase the assertion timeout and test timeout for the particular test. This should be a really quick fix, but I've not yet applied it and have opened this ticked as a TODO, as a lot of work is currently not merged in and I'd like the update to be a simple patch, rather than depending on other branches being mergeds.

The test in question is: Cloud functions -> Data triggers -> createAttendanceForBooking -> should create attendance entry for booking and not overwrite the existing data in slot
The metric (3/36 failure) can be found in the CI workflow: https://github.com/eisbuk/EisBuk/actions/runs/5118822733/jobs/9203298814

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant