Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HOLD for payment 2023-10-24] [Reassure] Migrate some of the GooglePlacesUtilsTest to reassure test framework #28585

Closed
mountiny opened this issue Oct 2, 2023 · 18 comments
Assignees
Labels
Awaiting Payment Auto-added when associated PR is deployed to production Bug Something is broken. Auto assigns a BugZero manager. Improvement Item broken or needs improvement. Task Weekly KSv2

Comments

@mountiny
Copy link
Contributor

mountiny commented Oct 2, 2023

Problem

We have implemented one of the GooglePlacesUtilsTests so it measures time of some function execution in a test framework which is not ideal for that. This test is repeatadly flaky as a result, for example look here https://github.com/Expensify/App/actions/runs/6377422632/job/17306058621

Solution

First, lets skip these tests and then migrate them to Reassure framework, once we can do that. Currently this exact type of tests is not supported according to Callstack

@mountiny mountiny added Daily KSv2 Bug Something is broken. Auto assigns a BugZero manager. labels Oct 2, 2023
@mountiny mountiny self-assigned this Oct 2, 2023
@melvin-bot
Copy link

melvin-bot bot commented Oct 2, 2023

Triggered auto assignment to @JmillsExpensify (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

@melvin-bot
Copy link

melvin-bot bot commented Oct 2, 2023

Bug0 Triage Checklist (Main S/O)

  • This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
  • This bug is not a duplicate report (check E/App issues and #expensify-bugs)
    • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
  • This bug is reproducible using the reproduction steps in the OP. S/O
    • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
    • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
  • This issue is filled out as thoroughly and clearly as possible
    • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
  • I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

@mountiny mountiny added Improvement Item broken or needs improvement. Task Bug Something is broken. Auto assigns a BugZero manager. and removed Bug Something is broken. Auto assigns a BugZero manager. labels Oct 2, 2023
@melvin-bot
Copy link

melvin-bot bot commented Oct 2, 2023

Current assignee @JmillsExpensify is eligible for the Bug assigner, not assigning anyone new.

@melvin-bot
Copy link

melvin-bot bot commented Oct 2, 2023

Bug0 Triage Checklist (Main S/O)

  • This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
  • This bug is not a duplicate report (check E/App issues and #expensify-bugs)
    • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
  • This bug is reproducible using the reproduction steps in the OP. S/O
    • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
    • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
  • This issue is filled out as thoroughly and clearly as possible
    • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
  • I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

@mountiny mountiny added Weekly KSv2 and removed Daily KSv2 labels Oct 2, 2023
@mountiny mountiny changed the title Migrate some of the GooglePlacesUtilsTest to reassure test framework [Reassure] Migrate some of the GooglePlacesUtilsTest to reassure test framework Oct 2, 2023
@koko57
Copy link
Contributor

koko57 commented Oct 2, 2023

@mountiny Commenting for @adhorodyski as Adam is ooo. 😃 (link)

@melvin-bot melvin-bot bot added Reviewing Has a PR in review Weekly KSv2 and removed Weekly KSv2 labels Oct 2, 2023
@koko57
Copy link
Contributor

koko57 commented Oct 3, 2023

@mountiny could you please assign @adhorodyski instead of me? 🙂

@mountiny
Copy link
Contributor Author

mountiny commented Oct 3, 2023

I made the PR to skip the flaky tests, no we just need to wait for Reassure to support such tests

@adhorodyski
Copy link
Contributor

@mountiny i will happily pick this up after we merge callstack/reassure#271 :)

@mountiny mountiny added Monthly KSv2 and removed Reviewing Has a PR in review Weekly KSv2 labels Oct 9, 2023
@mountiny
Copy link
Contributor Author

mountiny commented Oct 9, 2023

@adhorodyski is there anything we can do to help with the PR? If you need to allocate time for it or another engineer it makes sense to me to push this one ahead

@adhorodyski
Copy link
Contributor

I now work 50/50 between the flashlist migration under #28902 & this PR.
We should have it merged shortly (matter of days) as only docs & a simple output formatter are missing, meaning the implementation is actually there.
I will definitely let you know if anything changes and I'll need any help with it!

@mountiny
Copy link
Contributor Author

mountiny commented Oct 9, 2023

Thanks Adam!

@adhorodyski
Copy link
Contributor

The new API from callstack/reassure#271 just got released under v0.10.0 🚀 - I will pick up this task shortly!

@JmillsExpensify
Copy link

Are any additional PRs due for this issue? It looks like no payments are due, so I think we might be ok to close it?

@melvin-bot melvin-bot bot added Weekly KSv2 Awaiting Payment Auto-added when associated PR is deployed to production and removed Weekly KSv2 labels Oct 17, 2023
@melvin-bot melvin-bot bot changed the title [Reassure] Migrate some of the GooglePlacesUtilsTest to reassure test framework [HOLD for payment 2023-10-24] [Reassure] Migrate some of the GooglePlacesUtilsTest to reassure test framework Oct 17, 2023
@melvin-bot
Copy link

melvin-bot bot commented Oct 17, 2023

Reviewing label has been removed, please complete the "BugZero Checklist".

@melvin-bot melvin-bot bot removed the Reviewing Has a PR in review label Oct 17, 2023
@melvin-bot
Copy link

melvin-bot bot commented Oct 17, 2023

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.3.85-4 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

If no regressions arise, payment will be issued on 2023-10-24. 🎊

After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

  • External issue reporter
  • Contributor that fixed the issue
  • Contributor+ that helped on the issue and/or PR

For reference, here are some details about the assignees on this issue:

  • @koko57 does not require payment (Contractor)
  • @adhorodyski does not require payment (Contractor)

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

  • Merged PR within 3 business days of assignment - 50% bonus
  • Merged PR more than 9 business days after assignment - 50% penalty

@melvin-bot
Copy link

melvin-bot bot commented Oct 17, 2023

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

  • [@mountiny] The PR that introduced the bug has been identified. Link to the PR:
  • [@mountiny] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
  • [@mountiny] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
  • [@koko57 / @adhorodyski] Determine if we should create a regression test for this bug.
  • [@koko57 / @adhorodyski] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
  • [@JmillsExpensify] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

@JmillsExpensify
Copy link

No payments are required on this issue, so the main question left is whether a regression is required. @koko57 / @adhorodyski thoughts?

@mountiny
Copy link
Contributor Author

Nope this was an update to automated tests so all good here, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Awaiting Payment Auto-added when associated PR is deployed to production Bug Something is broken. Auto assigns a BugZero manager. Improvement Item broken or needs improvement. Task Weekly KSv2
Projects
Development

No branches or pull requests

4 participants