Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hani/Stabilize test sap google adclick #339

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Hani/Stabilize test sap google adclick #339

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
54acfe0
stabilize test sap google adclick
Nov 20, 2024
7918ad3
xfail on linux
Nov 27, 2024
09288e1
Merge branch 'main' into Hani/stabilize_test_sap_google_adclick
Nov 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion tests/address_bar_and_search/test_sap_google_adclick.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import platform
from time import sleep

import pytest
Expand All @@ -13,7 +14,7 @@ def test_case():
return "1365108"


@pytest.mark.unstable
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is failing in Linux, if you can't stabilize the test there, we can change the decorator to:

import platform
@pytest.mark.xfail(platform.system() == "Linux", reason="Telemetry testing unstable in Linux")

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test is failing on all OS's randomly if I run the test many times, sometimes search results don't return an ad and thats why the test can't see "search-result" locator.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In CI, it really looks like it's only failing in Linux--you're seeing fails in MacOS and Win?

Is there a way we can force there to be an ad, or re-run the check for SAP info multiple times without failing? Like run it 5 times and if it never gets anything it fails, but if it gets it once, it passes?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Locally today the test passed everytime after running it up to 10 times on all 3 OS's, so I went with the first solution and mark it unstable on Linux.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge main into your branch. That should fix the Linux CI find toolbar test failure.

@pytest.mark.xfail(platform.system() == "Linux", reason="Telemetry testing unstable in Linux")
def test_sap_google_adclick(driver: Firefox):
"""
C1365108, Test SAP Google adclick - URL bar - US
Expand Down