Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CV Eval - Deletion #14678

Merged
merged 4 commits into from
May 3, 2024
Merged

CV Eval - Deletion #14678

merged 4 commits into from
May 3, 2024

Conversation

sambible
Copy link
Contributor

@sambible sambible commented Apr 8, 2024

Big PR here, lots of removals. This is deleting all cases marked as such as part of the CV Component Eval.

If you'd like to, you can review the cases deleted in the Pheonix Component Feedback sheet. The primary reason for deletion for a case is either

  1. Dupe coverage that is picked up in another case
  2. Older Test that isn't longer relevant ( i.e. Pulp upgrade made some cases irrelevant )

@sambible sambible added AutoMerge_Cherry_Picked The cherrypicked PRs of master PR would be automerged if all checks passing 6.14.z Introduced in or relating directly to Satellite 6.14 6.15.z Introduced in or relating directly to Satellite 6.15 labels Apr 8, 2024
@sambible sambible self-assigned this Apr 8, 2024
@sambible sambible requested a review from a team as a code owner April 8, 2024 19:08
@sambible sambible added CherryPick PR needs CherryPick to previous branches and removed AutoMerge_Cherry_Picked The cherrypicked PRs of master PR would be automerged if all checks passing labels Apr 8, 2024
@Griffin-Sullivan
Copy link
Contributor

Older Test that isn't longer relevant ( i.e. Pulp upgrade made some cases irrelevant )

I tried bringing this problem up in a call once and I don't think I got the point across. I imagine this is an issue we will need to find a good solution for whether it's automated or a new process for the team each time we branch.

Copy link
Contributor

@vijaysawant vijaysawant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looked over 2 files only
api/test_contentview.py & test_contentviewfilter.py
query for @sambible

tests/foreman/api/test_contentviewfilter.py Show resolved Hide resolved
assert self.yumcv.repository[0].read().name == REPOS['rhst7']['name']

@pytest.mark.tier2
def test_positive_add_rh_custom_spin(self, target_sat):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this test consider by mistake, I don't see any recommendation in sheet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just added it back.

Copy link
Contributor

@vijaysawant vijaysawant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack.
I cross verified all deleted tests with tests present in sheet.
Put my understanding for 2 tests as part review, please take a look.

@@ -788,78 +788,6 @@ def test_negative_update_with_name(self, new_name, content_view, module_target_s
{'content-view-id': content_view['id'], 'name': new_name}
)

@pytest.mark.tier1
def test_negative_update_with_same_name(self, module_org, content_view, module_target_sat):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this deletion must be due to low severity of test, don't see any qe-recommendation in the sheet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

marked as QE delete

)

@pytest.mark.tier1
def test_negative_update_inclusion(self, module_org, content_view, module_target_sat):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this deletion must be due to low severity of test, don't see any qe-recommendation in the sheet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also listed as delete on sheet

Copy link
Contributor

@vsedmik vsedmik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few comments left regarding missing UI counterpats removal. Do we remove the test_contentview_old.py entirely in another PR?

@@ -282,22 +254,6 @@ def test_positive_add_sha512_rpm(self, content_view, module_org, module_target_s
class TestContentViewCreate:
"""Create tests for content views."""

@pytest.mark.parametrize('composite', [True, False])
@pytest.mark.tier1
def test_positive_create_composite(self, composite, target_sat):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the eval spreadsheet I can see removal for all - api, cli and ui - but I don't see removal from UI here.
Did we miss it or was it intetional?

@@ -369,43 +327,6 @@ def test_positive_add_synced_docker_repo(self, module_org, module_product, modul
content_view = content_view.update(['repository'])
assert repo.id in [repo_.id for repo_ in content_view.repository]

@pytest.mark.tier2
def test_positive_add_docker_repo_to_ccv(self, module_org, module_target_sat):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar here, in the eval sheet I can see the UI counterpart for removal too, but it's missing in this PR.

Comment on lines -593 to -613
def test_positive_publish_multiple_with_docker_repo(self, content_view, module_target_sat):
"""Add Docker-type repository to content view and publish it multiple
times.

:id: 33c1b2ee-ae8a-4a7e-8254-123d97aaaa58

:expectedresults: One repository is created with a Docker upstream
repository and the product is added to a content view which is then
published multiple times.
"""
assert len(content_view['versions']) == 0

publish_amount = randint(2, 5)
for _ in range(publish_amount):
module_target_sat.cli.ContentView.publish({'id': content_view['id']})
content_view = module_target_sat.cli.ContentView.info({'id': content_view['id']})
assert len(content_view['versions']) == publish_amount

@pytest.mark.tier2
def test_positive_publish_multiple_with_docker_repo_composite(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I'm removing the whole file in another PR

@vsedmik vsedmik merged commit ef09bc7 into SatelliteQE:master May 3, 2024
8 checks passed
@dosas dosas mentioned this pull request May 27, 2024
This was referenced May 28, 2024
jyejare pushed a commit to jyejare/robottelo that referenced this pull request Oct 19, 2024
* Deleting all test cases marked as such in CV Eval

* Add back mistakenly removed test

* Remove unnecessary imports
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.14.z Introduced in or relating directly to Satellite 6.14 6.15.z Introduced in or relating directly to Satellite 6.15 CherryPick PR needs CherryPick to previous branches
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants