Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CompEval] API modifications for errata-management #13515

Merged
merged 5 commits into from
Jan 18, 2024

Conversation

damoore044
Copy link
Contributor

@damoore044 damoore044 commented Dec 20, 2023

Problem Statement

API errata has several outdated tests to be modified, combined, or removed for component evaluation.
8-12 consistent failures in 6.15.0 CI, most in setup of hosts, errata/packages, or necessary components.

Solution

  • Updated hosts to register with global registration.

  • rhel8 hosts using rh repos and packages for rhel8, setup_content fixture now using the same.

  • Combine the following view, sort, and filter scenarios per comp-eval, into just test_positive_list_sorted_filtered:
    test_positive_list - test_positive_list_updated - test_positive_sorted_issue_date_and_filter_by_cve

  • Helper method to api publish & promote a content-view,
    wait for task(s) after content-view.publish()

  • Fix api_factory method wait_for_errata_applicability_task, which was using an old search format to find applicability tasks.

  • Propose marking the following as an End-to-End test, as it touches all components needed for errata,
    and performs a common and critical customer use-case; of installing errata to hosts in host collection:
    test_positive_install_in_hc[rhel params for 7, 8, 9]

  • test_apply_modular_errata_using_default_content_view: Removed this outdated test case-
    as the DefaultCV cannot be edited, published, promoted etc. A follow up introduction could cover this
    with a test_negative scenario.

Prior Failures - Resolved

  • test_errata_installation_with_swidtags : Unsure if we can use some other module_repos_collection ,
    that does not touch module_entitlement_manifest_org? output below :

    Passing, I made a new fixture: function_repos_collection_with_manifest in fixtures>repository.py, using function_sca_manifest_org.

PRT Case

trigger: test-robottelo
pytest: tests/foreman/api/test_errata.py

@damoore044 damoore044 added CherryPick PR needs CherryPick to previous branches Stream Introduced in or relating directly to Satellite Stream/Master 6.15.z Introduced in or relating directly to Satellite 6.15 labels Dec 20, 2023
@damoore044 damoore044 self-assigned this Dec 20, 2023
@damoore044 damoore044 force-pushed the api-errata-eval branch 2 times, most recently from d2ae8da to efcb401 Compare December 21, 2023 19:05
@damoore044
Copy link
Contributor Author

damoore044 commented Dec 21, 2023

Last Build [5430] vs 6.15.0 CI - Jan 18, 2024, 2:17 PM

Pass: 12 [+6]
Skip: 0
Fail: 0 [-9]

Total: 12 vs 15 (3 sorting tests were merged intotest_positive_list_sorted_filtered)

  • Removed outdated test scenario test_apply_modular_errata_using_default_content_view .

@damoore044 damoore044 force-pushed the api-errata-eval branch 2 times, most recently from 9166aca to 1e8c7c0 Compare December 21, 2023 19:59
@damoore044 damoore044 marked this pull request as ready for review December 21, 2023 22:21
@damoore044 damoore044 requested review from a team as code owners December 21, 2023 22:21
@damoore044 damoore044 force-pushed the api-errata-eval branch 3 times, most recently from 3eefd9b to bb18a48 Compare December 27, 2023 21:58
@damoore044 damoore044 force-pushed the api-errata-eval branch 2 times, most recently from 2f8c308 to 5951402 Compare January 7, 2024 01:34
Copy link
Member

@JacobCallahan JacobCallahan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few other things

Copy link
Contributor

@vijaysawant vijaysawant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few suggestions only, no blocker.

@damoore044 damoore044 force-pushed the api-errata-eval branch 5 times, most recently from 5740154 to 6c4c990 Compare January 11, 2024 17:15
@damoore044 damoore044 force-pushed the api-errata-eval branch 2 times, most recently from 2977293 to 5632656 Compare January 16, 2024 13:49
@damoore044 damoore044 requested a review from a team January 16, 2024 17:43
@jyejare
Copy link
Member

jyejare commented Jan 17, 2024

@damoore044 If the test failure is due to product changes, feel free to remove it and for the negative test that you wanted to do, add a new test for it!

This way in reporting tools new test will be removed and new tests would be added with unique id!

Also, the change is applicable for all zstream versions ? if yes, please raise a separate PR for this change and we will merge this PR with single failure :)

@damoore044
Copy link
Contributor Author

damoore044 commented Jan 17, 2024

@jyejare From discussing with the QE team, we can remove this outdated test test_apply_modular_errata_using_default_content_view , and supporting fixtures in this module,
I will circle back for a new test_negative scenario as part of continuous improvements.
Would that be appropriate to satisfy requirements here?

@damoore044
Copy link
Contributor Author

damoore044 commented Jan 17, 2024

@jyejare From local testing, this MR will be good to cherry-pick into 6.15.z, 6.14.z.
Can we try auto-CP with prt run for each ystream mr?

@damoore044 damoore044 added the 6.14.z Introduced in or relating directly to Satellite 6.14 label Jan 17, 2024
@jyejare
Copy link
Member

jyejare commented Jan 17, 2024

@damoore044 agree with merging this PR with removing that test case.

Also add cherrypick labels for branches and a cherrypick label itself for auto CP.

@damoore044 damoore044 added the AutoMerge_Cherry_Picked The cherrypicked PRs of master PR would be automerged if all checks passing label Jan 17, 2024
Copy link
Contributor

@vsedmik vsedmik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, in PRT all tests passed. Left just one non-blocking nitpick comment but I think we are good to merge.

Good work @damoore044!

@damoore044
Copy link
Contributor Author

trigger: test-robottelo
pytest: tests/foreman/api/test_errata.py

@JacobCallahan JacobCallahan merged commit 48dc4d0 into SatelliteQE:master Jan 18, 2024
5 checks passed
github-actions bot pushed a commit that referenced this pull request Jan 18, 2024
* api-errata-updates, component-eval, global registration and host content

* Combine the list, sort, and filter tests

* Add CU scenario for puma worker count

pre-commit fix

* Docstring fixes

verify key exists in task, prior to accessing

* Remove outdated Default_CV scenario

---------

Co-authored-by: Griffin Sullivan <[email protected]>
(cherry picked from commit 48dc4d0)
shweta83 pushed a commit to shweta83/robottelo that referenced this pull request Apr 10, 2024
* api-errata-updates, component-eval, global registration and host content

* Combine the list, sort, and filter tests

* Add CU scenario for puma worker count

pre-commit fix

* Docstring fixes

verify key exists in task, prior to accessing

* Remove outdated Default_CV scenario

---------

Co-authored-by: Griffin Sullivan <[email protected]>
@damoore044 damoore044 deleted the api-errata-eval branch February 20, 2025 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.14.z Introduced in or relating directly to Satellite 6.14 6.15.z Introduced in or relating directly to Satellite 6.15 AutoMerge_Cherry_Picked The cherrypicked PRs of master PR would be automerged if all checks passing CherryPick PR needs CherryPick to previous branches Stream Introduced in or relating directly to Satellite Stream/Master
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants