Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add filter for ldap related tests #14540

Merged
merged 1 commit into from
Apr 9, 2024
Merged

Add filter for ldap related tests #14540

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pytest_plugins/markers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ def pytest_configure(config):
"include_capsule: For satellite-maintain tests to run on Satellite and Capsule both",
"capsule_only: For satellite-maintain tests to run only on Capsules",
"manifester: Tests that require manifester",
"ldap: Tests related to ldap authentication",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dosas , I think we are going too deep with adding markers to filter tests by adding explicit markers.

BTW we have similar implementation in robottelo that adds marker on each test based on component doc tag implemented here https://github.com/SatelliteQE/robottelo/blob/master/pytest_plugins/metadata_markers.py#L109

With this you should be able to opt in / opt out tests based on component name including ldap. If this suits your need, please close the PR.

Copy link
Collaborator Author

@dosas dosas Mar 28, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sadly this does not fit my needs.

The tests I want to filter are the ones using create_ldap fixture

pytest --co -q -m "ldap" tests/foreman/api
tests/foreman/api/test_role.py::TestCannedRole::test_negative_access_entities_from_ldap_org_admin
tests/foreman/api/test_role.py::TestCannedRole::test_negative_access_entities_from_ldap_user
tests/foreman/api/test_role.py::TestCannedRole::test_positive_assign_org_admin_to_ldap_user_group
tests/foreman/api/test_role.py::TestCannedRole::test_negative_assign_org_admin_to_ldap_user_group
tests/foreman/api/test_user.py::TestActiveDirectoryUser::test_positive_create_in_ldap_mode[0]
tests/foreman/api/test_user.py::TestActiveDirectoryUser::test_positive_create_in_ldap_mode[1]
tests/foreman/api/test_user.py::TestActiveDirectoryUser::test_positive_create_in_ldap_mode[2]
tests/foreman/api/test_user.py::TestActiveDirectoryUser::test_positive_create_in_ldap_mode[3]
tests/foreman/api/test_user.py::TestActiveDirectoryUser::test_positive_create_in_ldap_mode[4]
tests/foreman/api/test_user.py::TestActiveDirectoryUser::test_positive_ad_basic_no_roles
tests/foreman/api/test_user.py::TestActiveDirectoryUser::test_positive_access_entities_from_ldap_org_admin
tests/foreman/api/test_user.py::TestFreeIPAUser::test_positive_ipa_basic_no_roles
tests/foreman/api/test_user.py::TestFreeIPAUser::test_positive_access_entities_from_ipa_org_admin

I also tried filtering by -k expression but this does not yield the right result

pytest --co -q --component=ldap tests/foreman/api
no tests collected

I doubt that I can change the component for https://github.com/SatelliteQE/robottelo/blob/master/tests/foreman/api/test_role.py#L1627 to ldap

I also have not found a way to not run tests belonging to a certain component, is this even possible?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dosas No ! You cant change the component names because they are tightly coupled with our internal component mappings system.

To filter out the component tests , you can use -m not <component_name>.

Copy link
Collaborator Author

@dosas dosas Mar 28, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dosas No ! You cant change the component names because they are tightly coupled with our internal component mappings system.

That is what I thought ;)

To filter out the component tests , you can use -m not <component_name>.

No, this does not work:

pytest --co -q -m "not AlternateContentSources" tests/foreman/api
2724/2806 tests collected (82 deselected)

pytest --co -q  tests/foreman/api
2724/2806 tests collected (82 deselected)

pytest --co -q --component=AlternateContentSources tests/foreman/api
7/2806 tests collected (2799 deselected)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh ! These component names as markers are applied as key=value pair styled and hence we cant filter them out from -m option. We can change that in future though or hope pytest supports that.

So for now I am ACKing you.

]
markers.extend(module_markers())
for marker in markers:
Expand Down
2 changes: 2 additions & 0 deletions tests/foreman/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ def pytest_collection_modifyitems(session, items, config):
for item in items:
if any("manifest" in f for f in getattr(item, "fixturenames", ())):
item.add_marker("manifester")
if any("ldap" in f for f in getattr(item, "fixturenames", ())):
item.add_marker("ldap")
# 1. Deselect tests marked with @pytest.mark.deselect
# WONTFIX BZs makes test to be dynamically marked as deselect.
deselect = item.get_closest_marker('deselect')
Expand Down
Loading