Filter test collection based on files changed in upstream PRs.

[tpapaioa]

Problem Statement

Filter tests based on the files modified in an upstream (foreman or katello) PR.

Solution

The new upstream_pr pytest plugin uses the PyGithub library to get the list of files modified by the given PR, and filters the test collection based on the mapping of upstream repo paths/filenames to test markers in robottelo.

For example, with the following new settings block settings.github:

$ cat conf/github.yaml
GITHUB:
  FOREMAN:
    REPO: foreman
    ORG: theforeman
    RULES:
      - PATH: "app/models/concerns/hostext/"
        MARKER: component
        MARKER_ARG: Hosts
      - PATH: "webpack/assets/javascripts/react_app/components/SettingUpdateModal/"
        MARKER: component
        MARKER_ARG: Settings
  KATELLO:
    REPO: katello
    ORG: Katello
    RULES:
      - PATH: "app/controllers/katello/api/v2/errata_controller.rb"
        MARKER: component
        MARKER_ARG: ErrataManagement

then the following Foreman PR, which modifies files under webpack/assets/javascripts/react_app/components/SettingUpdateModal/, should filter the tests on the Settings component. In this case, --upstream-pr foreman/9969 is equivalent to --component Settings:

Fixes #37013 - change the 'All hosts' menu item's url based on the new_host_page setting
theforeman/foreman#9969

>>> FOREMAN_PR_ID = 9969
>>> from github import Github
>>> from pprint import pp
>>> pr = Github().get_repo("theforeman/foreman").get_pull(FOREMAN_PR_ID)
>>> filenames = [file.filename for commit in pr.get_commits() for file in commit.files]
>>> pp(filenames)
['app/registries/foreman/settings/general.rb',
 'webpack/assets/javascripts/react_app/Root/Context/ForemanContext.js',
 'webpack/assets/javascripts/react_app/Root/ReactApp.js',
 'webpack/assets/javascripts/react_app/components/Layout/LayoutConstants.js',
 'webpack/assets/javascripts/react_app/components/Layout/LayoutHelper.js',
 'webpack/assets/javascripts/react_app/components/Layout/Navigation.js',
 'webpack/assets/javascripts/react_app/components/Layout/index.js',
 'webpack/assets/javascripts/react_app/components/SettingUpdateModal/SettingUpdateModalConstants.js',
 'webpack/assets/javascripts/react_app/components/SettingUpdateModal/components/SettingForm/SettingForm.js',
 'webpack/test_setup.js']

$ pytest tests/foreman --co --upstream-pr foreman/9969
[...]
=== 54/5426 tests collected (5372 deselected) in 16.91s ===

$ pytest tests/foreman --co --component Settings
[...]
=== 54/5426 tests collected (5372 deselected) in 15.14s ===

Similarly, for this Katello PR:

Fixes #37394 - content_view_filter_id only works for ID-type filters
Katello/katello#10985

the file app/controllers/katello/api/v2/errata_controller.rb is matched exactly, and tests with the ErrataManagement component are selected:

>>> KATELLO_PR_ID = 10985
>>> pr = Github().get_repo("Katello/katello").get_pull(KATELLO_PR_ID)
>>> filenames = [file.filename for commit in pr.get_commits() for file in commit.files]
>>> pp(filenames)
['app/controllers/katello/api/v2/debs_controller.rb',
 'app/controllers/katello/api/v2/errata_controller.rb',
 'app/controllers/katello/api/v2/packages_controller.rb']

$ pytest tests/foreman --co --upstream-pr katello/10985
[...]
=== 70/5426 tests collected (5356 deselected) in 15.45s ===

$ pytest tests/foreman --co --component ErrataManagement
[...]
== 70/5426 tests collected (5356 deselected) in 13.78s ===

[omkarkhatavkar]

adding reminder here that, this needs to be added in the CI code base as well.

[JacobCallahan]

I'm unsure of the name here. github might be a bit too vague. can you think of something more specific for this config to be called?

[tpapaioa]

Changed to github_repos.

[github-actions]

This pull request has not been updated in the past 45 days.

[github-actions]

This pull request is now being closed after stale warnings.