Add support for reporting/commenting test results to the Jira issue

[jameerpathan111]

What does this do?

• This PR adds support for reporting/commenting test results to Jira based on issues referenced in the Verifies or BlockedBy doc fields.

Primary use case

1. Add a Verifies or BlockedBy doc field to the test, mapping it to a Jira RFE/Bug. For example, :Verifies: SAT-25230 or :BlockedBy: SAT-24796.
   • For BlockedBy, the test will be collected only if the corresponding Jira issue is in the Review (ON_QA) state. This behavior can be overridden by using the --blocked-by SAT-25230 option.
2. Set ENABLE_COMMENT to true in the jira.yaml configuration file.
3. Use the --jira-comments pytest option.

Note:

• To prevent accidental usage of this functionality, the user is required to perform both steps 2 and 3.(Open to suggestions)
• By default, the comment will be added on issues in "Review" and "Release Pending" state, but this could be overridden by setting issue_status in jira.conf.

PRT example

Assuming the test has a Verifies or BlockedBy doc field set and you want to comment on issues in "New" or "In Progress" state.

trigger: test-robottelo
pytest: tests/foreman/api/test_ping.py --jira-comments
env:
    ROBOTTELO_jira__enable_comment: true
    ROBOTTELO_jira__issue_status: ['New', 'Review']

Related Issues

• SAT-24954
• satellite-jenkins#1341

[jyejare]

@jameerpathan111 Quick Comment: Please remove Jira Comment setting from file and keep only pytest option.

Reason: A local user could forget that he has enabled the setting in file and unintentionally the comments would be added to Jira card until noticed.

[jyejare]

Reviewed from high view, deep code review pending.

[JacobCallahan]

Overall ACK on the contents here. Though I would still like to know what other comment types are available besides "group" and how each differs from the others.

[jyejare]

Overall looks good.

But ACK pending comment to remove duplicate commenting settings and Jira Comment example.

[jameerpathan111]

Overall ACK on the contents here. Though I would still like to know what other comment types are available besides "group" and how each differs from the others.

There are two types, group and role and then you need to provide the group/role the comment is visible. See https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-issue-comments/#api-rest-api-3-issue-issueidorkey-comment-post

[jameerpathan111]

Please don't merge this before I remove my last commit, I'm doing some more testing with PRT.

[Satellite-QE]

PRT Result

Build Number: 7197
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/api/test_ping.py --jira-comments --external-logging
Test Result : ============= 1 failed, 1 passed, 31 warnings in 752.25s (0:12:32) =============

[Satellite-QE]

PRT Result

Build Number: 7199
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/api/test_ping.py --jira-comments --external-logging
Test Result : ============= 1 failed, 1 passed, 30 warnings in 680.51s (0:11:20) =============

[Satellite-QE]

PRT Result

Build Number: 7200
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/api/test_ping.py --jira-comments --external-logging
Test Result : ============= 1 failed, 1 passed, 28 warnings in 695.33s (0:11:35) =============

[jameerpathan111]

trigger: test-robottelo
pytest: tests/foreman/api/test_ping.py --jira-comments
env:
    ROBOTTELO_jira__enable_comment: true
    ROBOTTELO_jira__comment_type: 'group'
    ROBOTTELO_jira__comment_visibility: 'Red Hat Employee'
    ROBOTTELO_jira__issue_status: ['Review', 'Release Pending']

[Satellite-QE]

PRT Result

Build Number: 7207
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/api/test_ping.py --jira-comments --external-logging
Test Result : ============= 1 failed, 1 passed, 33 warnings in 796.04s (0:13:16) =============

[jameerpathan111]

Ready for merge. Tested the PR, see PRT run 7207 for the result.
Comment added by the job on the jira issue:

This is an automated comment from job/user: <prt-url>/7207/ for a Robottelo test run.
Satellite/Capsule: stream Snap: 60.0
Result for tests linked with issue: [SAT-24796]
tests/foreman/api/test_ping.py::test_positive_ping : passed
tests/foreman/api/test_ping.py::test_positive_random_test : failed