Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update junit.py #3496

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Update junit.py #3496

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4b5ac5e
Update junit.py
shekharsorot Nov 5, 2024
9ecf795
Update junit.py
shekharsorot Nov 11, 2024
6117738
Update junit.py
shekharsorot Nov 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 26 additions & 9 deletions lisa/notifiers/junit.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ class JUnitSchema(schema.Notifier):
path: str = "lisa.junit.xml"
# respect the original behavior, include subtest by default
include_subtest: bool = True
# show passed case 'message' and 'stacktraces'
include_passed_messages: bool = False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't need to be False by default, so the logic is more consistent for different types.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I marked it false to ensure current output is not modified in any way. This is because i expect users to be parsing this output in their implementation of LISA.
Marking passed and skipped flag as false by default allows us to maintain some level of compat with existing implementation while allowing customization for new implementations.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If no scenario needs the stack trace for passed test cases, it doesn't need the flag.

# show 'message' for failed cases and 'stacktrace'
include_failed_messages: bool = True
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When should it be set to False? If no scenario, it can be included always.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default behavior is to include messages and stacktraces for Failed tests/subcases.
Ideally we want messages for skipped tests.
Passed tests/cases should be optional and extension can bundle additional information in stacktrace

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the scenarios, which want set failed/skippped to false, and set passed to true. So the three flags are not needed. The logic always include stackstrace for failed and skipped, but never for passed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

understood. I will modify code to only implement logic for successful test cases.

# show 'message' for skipped cases
include_skipped_messages: bool = True


class _TestSuiteInfo:
Expand Down Expand Up @@ -291,6 +297,8 @@ def _add_test_case_result(
class_name: str,
elapsed: float,
) -> None:
# Creating a JunitSchema object to read the passed parameters to Junit class
runbook: JUnitSchema = cast(JUnitSchema, self.runbook)
testsuite_info = self._testsuites_info.get(suite_full_name)
if not testsuite_info:
raise LisaException("Test suite not started.")
Expand All @@ -302,18 +310,27 @@ def _add_test_case_result(

if message.status == TestStatus.FAILED:
failure = ET.SubElement(testcase, "failure")
failure.attrib["message"] = message.message
failure.text = message.stacktrace

if runbook.include_failed_messages:
failure.attrib["message"] = message.message
# failure.text = message.stacktrace
squirrelsc marked this conversation as resolved.
Show resolved Hide resolved
if str(message.stacktrace) != "":
squirrelsc marked this conversation as resolved.
Show resolved Hide resolved
failure.attrib["stacktrace"] = str(message.stacktrace)
squirrelsc marked this conversation as resolved.
Show resolved Hide resolved
testsuite_info.failed_count += 1
squirrelsc marked this conversation as resolved.
Show resolved Hide resolved

elif (
message.status == TestStatus.SKIPPED
or message.status == TestStatus.ATTEMPTED
):
elif message.status in [TestStatus.SKIPPED, TestStatus.ATTEMPTED]:
skipped = ET.SubElement(testcase, "skipped")
skipped.attrib["message"] = message.message

if runbook.include_skipped_messages:
skipped.attrib["message"] = message.message
if str(message.stacktrace) != "":
skipped.attrib["stacktrace"] = str(message.stacktrace)
# Only add XML sub element if include_passed_messages is True in runbook
# By default, its assumed subtest is a pass
elif message.status == TestStatus.PASSED and runbook.include_passed_messages:
passed = ET.SubElement(testcase, "passed")
passed.attrib["message"] = message.message
# passed.text = message.stacktrace
if str(message.stacktrace) != "":
passed.attrib["stacktrace"] = str(message.stacktrace)
testsuite_info.test_count += 1

# Write out current results to file.
Expand Down
Loading