Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The 'errors' record for series now counts all errors for tests/series. #785

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

The 'errors' record for series now counts all errors for tests/series. #785

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 16 additions & 7 deletions lib/pavilion/series/info.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,21 +149,30 @@ def errors(self) -> int:

status_obj = self._get_status_file()

error_values = [
'ERROR',
'CANCELLED',
'TIMEOUT',
]

errors = 0
for status in status_obj.history():
if status.state in (status_file.SERIES_STATES.ERROR,
status_file.SERIES_STATES.BUILD_ERROR,
status_file.SERIES_STATES.CREATION_ERROR,
status_file.SERIES_STATES.KICKOFF_ERROR):
errors += 1
for error_val in error_values:
if error_val in status.state:
Copy link
Collaborator

@hwikle-lanl hwikle-lanl Sep 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using str.__contains__ here feels a bit kludgey and error-prone. A better solution would be to implement a mechanism to separate statuses into ERROR, CANCELLED, and TIMEOUT classes, either by:

a) Creating something like a class field on TestStatusInfo that takes on an enum value and testing against that; or

b) Implementing three static methods on StatesStruct that return sets of ERROR, CANCELLED, and TIMEOUT statuses respectively and then checking whether status.state is contained within each set (or simply the union of the three sets).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The second option feels cleaner to me.

errors += 1
break

for test_path in self._tests:
test_info = self.test_info(test_path)
if test_info is None:
continue

if test_info.result == TestRun.ERROR:
errors += 1
# Look for any bad test states
for error_val in error_values:
for state in test_info.state_history:
if error_val in state.state:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto here.

errors += 1
break

return errors

Expand Down
4 changes: 4 additions & 0 deletions test/tests/series_tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,10 @@ def test_ignore_errors(self):
else:
self.assertEqual(test.result, None)

# Check that the info object lists the proper number of errors.
series_info = series.SeriesInfo(self.pav_cfg, series_obj.path)
self.assertEqual(series_info.errors, 9)

def test_series_conditionals_only_if_ok(self):
"""Test that adding a conditional that always matches produces tests that
run when expected."""
Expand Down
Loading