The status of the skipped test case is pass in html Report of Katalon 5.10.1

[ghost]

Preconditions:

• There are 2 test cases have been created
• There is a test suite which contains 2 test cases above
• Katalon version 5.10.1 is installed

Steps to reproduce:

• Skip one test case in the test suite in precondition by using Test Listener
• Run the test suite
• Verify the status of 2 test cases in generated html report (under Reports folder)

Expected Behavior:

The status of html report should specify that 1 test case passed and 1 skipped

Actual Behavior: The status in html report is:

Status: 2 test total, 2 passed, 0 failed, 0 error, 0 incomplete

[trongbui]

https://katalon.zendesk.com/agent/tickets/219

One of the new feature introduced in Katalon studio 5.10.1 (testCaseContext.skipThisTestCase()) is really nice to have, and we have been experimenting with it. However, one problem we see with this is:

• In the logs area, we do not see the overall count of skipped tests (besides count of passed, failed and error tests).
• In the HTML report, the skipped tests appear to be marked as "Passed".

I believe this is a bug and needs to be addressed to generate reports with right information.

Requesting to please note this aspect.

One more thing, if there is provision to mention the reason for skipping in the form of a message, that would be great. This reason could be seen in the logs.
e.g.
String reasonToSkip = "This functional flow is not supported on browser A"
testCaseContext.skipThisTestCase(reasonToSkip)

[trongbui]

Just wondering about, approximately when could we expect the fix for this issue?

[dungleonhart]

This has been planned for 6.1 release. We might add API for users to manually mark the status of skipped test cases or really skip them from the report.

[trongbui]

For this one, I would really appreciate if the html report mentions the test case as Skipped, and we are allowed to pass the reason (e.g. in the form of string message). It is ok if we are required to write a specific statement in the script to mark it that way in html report.

That will avoid the confusion while analyzing the logs and html report (they will be consistent).

So all in all and as possible test cases, the skip status needs to be consistent at 3 locations (based on my understanding):

1. The logs
2. The reports - html report (may be the CSV or PDFs would be automatically taken care?)
3. The report when opened in katalon view

Thanks and regards,
Tanmay

[akshayuascs]

Any update on this issue?

[matt-piasecki]

This issue persists in version 7. Logs will display correctly at the end of test execution, but the report is still inconsistent with the actual results of execution.

[duyluonganh]

Hi all,

Displaying SKIP status has been supported in Katalon Studio v7.8 onward. Please check for update.

References: https://docs.katalon.com/katalon-studio/new/version-70.html#improvements-3