Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add test results output #949

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
9104c25
Test summary output
mihaisee Jun 13, 2023
86ba246
Removed debug logs
mihaisee Jun 15, 2023
212e2ff
Added some documentation for the testResultsObject output
mihaisee Jun 15, 2023
1e424a0
Test cypress testResultsObject output in example-recording.yml
mihaisee Jun 15, 2023
8188a63
Test cypress testResultsObject output in example-recording.yml
mihaisee Jun 15, 2023
63fc380
Skip parsing testResultsObject if empty
mihaisee Jun 16, 2023
3c0abeb
Update example-recording.yml
mihaisee Jun 21, 2023
53a3e26
Merge branch 'cypress-io:master' into feat/add-test-reults-output
mihaisee Jun 21, 2023
f9794b0
Update .gitignore
mihaisee Jun 26, 2023
06567f7
Merge branch 'master' into feat/add-test-reults-output
mihaisee Aug 1, 2023
b116a60
Merge branch 'master' into feat/add-test-reults-output
mihaisee Oct 4, 2023
ab2c53e
Removed the calculated success status from testResultsObject
mihaisee Oct 5, 2023
087d18b
Update docs and workflow to reflect removal of success property from …
mihaisee Oct 5, 2023
3a8d645
Updated testResultsObject to testResults default
mihaisee Oct 11, 2023
9624c3f
Merge branch 'master' into feat/add-test-reults-output
mihaisee Oct 11, 2023
4d47501
Updated testResultsObject to testResults default - update workflow
mihaisee Oct 11, 2023
f4402c5
Merge branch 'feat/add-test-reults-output' of github.com:mihaisee/git…
mihaisee Oct 11, 2023
7421400
Updated testResultsObject to testResults default - update workflow
mihaisee Oct 11, 2023
84f0d2a
Update .github/workflows/example-recording.yml
mihaisee Oct 12, 2023
18af1c0
Merge branch 'master' into feat/add-test-reults-output
mihaisee Oct 12, 2023
185d2ef
Update README.md
mihaisee Oct 12, 2023
7b06979
Merge branch 'master' into feat/add-test-reults-output
mihaisee Oct 20, 2023
4f33b71
Update action.yml
mihaisee Oct 20, 2023
d144906
updating to runResults
mschile Oct 23, 2023
6f3e59e
updating to runResults
mschile Oct 23, 2023
efff8c9
Merge branch 'master' into feat/add-test-reults-output
emilyrohrbough Oct 26, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/workflows/example-recording.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,29 @@ jobs:
echo Cypress finished with: ${{ steps.cypress.outcome }}
echo See results at ${{ steps.cypress.outputs.resultsUrl }}

# print the `runResults` output directly as string
- name: Print Cypress run results output
run: |
echo Cypress run results: ${{ steps.cypress.outputs.runResults }}

# parse the `runResults` output as JSON.
# Note that the output is a string and empty if there is a custom command passed to the action and the command runs through CLI
- name: Parse the Cypress run results output
uses: actions/github-script@v6
with:
script: |
if (${{ steps.cypress.outputs.runResults }} == "") {
console.log('No run results found')
return
}

const runResults = JSON.parse('${{ steps.cypress.outputs.runResults }}')
console.log("TotalPassed: " + runResults.totalPassed)
console.log("TotalFailed: " + runResults.totalFailed)
console.log("TotalPending: " + runResults.totalPending)
console.log("TotalSkipped: " + runResults.totalSkipped)
console.log("TotalDuration: " + runResults.totalDuration)

group:
runs-on: ubuntu-22.04
needs: [check-record-key]
Expand Down
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1382,6 +1382,23 @@ This is an example of using the step output `resultsUrl`:

The GitHub step output `dashboardUrl` is deprecated. Cypress Dashboard is now [Cypress Cloud](https://on.cypress.io/cloud-introduction).

This action also sets a GitHub step output `runResults` which contains a stringified JSON object.

This is an example of parsing the `runResults`.

```yaml
...
steps:
- name: Parse run results
uses: actions/github-script@v6
with:
script: |
const runResults = JSON.parse('${{ steps.cypress-run.outputs.runResults }}')
console.log(runResults.totalPassed)
...
```
Structure of the run results object can be found [here](https://docs.cypress.io/guides/guides/module-api#Results).

[![recording example](https://github.com/cypress-io/github-action/workflows/example-recording/badge.svg?branch=master)](.github/workflows/example-recording.yml)

**Note:** every GitHub workflow step can have `outcome` and `conclusion` properties. See the GitHub [Contexts](https://docs.github.com/en/actions/learn-github-actions/contexts) documentation section [steps context](https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#steps-context). In particular, the `outcome` or `conclusion` value can be `success`, `failure`, `cancelled`, or `skipped` which you can use in any following steps.
Expand Down
2 changes: 2 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,8 @@ outputs:
description: 'Cypress Cloud URL if the run was recorded (deprecated)'
resultsUrl:
description: 'Cypress Cloud URL if the run was recorded'
runResults:
description: 'Cypress run results (stringified JSON object)'
runs:
using: 'node20'
main: 'dist/index.js'
Expand Down
2 changes: 2 additions & 0 deletions dist/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -75212,6 +75212,8 @@ const runTests = async () => {
debug(`Cypress options ${JSON.stringify(cypressOptions)}`)

const onTestsFinished = (testResults) => {
core.setOutput('runResults', testResults)

const resultsUrl = testResults.runUrl
process.chdir(startWorkingDirectory)

Expand Down
2 changes: 2 additions & 0 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -816,6 +816,8 @@ const runTests = async () => {
debug(`Cypress options ${JSON.stringify(cypressOptions)}`)

const onTestsFinished = (testResults) => {
core.setOutput('runResults', testResults)
Copy link
Member

@emilyrohrbough emilyrohrbough Oct 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed with the team. For simplicity, we would like to scope the exposed runResults stringified object to version, totalDuration, totalFailed, totalPassed, totalPending, totalSkipped, totalSuites and totalTests. The results object can get quite large depending on the size of your test suite and if more information is needed, it can be parsed and save in the afterRun hook.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@emilyrohrbough can logging the entire results object be an optional input parameter instead? e.g.

output-entire-results-object: true 

This way, by default, a summary is output but still leaves us the option to get the entire object. This way one does not need to go in and change all the test files to log an object (only needs a version update and addition of a parameter)
CC @mihaisee


const resultsUrl = testResults.runUrl
process.chdir(startWorkingDirectory)

Expand Down