Regression Tests #491
DafyddLlyrregression-tests.yml
on: workflow_dispatch
Run Integration tests
5m 32s
Run API Tests
2m 48s
Run React Tests
5m 55s
E2E tests
1h 7m
Notify On Failure
0s
Annotations
10 errors, 22 warnings, and 1 notice
|
[chromium] › create-flow/create-flow.spec.ts:165:7 › Navigation › Publish a flow:
e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts#L1
1) [chromium] › create-flow/create-flow.spec.ts:165:7 › Navigation › Publish a flow ──────────────
Test timeout of 45000ms exceeded.
|
|
[chromium] › create-flow/create-flow.spec.ts:165:7 › Navigation › Publish a flow:
e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts#L174
1) [chromium] › create-flow/create-flow.spec.ts:165:7 › Navigation › Publish a flow ──────────────
Error: locator.click: Test ended.
Call log:
- waiting for getByRole('button', { name: 'PUBLISH', exact: true })
172 |
173 | page.getByRole("button", { name: "CHECK FOR CHANGES TO PUBLISH" }).click();
> 174 | page.getByRole("button", { name: "PUBLISH", exact: true }).click();
| ^
175 |
176 | const previewLink = page.getByRole("link", {
177 | name: "Open published service",
at /home/runner/work/planx-new/planx-new/e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts:174:64
|
|
[chromium] › create-flow/create-flow.spec.ts:165:7 › Navigation › Publish a flow:
e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts#L1
1) [chromium] › create-flow/create-flow.spec.ts:165:7 › Navigation › Publish a flow ──────────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Test timeout of 45000ms exceeded.
|
|
[chromium] › create-flow/create-flow.spec.ts:165:7 › Navigation › Publish a flow:
e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts#L173
1) [chromium] › create-flow/create-flow.spec.ts:165:7 › Navigation › Publish a flow ──────────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: locator.click: Test ended.
Call log:
- waiting for getByRole('button', { name: 'CHECK FOR CHANGES TO PUBLISH' })
171 | await page.goto(`/${context.team.slug}/${serviceProps.slug}`);
172 |
> 173 | page.getByRole("button", { name: "CHECK FOR CHANGES TO PUBLISH" }).click();
| ^
174 | page.getByRole("button", { name: "PUBLISH", exact: true }).click();
175 |
176 | const previewLink = page.getByRole("link", {
at /home/runner/work/planx-new/planx-new/e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts:173:72
|
|
[chromium] › create-flow/create-flow.spec.ts:165:7 › Navigation › Publish a flow:
e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts#L1
1) [chromium] › create-flow/create-flow.spec.ts:165:7 › Navigation › Publish a flow ──────────────
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Test timeout of 45000ms exceeded.
|
|
[chromium] › create-flow/create-flow.spec.ts:165:7 › Navigation › Publish a flow:
e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts#L173
1) [chromium] › create-flow/create-flow.spec.ts:165:7 › Navigation › Publish a flow ──────────────
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: locator.click: Test ended.
Call log:
- waiting for getByRole('button', { name: 'CHECK FOR CHANGES TO PUBLISH' })
171 | await page.goto(`/${context.team.slug}/${serviceProps.slug}`);
172 |
> 173 | page.getByRole("button", { name: "CHECK FOR CHANGES TO PUBLISH" }).click();
| ^
174 | page.getByRole("button", { name: "PUBLISH", exact: true }).click();
175 |
176 | const previewLink = page.getByRole("link", {
at /home/runner/work/planx-new/planx-new/e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts:173:72
|
|
[chromium] › create-flow/create-flow.spec.ts:182:7 › Navigation › Cannot preview an offline flow:
e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts#L1
2) [chromium] › create-flow/create-flow.spec.ts:182:7 › Navigation › Cannot preview an offline flow
Test timeout of 45000ms exceeded.
|
|
[chromium] › create-flow/create-flow.spec.ts:182:7 › Navigation › Cannot preview an offline flow:
e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts#L198
2) [chromium] › create-flow/create-flow.spec.ts:182:7 › Navigation › Cannot preview an offline flow
Error: expect(locator).toBeVisible()
Locator: getByRole('heading', { name: 'Offline', level: 1 })
Expected: visible
Received: hidden
Call log:
- expect.toBeVisible with timeout 45000ms
- waiting for getByRole('heading', { name: 'Offline', level: 1 })
196 | await expect(
197 | page.getByRole("heading", { level: 1, name: "Offline" }),
> 198 | ).toBeVisible();
| ^
199 | });
200 |
201 | test("Turn a flow online", async ({ browser }) => {
at /home/runner/work/planx-new/planx-new/e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts:198:7
|
|
[chromium] › create-flow/create-flow.spec.ts:182:7 › Navigation › Cannot preview an offline flow:
e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts#L1
2) [chromium] › create-flow/create-flow.spec.ts:182:7 › Navigation › Cannot preview an offline flow
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Test timeout of 45000ms exceeded.
|
|
[chromium] › create-flow/create-flow.spec.ts:182:7 › Navigation › Cannot preview an offline flow:
e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts#L198
2) [chromium] › create-flow/create-flow.spec.ts:182:7 › Navigation › Cannot preview an offline flow
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toBeVisible()
Locator: getByRole('heading', { name: 'Offline', level: 1 })
Expected: visible
Received: hidden
Call log:
- expect.toBeVisible with timeout 45000ms
- waiting for getByRole('heading', { name: 'Offline', level: 1 })
196 | await expect(
197 | page.getByRole("heading", { level: 1, name: "Offline" }),
> 198 | ).toBeVisible();
| ^
199 | });
200 |
201 | test("Turn a flow online", async ({ browser }) => {
at /home/runner/work/planx-new/planx-new/e2e/tests/ui-driven/src/create-flow/create-flow.spec.ts:198:7
|
|
Run API Tests
The following actions uses Node.js version which is deprecated and will be forced to run on node20: andstor/file-existence-action@v2. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
|
|
Run API Tests:
api.planx.uk/lib/notify/index.ts#L73
Unexpected any. Specify a different type
|
|
|
|
Run API Tests:
api.planx.uk/modules/auth/strategy/google.ts#L20
Unexpected any. Specify a different type
|
|
Run API Tests:
api.planx.uk/modules/flows/validate/service/fileTypes.ts#L73
Unexpected any. Specify a different type
|
|
Run API Tests:
api.planx.uk/modules/gis/service/classifiedRoads.test.ts#L20
Disabled test suite
|
|
Run API Tests:
api.planx.uk/modules/gis/service/classifiedRoads.ts#L129
Unexpected any. Specify a different type
|
|
Run API Tests:
api.planx.uk/modules/gis/service/classifiedRoads.ts#L150
Unexpected any. Specify a different type
|
|
Run API Tests:
api.planx.uk/modules/gis/service/digitalLand.ts#L80
Unexpected any. Specify a different type
|
|
Run API Tests:
api.planx.uk/modules/gis/service/digitalLand.ts#L81
Unexpected any. Specify a different type
|
|
Run API Tests:
api.planx.uk/modules/gis/service/digitalLand.ts#L115
Unexpected any. Specify a different type
|
|
Run Integration tests
The following actions uses Node.js version which is deprecated and will be forced to run on node20: andstor/file-existence-action@v2. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
|
|
Run React Tests
The following actions uses Node.js version which is deprecated and will be forced to run on node20: andstor/file-existence-action@v2. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
|
|
E2E tests:
e2e/tests/ui-driven/src/invite-to-pay/agent.spec.ts#L93
Unexpected use of networkidle
|
|
E2E tests:
e2e/tests/ui-driven/src/invite-to-pay/nominee.spec.ts#L87
Unexpected use of networkidle
|
|
E2E tests:
e2e/tests/ui-driven/src/invite-to-pay/nominee.spec.ts#L96
Unexpected use of networkidle
|
|
E2E tests:
e2e/tests/ui-driven/src/invite-to-pay/nominee.spec.ts#L131
Unexpected use of networkidle
|
|
Slow Test:
e2e/tests/ui-driven/[chromium] › invite-to-pay/agent.spec.ts#L1
e2e/tests/ui-driven/[chromium] › invite-to-pay/agent.spec.ts took 13.5m
|
|
Slow Test:
e2e/tests/ui-driven/[chromium] › pay.spec.ts#L1
e2e/tests/ui-driven/[chromium] › pay.spec.ts took 13.5m
|
|
Slow Test:
e2e/tests/ui-driven/[chromium] › create-flow/create-flow.spec.ts#L1
e2e/tests/ui-driven/[chromium] › create-flow/create-flow.spec.ts took 9.1m
|
|
Slow Test:
e2e/tests/ui-driven/[chromium] › save-and-return.spec.ts#L1
e2e/tests/ui-driven/[chromium] › save-and-return.spec.ts took 9.1m
|
|
Slow Test:
e2e/tests/ui-driven/[chromium] › sections.spec.ts#L1
e2e/tests/ui-driven/[chromium] › sections.spec.ts took 6.8m
|
|
🎭 Playwright Run Summary
24 failed
[chromium] › create-flow/create-flow.spec.ts:165:7 › Navigation › Publish a flow ───────────────
[chromium] › create-flow/create-flow.spec.ts:182:7 › Navigation › Cannot preview an offline flow
[chromium] › create-flow/create-flow.spec.ts:201:7 › Navigation › Turn a flow online ───────────
[chromium] › create-flow/create-flow.spec.ts:228:7 › Navigation › Can preview a published flow ─
[chromium] › invite-to-pay/agent.spec.ts:45:7 › Agent journey @regression › agent can send a payment request
[chromium] › invite-to-pay/agent.spec.ts:79:7 › Agent journey @regression › agent cannot send a payment request after initialising a normal payment
[chromium] › invite-to-pay/agent.spec.ts:98:7 › Agent journey @regression › agent cannot make changes after sending a payment request - session is locked
[chromium] › invite-to-pay/agent.spec.ts:125:7 › Agent journey @regression › reconciliation does not apply to sessions with open payment requests
[chromium] › invite-to-pay/agent.spec.ts:162:7 › Agent journey @regression › agent cannot make payment after sending a payment request in another tab
[chromium] › invite-to-pay/agent.spec.ts:194:7 › Agent journey @regression › agent cannot generate payment request after starting payment in another tab
[chromium] › invite-to-pay/nominee.spec.ts:46:7 › Nominee journey @regression › responding to a valid payment request
[chromium] › pay.spec.ts:53:7 › Gov Pay integration @regression › a successful payment ─────────
[chromium] › pay.spec.ts:86:7 › Gov Pay integration @regression › a retry attempt for a failed GOV.UK payment
[chromium] › pay.spec.ts:145:7 › Gov Pay integration @regression › a retry attempt for a cancelled GOV.UK payment
[chromium] › pay.spec.ts:196:7 › Gov Pay integration @regression › a retry attempt for an abandoned GOV.UK payment
[chromium] › pay.spec.ts:254:7 › Gov Pay integration @regression › a retry attempt for an abandoned and then cancelled GOV.UK payment
[chromium] › pay.spec.ts:297:7 › Gov Pay integration @regression › navigating back to the pay component after a successful payment
[chromium] › save-and-return.spec.ts:66:9 › Save and return › resuming a saved application › the application presents a review screen on return
[chromium] › save-and-return.spec.ts:92:9 › Save and return › resuming a saved application › the application resumes from the last unanswered question
[chromium] › save-and-return.spec.ts:119:9 › Save and return › session reconciliation › the application resumes with no modifications
[chromium] › save-and-return.spec.ts:142:9 › Save and return › session reconciliation › the application resumes from the first modified question
[chromium] › sections.spec.ts:314:9 › Section statuses › simple save and return › not started, ready to start and complete statuses
[chromium] › sections.spec.ts:373:9 › Section statuses › simple save and return › the ready to continue status
[chromium] › sections.spec.ts:447:9 › Section statuses › save and return with service changes (reconciliation) › needs new information and started
14 passed (57.6m)
|
Artifacts
Produced during runtime
| Name | Size | |
|---|---|---|
|
playwright-report
Expired
|
103 MB |
|