[ui, ci] UI test look-back and comparison for PRs #24604

philrenaud · 2024-12-04T14:52:13Z

Description

Phase 2 of the project to fix and improve our ember test audits. Phase 1 can be found here.

…checkmark

.github/workflows/test-ui.yml

…them and main

.github/workflows/test-ui.yml

+
+      # Try to list available artifacts first
+      - name: List artifacts
+        uses: actions/github-script@v7


github-actions · 2024-12-05T15:31:45Z

Test Time Analysis

Total Tests: 1606
Significantly Slower: 64
Significantly Faster: 55

Most Significant Changes:

Integration | Helper | trim-path: it will remove a suffix slash

Current: 215ms
Historical Avg: 39.84615384615385ms
Change: 439.6%

Integration | Component | list pagination: when there is less pages than the total spread amount

Current: 215ms
Historical Avg: 43ms
Change: 400.0%

Unit | Ability | token: A non-management user can do nothing with tokens

Current: 120ms
Historical Avg: 25.923076923076923ms
Change: 362.9%

Integration | Component | das/recommendation-card: it renders diff calculations in a sentence with no aggregation for one allocatio

Current: 315ms
Historical Avg: 92.84615384615384ms
Change: 239.3%

Integration | Component | multi-select dropdown: list options have a zero tabindex and are therefore sequentially navigable

Current: 168ms
Historical Avg: 55.46153846153846ms
Change: 202.9%

github-actions · 2024-12-05T15:46:07Z

Test Time Analysis

Total Tests: 1606
Significantly Slower: 25
Significantly Faster: 10

Most Significant Changes:

Integration | Helper | trim-path: it will remove a suffix slash

Current: 215ms
Historical Avg: 39.42857142857143ms
Change: 445.3%

Integration | Component | das/recommendation-card: it renders diff calculations in a sentence with no aggregation for one allocatio

Current: 315ms
Historical Avg: 94.28571428571429ms
Change: 234.1%

Integration | Component | TopoViz::Datacenter: args get passed down to the TopViz::Node children

Current: 317ms
Historical Avg: 125.35714285714286ms
Change: 152.9%

Integration | Component | das/recommendation-card: it disables a resource’s toggle all toggle when there are no recommendations for it

Current: 144ms
Historical Avg: 59ms
Change: 144.1%

Integration | Component | line-chart: annotations will have staggered heights when too close to be positioned side-by-side

Current: 550ms
Historical Avg: 261.42857142857144ms
Change: 110.4%

github-actions · 2024-12-05T15:48:06Z

Test Time Analysis

Total Tests: 1606
Significantly Slower: 65
Significantly Faster: 56

Most Significant Changes:

Integration | Helper | trim-path: it will remove a suffix slash

Current: 215ms
Historical Avg: 39.42857142857143ms
Change: 445.3%

Integration | Component | list pagination: when there is less pages than the total spread amount

Current: 215ms
Historical Avg: 43.07142857142857ms
Change: 399.2%

Unit | Ability | token: A non-management user can do nothing with tokens

Current: 120ms
Historical Avg: 25.071428571428573ms
Change: 378.6%

Integration | Component | das/recommendation-card: it renders diff calculations in a sentence with no aggregation for one allocatio

Current: 315ms
Historical Avg: 94.28571428571429ms
Change: 234.1%

Integration | Component | multi-select dropdown: list options have a zero tabindex and are therefore sequentially navigable

Current: 168ms
Historical Avg: 54.57142857142857ms
Change: 207.9%

github-actions · 2024-12-05T16:00:14Z

Test Time Analysis

Total Tests: 1606
Significantly Slower: 59
Significantly Faster: 59

Most Significant Changes:

Integration | Helper | trim-path: it will remove a suffix slash

Current: 215ms
Historical Avg: 39.2ms
Change: 448.5%

Unit | Ability | token: A non-management user can do nothing with tokens

Current: 120ms
Historical Avg: 26.2ms
Change: 358.0%

Integration | Component | list pagination: when there is less pages than the total spread amount

Current: 215ms
Historical Avg: 56.93333333333333ms
Change: 277.6%

Integration | Component | trigger > Asynchronous Interactions: it can trigger an asynchronous action

Current: 546ms
Historical Avg: 158.46666666666667ms
Change: 244.6%

Integration | Component | das/recommendation-card: it renders diff calculations in a sentence with no aggregation for one allocatio

Current: 315ms
Historical Avg: 99.6ms
Change: 216.3%

github-actions · 2024-12-05T16:05:23Z

Test Time Analysis

Total Tests: 1606
Significantly Slower: 59
Significantly Faster: 59

Most Significant Changes:

Integration | Helper | trim-path: it will remove a suffix slash

Current: 215ms
Historical Avg: 39.2ms
Change: 448.5%

Unit | Ability | token: A non-management user can do nothing with tokens

Current: 120ms
Historical Avg: 26.2ms
Change: 358.0%

Integration | Component | list pagination: when there is less pages than the total spread amount

Current: 215ms
Historical Avg: 56.93333333333333ms
Change: 277.6%

Integration | Component | trigger > Asynchronous Interactions: it can trigger an asynchronous action

Current: 546ms
Historical Avg: 158.46666666666667ms
Change: 244.6%

Integration | Component | das/recommendation-card: it renders diff calculations in a sentence with no aggregation for one allocatio

Current: 315ms
Historical Avg: 99.6ms
Change: 216.3%

github-actions · 2024-12-05T16:13:06Z

Test Time Analysis

Total Tests: 1606
Significantly Slower: 58
Significantly Faster: 62

Most Significant Changes:

Integration | Helper | trim-path: it will remove a suffix slash

Current: 215ms
Historical Avg: 39.625ms
Change: 442.6%

Unit | Ability | token: A non-management user can do nothing with tokens

Current: 120ms
Historical Avg: 25.375ms
Change: 372.9%

Integration | Component | list pagination: when there is less pages than the total spread amount

Current: 215ms
Historical Avg: 55.8125ms
Change: 285.2%

Integration | Component | trigger > Asynchronous Interactions: it can trigger an asynchronous action

Current: 546ms
Historical Avg: 157.8125ms
Change: 246.0%

Integration | Component | das/recommendation-card: it renders diff calculations in a sentence with no aggregation for one allocatio

Current: 315ms
Historical Avg: 96.375ms
Change: 226.8%

github-actions · 2024-12-05T16:28:01Z

Test Time Analysis

Total Tests: 1606
Significantly Slower: 61
Significantly Faster: 64

Most Significant Changes:

Unit | Ability | token: A non-management user can do nothing with tokens

Current: 120ms
Historical Avg: 24.11111111111111ms
Change: 397.7%
Trend: ▁▁▁▂▁▁▁█▁▁▁▁▁▁▁▁▁▁ ↑ 🔴

Integration | Helper | trim-path: it will remove a suffix slash

Current: 215ms
Historical Avg: 47.5ms
Change: 352.6%
Trend: ▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁█▁ ↑ 🔴

Integration | Component | list pagination: when there is less pages than the total spread amount

Current: 215ms
Historical Avg: 53.94444444444444ms
Change: 298.6%
Trend: ▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ↑ 🔴

Integration | Component | trigger > Asynchronous Interactions: it can trigger an asynchronous action

Current: 546ms
Historical Avg: 152.22222222222223ms
Change: 258.7%
Trend: ▁▁▁▁▂▁█▁▂▁▅▁▄▁▂▁▂▁ ↑ 🔴

Integration | Component | das/recommendation-card: it renders diff calculations in a sentence with no aggregation for one allocatio

Current: 315ms
Historical Avg: 93.44444444444444ms
Change: 237.1%
Trend: ▃▁▁▂▁▁▁▂▂▁█▁▂▁▁▃▂▂ ↑ 🔴

philrenaud added 2 commits December 4, 2024 09:37

Continue-on-error placement was causing failing exams to get a green …

7b8ca14

…checkmark

First pass at lookback step

bea9b3a

philrenaud self-assigned this Dec 4, 2024

github-advanced-security bot found potential problems Dec 4, 2024

View reviewed changes

.github/workflows/test-ui.yml Fixed Show fixed Hide fixed

.github/workflows/test-ui.yml Fixed Show fixed Hide fixed

.github/workflows/test-ui.yml Fixed Show fixed Hide fixed

.github/workflows/test-ui.yml Fixed Show fixed Hide fixed

Pinned github action artifacts

4b984ad

vercel bot deployed to Preview – nomad-ui December 4, 2024 15:02 View deployment

no-op to trigger ui tests

cdbff0b

vercel bot deployed to Preview – nomad-ui December 4, 2024 15:04 View deployment

Retain currently-run PR results for 7 days and differentiate between …

65fe72a

…them and main

vercel bot deployed to Preview – nomad-ui December 4, 2024 15:38 View deployment

vercel bot deployed to Preview – nomad-ui December 4, 2024 15:59 View deployment

github-advanced-security bot found potential problems Dec 4, 2024

View reviewed changes

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from 6aacf40 to 0d96528 Compare December 4, 2024 16:01

vercel bot deployed to Preview – nomad-ui December 4, 2024 16:02 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from 0d96528 to 9ddf7c6 Compare December 4, 2024 16:09

vercel bot deployed to Preview – nomad-ui December 4, 2024 16:10 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from 9ddf7c6 to 3226226 Compare December 4, 2024 16:20

vercel bot deployed to Preview – nomad-ui December 4, 2024 16:21 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from 3226226 to b71f943 Compare December 4, 2024 16:25

vercel bot deployed to Preview – nomad-ui December 4, 2024 16:26 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from b71f943 to b59eb3b Compare December 4, 2024 16:27

vercel bot deployed to Preview – nomad-ui December 4, 2024 16:28 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from b59eb3b to 5b76906 Compare December 4, 2024 16:54

vercel bot deployed to Preview – nomad-ui December 4, 2024 16:55 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from 5b76906 to a7b089e Compare December 4, 2024 16:56

vercel bot deployed to Preview – nomad-ui December 4, 2024 16:57 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from a7b089e to 9155550 Compare December 4, 2024 16:58

vercel bot deployed to Preview – nomad-ui December 4, 2024 16:59 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from 9155550 to b792afc Compare December 4, 2024 16:59

vercel bot deployed to Preview – nomad-ui December 4, 2024 17:00 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from b892be0 to 368dd7f Compare December 5, 2024 15:07

vercel bot deployed to Preview – nomad-ui December 5, 2024 15:08 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from 368dd7f to 7e6e8c6 Compare December 5, 2024 15:14

vercel bot deployed to Preview – nomad-ui December 5, 2024 15:15 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from 7e6e8c6 to f828286 Compare December 5, 2024 15:29

vercel bot deployed to Preview – nomad-ui December 5, 2024 15:30 View deployment

Debugging run

90eae68

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from f828286 to 90eae68 Compare December 5, 2024 15:31

vercel bot deployed to Preview – nomad-ui December 5, 2024 15:32 View deployment

vercel bot deployed to Preview – nomad-ui December 5, 2024 15:46 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from 3e1b978 to 3f15f3c Compare December 5, 2024 15:47

vercel bot deployed to Preview – nomad-ui December 5, 2024 15:48 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from 3f15f3c to 1386019 Compare December 5, 2024 15:59

vercel bot deployed to Preview – nomad-ui December 5, 2024 16:00 View deployment

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from 1386019 to b123921 Compare December 5, 2024 16:04

vercel bot deployed to Preview – nomad-ui December 5, 2024 16:06 View deployment

A little debugging for historical timings

aed33ae

philrenaud force-pushed the b-ui/ember-test-lookback-and-compare branch from b123921 to aed33ae Compare December 5, 2024 16:12

vercel bot deployed to Preview – nomad-ui December 5, 2024 16:13 View deployment

OK but what if I made a data visualization for very good reasons

b8f9c28

vercel bot deployed to Preview – nomad-ui December 5, 2024 16:28 View deployment

This was referenced Dec 6, 2024

[ui, ci] Ember Exam failures should cause overall test failures #24603

Open

Remove the Ember Test Audit workflow #24637

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ui, ci] UI test look-back and comparison for PRs #24604

[ui, ci] UI test look-back and comparison for PRs #24604

philrenaud commented Dec 4, 2024

github-actions bot commented Dec 5, 2024

github-actions bot commented Dec 5, 2024

github-actions bot commented Dec 5, 2024

github-actions bot commented Dec 5, 2024

github-actions bot commented Dec 5, 2024

github-actions bot commented Dec 5, 2024

github-actions bot commented Dec 5, 2024

[ui, ci] UI test look-back and comparison for PRs #24604

Are you sure you want to change the base?

[ui, ci] UI test look-back and comparison for PRs #24604

Conversation

philrenaud commented Dec 4, 2024

Description

github-actions bot commented Dec 5, 2024

Test Time Analysis

Most Significant Changes:

github-actions bot commented Dec 5, 2024

Test Time Analysis

Most Significant Changes:

github-actions bot commented Dec 5, 2024

Test Time Analysis

Most Significant Changes:

github-actions bot commented Dec 5, 2024

Test Time Analysis

Most Significant Changes:

github-actions bot commented Dec 5, 2024

Test Time Analysis

Most Significant Changes:

github-actions bot commented Dec 5, 2024

Test Time Analysis

Most Significant Changes:

github-actions bot commented Dec 5, 2024

Test Time Analysis

Most Significant Changes: