[Obs AI Assistant] Attempt to fix flaky KB migration test #206324
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
viduni94
added
release_note:skip
|
Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant) |
botelastic
bot
added
the
ci:project-deploy-observability
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#7698[✅] x-pack/test/observability_ai_assistant_api_integration/enterprise/config.ts: 200/200 tests passed. |
|
What happens if you run the flaky test runner on main? I want to be sure that retrying actually improves this |
viduni94
force-pushed
the
attempt-to-fix-flaky-KB-migration-test
branch
from
b14c8bd to
36b628e
Compare
@sorenlouv
The expectation behind adding a The flakiness was resolved as seen in this test run, after adding the retry statement. |
sorenlouv
approved these changes
Jan 13, 2025
viduni94
force-pushed
the
attempt-to-fix-flaky-KB-migration-test
branch
from
5bad10e to
6a0a28d
Compare
💚 Build Succeeded
Metrics [docs]
History
cc @viduni94 |
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#7708[✅] x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts: 100/100 tests passed. |
|
Starting backport for target branches: 8.x |
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
sorenlouv
added a commit
that referenced
this pull request
Jan 14, 2025
) (#206543) # Backport This will backport the following commits from `main` to `8.x`: - [[Obs AI Assistant] Attempt to fix flaky KB migration test (#206324)](#206324) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Viduni Wickramarachchi","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-01-13T20:12:58Z","message":"[Obs AI Assistant] Attempt to fix flaky KB migration test (#206324)\n\n## Summary\r\n\r\nCloses https://github.com/elastic/kibana/issues/202926\r\n\r\n### Problem\r\nThe KB migration test appears to be flaky sometimes, because the\r\nmigration takes a few seconds to run. Therefore, when the test expects\r\nto have semantic_text for all entries, it fails.\r\n\r\n### Solution\r\nAdding a retry statement to avoid the failure in cases where the\r\nmigration takes a little while to run.\r\n\r\n### Checklist\r\n\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n- [x] The PR description includes the appropriate Release Notes section,\r\nand the correct `release_note:*` label is applied per the\r\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"95b76dc12b11bc5e147bb6e6628cec471fbc02e3","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Obs AI Assistant","ci:project-deploy-observability","backport:version","v8.18.0"],"number":206324,"url":"https://github.com/elastic/kibana/pull/206324","mergeCommit":{"message":"[Obs AI Assistant] Attempt to fix flaky KB migration test (#206324)\n\n## Summary\r\n\r\nCloses https://github.com/elastic/kibana/issues/202926\r\n\r\n### Problem\r\nThe KB migration test appears to be flaky sometimes, because the\r\nmigration takes a few seconds to run. Therefore, when the test expects\r\nto have semantic_text for all entries, it fails.\r\n\r\n### Solution\r\nAdding a retry statement to avoid the failure in cases where the\r\nmigration takes a little while to run.\r\n\r\n### Checklist\r\n\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n- [x] The PR description includes the appropriate Release Notes section,\r\nand the correct `release_note:*` label is applied per the\r\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"95b76dc12b11bc5e147bb6e6628cec471fbc02e3"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/206324","number":206324,"mergeCommit":{"message":"[Obs AI Assistant] Attempt to fix flaky KB migration test (#206324)\n\n## Summary\r\n\r\nCloses https://github.com/elastic/kibana/issues/202926\r\n\r\n### Problem\r\nThe KB migration test appears to be flaky sometimes, because the\r\nmigration takes a few seconds to run. Therefore, when the test expects\r\nto have semantic_text for all entries, it fails.\r\n\r\n### Solution\r\nAdding a retry statement to avoid the failure in cases where the\r\nmigration takes a little while to run.\r\n\r\n### Checklist\r\n\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n- [x] The PR description includes the appropriate Release Notes section,\r\nand the correct `release_note:*` label is applied per the\r\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"95b76dc12b11bc5e147bb6e6628cec471fbc02e3"}},{"branch":"8.x","label":"v8.18.0","labelRegex":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Viduni Wickramarachchi <[email protected]>
viduni94
added a commit
to viduni94/kibana
that referenced
this pull request
Jan 23, 2025
…6324) ## Summary Closes elastic#202926 ### Problem The KB migration test appears to be flaky sometimes, because the migration takes a few seconds to run. Therefore, when the test expects to have semantic_text for all entries, it fails. ### Solution Adding a retry statement to avoid the failure in cases where the migration takes a little while to run. ### Checklist - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Closes #202926
Problem
The KB migration test appears to be flaky sometimes, because the migration takes a few seconds to run. Therefore, when the test expects to have semantic_text for all entries, it fails.
Solution
Adding a retry statement to avoid the failure in cases where the migration takes a little while to run.
Checklist
release_note:*label is applied per the guidelines