[8.x] [Synthetics] Improve overview page performance !! (#201275) #203892
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## Summary Improve overview page performance !! Right now UI works for few hundred to 1000 monitors, but it starts degrading after that, this PR makes sure, we refactor queries in such a way that it scale up to 10k-20k monitors easily. ### Queries before Before this PR, we were doing 2 steps queries, first fetch all saved objects and the fetch all summary documents by passings all ids from first phase. This meant that let's say if we have 20k saved objects, first we will need to page through all of them to even start fetching summaries. To fetch summary documents, we were using `top_hits` query which can be memory expensive. ### Queries now In this PR we fetch summaries and saved objects in parallel, since we have space id on documents as well, there was no need to do 2 step queries. Now we fetch both things in parallel and then we hydrate saved object data from summary data. In this PR now we are using top_metrics query to fetch each monitor status instead of `top_hits` I tested on about 20k monitors, app performs reasoably well after the PR <img width="1920" alt="image" src="https://github.com/user-attachments/assets/c143e196-59a4-45b4-86b7-bd22ac4c5d4b"> On a very slow cluster on which kibana is local against a remote cluster ### After <img width="1920" alt="image" src="https://github.com/user-attachments/assets/540d0cdf-2f8c-44d1-af76-81953d9ca0ff"> ### Before <img width="1918" alt="image" src="https://github.com/user-attachments/assets/5fdc314d-bb59-4137-9397-d8aee6bd4806"> --------- Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit b4ccb0c)
botelastic
bot
added
ci:project-deploy-observability
|
Pinging @elastic/obs-ux-management-team (Team:obs-ux-management) |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]
History
cc @shahzad31 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto8.x:Questions ?
Please refer to the Backport tool documentation