Skip to content

Commit

Permalink
update document
Browse files Browse the repository at this point in the history
Signed-off-by: Lu Yu <[email protected]>
  • Loading branch information
BionIT committed May 16, 2024
1 parent e3600f7 commit eb177c4
Show file tree
Hide file tree
Showing 19 changed files with 96 additions and 103 deletions.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
layout: post
title: "Top 3 Scenarios Where Managing Multiple Clusters with One OpenSearch Dashboard is a Life Saver"
authors:
- nluyu
date: 2024-05-16
categories:
- feature
meta_keyword: OpenSearch multiple data source, OpenSearch Dashboards
meta_description: This blog provides a quick glance at the accumulated features of multi-data sources from version 2.4 to 2.14 and how these features empower your productivity with unified experience.
---

OpenSearch Dashboards, the visualization tool for your OpenSearch data, offers the multiple data sources feature. This feature enables you to consolidate and visualize data from various OpenSearch clusters within a unified dashboard. By integrating disparate sources into a single view, multiple data sources streamlines data analysis workflows, eliminates the need for complex data consolidation processes, and enhances productivity. With the ability to connect to and visualize data from multiple OpenSearch clusters, you can unlock new levels of observability, governance, and data-driven decision-making.

Multiple data sources is available in OpenSearch version 2.4 and later. Multiple data sources supports authentication types (AWS SigV4, basic authentication), Vega visualizations, and data management features (saved objects importing). With OpenSearch 2.14, we have expanded multiple data sources support for nine dashboard features (index management, machine learning (ML), search relevance, anomaly detection, maps, security, notification, query workbench, and trace analytics) and certain visualizations (time-series visual builder [TSVB] and region maps). If you are managing an environment with multiple clusters and use one cluster more than the others, you now have functionality to set a default cluster that can be referenced by all dashboard features.

This blog explains the top three scenarios to enable one OpenSearch Dashboards instance to handle data from multiple data sources.

## Scenario 1: Unified analytics across distributed data sources
For enterprises with data scattered across various clusters, users previously had to establish temporary ingestion pipelines to consolidate data into a central repository for analysis. However, with the introduction of multiple data sources support, OpenSearch Dashboards now enables users to query and visualize data from multiple sources within a single dashboards. This feature supports a range of visualization types, like Vega, TSVB, Region Map, and index pattern-based visualizations, providing a comprehensive and cohesive view of dispersed data.

The following images show how OpenSearch Dashboards can present data from various sources in a unified dashboard view.

Users can perform query analytics on different clusters using the data source picker.

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/discover.png"/>

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/dashboards_overall.png"/>

The following image shows a TSVB visualization created from disparate data sources.

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/tsvb_overall.png"/>

By simply specifying a new data_source_name field within the Vega specification url field, users can create Vega visualization with data from one or multiple specific data sources by name. An example specification setting is shown in the following image.

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/vega_overall.png"/>

Multiple data sources supports adding a prefix to the index pattern name, improving ease of identifying the pattern during its creation.

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/maps_1.png"/>

Also, the referenced data sources from the current map layers are indicated by clicking on the icon on the top navigation bar.

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/maps_2.png"/>

## Scenario 2: Centralized data management and governance

In addition to analyzing multiple data sources in a unified dashboard, you can manage remote clusters operationally. You can manage your indexes, security policies, notifications, and ML models and then select the target data source from the unified dashboard.

OpenSearch Dashboards allows you to perform all index operations, such as creating, reading, updating, and deleting indexes, as well as managing index mappings, templates, and aliases, across multiple data sources from a single instance.

You can switch between different data sources using the upper-right navigation bar, select a specific index, and choose the desired operation. Selecting on an index name will display detailed information about that index. The data source id, which is the unique identifier (UUID) is embedded in the URL, allowing you to bookmark and share specific pages directly.

The following images show the data source picker in the top right hand corner within Index State Management.

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/indexes_overview.png"/>

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/detailed_index.png"/>

OpenSearch Dashboards supports managing security across multiple data sources. Administrators can centrally manage roles and internal users for different clusters by selecting the desired cluster from the DATA SOURCES dropdown menu, as shown in the following image.

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/roles_overview.png"/>

Users can also manage notification channels, as shown in the following image.

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/channels_overview.png"/>

Users can manage remote ML models, as shown in the following image. The pre-trained models are deployed to the selected cluster through the Dev Tools console.

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/models_overview.png"/>

The Dev Tools console, shown in the following image, is integrated with the multiple data sources feature. Users can select the target data source and execute commands to query ot modify the backend data directly from a central location.

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/devtool.png"/>

Multiple data sources enable OpenSearch Dashboards to provide a centralized interface to create, view, and modify anomaly detectors from various OpenSearch clusters, streamlining the management process, as shown in the following image.

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/detector_detail.png"/>


## Scenario 3: Comparing search query results across clusters

For users with multiple OpenSearch clusters handling search traffic, multiple data sources enables OpenSearch Dashboards with the ability to compare search results across these clusters within a single instance by using the Data Source picker in the Compare search results application.

<img src="/assets/media/blog-images/2024-05-16-top3-scenarios-where-managing-multiple-clusters-with-one-opensearch-dashboards-is-a-life-saver/compare_queries.png"/>


## Summary
The multiple data sources feature in OpenSearch Dashboards represents a significant advancement in data analysis and management. By consolidating access to distributed data sources within a single interface, it enables users to gain comprehensive insights, streamline operations, and make informed decisions more efficiently. As organizations face challenges in managing and analyzing data across disparate systems, this feature positions OpenSearch Dashboards as a powerful solution for centralized data observability, governance, and exploration. With the ability to unify data from multiple clusters, users can unlock new levels of productivity, collaboration, and data-driven decision-making across their data ecosystem.

If you have additional scenarios you’d like us to cover, let us know.

Special thanks to Ramakrishna Chilaka, Prabhat Sharma, Derek Ho, Junqiu Lei, Jackie Han, Riya Saxena, Sumukh Hanumantha Swamy, Lin Wang, Ella Zhu, Emma Jin, Huy Nguyen, Zhongnan Su for their work to enable multiple data sources support for OpenSearch Dashboards.



0 comments on commit eb177c4

Please sign in to comment.