Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Import Data from Global Hashmap and retrieve Datasource details #1083

Merged
merged 36 commits into from
Feb 19, 2024
Merged

Import Data from Global Hashmap and retrieve Datasource details #1083

merged 36 commits into from
Feb 19, 2024

Conversation

shreyabiradar07
Copy link
Contributor

@shreyabiradar07 shreyabiradar07 commented Jan 19, 2024 •
edited
Loading

This PR implements Import Data workflow from the Datasource collection and retrieval of Datasource details - cluster level metadata. Builds on top of PR 1080

Docker image - docker.io/shbiradar/datasource_details:kruize_local

@shreyabiradar07 shreyabiradar07 added the kruize-local Tag for mentioning all the PR's and issues raised which covers the kruize local monitoring usecase label Jan 19, 2024
@shreyabiradar07
Refactor code by encapsulating common API functionality for Prometheu…
0c71bf4 
…s data retrieval
@shreyabiradar07
Refactor interface methods for Prometheus data extraction
b26b4fa
@shreyabiradar07
Update extract methods to specifically extract Prometheus data values…
55f2870 
… and objects
@shreyabiradar07
Convert DataSourceDetailsOperator to singleton for restricted access …
fee27ff 
…to the list of DataSourceDetailsInfo objects
@shreyabiradar07
Add comments explaining DataSourceQueries and PromQLDataSourceQueries…
a1b45dc 
… classes
@shreyabiradar07
Update return type to JsonArray and rename function to getPrometheusD…
0428f61 
…ataResultArray
@shreyabiradar07
Rename function to getPrometheusDataValue
5fd8e3f
@shreyabiradar07
Rename and update function documentation
a6c063a
@shreyabiradar07
Rename createExperimentsFromDataSource to ImportDataFromDataSource an…
dd0becb 
…d comment out data import process
@shreyabiradar07 shreyabiradar07 changed the base branch from mvp_demo to kruize_local_poc January 29, 2024 09:52
@shreyabiradar07 shreyabiradar07 changed the title Create experiments and retrieve Datasource details on Initialization Import Data from Global Hashmap and retrieve Datasource details Jan 29, 2024
@dinogun
Copy link
Contributor

dinogun commented Jan 29, 2024

@shreyabiradar07 looks like you need to fix conflicts

@shreyabiradar07
Merge branch 'kruize_local_add_datasource' into 'retreive_datasource_…
58a1a01 
…details'

# Conflicts:
        src/main/java/com/autotune/common/datasource/PrometheusDataSource.java
        src/main/java/com/autotune/experimentManager/handler/MetricCollectionHandler.java
        src/main/java/com/autotune/common/datasource/PrometheusDataOperator.java
@shreyabiradar07
Add methods to execute queries on datasources
61b4a89
@shreyabiradar07
Optimize code by replacing strings with constants
1ff1a92
@shreyabiradar07
implement functionality to import data for a specified data source
6f845f7
@shreyabiradar07
Add validation for query result array
92eff8c
@shreyabiradar07
Optimise code by adding validation constants and exception
8d9e594
@shreyabiradar07 shreyabiradar07 marked this pull request as ready for review January 30, 2024 15:34
@dinogun
Copy link
Contributor

dinogun commented Jan 31, 2024

@shreyabiradar07 Please fix the conflicts

@shreyabiradar07
Merge remote-tracking branch 'upstream/kruize_local_poc' into 'retrie…
c3516c5 
…ve-datasource-details'

# Conflicts:
#       src/main/java/com/autotune/common/datasource/DataSourceOperator.java
#       src/main/java/com/autotune/common/datasource/DataSourceOperatorImpl.java
#       src/main/java/com/autotune/common/datasource/KruizeDataSourceOperator.java
#       src/main/java/com/autotune/common/datasource/prometheus/PrometheusDataOperatorImpl.java
@shreyabiradar07
Add exceptions to handle data source details
d76b029
@shreyabiradar07
Refactor datasource metadata population validation logic
76d4ac3
@shreyabiradar07
Refactor 'DataSourceDetailsInfo' and nested objects to use HashMap
4708484
@shreyabiradar07
Update 'DataSourceDetailsInfo' data population with namespace metadata
d80a641
@shreyabiradar07
Refactor DataSourceDetails classes and delete unused setter functions
8525835
@shreyabiradar07
Add functionalities for population of workload and container metadata…
4e6c13a 
… and update data validation
@shreyabiradar07
Update comments and refactor exception handling
1bf18bb
@shreyabiradar07
Add debug statements and replace strings with constants
aaa4782
@shreyabiradar07
Set cluster name as default and refactor nested if statements
ca9d759
dinogun
dinogun approved these changes Feb 19, 2024
View reviewed changes
Copy link
Contributor

@dinogun dinogun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dinogun dinogun merged commit e9d9a94 into kruize:kruize_local_poc Feb 19, 2024
10 of 11 checks passed
@rbadagandi1 rbadagandi1 added this to the Kruize 0.0.21_rm Release milestone Apr 24, 2024
@rbadagandi1 rbadagandi1 added remote_monitoring and removed kruize-local Tag for mentioning all the PR's and issues raised which covers the kruize local monitoring usecase labels Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dinogun dinogun dinogun approved these changes

Assignees

@shreyabiradar07 shreyabiradar07

Labels
remote_monitoring
Projects
Monitoring
Status: Done
Milestone
Kruize 0.0.21_rm Release
Development

Successfully merging this pull request may close these issues.
3 participants
@shreyabiradar07 @dinogun @rbadagandi1