Code

[Draft] Original changes against main#13

Code

Open

Arpit-Bandejiya wants to merge 153 commits intomainshiv0408/OpenSearch:mainfrom remote_state_collabshiv0408/OpenSearch:remote_state_collabCopy head branch name to clipboard

+10,255-1,700

Commits

Commits on Feb 6, 2024

• Split the cluster state remote global metadata file to metadata attribute files

  Show description for 6bf7bc9
  shiv0408

  committed
  6bf7bc9

  Copy full SHA for 6bf7bc9

Commits on Feb 20, 2024

• Upload all the metadata attributes in parallel

  Show description for 5d6a0ad
  shiv0408

  committed
  5d6a0ad

  Copy full SHA for 5d6a0ad

• Added javadocs

  Show description for 279dbbe
  shiv0408

  committed
  279dbbe

  Copy full SHA for 279dbbe

Commits on Mar 13, 2024

• Address PR comment

  Show description for 0542849
  shiv0408

  committed
  0542849

  Copy full SHA for 0542849

• Add Integ test

  Show description for c9fb25b
  shiv0408

  committed
  c9fb25b

  Copy full SHA for c9fb25b

• Added a bwc test

  Show description for fc270d1
  shiv0408

  committed
  fc270d1

  Copy full SHA for fc270d1

Commits on Mar 15, 2024

• Address PR comments

  Show description for adb4cf2
  shiv0408

  committed
  adb4cf2

  Copy full SHA for adb4cf2

• Modify Custom's fromXContent

  Show description for 0b38736
  shiv0408

  committed
  0b38736

  Copy full SHA for 0b38736

Commits on Mar 18, 2024

• Remove ClusterMetadataManifest constructor

  Show description for c86c0f1
  shiv0408

  committed
  c86c0f1

  Copy full SHA for c86c0f1

Commits on Mar 19, 2024

• Added tests

  Show description for 3ed92e5
  shiv0408

  committed
  3ed92e5

  Copy full SHA for 3ed92e5

• remove stale global metadata files

  Show description for e1f517e
  shiv0408

  committed
  e1f517e

  Copy full SHA for e1f517e

• spotless apply

  Show description for cd5c9a5
  shiv0408

  committed
  cd5c9a5

  Copy full SHA for cd5c9a5

• Merge branch 'main' into cluster_state_split

  shiv0408

  committed
  2bd97d7

  Copy full SHA for 2bd97d7

Commits on Apr 3, 2024

• Merge branch 'main' into remote_state_collab

  shiv0408

  committed
  8008ab2

  Copy full SHA for 8008ab2

• Refactored RemoteClusterStateService into smaller files

  Show description for 33cfeb0
  shiv0408

  committed
  33cfeb0

  Copy full SHA for 33cfeb0

Commits on Apr 8, 2024

• Merge branch 'main' into cluster_state_split

  shiv0408

  committed
  8244c6d

  Copy full SHA for 8244c6d

Commits on Apr 9, 2024

• Optimize stale file deletion

  Show description for 5f62745
  shiv0408

  committed
  5f62745

  Copy full SHA for 5f62745

Commits on Apr 19, 2024

• Initial commit for RemoteRoutingTableService setup

  Show description for d5a17ed
  

  himshikha

  authored and

  Himshikha Gupta

  committed
  d5a17ed

  Copy full SHA for d5a17ed

Commits on Apr 22, 2024

• Adds unit test for remote routing setup

  Show description for 9176015
  

  Himshikha Gupta

  committed
  9176015

  Copy full SHA for 9176015

Commits on Apr 23, 2024

• Initial commit for index routing table manifest

  Show description for a17cf06
  

  Bukhtawar

  authored and

  Himshikha Gupta

  committed
  a17cf06

  Copy full SHA for a17cf06

• Changes for IndexRoutingTableHeader

  Show description for 3dd93b4
  

  Bukhtawar

  authored and

  Himshikha Gupta

  committed
  3dd93b4

  Copy full SHA for 3dd93b4

• Revert unintentional changes for IndexRoutingTableHeader

  Show description for 933cdc2
  

  Bukhtawar

  authored and

  Himshikha Gupta

  committed
  933cdc2

  Copy full SHA for 933cdc2

• Changes for IndexRoutingTableInputStream

  Show description for f35f0dc
  

  Bukhtawar

  authored and

  Himshikha Gupta

  committed
  f35f0dc

  Copy full SHA for f35f0dc

Commits on Apr 29, 2024

• Merge branch 'main' into cluster_state_split

  Show description for 425cf20
  shiv0408

  committed
  425cf20

  Copy full SHA for 425cf20

• fix build after merge

  Show description for 8efc1ed
  shiv0408

  committed
  8efc1ed

  Copy full SHA for 8efc1ed

Commits on Apr 30, 2024

• Merge branch 'main' into cluster_state_split

  shiv0408

  committed
  6c637c0

  Copy full SHA for 6c637c0

• Optimize remote state stale file deletion

  Show description for 55ed245
  shiv0408

  committed
  55ed245

  Copy full SHA for 55ed245

• Added UT

  Show description for bef9d2e
  shiv0408

  committed
  bef9d2e

  Copy full SHA for bef9d2e

• Refactored into a clean up manager file

  Show description for 6bed64b
  shiv0408

  committed
  6bed64b

  Copy full SHA for 6bed64b

• Add UT

  Show description for bb4b71f
  shiv0408

  committed
  bb4b71f

  Copy full SHA for bb4b71f

• Modify the Integ test

  Show description for 1febb68
  shiv0408

  committed
  1febb68

  Copy full SHA for 1febb68

• Address PR comments

  Show description for 494aacc
  shiv0408

  committed
  494aacc

  Copy full SHA for 494aacc

Commits on May 2, 2024

• Address further PR comment

  Show description for 928b650
  shiv0408

  committed
  928b650

  Copy full SHA for 928b650

• apply spotless

  Show description for 2ebfc6d
  shiv0408

  committed
  2ebfc6d

  Copy full SHA for 2ebfc6d

• Address PR comment

  Show description for 8f5d7d7
  shiv0408

  committed
  8f5d7d7

  Copy full SHA for 8f5d7d7

• Add changelog

  Show description for 4e3c9e9
  shiv0408

  committed
  4e3c9e9

  Copy full SHA for 4e3c9e9

• apply spotless

  Show description for 5ea5dbb
  shiv0408

  committed
  5ea5dbb

  Copy full SHA for 5ea5dbb

Commits on May 3, 2024

• removed unnecessary method

  Show description for fb0b6aa
  shiv0408

  committed
  fb0b6aa

  Copy full SHA for fb0b6aa

Commits on May 6, 2024

• Input stream fixes

  

  Himshikha Gupta

  committed
  cae6632

  Copy full SHA for cae6632

• Checksum calculation at end of file

  

  Himshikha Gupta

  committed
  40fec5f

  Copy full SHA for 40fec5f

Commits on May 7, 2024

• Add toXContent and fromXContent for DiscoveryNode and DiscoveryNodes

  Show description for d5fc147
  shiv0408

  committed
  d5fc147

  Copy full SHA for d5fc147

• fixing checksum

  

  Himshikha Gupta

  committed
  b9887b4

  Copy full SHA for b9887b4

• test fix

  Show description for 15d06bc
  shiv0408

  committed
  15d06bc

  Copy full SHA for 15d06bc

Commits on May 9, 2024

• Add to and from XContent to ClusterBlock and ClusterBlocks

  Show description for 87625d3
  shiv0408

  committed
  87625d3

  Copy full SHA for 87625d3

Commits on May 10, 2024

• Address further PR comments

  Show description for 032ced2
  shiv0408

  committed
  032ced2

  Copy full SHA for 032ced2

• Merge branch 'main' into cluster_state_split

  shiv0408

  committed
  dd76f39

  Copy full SHA for dd76f39

Commits on May 13, 2024

• Fix test failures

  Show description for e81db21
  shiv0408

  committed
  e81db21

  Copy full SHA for e81db21

• Made minor requested changes

  Show description for 04140ff
  shiv0408

  committed
  04140ff

  Copy full SHA for 04140ff

• Merge branch 'main' into async_delete

  Show description for 1a6940f
  shiv0408

  committed
  1a6940f

  Copy full SHA for 1a6940f

• Merge branch 'main' into cluster_state_split

  shiv0408

  committed
  b20537a

  Copy full SHA for b20537a

Commits on May 14, 2024

• Add read flow for IndexRoutingTable

  Show description for c20c6ac
  

  Arpit-Bandejiya

  authored and

  Himshikha Gupta

  committed
  c20c6ac

  Copy full SHA for c20c6ac

• Moving routing table version from IndexRouting stream to manifest

  Show description for 381630b
  

  Himshikha Gupta

  committed
  381630b

  Copy full SHA for 381630b

• Merge remote-tracking branch 'shiv0408/OpenSearch/cluster_state_split' into async-write

  

  Himshikha Gupta

  committed
  eb1b7ec

  Copy full SHA for eb1b7ec

• Draft changes for Diff manifest upload

  Show description for a10b062
  shiv0408

  committed
  a10b062

  Copy full SHA for a10b062

• Download cluster state from remote

  soosinha

  committed
  acdb97e

  Copy full SHA for acdb97e

• Fix default time interval

  Show description for 2025f70
  shiv0408

  committed
  2025f70

  Copy full SHA for 2025f70

Commits on May 15, 2024

• Merge remote-tracking branch 'shiv0408/OpenSearch/remote_state_collab' into async-write

  

  Himshikha Gupta

  committed
  289feb4

  Copy full SHA for 289feb4

• Merging from remote cluster state

  

  Himshikha Gupta

  committed
  b09bce7

  Copy full SHA for b09bce7

• Download cluster state from remote

  

  soosinha

  authored and

  Himshikha Gupta

  committed
  df8ddad

  Copy full SHA for df8ddad

• Added parser for ClusterDiffManifest

  Show description for df3c088
  shiv0408

  committed
  df3c088

  Copy full SHA for df3c088

• Merge branch 'main' into async_delete

  Show description for 3ff82b3
  shiv0408

  committed
  3ff82b3

  Copy full SHA for 3ff82b3

• Merge remote-tracking branch 'shiv/async-write' into remote_state_collab

  Arpit-Bandejiya

  committed
  96cf135

  Copy full SHA for 96cf135

• Fixig build failures

  

  Himshikha Gupta

  committed
  774f0c8

  Copy full SHA for 774f0c8

• Merge pull request #9 from himshikha/async-write

  Show description for bf54d72
  himshikha

  authored
  bf54d72

  Copy full SHA for bf54d72

• spotless changes for ClusterBlocks

  Show description for 5febe86
  shiv0408

  committed
  5febe86

  Copy full SHA for 5febe86

Commits on May 16, 2024

• Merge branch 'main' into async_delete

  shiv0408

  committed
  12632fc

  Copy full SHA for 12632fc

• remove duplicate diff object from manifest

  Show description for bd45e85
  shiv0408

  committed
  bd45e85

  Copy full SHA for bd45e85

• Add remote-routing table diff

  Show description for 54f1f3d
  Arpit-Bandejiya

  committed
  54f1f3d

  Copy full SHA for 54f1f3d

• Merge branch 'main' into async_delete

  shiv0408

  committed
  3825951

  Copy full SHA for 3825951

• Add transport call to publish remote state

  soosinha

  committed
  f222634

  Copy full SHA for f222634

• Fix EmptyMap warning in StreamReader

  Show description for 560ecb8
  Arpit-Bandejiya

  committed
  560ecb8

  Copy full SHA for 560ecb8

• Add basic read-remote-routing flow

  Show description for 4c33720
  Arpit-Bandejiya

  committed
  4c33720

  Copy full SHA for 4c33720

Commits on May 17, 2024

• Move diff manifest to new class and fix fromXContent

  Show description for 3186372
  shiv0408

  committed
  3186372

  Copy full SHA for 3186372

• Fix cluster state publication

  soosinha

  committed
  8912b0c

  Copy full SHA for 8912b0c

• Clean up global metadata attribute objects from remote

  Show description for b604b09
  shiv0408

  committed
  b604b09

  Copy full SHA for b604b09

• Refactor remote state

  soosinha

  committed
  dc204c1

  Copy full SHA for dc204c1

Commits on May 18, 2024

• Add Code for reading from diff file and constructing the Routing Table

  Show description for 4056877
  Arpit-Bandejiya

  committed
  4056877

  Copy full SHA for 4056877

• Add Remote Routing read in remote cluster state publication

  Show description for 70fae29
  Arpit-Bandejiya

  committed
  70fae29

  Copy full SHA for 70fae29

Commits on May 19, 2024

• Add clusterUUID when downloading state

  soosinha

  committed
  093d926

  Copy full SHA for 093d926

Commits on May 20, 2024

• Add fixes for remote routing table

  Show description for 034f54b
  Arpit-Bandejiya

  committed
  034f54b

  Copy full SHA for 034f54b

• Merge branch 'async_delete' into remote_state_collab

  Show description for 8201f2a
  shiv0408

  committed
  8201f2a

  Copy full SHA for 8201f2a

• Fix build failure after merge

  Show description for ce63a5a
  shiv0408

  committed
  ce63a5a

  Copy full SHA for ce63a5a

Commits on May 21, 2024

• Added support for reading cluster state files in parallel

  Show description for eeff23f
  shiv0408

  committed
  eeff23f

  Copy full SHA for eeff23f

• Read diff files asynchronously

  Show description for 7242dad
  shiv0408

  committed
  7242dad

  Copy full SHA for 7242dad

• add missed semicolon

  Show description for 44e1d25
  shiv0408

  committed
  44e1d25

  Copy full SHA for 44e1d25

• Fix tests after merge

  Show description for 9982d79
  shiv0408

  committed
  9982d79

  Copy full SHA for 9982d79

• Enable Routing Table parallel read

  Show description for 6d33900
  Arpit-Bandejiya

  committed
  6d33900

  Copy full SHA for 6d33900

Commits on May 22, 2024

• Create new remote state interfaces and add implementations

  Show description for 99687b0
  soosinha

  committed
  99687b0

  Copy full SHA for 99687b0

• Remove RemoteObject Store interface

  Show description for 3e1f31f
  soosinha

  committed
  3e1f31f

  Copy full SHA for 3e1f31f

Commits on May 23, 2024

• Reverting dev testing configuration

  soosinha

  committed
  043a170

  Copy full SHA for 043a170

• Refactor discoverynodes and clusterblocks

  soosinha

  committed
  3e3dfc8

  Copy full SHA for 3e3dfc8

• Async write flow for routing table

  

  Himshikha Gupta

  committed
  89a10ac

  Copy full SHA for 89a10ac

• Merge pull request #11 from himshikha/async-write

  Show description for 6ae16f5
  himshikha

  authored
  6ae16f5

  Copy full SHA for 6ae16f5

• Add metadata version and change codec version to V3

  Show description for 8f0902c
  shiv0408

  committed
  8f0902c

  Copy full SHA for 8f0902c

• Minor Refactoring for RemoteReadResults

  Show description for 7d630e2
  Arpit-Bandejiya

  committed
  7d630e2

  Copy full SHA for 7d630e2

• Set metadata version in cluster state

  soosinha

  committed
  c1cd20c

  Copy full SHA for c1cd20c

• remove excessive logging

  Show description for bb8c208
  Arpit-Bandejiya

  committed
  bb8c208

  Copy full SHA for bb8c208

• Merge remote-tracking branch 'refs/remotes/shiv/remote_state_collab' into remote_state_collab

  Arpit-Bandejiya

  committed
  5dc642e

  Copy full SHA for 5dc642e

• Fix build issue

  Show description for a9ddead
  Arpit-Bandejiya

  committed
  a9ddead

  Copy full SHA for a9ddead

• Fixing build

  soosinha

  committed
  f7fef97

  Copy full SHA for f7fef97

• Add custom metadata diff

  Show description for ae2ed6b
  shiv0408

  committed
  ae2ed6b

  Copy full SHA for ae2ed6b

• Add RemoteIndexMetadata unit tests

  Show description for a96304d
  soosinha

  committed
  a96304d

  Copy full SHA for a96304d

Commits on May 24, 2024

• Enable remote routing by default with remote state

  Show description for 6d1990c
  Arpit-Bandejiya

  committed
  6d1990c

  Copy full SHA for 6d1990c

• Correct the diff creation logic

  Show description for e022d78
  soosinha

  committed
  e022d78

  Copy full SHA for e022d78

Commits on May 25, 2024

• Delete stale index routing files

  

  Shailendra Singh

  committed
  ca2640b

  Copy full SHA for ca2640b

• Fix stale file blob path for indices routing

  

  Shailendra Singh

  committed
  9aa707e

  Copy full SHA for 9aa707e

• Use isRemoteStateNode condition

  Show description for 3aaa6a8
  soosinha

  committed
  3aaa6a8

  Copy full SHA for 3aaa6a8

• Add metadata version

  soosinha

  committed
  5d70c59

  Copy full SHA for 5d70c59

• Correct blob path for download

  soosinha

  committed
  6a4c721

  Copy full SHA for 6a4c721

• Custom toXContent implementations

  Show description for fda3031
  shiv0408

  committed
  fda3031

  Copy full SHA for fda3031

• Add indexMetadataListener count to latch

  Show description for 1ea5b01
  shiv0408

  committed
  1ea5b01

  Copy full SHA for 1ea5b01

Commits on May 26, 2024

• Change RemoteObject interface signature

  Show description for 9d687fa
  soosinha

  committed
  9d687fa

  Copy full SHA for 9d687fa

• refactor manifest manager

  Show description for 677b632
  soosinha

  committed
  677b632

  Copy full SHA for 677b632

• Refactor cleanup manager

  Show description for e12131d
  soosinha

  committed
  e12131d

  Copy full SHA for e12131d

Commits on May 27, 2024

• Prevent loading ephemeral objects in restore flow

  Show description for f4f773f
  soosinha

  committed
  f4f773f

  Copy full SHA for f4f773f

• Add transient settings to manifest

  Show description for 14a279d
  shiv0408

  committed
  14a279d

  Copy full SHA for 14a279d

Commits on May 28, 2024

• Create RemoteObjectStore class and corresponding implementations

  soosinha

  committed
  523889c

  Copy full SHA for 523889c

Commits on May 29, 2024

• Add remote state implementation unit tests

  Show description for 9b1cab6
  soosinha

  committed
  9b1cab6

  Copy full SHA for 9b1cab6

• Update interface to use entity U

  Show description for 9aeea21
  soosinha

  committed
  9aeea21

  Copy full SHA for 9aeea21

Commits on May 30, 2024

• Remove duplicate tests

  Show description for 1952bac
  shiv0408

  committed
  1952bac

  Copy full SHA for 1952bac

• Add parsing for DiffableStringMap

  Show description for fc9160c
  shiv0408

  committed
  fc9160c

  Copy full SHA for fc9160c

• Add status to cluster block serialization

  Show description for 35970d9
  shiv0408

  committed
  35970d9

  Copy full SHA for 35970d9

• fix cluster block tests

  Show description for e541318
  shiv0408

  committed
  e541318

  Copy full SHA for e541318

• Minor refactoring for read flow

  Show description for e8fec03
  Arpit-Bandejiya

  committed
  e8fec03

  Copy full SHA for e8fec03

Commits on May 31, 2024

• Upload consistent settings and customs object to remote

  Show description for 00f0249
  shiv0408

  committed
  00f0249

  Copy full SHA for 00f0249

Commits on Jun 3, 2024

• Address remote state interface comments

  soosinha

  committed
  bb335f6

  Copy full SHA for bb335f6

• Implement diffUtils in RemoteRoutingTable

  Show description for 6776c03
  Arpit-Bandejiya

  committed
  6776c03

  Copy full SHA for 6776c03

• refactoring in remote routing table service

  Show description for 65a04a2
  Arpit-Bandejiya

  committed
  65a04a2

  Copy full SHA for 65a04a2

Commits on Jun 4, 2024

• Fix snapshot de/ser

  Show description for 2afe186
  shiv0408

  committed
  2afe186

  Copy full SHA for 2afe186

• Fix snapshot de/ser

  Show description for 633b183
  shiv0408

  committed
  633b183

  Copy full SHA for 633b183

• Add UTs for RemoteRoutingTableService

  Show description for 1c0bbf9
  Arpit-Bandejiya

  committed
  1c0bbf9

  Copy full SHA for 1c0bbf9

• Merge remote-tracking branch 'refs/remotes/shiv/remote_state_collab' into remote_state_collab

  Arpit-Bandejiya

  committed
  f15da96

  Copy full SHA for f15da96

Commits on Jun 5, 2024

• Make arraylist synchronized

  Show description for 0333312
  soosinha

  committed
  0333312

  Copy full SHA for 0333312

• Update remote publication to skip download on master node

  soosinha

  committed
  148eca5

  Copy full SHA for 148eca5

• Add unit tests for uploadBlobAsync

  soosinha

  committed
  23f49e5

  Copy full SHA for 23f49e5

• Address interface comments

  soosinha

  committed
  7f2391c

  Copy full SHA for 7f2391c

• read cluster state custom from diff

  Show description for caf61a6
  shiv0408

  committed
  caf61a6

  Copy full SHA for caf61a6

• Remove get/set method from interface

  Show description for e1eddbe
  soosinha

  committed
  e1eddbe

  Copy full SHA for e1eddbe

Commits on Jun 6, 2024

• Merge branch 'main' into remote_state_collab

  soosinha

  committed
  944f8b2

  Copy full SHA for 944f8b2

• disable checksum for debugging

  soosinha

  committed
  8b71d2c

  Copy full SHA for 8b71d2c

• Use byte buffer

  soosinha

  committed
  87ade4c

  Copy full SHA for 87ade4c

Commits on Jun 7, 2024

• Use InputStreams rather than XContent for serialization

  Show description for 5c3e28b
  shiv0408

  committed
  5c3e28b

  Copy full SHA for 5c3e28b

• Add Remote Entity implementation unit tests

  soosinha

  committed
  3147aa3

  Copy full SHA for 3147aa3

• Add unit tests for Remote Entity implementation

  Show description for 0742523
  soosinha

  committed
  0742523

  Copy full SHA for 0742523

• Add publication flag and remote routing table check

  Show description for 26a12cf
  

  Himshikha Gupta

  authored and

  soosinha

  committed
  26a12cf

  Copy full SHA for 26a12cf

• Add manifest file name

  Show description for 5b83599
  soosinha

  committed
  5b83599

  Copy full SHA for 5b83599

• Renaming test files

  Show description for e47d285
  soosinha

  committed
  e47d285

  Copy full SHA for e47d285

• Send manifest file name

  Show description for 79eb089
  soosinha

  committed
  79eb089

  Copy full SHA for 79eb089

• Send manifest file name in remote publish request

  Show description for fa24988
  soosinha

  committed
  fa24988

  Copy full SHA for fa24988

Commits on Jun 8, 2024

• Create serde utility for Writable classes

  Show description for 5c6f418
  soosinha

  committed
  5c6f418

  Copy full SHA for 5c6f418

• ChecksumWritableBlobStoreFormat Unit test

  Show description for a813834
  soosinha

  committed
  a813834

  Copy full SHA for a813834

• Fixing build

  Show description for e8c8413
  soosinha

  committed
  e8c8413

  Copy full SHA for e8c8413

Commits on Jun 10, 2024

• Handle read ClusterState for V1 manifest

  Show description for 4def30f
  shiv0408

  committed
  4def30f

  Copy full SHA for 4def30f