Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clickhouse peer: specify S3 fields #1205

Merged
merged 4 commits into from
Feb 5, 2024
Merged

Conversation

Amogh-Bharadwaj
Copy link
Contributor

@Amogh-Bharadwaj Amogh-Bharadwaj commented Feb 5, 2024

This PR refactors the Clickhouse peer to now have the S3 fields - object path, access key id, secret key and region:

message ClickhouseConfig{
  string host = 1;
  uint32 port = 2;
  string user = 3;
  string password = 4;
  string database = 5;
  string s3_path = 6; // path to S3 bucket which will store avro files
  string access_key_id = 7;
  string secret_access_key = 8;
  string region = 9;
}

The UI for Clickhouse peer creation has been updated accordingly:
Screenshot 2024-02-05 at 5 36 55 PM

Validation for the S3 stage is also added as part of validate peer for clickhouse.

Nexus code updated accordingly.

This PR has been functionally tested by validating and creating a clickhouse peer followed by performing a basic CDC PG -> CH mirror

@serprex serprex requested a review from pankaj-peerdb February 5, 2024 14:12
@iskakaushik iskakaushik merged commit a2c1f69 into main Feb 5, 2024
9 checks passed
@iskakaushik iskakaushik deleted the clickhouse-peer-refactor branch February 5, 2024 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants