Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interopability With GCS for S3 QRep #502

Merged
merged 2 commits into from
Oct 11, 2023
Merged

Interopability With GCS for S3 QRep #502

merged 2 commits into from
Oct 11, 2023

Conversation

Amogh-Bharadwaj
Copy link
Contributor

@Amogh-Bharadwaj Amogh-Bharadwaj commented Oct 11, 2023

Support for using GCS buckets in S3 QRep

The S3 Peer now looks like:

CREATE PEER s3_gcs_interop_peer FROM S3 WITH
(
    url = 's3://<gcs_bucket_name>/<prefix>',
    access_key_id = '<access_key_id>',
    secret_access_key = '<secret>',
    region = 'auto' -- preferable for GCS,
    endpoint = 'https://storage.googleapis.com' -- must for GCS
);

Note that everything apart from url here is an optional field. If you leave any/all of the other fields out, they will be picked up from the docker-compose.yml file.

The conditions mentioned here still hold - ensure the GCS bucket you specify above exists already.

You can also set the AWS environment variables in docker-compose.yml

Fixes #501

@Amogh-Bharadwaj Amogh-Bharadwaj merged commit 1716500 into main Oct 11, 2023
@iskakaushik iskakaushik modified the milestone: oct-2 to oct-13 Oct 11, 2023
@serprex serprex deleted the s3-gcs-interop branch July 19, 2024 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow GCS as a QRep Destination via S3 compatible API
2 participants