Skip to content

Commit

Permalink
Add prefix to GCS and S3 uri separators
Browse files Browse the repository at this point in the history
Storage adapters for both GCS and S3 use a separator for their URIs.
The symbols in each adapter use a prefix to avoid collisions.
However, this is not the case for the symbol kSep.
This change fixes such potential collision.

Fixes #8372
  • Loading branch information
tigrux committed Jan 23, 2024
1 parent 3949a3b commit 6e34e5d
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 6 deletions.
6 changes: 3 additions & 3 deletions velox/connectors/hive/storage_adapters/gcs/GCSUtil.h
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
namespace facebook::velox {

namespace {
constexpr const char* kSep{"/"};
constexpr std::string_view kGCSSep{"/"};
constexpr std::string_view kGCSScheme{"gs://"};

} // namespace
Expand All @@ -36,7 +36,7 @@ inline void setBucketAndKeyFromGCSPath(
const std::string& path,
std::string& bucket,
std::string& key) {
auto firstSep = path.find_first_of(kSep);
auto firstSep = path.find_first_of(kGCSSep);
bucket = path.substr(0, firstSep);
key = path.substr(firstSep + 1);
}
Expand All @@ -45,7 +45,7 @@ inline std::string gcsURI(const std::string& bucket) {
}

inline std::string gcsURI(const std::string& bucket, const std::string& key) {
return gcsURI(bucket) + kSep + key;
return gcsURI(bucket) + std::string(kGCSSep) + key;
}

inline std::string gcsPath(const std::string_view& path) {
Expand Down
6 changes: 3 additions & 3 deletions velox/connectors/hive/storage_adapters/s3fs/S3Util.h
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
namespace facebook::velox {

namespace {
constexpr std::string_view kSep{"/"};
constexpr std::string_view kS3Sep{"/"};
// AWS S3 EMRFS, Hadoop block storage filesystem on-top of Amazon S3 buckets.
constexpr std::string_view kS3Scheme{"s3://"};
// This should not be mixed with s3 nor the s3a.
Expand Down Expand Up @@ -81,7 +81,7 @@ inline void getBucketAndKeyFromS3Path(
const std::string& path,
std::string& bucket,
std::string& key) {
auto firstSep = path.find_first_of(kSep);
auto firstSep = path.find_first_of(kS3Sep);
bucket = path.substr(0, firstSep);
key = path.substr(firstSep + 1);
}
Expand All @@ -98,7 +98,7 @@ inline std::string s3URI(const std::string& bucket) {
}

inline std::string s3URI(const std::string& bucket, const std::string& key) {
return s3URI(bucket) + "/" + key;
return s3URI(bucket) + std::string(kS3Sep) + key;
}

inline std::string s3Path(const std::string_view& path) {
Expand Down

0 comments on commit 6e34e5d

Please sign in to comment.