Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [OssResourceManager] The root path of the OSS configuration must not start with "/", which conflicts with the configuration of the History Server. #4088

Open
2 of 3 tasks
jiangwwwei opened this issue Dec 24, 2024 · 1 comment
Labels
Bug Something isn't working Invalid Invalid

Comments

@jiangwwwei
Copy link
Contributor

jiangwwwei commented Dec 24, 2024

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

image

Dinky's OssResourceManager relies on the aws-sdk implementation, and when constructing requests using its API, the file path is the Object Name, which should not start with a "/" (the aws SDK will automatically add a "/" at the beginning of the object name when building the HTTP request). If the root path in Dinky's Oss configuration starts with "/", it will cause an error indicating an invalid object name.
image

However, the HistoryServer's request to oss is implemented by flink-s3-fs-presto, and it requests the direct path. If a "/" is not added at the beginning, it will report an error stating that it is not an absolute path.

What you expected to happen

Both the resource manager and the history server can access OSS normally.

How to reproduce

  1. Use OSS Resource
  2. In the root path configuration, add and do not add "/"
  3. Observe the file upload/download status of the Resource and the operation of the History Server.

Anything else

It is worth mentioning: In OssResourceManager, the "/" before the path in getFullDirectoryStructure() is specifically handled and removed, but other methods do not have this treatment.
image

Version

1.2.0

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@jiangwwwei jiangwwwei added Bug Something isn't working Waiting for reply Waiting for reply labels Dec 24, 2024
@aiwenmo aiwenmo removed the Waiting for reply Waiting for reply label Jan 21, 2025
@github-actions github-actions bot added the Invalid Invalid label Mar 1, 2025
Copy link

github-actions bot commented Mar 1, 2025

Hello @, this issue has not been active for more than 30 days. This issue will be closed in 7 days if there is no response. If you have any questions, you can comment and reply.

你好 @, 这个 issue 30 天内没有活跃,7 天后将关闭,如需回复,可以评论回复。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working Invalid Invalid
Projects
None yet
Development

No branches or pull requests

2 participants