Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Other]: Upload Failure Due to RPC Task Distribution Across Multiple Minio Servers #2480

Closed
zhaolibo1989 opened this issue Aug 5, 2024 · 4 comments
Labels
other Indicates an issue or PR lacks a `triage/foo` label and requires one.

Comments

@zhaolibo1989
Copy link
Contributor

What would you like to share?

老版本(v3.4.0),我测试发现当启用两个 minio 时(对应的 rpc 服务也有两个),发图片消息可能出现 SDK 上传文件过程(如 )使用了不同的 rpc 服务,最终导致消息发送失败。例如 InitiateMultipartUpload 和 CompleteMultipartUpload 方法的执行过程被分配到了不同的 RPC 服务,导致 CompleteMultipartUpload 过程出现“The specified key does not exist.”的错误。

1

2

我注意到,在新版本中增加了 setURLPrefix 的代码。但不太确定这些代码的具体作用:

func (o *ThirdApi) CompleteMultipartUpload(c *gin.Context) {
	opt := setURLPrefixOption(third.ThirdClient.CompleteMultipartUpload, func(req *third.CompleteMultipartUploadReq) error {
		return setURLPrefix(c, &req.UrlPrefix)
	})
	a2r.Call(third.ThirdClient.CompleteMultipartUpload, o.Client, c, opt)
}

最后,我怎么才能在 v3.4.0 版本上解决这个问题?

Additional information

No response

@zhaolibo1989 zhaolibo1989 added the other Indicates an issue or PR lacks a `triage/foo` label and requires one. label Aug 5, 2024
@zhaolibo1989
Copy link
Contributor Author

这种问题在最新版本中解决了吗?

@kubbot
Copy link
Contributor

kubbot commented Aug 5, 2024

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Is this issue resolved in the latest version?

@zhaolibo1989
Copy link
Contributor Author

zhaolibo1989 commented Aug 5, 2024

谢谢 @OpenIM-sk

如果是为给 minio 扩容,可以直接做minio集群。https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html

openim只对接minio集群 。

如果集群不能解决需求,可以 自己上传文件,自己管理, 灵活度最高。 生成url后,再createFileMessageByURL。

@zhaolibo1989
Copy link
Contributor Author

I have an idea to slightly modify it so that the API can directly access the S3 service without going through the RPC service as an intermediary. This way, when different APIs are launched, they can be directly "bound" to the S3 service, and as long as the APP accesses the same API service, it can use the S3 service normally.
However, this requires starting an API for each S3 service. In our scenario, we have almost completely launched a second instance of OpenIM-Server, except for the following components: zookeeper, mongo, redis, and mysql.

Code look like this:

ThirdApi-struct
InitiateMultipartUpload
CompleteMultipartUpload

That's all, enjoy this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
other Indicates an issue or PR lacks a `triage/foo` label and requires one.
Projects
None yet
Development

No branches or pull requests

3 participants