From 4a383075815435b921217dcfc4a02eb3eaf296e4 Mon Sep 17 00:00:00 2001 From: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Date: Wed, 24 Apr 2024 12:11:36 +0800 Subject: [PATCH 1/6] 3.6.1 code conventions (#2203) * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * refactor: webhooks update. * refactor: kafka update. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * refactor: kafka update. * refactor: kafka update. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * Windows can compile and run. * Windows can compile and run. * refactor: kafka update. * feat: msg cache split * refactor: webhooks update * refactor: webhooks update * refactor: friends update * refactor: group update * refactor: third update * refactor: api update * refactor: crontab update * refactor: msggateway update * mage * mage * refactor: all module update. * check * refactor: all module update. * load config * load config * load config * load config * refactor: all module update. * refactor: all module update. * refactor: all module update. * refactor: all module update. * refactor: all module update. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * refactor: all module update. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * update tools * update tools * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * update protocol * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: api remove token auth by redis directly. * Code Refactoring * refactor: websocket auth change to call rpc of auth. * refactor: kick online user and remove token change to call auth rpc. * refactor: kick online user and remove token change to call auth rpc. * refactor: remove msggateway redis. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor webhook * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor webhook * refactor: cmd update. * refactor: cmd update. * fix: runtime: goroutine stack exceeds * refactor: cmd update. * refactor notification * refactor notification * refactor * refactor: cmd update. * refactor: cmd update. * refactor * refactor * refactor * protojson * protojson * protojson * go mod * wrapperspb * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: context update. * refactor: websocket update info. * refactor: websocket update info. * refactor: websocket update info. * refactor: websocket update info. * refactor: api name change. * refactor: debug info. * refactor: debug info. * refactor: debug info. * fix: update file * refactor * refactor * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * fix: callback update. * fix: callback update. * refactor * fix: update message. * fix: msg cache timeout. * refactor * refactor * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: push update. * refactor * refactor * fix: push update. * fix: websocket handle error remove when upgrade error. * fix: priority url * fix: minio config * refactor: add zk logger. * refactor * fix: minio config * refactor * remove \r * remove \r * remove \r * remove \r * remove \r * remove \r * remove \r * remove \r * remove \r * remove \r * fix bug: get localIP * refactor * refactor * refactor * refactor: remove zk logger. * refactor: update tools version. * refactor * refactor: update server version to 3.7.0. * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor: zk log debug. * refactor: zk log debug. * refactor: zk log debug. * refactor: zk log debug. * refactor: zk log debug. * refactor * refactor * refactor * refactor: log level change. * refactor: 3.7.0 code conventions. --------- Co-authored-by: skiffer-git <44203734@qq.com> Co-authored-by: withchao <993506633@qq.com> Co-authored-by: root --- .dockerignore | 5 +- .gitattributes | 1 + Dockerfile | 59 +++++++++++------ config/README.md | 63 +++++++++++++++++++ config/minio.yml | 8 +-- go.mod | 9 +-- go.sum | 8 +-- internal/api/init.go | 4 -- internal/msgtransfer/init.go | 5 -- .../msgtransfer/online_history_msg_handler.go | 2 +- internal/push/push_handler.go | 6 +- internal/rpc/third/third.go | 16 ----- pkg/common/config/config.go | 23 ++++--- pkg/common/config/load_config_test.go | 4 +- pkg/common/config/version | 2 +- .../discoveryregister/discoveryregister.go | 1 + pkg/localcache/cache.go | 4 +- pkg/localcache/go.mod | 5 -- pkg/localcache/go.sum | 1 - pkg/localcache/option.go | 2 +- pkg/rpccache/conversation.go | 2 +- pkg/rpccache/friend.go | 2 +- pkg/rpccache/group.go | 2 +- pkg/rpccache/user.go | 2 +- scripts/docker-start-all.sh | 9 +-- 25 files changed, 148 insertions(+), 97 deletions(-) create mode 100644 .gitattributes create mode 100644 config/README.md delete mode 100644 pkg/localcache/go.mod delete mode 100644 pkg/localcache/go.sum diff --git a/.dockerignore b/.dockerignore index 263798e07e..cf6c3cd596 100644 --- a/.dockerignore +++ b/.dockerignore @@ -5,9 +5,8 @@ .git # Ignore build artifacts -_output/ logs/ - +_output/ # Ignore non-essential documentation README.md README-zh_CN.md @@ -18,8 +17,6 @@ CHANGELOG/ # Ignore testing and linting configuration .golangci.yml -# Ignore deployment-related files -docker-compose.yaml # Ignore assets assets/ diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000000..dfdb8b771c --- /dev/null +++ b/.gitattributes @@ -0,0 +1 @@ +*.sh text eol=lf diff --git a/Dockerfile b/Dockerfile index 9b10212165..746dddf65a 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,30 +1,49 @@ -# Build Stage -FROM golang:1.20 AS builder +# Use Go 1.21 Alpine as the base image for building the application +FROM golang:1.21-alpine as builder -# Set go mod installation source and proxy -ARG GO111MODULE=on +# Define the base directory for the application as an environment variable +ENV SERVER_DIR=/openim-server -ENV GO111MODULE=$GO111MODULE -ENV GOPROXY=$GOPROXY +# Set the working directory inside the container based on the environment variable +WORKDIR $SERVER_DIR -# Set up the working directory -WORKDIR /openim/openim-server +# Set the Go proxy to improve dependency resolution speed +ENV GOPROXY=https://goproxy.io,direct +# Copy all files from the current directory into the container +COPY . . -# Copy all files to the container -ADD . . +RUN go mod download -RUN make clean -RUN make build +# Install Mage to use for building the application +RUN go install github.com/magefile/mage@v1.15.0 -FROM ghcr.io/openim-sigs/openim-ubuntu-image:latest +# Optionally build your application if needed +RUN mage build -WORKDIR ${SERVER_WORKDIR} +# Using Alpine Linux with Go environment for the final image +FROM golang:1.21-alpine -# Copy scripts and binary files to the production image -COPY --from=builder ${OPENIM_SERVER_BINDIR} /openim/openim-server/_output/bin -COPY --from=builder ${OPENIM_SERVER_CMDDIR} /openim/openim-server/scripts -COPY --from=builder ${SERVER_WORKDIR}/config /openim/openim-server/config -COPY --from=builder ${SERVER_WORKDIR}/deployments /openim/openim-server/deployments +# Install necessary packages, such as bash +RUN apk add --no-cache bash -CMD ["/openim/openim-server/scripts/docker-start-all.sh"] +# Set the environment and work directory +ENV SERVER_DIR=/openim-server +WORKDIR $SERVER_DIR + + +# Copy the compiled binaries and mage from the builder image to the final image +COPY --from=builder $SERVER_DIR/_output $SERVER_DIR/_output +COPY --from=builder $SERVER_DIR/config $SERVER_DIR/config +COPY --from=builder /go/bin/mage /usr/local/bin/mage +COPY --from=builder $SERVER_DIR/magefile_windows.go $SERVER_DIR/ +COPY --from=builder $SERVER_DIR/magefile_unix.go $SERVER_DIR/ +COPY --from=builder $SERVER_DIR/magefile.go $SERVER_DIR/ +COPY --from=builder $SERVER_DIR/start-config.yml $SERVER_DIR/ +COPY --from=builder $SERVER_DIR/go.mod $SERVER_DIR/ +COPY --from=builder $SERVER_DIR/go.sum $SERVER_DIR/ + +RUN go get github.com/openimsdk/gomake@v0.0.9 + +# Set the command to run when the container starts +ENTRYPOINT ["sh", "-c", "mage start && tail -f /dev/null"] diff --git a/config/README.md b/config/README.md new file mode 100644 index 0000000000..048d7fe362 --- /dev/null +++ b/config/README.md @@ -0,0 +1,63 @@ +--- +title: 'OpenIM Configuration Files and Common Configuration Item Modifications Guide' + +## Configuration Files Explanation + +| Configuration File | Description | +| ------------------------------- | ------------------------------------------------------------ | +| **kafka.yml** | Configurations for Kafka username, password, address, etc. | +| **redis.yml** | Configurations for Redis password, address, etc. | +| **minio.yml** | Configurations for MinIO username, password, address, and external IP/domain; failing to modify external IP or domain may cause image file sending failures | +| **zookeeper.yml** | Configurations for ZooKeeper user, password, address, etc. | +| **mongodb.yml** | Configurations for MongoDB username, password, address, etc. | +| **log.yml** | Configurations for log level and storage directory. | +| **notification.yml** | Configurations for events like adding friends, creating groups, etc. | +| **share.yml** | Common configurations needed by various OpenIM services, such as secret. | +| **webhooks.yml** | Configurations for URLs in Webhook. | +| **local-cache.yml** | Local cache configurations. | +| **openim-rpc-third.yml** | Configurations for listening IP, port, and storage settings for images and videos in openim-rpc-third service. | +| **openim-rpc-user.yml** | Configurations for listening IP and port in openim-rpc-user service. | +| **openim-api.yml** | Configurations for listening IP, port, etc., in openim-api service. | +| **openim-crontask.yml** | Configurations for openim-crontask service. | +| **openim-msggateway.yml** | Configurations for listening IP, port, etc., in openim-msggateway service. | +| **openim-msgtransfer.yml** | Configurations for openim-msgtransfer service. | +| **openim-push.yml** | Configurations for listening IP, port, and offline push settings in openim-push service. | +| **openim-rpc-auth.yml** | Configurations for listening IP, port, and token expiration settings in openim-rpc-auth service. | +| **openim-rpc-conversation.yml** | Configurations for listening IP, port, etc., in openim-rpc-conversation service. | +| **openim-rpc-friend.yml** | Configurations for listening IP, port, etc., in openim-rpc-friend service. | +| **openim-rpc-group.yml** | Configurations for listening IP, port, etc., in openim-rpc-group service. | +| **openim-rpc-msg.yml** | Configurations for listening IP, port, and whether to verify friendship before sending messages in openim-rpc-msg service. | + +## Common Configuration Item Modifications + +| Configuration Item Modification | Configuration File | +| ----------------------------------------------------- | ----------------------- | +| Using MinIO for image and video file object storage | `minio.yml` | +| Adjusting production environment logs | `log.yml` | +| Verifying friendship before sending messages | `openim-rpc-msg.yml` | +| Modifying secret | `share.yml` | +| Using OSS, COS, AWS, Kodo for image and video storage | `openim-rpc-third.yml` | +| Setting multiple login policy | `openim-msggateway.yml` | +| Setting up offline push | `openim-push.yml` | + +## Starting Multiple Instances of an OpenIM Service + +To start multiple instances of an OpenIM service, simply increase the corresponding port numbers and modify the `start-config.yml` file in the project root directory. Restart the service to take effect. For example, the configuration to start 2 instances of `openim-rpc-user` is as follows: + +```yaml +rpc: + registerIP: '' + listenIP: 0.0.0.0 + ports: [ 10110, 10111 ] + +prometheus: + enable: true + ports: [ 20100, 20101 ] +``` + +Modify `start-config.yml`: + +```yaml +serviceBinaries: + openim-rpc-user: 2 +``` diff --git a/config/minio.yml b/config/minio.yml index 0cabb7f36a..0a80bb845c 100644 --- a/config/minio.yml +++ b/config/minio.yml @@ -1,9 +1,7 @@ bucket: "openim" -port: 10005 accessKeyID: "root" secretAccessKey: "openIM123" sessionToken: '' -internalIP: localhost -externalIP: 150.109.93.151 -url: https://image.rentsoft.cn/ -publicRead: false +internalAddress: "minio:9000" +externalAddress: "http://external_ip:10005" +publicRead: false \ No newline at end of file diff --git a/go.mod b/go.mod index 8ada8f2e77..891125a899 100644 --- a/go.mod +++ b/go.mod @@ -13,9 +13,8 @@ require ( github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 github.com/lestrrat-go/file-rotatelogs v2.4.0+incompatible // indirect github.com/mitchellh/mapstructure v1.5.0 - github.com/openimsdk/localcache v0.0.1 github.com/openimsdk/protocol v0.0.64 - github.com/openimsdk/tools v0.0.47-alpha.42 + github.com/openimsdk/tools v0.0.49-alpha.2 github.com/pkg/errors v0.9.1 // indirect github.com/prometheus/client_golang v1.18.0 github.com/stretchr/testify v1.9.0 @@ -32,9 +31,10 @@ require ( github.com/IBM/sarama v1.43.0 github.com/fatih/color v1.14.1 github.com/go-redis/redis v6.15.9+incompatible + github.com/hashicorp/golang-lru/v2 v2.0.7 github.com/kelindar/bitmap v1.5.2 github.com/likexian/gokit v0.25.13 - github.com/openimsdk/gomake v0.0.6 + github.com/openimsdk/gomake v0.0.9 github.com/redis/go-redis/v9 v9.4.0 github.com/robfig/cron/v3 v3.0.1 github.com/shirou/gopsutil v3.21.11+incompatible @@ -83,7 +83,6 @@ require ( github.com/hashicorp/errwrap v1.1.0 // indirect github.com/hashicorp/go-multierror v1.1.1 // indirect github.com/hashicorp/go-uuid v1.0.3 // indirect - github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect github.com/hashicorp/hcl v1.0.0 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/jcmturner/aescts/v2 v2.0.0 // indirect @@ -173,5 +172,3 @@ require ( golang.org/x/crypto v0.21.0 // indirect gopkg.in/ini.v1 v1.67.0 // indirect ) - -replace github.com/openimsdk/localcache => ./pkg/localcache diff --git a/go.sum b/go.sum index 0ef6209fc7..5dc0cc524c 100644 --- a/go.sum +++ b/go.sum @@ -266,12 +266,12 @@ github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1y github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY= github.com/onsi/gomega v1.18.1 h1:M1GfJqGRrBrrGGsbxzV5dqM2U2ApXefZCQpkukxYRLE= github.com/onsi/gomega v1.18.1/go.mod h1:0q+aL8jAiMXy9hbwj2mr5GziHiwhAIQpFmmtT5hitRs= -github.com/openimsdk/gomake v0.0.6 h1:bJmQWDHBj8PQ7oGJ2SL3Gsx0k5CdI/BPfGzlGcV105s= -github.com/openimsdk/gomake v0.0.6/go.mod h1:PndCozNc2IsQIciyn9mvEblYWZwJmAI+06z94EY+csI= +github.com/openimsdk/gomake v0.0.9 h1:ouf25ygN2PMQ68Gfgns/EQRPiLPnp+77SIr68GfE+n4= +github.com/openimsdk/gomake v0.0.9/go.mod h1:PndCozNc2IsQIciyn9mvEblYWZwJmAI+06z94EY+csI= github.com/openimsdk/protocol v0.0.64 h1:OrjSs4CgKN9VLvJvrAsc37O7Ru0E0VllXZQSmG/ab7U= github.com/openimsdk/protocol v0.0.64/go.mod h1:OZQA9FR55lseYoN2Ql1XAHYKHJGu7OMNkUbuekrKCM8= -github.com/openimsdk/tools v0.0.47-alpha.42 h1:wM6t9otTLhXECq8aQcYaZGvBgo/ZAmbNTqVt3g3NHGg= -github.com/openimsdk/tools v0.0.47-alpha.42/go.mod h1:P4oGP1Pd+d4ctbLD5U/XQTgl8yu8Hd3skx640Fr69ko= +github.com/openimsdk/tools v0.0.49-alpha.2 h1:8IfV6o2ySU7C54sh/MG7ctEp1h3lSNe03OCUDWSk5Ws= +github.com/openimsdk/tools v0.0.49-alpha.2/go.mod h1:P4oGP1Pd+d4ctbLD5U/XQTgl8yu8Hd3skx640Fr69ko= github.com/pelletier/go-toml/v2 v2.1.0 h1:FnwAJ4oYMvbT/34k9zzHuZNrhlz48GB3/s6at6/MHO4= github.com/pelletier/go-toml/v2 v2.1.0/go.mod h1:tJU2Z3ZkXwnxa4DPO899bsyIoywizdUvyaeZurnPPDc= github.com/pierrec/lz4/v4 v4.1.21 h1:yOVMLb6qSIDP67pl/5F7RepeKYu/VmTyEXvuMI5d9mQ= diff --git a/internal/api/init.go b/internal/api/init.go index dcce69a12f..6e784da9a9 100644 --- a/internal/api/init.go +++ b/internal/api/init.go @@ -64,10 +64,6 @@ func Start(ctx context.Context, index int, config *Config) error { return errs.WrapMsg(err, "failed to register discovery service") } - if err = client.CreateRpcRootNodes(config.Share.RpcRegisterName.GetServiceNames()); err != nil { - return errs.WrapMsg(err, "failed to create RPC root nodes") - } - var ( netDone = make(chan struct{}, 1) netErr error diff --git a/internal/msgtransfer/init.go b/internal/msgtransfer/init.go index 685e847b86..68d953e902 100644 --- a/internal/msgtransfer/init.go +++ b/internal/msgtransfer/init.go @@ -80,11 +80,6 @@ func Start(ctx context.Context, index int, config *Config) error { if err != nil { return err } - - if err := client.CreateRpcRootNodes(config.Share.RpcRegisterName.GetServiceNames()); err != nil { - return err - } - client.AddOption(mw.GrpcClient(), grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithDefaultServiceConfig(fmt.Sprintf(`{"LoadBalancingPolicy": "%s"}`, "round_robin"))) //todo MsgCacheTimeout diff --git a/internal/msgtransfer/online_history_msg_handler.go b/internal/msgtransfer/online_history_msg_handler.go index 9960aa5189..8691e92ab6 100644 --- a/internal/msgtransfer/online_history_msg_handler.go +++ b/internal/msgtransfer/online_history_msg_handler.go @@ -394,7 +394,7 @@ func (och *OnlineHistoryRedisConsumerHandler) ConsumeClaim( break } } - log.ZDebug(context.Background(), "online new session msg come", "highWaterMarkOffset", + log.ZInfo(context.Background(), "online new session msg come", "highWaterMarkOffset", claim.HighWaterMarkOffset(), "topic", claim.Topic(), "partition", claim.Partition()) var ( diff --git a/internal/push/push_handler.go b/internal/push/push_handler.go index 2246fb3f68..e09b0b2ec3 100644 --- a/internal/push/push_handler.go +++ b/internal/push/push_handler.go @@ -96,7 +96,7 @@ func (c *ConsumerHandler) handleMs2PsChat(ctx context.Context, msg []byte) { var err error switch msgFromMQ.MsgData.SessionType { case constant.ReadGroupChatType: - err = c.Push2SuperGroup(ctx, pbData.MsgData.GroupID, pbData.MsgData) + err = c.Push2Group(ctx, pbData.MsgData.GroupID, pbData.MsgData) default: var pushUserIDList []string isSenderSync := datautil.GetSwitchFromOptions(pbData.MsgData.Options, constant.IsSenderSync) @@ -108,7 +108,7 @@ func (c *ConsumerHandler) handleMs2PsChat(ctx context.Context, msg []byte) { err = c.Push2User(ctx, pushUserIDList, pbData.MsgData) } if err != nil { - log.ZError(ctx, "push failed", err, "msg", pbData.String()) + log.ZWarn(ctx, "push failed", err, "msg", pbData.String()) } } @@ -179,7 +179,7 @@ func (c *ConsumerHandler) shouldPushOffline(_ context.Context, msg *sdkws.MsgDat return true } -func (c *ConsumerHandler) Push2SuperGroup(ctx context.Context, groupID string, msg *sdkws.MsgData) (err error) { +func (c *ConsumerHandler) Push2Group(ctx context.Context, groupID string, msg *sdkws.MsgData) (err error) { log.ZDebug(ctx, "Get super group msg from msg_transfer and push msg", "msg", msg.String(), "groupID", groupID) var pushToUserIDs []string if err = c.webhookBeforeGroupOnlinePush(ctx, &c.config.WebhooksConfig.BeforeGroupOnlinePush, groupID, msg, diff --git a/internal/rpc/third/third.go b/internal/rpc/third/third.go index ad87fa6f5c..9bf8cafa9e 100644 --- a/internal/rpc/third/third.go +++ b/internal/rpc/third/third.go @@ -18,7 +18,6 @@ import ( "context" "fmt" "github.com/openimsdk/open-im-server/v3/pkg/common/config" - "net/url" "time" "github.com/openimsdk/open-im-server/v3/pkg/common/db/cache" @@ -29,7 +28,6 @@ import ( "github.com/openimsdk/tools/db/mongoutil" "github.com/openimsdk/tools/db/redisutil" "github.com/openimsdk/tools/discovery" - "github.com/openimsdk/tools/errs" "github.com/openimsdk/tools/s3" "github.com/openimsdk/tools/s3/cos" "github.com/openimsdk/tools/s3/minio" @@ -38,7 +36,6 @@ import ( ) type thirdServer struct { - apiURL string thirdDatabase controller.ThirdDatabase s3dataBase controller.S3Database userRpcClient rpcclient.UserRpcClient @@ -73,18 +70,6 @@ func Start(ctx context.Context, config *Config, client discovery.SvcDiscoveryReg if err != nil { return err } - apiURL := config.MinioConfig.URL - if apiURL == "" { - return errs.Wrap(fmt.Errorf("api is empty")) - } - if _, err := url.Parse(config.MinioConfig.URL); err != nil { - return err - } - if apiURL[len(apiURL)-1] != '/' { - apiURL += "/" - } - apiURL += "object/" - // Select the oss method according to the profile policy enable := config.RpcConfig.Object.Enable var o s3.Interface @@ -103,7 +88,6 @@ func Start(ctx context.Context, config *Config, client discovery.SvcDiscoveryReg } cache.InitLocalCache(&config.LocalCacheConfig) third.RegisterThirdServer(server, &thirdServer{ - apiURL: apiURL, thirdDatabase: controller.NewThirdDatabase(cache.NewThirdCache(rdb), logdb), userRpcClient: rpcclient.NewUserRpcClient(client, config.Share.RpcRegisterName.User, config.Share.IMAdminUserID), s3dataBase: controller.NewS3Database(rdb, o, s3db), diff --git a/pkg/common/config/config.go b/pkg/common/config/config.go index 19edccbab2..df2639cab3 100644 --- a/pkg/common/config/config.go +++ b/pkg/common/config/config.go @@ -22,6 +22,7 @@ import ( "github.com/openimsdk/tools/s3/cos" "github.com/openimsdk/tools/s3/minio" "github.com/openimsdk/tools/s3/oss" + "net" "time" ) @@ -52,13 +53,11 @@ type Log struct { type Minio struct { Bucket string `mapstructure:"bucket"` - Port int `mapstructure:"port"` AccessKeyID string `mapstructure:"accessKeyID"` SecretAccessKey string `mapstructure:"secretAccessKey"` SessionToken string `mapstructure:"sessionToken"` - InternalIP string `mapstructure:"internalIP"` - ExternalIP string `mapstructure:"externalIP"` - URL string `mapstructure:"url"` + InternalAddress string `mapstructure:"internalAddress"` + ExternalAddress string `mapstructure:"externalAddress"` PublicRead bool `mapstructure:"publicRead"` } @@ -477,16 +476,24 @@ func (k *Kafka) Build() *kafka.Config { } } func (m *Minio) Build() *minio.Config { - return &minio.Config{ + conf := minio.Config{ Bucket: m.Bucket, - Endpoint: fmt.Sprintf("http://%s:%d", m.InternalIP, m.Port), AccessKeyID: m.AccessKeyID, SecretAccessKey: m.SecretAccessKey, SessionToken: m.SessionToken, - SignEndpoint: fmt.Sprintf("http://%s:%d", m.ExternalIP, m.Port), PublicRead: m.PublicRead, } - + if _, _, err := net.SplitHostPort(m.InternalAddress); err == nil { + conf.Endpoint = fmt.Sprintf("http://%s", m.InternalAddress) + } else { + conf.Endpoint = m.InternalAddress + } + if _, _, err := net.SplitHostPort(m.ExternalAddress); err == nil { + conf.SignEndpoint = fmt.Sprintf("http://%s", m.ExternalAddress) + } else { + conf.SignEndpoint = m.ExternalAddress + } + return &conf } func (c *Cos) Build() *cos.Config { return &cos.Config{ diff --git a/pkg/common/config/load_config_test.go b/pkg/common/config/load_config_test.go index 575448393f..256214565b 100644 --- a/pkg/common/config/load_config_test.go +++ b/pkg/common/config/load_config_test.go @@ -31,6 +31,8 @@ func TestLoadOpenIMRpcUserConfig(t *testing.T) { var user User err := LoadConfig("../../../config/openim-rpc-user.yml", "IMENV_OPENIM_RPC_USER", &user) assert.Nil(t, err) + //export IMENV_OPENIM_RPC_USER_RPC_LISTENIP="0.0.0.0" assert.Equal(t, "0.0.0.0", user.RPC.ListenIP) - assert.Equal(t, []int{10110}, user.RPC.Ports) + //export IMENV_OPENIM_RPC_USER_RPC_PORTS="10110,10111,10112" + assert.Equal(t, []int{10110, 10111, 10112}, user.RPC.Ports) } diff --git a/pkg/common/config/version b/pkg/common/config/version index 084e244cea..240bba9069 100644 --- a/pkg/common/config/version +++ b/pkg/common/config/version @@ -1 +1 @@ -3.6.0 \ No newline at end of file +3.7.0 \ No newline at end of file diff --git a/pkg/common/discoveryregister/discoveryregister.go b/pkg/common/discoveryregister/discoveryregister.go index b664ee34eb..38d7382fa6 100644 --- a/pkg/common/discoveryregister/discoveryregister.go +++ b/pkg/common/discoveryregister/discoveryregister.go @@ -33,6 +33,7 @@ const ( func NewDiscoveryRegister(zookeeperConfig *config.ZooKeeper, share *config.Share) (discovery.SvcDiscoveryRegistry, error) { switch share.Env { case zookeeperConst: + return zookeeper.NewZkClient( zookeeperConfig.Address, zookeeperConfig.Schema, diff --git a/pkg/localcache/cache.go b/pkg/localcache/cache.go index 91c490922f..0e040ad389 100644 --- a/pkg/localcache/cache.go +++ b/pkg/localcache/cache.go @@ -19,8 +19,8 @@ import ( "hash/fnv" "unsafe" - "github.com/openimsdk/localcache/link" - "github.com/openimsdk/localcache/lru" + "github.com/openimsdk/open-im-server/v3/pkg/localcache/link" + "github.com/openimsdk/open-im-server/v3/pkg/localcache/lru" ) type Cache[V any] interface { diff --git a/pkg/localcache/go.mod b/pkg/localcache/go.mod deleted file mode 100644 index 5f0793042e..0000000000 --- a/pkg/localcache/go.mod +++ /dev/null @@ -1,5 +0,0 @@ -module github.com/openimsdk/localcache - -go 1.19 - -require github.com/hashicorp/golang-lru/v2 v2.0.7 diff --git a/pkg/localcache/go.sum b/pkg/localcache/go.sum deleted file mode 100644 index b1d8cf8c44..0000000000 --- a/pkg/localcache/go.sum +++ /dev/null @@ -1 +0,0 @@ -github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= diff --git a/pkg/localcache/option.go b/pkg/localcache/option.go index e60d5aa07b..00bb9d0441 100644 --- a/pkg/localcache/option.go +++ b/pkg/localcache/option.go @@ -18,7 +18,7 @@ import ( "context" "time" - "github.com/openimsdk/localcache/lru" + "github.com/openimsdk/open-im-server/v3/pkg/localcache/lru" ) func defaultOption() *option { diff --git a/pkg/rpccache/conversation.go b/pkg/rpccache/conversation.go index 196657b4b3..55897a8dae 100644 --- a/pkg/rpccache/conversation.go +++ b/pkg/rpccache/conversation.go @@ -17,9 +17,9 @@ package rpccache import ( "context" - "github.com/openimsdk/localcache" "github.com/openimsdk/open-im-server/v3/pkg/common/cachekey" "github.com/openimsdk/open-im-server/v3/pkg/common/config" + "github.com/openimsdk/open-im-server/v3/pkg/localcache" "github.com/openimsdk/open-im-server/v3/pkg/rpcclient" pbconversation "github.com/openimsdk/protocol/conversation" "github.com/openimsdk/tools/errs" diff --git a/pkg/rpccache/friend.go b/pkg/rpccache/friend.go index 557b5cffc0..3e9e7863ab 100644 --- a/pkg/rpccache/friend.go +++ b/pkg/rpccache/friend.go @@ -17,9 +17,9 @@ package rpccache import ( "context" - "github.com/openimsdk/localcache" "github.com/openimsdk/open-im-server/v3/pkg/common/cachekey" "github.com/openimsdk/open-im-server/v3/pkg/common/config" + "github.com/openimsdk/open-im-server/v3/pkg/localcache" "github.com/openimsdk/open-im-server/v3/pkg/rpcclient" "github.com/openimsdk/tools/log" "github.com/redis/go-redis/v9" diff --git a/pkg/rpccache/group.go b/pkg/rpccache/group.go index daf76a7a01..7ba22beb82 100644 --- a/pkg/rpccache/group.go +++ b/pkg/rpccache/group.go @@ -17,9 +17,9 @@ package rpccache import ( "context" - "github.com/openimsdk/localcache" "github.com/openimsdk/open-im-server/v3/pkg/common/cachekey" "github.com/openimsdk/open-im-server/v3/pkg/common/config" + "github.com/openimsdk/open-im-server/v3/pkg/localcache" "github.com/openimsdk/open-im-server/v3/pkg/rpcclient" "github.com/openimsdk/protocol/sdkws" "github.com/openimsdk/tools/errs" diff --git a/pkg/rpccache/user.go b/pkg/rpccache/user.go index b31f187dba..0a7a4e4b84 100644 --- a/pkg/rpccache/user.go +++ b/pkg/rpccache/user.go @@ -17,9 +17,9 @@ package rpccache import ( "context" - "github.com/openimsdk/localcache" "github.com/openimsdk/open-im-server/v3/pkg/common/cachekey" "github.com/openimsdk/open-im-server/v3/pkg/common/config" + "github.com/openimsdk/open-im-server/v3/pkg/localcache" "github.com/openimsdk/open-im-server/v3/pkg/rpcclient" "github.com/openimsdk/protocol/sdkws" "github.com/openimsdk/tools/errs" diff --git a/scripts/docker-start-all.sh b/scripts/docker-start-all.sh index 8d91cd7715..116038b536 100755 --- a/scripts/docker-start-all.sh +++ b/scripts/docker-start-all.sh @@ -21,9 +21,6 @@ #fixme This scripts is the total startup scripts #fixme The full name of the shell scripts that needs to be started is placed in the need_to_start_server_shell array -OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/.. -source "${OPENIM_ROOT}/scripts/install/common.sh" -openim::log::info "\n# Use Docker to start all OpenIM service" -trap 'openim::util::onCtrlC' INT -"${OPENIM_ROOT}"/scripts/start-all.sh 2>&1 & -tail -f ${DOCKER_LOG_FILE} +cd /openim/openim-server +mage start +tail -f /dev/null From 837e365a800cc069ac653046e865708abe74510a Mon Sep 17 00:00:00 2001 From: skiffer-git <72860476+skiffer-git@users.noreply.github.com> Date: Wed, 24 Apr 2024 15:43:03 +0800 Subject: [PATCH 2/6] update go.mod go.sum (#2209) * remove \r * remove \r * remove \r * remove \r * remove \r * remove \r * remove \r * remove \r * remove \r * remove \r * fix bug: get localIP * update some ci file (#2200) * Update openimci.yml * Update golangci-lint.yml * Update e2e-test.yml * 3.6.1 code conventions (#2202) * refactor: webhooks update. * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * feat: s3 api addr * refactor: webhooks update. * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * refactor: webhooks update. * refactor: kafka update. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * refactor: kafka update. * refactor: kafka update. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * Windows can compile and run. * Windows can compile and run. * refactor: kafka update. * feat: msg cache split * refactor: webhooks update * refactor: webhooks update * refactor: friends update * refactor: group update * refactor: third update * refactor: api update * refactor: crontab update * refactor: msggateway update * mage * mage * refactor: all module update. * check * refactor: all module update. * load config * load config * load config * load config * refactor: all module update. * refactor: all module update. * refactor: all module update. * refactor: all module update. * refactor: all module update. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * refactor: all module update. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * update tools * update tools * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * update protocol * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: api remove token auth by redis directly. * Code Refactoring * refactor: websocket auth change to call rpc of auth. * refactor: kick online user and remove token change to call auth rpc. * refactor: kick online user and remove token change to call auth rpc. * refactor: remove msggateway redis. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor webhook * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor webhook * refactor: cmd update. * refactor: cmd update. * fix: runtime: goroutine stack exceeds * refactor: cmd update. * refactor notification * refactor notification * refactor * refactor: cmd update. * refactor: cmd update. * refactor * refactor * refactor * protojson * protojson * protojson * go mod * wrapperspb * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: context update. * refactor: websocket update info. * refactor: websocket update info. * refactor: websocket update info. * refactor: websocket update info. * refactor: api name change. * refactor: debug info. * refactor: debug info. * refactor: debug info. * fix: update file * refactor * refactor * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * fix: callback update. * fix: callback update. * refactor * fix: update message. * fix: msg cache timeout. * refactor * refactor * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: push update. * refactor * refactor * fix: push update. * fix: websocket handle error remove when upgrade error. * fix: priority url * fix: minio config * refactor: add zk logger. * refactor * fix: minio config * refactor * refactor * refactor * refactor * refactor: remove zk logger. * refactor: update tools version. * refactor * refactor: update server version to 3.7.0. * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor: zk log debug. * refactor: zk log debug. * refactor: zk log debug. * refactor: zk log debug. * refactor: zk log debug. * refactor * refactor * refactor * refactor: log level change. * refactor: 3.7.0 code conventions. --------- Co-authored-by: skiffer-git <44203734@qq.com> Co-authored-by: withchao <993506633@qq.com> * update go.mod go.sum * Remove Chinese comments * user localhost for minio * user localhost for minio * Remove Chinese comments * Remove Chinese comments * Remove Chinese comments * Set up 4 instances of transfer * Set up 4 instances of transfer * Add comments to the configuration file * Add comments to the configuration file --------- Co-authored-by: root Co-authored-by: xuan <146319162+wxuanF@users.noreply.github.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <993506633@qq.com> --- .github/workflows/e2e-test.yml | 45 ++- .github/workflows/golangci-lint.yml | 8 +- .github/workflows/openimci.yml | 381 +++++++-------------- config/kafka.yml | 18 + config/minio.yml | 13 +- config/mongodb.yml | 9 +- config/notification.yml | 9 - config/openim-msggateway.yml | 11 + config/openim-msgtransfer.yml | 3 + config/openim-push.yml | 9 +- config/openim-rpc-auth.yml | 7 +- config/openim-rpc-conversation.yml | 8 +- config/openim-rpc-friend.yml | 5 + config/openim-rpc-group.yml | 6 +- config/openim-rpc-msg.yml | 8 +- config/openim-rpc-third.yml | 7 + go.mod | 2 +- go.sum | 19 + internal/push/offlinepush/offlinepusher.go | 2 +- internal/push/push_handler.go | 2 +- start-config.yml | 4 +- 21 files changed, 286 insertions(+), 290 deletions(-) diff --git a/.github/workflows/e2e-test.yml b/.github/workflows/e2e-test.yml index f98221e41b..62df69ed5c 100644 --- a/.github/workflows/e2e-test.yml +++ b/.github/workflows/e2e-test.yml @@ -73,14 +73,22 @@ jobs: - name: Docker Operations run: | - sudo make init sudo docker compose up -d + sudo bash bootstrap.sh + sudo mage sudo sleep 20 - name: Module Operations run: | - sudo make tidy - sudo make tools.verify.go-gitlint + sudo go mod tidy + echo "===========> Verifying go-gitlint is installed" + if [ ! -f ./_output/tools/go-gitlint ]; then + export GOBIN=$(pwd)/_output/tools + echo "===========> Installing The default installation path is /home/ubuntu/DF/open-im-server/_output/tools/go-gitlint" + sudo go install github.com/marmotedu/go-gitlint/cmd/go-gitlint@latest + echo "===========> go-gitlint is installed in /home/ubuntu/DF/open-im-server/_output/tools/go-gitlint" + fi + - name: Build, Start(make build && make start) run: | @@ -90,7 +98,8 @@ jobs: run: | sudo ./scripts/install/install.sh -s - - name: Exec OpenIM API test (make test-api) +# - name: Exec OpenIM API test (make test-api) + - name: Exec OpenIM test (make test) run: | mkdir -p ./tmp touch ./tmp/test.md @@ -98,23 +107,27 @@ jobs: echo "## OpenIM API Test" >> ./tmp/test.md echo "
Command Output for OpenIM API Test" >> ./tmp/test.md echo "
" >> ./tmp/test.md
-        sudo make test-api | tee -a ./tmp/test.md
+        echo "===========> Run api test"
+        ./scripts/install/test.sh
+        echo "===========> Run api test" >> ./tmp/test.md
+        ./scripts/install/test.sh >> ./tmp/test.md
         echo "
" >> ./tmp/test.md echo "
" >> ./tmp/test.md - sudo make test-api + echo "===========> Run api test" + ./scripts/install/test.sh - - name: Exec OpenIM E2E Test (make test-e2e) - run: | - echo "" >> ./tmp/test.md - echo "## OpenIM E2E Test" >> ./tmp/test.md - echo "
Command Output for OpenIM E2E Test" >> ./tmp/test.md - echo "
" >> ./tmp/test.md
-        sudo make test-e2e | tee -a ./tmp/test.md
-        echo "
" >> ./tmp/test.md - echo "
" >> ./tmp/test.md + # - name: Exec OpenIM E2E Test (make test-e2e) + # run: | + # echo "" >> ./tmp/test.md + # echo "## OpenIM E2E Test" >> ./tmp/test.md + # echo "
Command Output for OpenIM E2E Test" >> ./tmp/test.md + # echo "
" >> ./tmp/test.md
+ #       sudo make test-e2e | tee -a ./tmp/test.md
+ #       echo "
" >> ./tmp/test.md + # echo "
" >> ./tmp/test.md - sudo make test-e2e + # sudo make test-e2e - name: Comment PR with file uses: thollander/actions-comment-pull-request@v2 diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index 31e491d6b6..64bd498c54 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -29,7 +29,13 @@ jobs: go-version: '1.21' cache: false - name: OpenIM Scripts Verification(make verify) - run: sudo make verify + run: | + cd scripts + for script in verify-*; do + if [ -x "$script" ]; then + ./"$script" + fi + done - name: golangci-lint uses: golangci/golangci-lint-action@v4.0.0 with: diff --git a/.github/workflows/openimci.yml b/.github/workflows/openimci.yml index f47283997d..033868a2ed 100644 --- a/.github/workflows/openimci.yml +++ b/.github/workflows/openimci.yml @@ -1,3 +1,4 @@ + # Copyright © 2023 OpenIM open source community. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -11,8 +12,7 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. - -name: OpenIM CI Aotu Build and Install +name: OpenIM CI Auto Build on: push: @@ -37,269 +37,154 @@ on: - "**.md" - "docs/**" -env: - GO_VERSION: "1.19" - GOLANGCI_VERSION: "v1.50.1" - jobs: - openim: - name: Test with go ${{ matrix.go_version }} on ${{ matrix.os }} - runs-on: ${{ matrix.os }} - permissions: - contents: write - pull-requests: write - environment: - name: openim - strategy: - matrix: - go_version: ["1.19","1.20","1.21"] - os: [ubuntu-latest] - steps: - - name: Setup - uses: actions/checkout@v4 - - - name: Set up Go ${{ matrix.go_version }} - uses: actions/setup-go@v5 - with: - go-version: ${{ matrix.go_version }} - id: go - - - name: Install Task - uses: arduino/setup-task@v1 - with: - version: '3.x' # If available, use the latest major version that's compatible - repo-token: ${{ secrets.GITHUB_TOKEN }} - - - name: Code Typecheck Detector - uses: kubecub/typecheck@main - - - name: Conformity Checker for Project - uses: kubecub/standardizer@main - - - name: Module Operations - run: | - sudo make tidy - sudo make tools.verify.go-gitlint - - - name: Format Code - run: sudo make format - continue-on-error: true - - - name: Generate Files - run: make gen - continue-on-error: true - - - name: Build Source - run: sudo make build - - - name: Build multiarch PLATFORMS - if: startsWith(github.ref, 'refs/heads/release-') - run: | - sudo make multiarch - - - name: Cleanup Build - run: sudo make clean - - - name: Set Current Directory - id: set_directory - run: echo "::set-output name=directory::$(pwd)" - continue-on-error: true - - - name: Collect and Display Test Coverage - id: collect_coverage - run: | - cd ${{ steps.set_directory.outputs.directory }} - make cover - echo "::set-output name=coverage_file::./_output/tmp/coverage.out" - echo "Test Coverage:" - cat ${{ steps.collect_coverage.outputs.coverage_file }} - continue-on-error: true - - openim-start: - name: Test OpenIM install/start on ${{ matrix.os }}-${{ matrix.arch }} - runs-on: ${{ matrix.os }} - permissions: - contents: write - pull-requests: write - environment: - name: openim - strategy: - matrix: - go_version: ["1.21"] - os: ["ubuntu-latest"] - steps: - - name: Checkout and Install OpenIM - uses: actions/checkout@v4 - - name: Install Task - uses: arduino/setup-task@v1 - with: - version: '3.x' # If available, use the latest major version that's compatible - repo-token: ${{ secrets.GITHUB_TOKEN }} - - name: Run OpenIM make install start - run: | - sudo make install - - # - name: Check the OpenIM environment and status - # run: | - # sudo docker images - # sudo docker ps - - name: Check the OpenIM environment and status - if: runner.os == 'Linux' && matrix.arch == 'amd64' - id: docker_info - run: | - sleep 30 - echo "images<> $GITHUB_ENV - sudo docker images >> $GITHUB_ENV - echo "EOF" >> $GITHUB_ENV - echo "containers<> $GITHUB_ENV - sudo docker ps >> $GITHUB_ENV - echo "EOF" >> $GITHUB_ENV - - - name: Comment PR - uses: thollander/actions-comment-pull-request@v2 - if: runner.os == 'Linux' && matrix.arch == 'amd64' - with: - message: | - > [!TIP] - > Run make install to check the status - - ### Docker Images: -
Click to expand docker images - ```bash - ${{ env.images }} - ``` -
- - ### Docker Processes: -
Click to expand docker ps - ```bash - ${{ env.containers }} - ``` -
- GITHUB_TOKEN: ${{ secrets.BOT_GITHUB_TOKEN }} - - execute-scripts: - name: Execute OpenIM Script On ${{ matrix.os }}-${{ matrix.arch }} - runs-on: ${{ matrix.os }} + build-linux: + name: Execute OpenIM Script On Linux + runs-on: ubuntu-latest permissions: - contents: write - pull-requests: write + contents: write + pull-requests: write environment: name: openim strategy: matrix: - go_version: ["1.21"] - os: ["ubuntu-latest", "macos-latest"] arch: [arm64, armv7, amd64] + steps: - - name: Checkout code - uses: actions/checkout@v4 - - - name: Set up Go ${{ matrix.go_version }} - uses: actions/setup-go@v5 - with: - go-version: ${{ matrix.go_version }} - id: go + - uses: actions/checkout@v3 - - name: Install Task - uses: arduino/setup-task@v1 + - name: Set up Go + uses: actions/setup-go@v4 with: - version: '3.x' # If available, use the latest major version that's compatible - repo-token: ${{ secrets.GITHUB_TOKEN }} - - # - name: Install latest Bash (macOS only) - # if: runner.os == 'macOS' && matrix.arch == 'arm64' - # run: | - # /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" - # brew update - - # brew install bash - # brew install gnu-sed - - # echo "/usr/local/bin" >> $GITHUB_PATH - # echo "$(brew --prefix)/opt/gnu-sed/libexec/gnubin" >> $GITHUB_PATH - # continue-on-error: true + go-version: '1.21' - - name: Set up Docker for Ubuntu - if: runner.os == 'Linux' + - name: Set up Docker for Linux run: | - sudo make init sudo docker compose up -d - sudo sleep 20 - - # - name: Set up Docker for macOS - # if: runner.os == 'macOS' && matrix.arch == 'arm64' - # run: | - # brew install --cask docker - # open /Applications/Docker.app - - # sleep 10 - # docker-compose --version || brew install docker-compose + sudo sleep 30 # Increased sleep time for better stability + timeout-minutes: 20 # Increased timeout for Docker setup - # docker-compose up -d - # sleep 20 - - name: Module Operations for Ubuntu - if: runner.os == 'Linux' - run: | - sudo make tidy - sudo make tools.verify.go-gitlint - - # - name: Module Operations for macOS - # if: runner.os == 'macOS' - # run: | - # make tidy - # make tools.verify.go-gitlint - - - name: Build, Start, Check Services and Print Logs for Ubuntu - if: runner.os == 'Linux' - run: | - sudo make build - sudo make start - sudo make check - - - name: Restart Services and Print Logs for Ubuntu - if: runner.os == 'Linux' && matrix.arch == 'amd64' - run: | - sudo make restart - sudo make check - - - name: Build, Start, Check Services and Print Logs for macOS - if: runner.os == 'macOS' && matrix.arch == 'arm64' - run: | - make build - - openim-test-build-image: - name: Build OpenIM Docker Image - runs-on: ubuntu-latest - permissions: - contents: write - pull-requests: write - environment: - name: openim - steps: - - name: Checkout code - uses: actions/checkout@v4 - - name: Set up Go ${{ matrix.go_version }} - uses: actions/setup-go@v5 - with: - go-version: ${{ matrix.go_version }} - id: go - - - name: Install Task - uses: arduino/setup-task@v1 - with: - version: '3.x' # If available, use the latest major version that's compatible - repo-token: ${{ secrets.GITHUB_TOKEN }} + - name: init + run: sudo bash bootstrap.sh + timeout-minutes: 20 - - name: Test Docker Build + - name: Build, Start, Check Services and Print Logs for Linux run: | - sudo make init - sudo make image + sudo mage + sudo mage start + sudo mage check + - - name: Get OpenIM Docker Images Status - id: docker_processes + - name: Restart Services and Print Logs run: | - sudo docker images - sudo docker ps + sudo mage stop + sudo mage start + sudo mage check + + +# build-mac: +# name: Execute OpenIM Script On macOS +# runs-on: macos-latest +# permissions: +# contents: write +# pull-requests: write +# environment: +# name: openim +# strategy: +# matrix: +# arch: [arm64, armv7, amd64] +# +# steps: +# - uses: actions/checkout@v3 + +# - name: Set up Go +# uses: actions/setup-go@v4 +# with: +# go-version: '1.21' + + +# while ! docker system info > /dev/null 2>&1; do +# echo "Waiting for Docker to start..." +# sleep 10 # Increased delay to ensure Docker starts properly +# done + +# - name: Install Docker +# run: | +# brew install docker +# brew install docker-compose +# sleep 10 +# docker-compose up -d +# sleep 30 +# timeout-minutes: 20 +# + +# - name: init +# run: sudo bash bootstrap.sh +# timeout-minutes: 20 + +# - name: Build, Start, Check Services and Print Logs for Linux +# run: | +# sudo mage +# sudo mage start +# sudo mage check + +# - name: Restart Services and Print Logs +# run: | +# sudo mage stop +# sudo mage start +# sudo mage check + +# build-windows: +# name: Execute OpenIM Script On Windows +# runs-on: windows-latest +# permissions: +# contents: write +# pull-requests: write +# environment: +# name: openim +# strategy: +# matrix: +# arch: [arm64, armv7, amd64] +# +# steps: +# - uses: actions/checkout@v3 + +# - name: Set up Go +# uses: actions/setup-go@v4 +# with: +# go-version: '1.21' + +# - name: Set up Docker for Windows +# run: | +# $images = @("zookeeper", "redis", "kafka") +# foreach ($image in $images) { +# $tag = "$image:latest" +# docker pull $tag | Out-Null +# if ($LASTEXITCODE -ne 0) { +# Write-Host "Skipping $image as it is not available for Windows" +# } else { +# Write-Host "Successfully pulled $image" +# } +# } +# docker compose up -d +# Start-Sleep -Seconds 30 +# timeout-minutes: 20 +# shell: pwsh + +# - name: init +# run: bootstrap.bat +# timeout-minutes: 20 + +# - name: Build, Start, Check Services and Print Logs for Linux +# run: | +# mage +# mage start +# mage check + +# - name: Restart Services and Print Logs +# run: | +# mage stop +# mage start +# mage check diff --git a/config/kafka.yml b/config/kafka.yml index b1773c80a8..d412e1be06 100644 --- a/config/kafka.yml +++ b/config/kafka.yml @@ -1,18 +1,36 @@ +# Username for authentication username: '' +# Password for authentication password: '' +# Producer acknowledgment settings producerAck: "" +# Compression type to use (e.g., none, gzip, snappy) compressType: "none" +# List of Kafka broker addresses address: [ localhost:19094 ] +# Kafka topic for Redis integration toRedisTopic: "toRedis" +# Kafka topic for MongoDB integration toMongoTopic: "toMongo" +# Kafka topic for push notifications toPushTopic: "toPush" +# Consumer group ID for Redis topic toRedisGroupID: redis +# Consumer group ID for MongoDB topic toMongoGroupID: mongo +# Consumer group ID for push notifications topic toPushGroupID: push +# TLS (Transport Layer Security) configuration tls: + # Enable or disable TLS enableTLS: false + # CA certificate file path caCrt: "" + # Client certificate file path clientCrt: "" + # Client key file path clientKey: "" + # Client key password clientKeyPwd: "" + # Whether to skip TLS verification (not recommended for production) insecureSkipVerify: false diff --git a/config/minio.yml b/config/minio.yml index 0a80bb845c..11a9ace354 100644 --- a/config/minio.yml +++ b/config/minio.yml @@ -1,7 +1,16 @@ +# Name of the bucket in MinIO bucket: "openim" +# Access key ID for MinIO authentication accessKeyID: "root" +# Secret access key for MinIO authentication secretAccessKey: "openIM123" +# Session token for MinIO authentication (optional) sessionToken: '' -internalAddress: "minio:9000" +# Internal address of the MinIO server +internalAddress: "localhost:10005" +# External address of the MinIO server, accessible from outside. Supports both HTTP and HTTPS using a domain name externalAddress: "http://external_ip:10005" -publicRead: false \ No newline at end of file +# Flag to enable or disable public read access to the bucket +publicRead: false + + diff --git a/config/mongodb.yml b/config/mongodb.yml index 12f1f66a55..98f5694e45 100644 --- a/config/mongodb.yml +++ b/config/mongodb.yml @@ -1,7 +1,14 @@ +# URI for database connection, leave empty if using address and credential settings directly uri: '' +# List of MongoDB server addresses address: [ localhost:37017 ] +# Name of the database database: openim_v3 +# Username for database authentication username: openIM +# Password for database authentication password: openIM123 +# Maximum number of connections in the connection pool maxPoolSize: 100 -maxRetry: 10 \ No newline at end of file +# Maximum number of retry attempts for a failed database connection +maxRetry: 10 diff --git a/config/notification.yml b/config/notification.yml index 1afb44e467..278376c244 100644 --- a/config/notification.yml +++ b/config/notification.yml @@ -17,32 +17,23 @@ # The options field 'isNotification' indicates if it's a notification. groupCreated: isSendMsg: true - # Reliability level of the message sending. # Set to 1 to send only when online, 2 for guaranteed delivery. reliabilityLevel: 1 - # This setting is effective only when 'isSendMsg' is true. # It controls whether to count unread messages. unreadCount: false - # Configuration for offline push notifications. offlinePush: # Enables or disables offline push notifications. enable: false - # Title for the notification when a group is created. title: "create group title" - # Description for the notification. desc: "create group desc" - # Additional information for the notification. ext: "create group ext" -# Content type is not added here. -# Content should use a JSON structure conforming to the protobuf format. - groupInfoSet: isSendMsg: false reliabilityLevel: 1 diff --git a/config/openim-msggateway.yml b/config/openim-msggateway.yml index ad6e180cca..0c92d83278 100644 --- a/config/openim-msggateway.yml +++ b/config/openim-msggateway.yml @@ -1,19 +1,30 @@ rpc: + # The IP address where this RPC service registers itself; if left blank, it defaults to the internal network IP registerIP: '' + # List of ports that the RPC service listens on; configuring multiple ports will launch multiple instances. These must match the number of configured prometheus ports ports: [ 10140 ] prometheus: + # Enable or disable Prometheus monitoring enable: true + # List of ports that Prometheus listens on; these must match the number of rpc.ports to ensure correct monitoring setup ports: [ 20112 ] +# IP address that the RPC/WebSocket service listens on; setting to 0.0.0.0 listens on both internal and external IPs. If left blank, it automatically uses the internal network IP listenIP: 0.0.0.0 longConnSvr: + # WebSocket listening ports, must match the number of rpc.ports ports: [ 10001 ] + # Maximum number of WebSocket connections websocketMaxConnNum: 100000 + # Maximum length of the entire WebSocket message packet websocketMaxMsgLen: 4096 + # WebSocket connection handshake timeout in seconds websocketTimeout: 10 +# 1: For Android, iOS, Windows, Mac, and web platforms, only one instance can be online at a time multiLoginPolicy: 1 + diff --git a/config/openim-msgtransfer.yml b/config/openim-msgtransfer.yml index 9cb1598b5b..07a7dc1ab1 100644 --- a/config/openim-msgtransfer.yml +++ b/config/openim-msgtransfer.yml @@ -1,3 +1,6 @@ prometheus: + # Enable or disable Prometheus monitoring enable: true + # List of ports that Prometheus listens on; each port corresponds to an instance of monitoring. Ensure these are managed accordingly + # Because four instances have been launched, four ports need to be specified ports: [ 20108, 20109, 20110, 20111 ] diff --git a/config/openim-push.yml b/config/openim-push.yml index 35e1b2c07d..a1abfcf889 100644 --- a/config/openim-push.yml +++ b/config/openim-push.yml @@ -1,14 +1,20 @@ rpc: + # The IP address where this RPC service registers itself; if left blank, it defaults to the internal network IP registerIP: '' + # IP address that the RPC service listens on; setting to 0.0.0.0 listens on both internal and external IPs. If left blank, it automatically uses the internal network IP listenIP: 0.0.0.0 + # List of ports that the RPC service listens on; configuring multiple ports will launch multiple instances. These must match the number of configured prometheus ports ports: [ 10170 ] prometheus: + # Enable or disable Prometheus monitoring enable: true + # List of ports that Prometheus listens on; these must match the number of rpc.ports to ensure correct monitoring setup ports: [ 20107 ] maxConcurrentWorkers: 3 -enable: getui +#"Use geTui for offline push notifications, or choose fcm or jpns; corresponding configuration settings must be specified." +enable: "geTui" geTui: pushUrl: "https://restapi.getui.com/v2/$appId" masterSecret: '' @@ -24,6 +30,7 @@ jpns: pushURL: '' pushIntent: '' +# iOS system push sound and badge count iosPush: pushSound: "xxx" badgeCount: true diff --git a/config/openim-rpc-auth.yml b/config/openim-rpc-auth.yml index a77b5457e8..2d861cd5ab 100644 --- a/config/openim-rpc-auth.yml +++ b/config/openim-rpc-auth.yml @@ -1,13 +1,18 @@ rpc: + # The IP address where this RPC service registers itself; if left blank, it defaults to the internal network IP registerIP: '' + # IP address that the RPC service listens on; setting to 0.0.0.0 listens on both internal and external IPs. If left blank, it automatically uses the internal network IP listenIP: 0.0.0.0 + # List of ports that the RPC service listens on; configuring multiple ports will launch multiple instances. These must match the number of configured prometheus ports ports: [ 10160 ] prometheus: + # Enable or disable Prometheus monitoring enable: true + # List of ports that Prometheus listens on; these must match the number of rpc.ports to ensure correct monitoring setup ports: [ 20106 ] tokenPolicy: - #token有效期，单位（天） + # Token validity period, in days expire: 90 diff --git a/config/openim-rpc-conversation.yml b/config/openim-rpc-conversation.yml index e2d9b6a536..a094bfac10 100644 --- a/config/openim-rpc-conversation.yml +++ b/config/openim-rpc-conversation.yml @@ -1,11 +1,13 @@ rpc: + # The IP address where this RPC service registers itself; if left blank, it defaults to the internal network IP registerIP: '' + # IP address that the RPC service listens on; setting to 0.0.0.0 listens on both internal and external IPs. If left blank, it automatically uses the internal network IP listenIP: 0.0.0.0 + # List of ports that the RPC service listens on; configuring multiple ports will launch multiple instances. These must match the number of configured prometheus ports ports: [ 10180 ] prometheus: + # Enable or disable Prometheus monitoring enable: true + # List of ports that Prometheus listens on; these must match the number of rpc.ports to ensure correct monitoring setup ports: [ 20105 ] - - - diff --git a/config/openim-rpc-friend.yml b/config/openim-rpc-friend.yml index 109e3f658f..7b829f971c 100644 --- a/config/openim-rpc-friend.yml +++ b/config/openim-rpc-friend.yml @@ -1,8 +1,13 @@ rpc: + # The IP address where this RPC service registers itself; if left blank, it defaults to the internal network IP registerIP: '' + # IP address that the RPC service listens on; setting to 0.0.0.0 listens on both internal and external IPs. If left blank, it automatically uses the internal network IP listenIP: 0.0.0.0 + # List of ports that the RPC service listens on; configuring multiple ports will launch multiple instances. These must match the number of configured prometheus ports ports: [ 10120 ] prometheus: + # Enable or disable Prometheus monitoring enable: true + # List of ports that Prometheus listens on; these must match the number of rpc.ports to ensure correct monitoring setup ports: [ 20104 ] diff --git a/config/openim-rpc-group.yml b/config/openim-rpc-group.yml index a70d6f96e2..78b44030e0 100644 --- a/config/openim-rpc-group.yml +++ b/config/openim-rpc-group.yml @@ -1,9 +1,13 @@ rpc: + # The IP address where this RPC service registers itself; if left blank, it defaults to the internal network IP registerIP: '' + # IP address that the RPC service listens on; setting to 0.0.0.0 listens on both internal and external IPs. If left blank, it automatically uses the internal network IP listenIP: 0.0.0.0 + # List of ports that the RPC service listens on; configuring multiple ports will launch multiple instances. These must match the number of configured prometheus ports ports: [ 10150 ] prometheus: + # Enable or disable Prometheus monitoring enable: true + # List of ports that Prometheus listens on; these must match the number of rpc.ports to ensure correct monitoring setup ports: [ 20103 ] - diff --git a/config/openim-rpc-msg.yml b/config/openim-rpc-msg.yml index de5e2324df..17ce26e9b2 100644 --- a/config/openim-rpc-msg.yml +++ b/config/openim-rpc-msg.yml @@ -1,13 +1,19 @@ rpc: + # The IP address where this RPC service registers itself; if left blank, it defaults to the internal network IP registerIP: '' + # IP address that the RPC service listens on; setting to 0.0.0.0 listens on both internal and external IPs. If left blank, it automatically uses the internal network IP listenIP: 0.0.0.0 + # List of ports that the RPC service listens on; configuring multiple ports will launch multiple instances. These must match the number of configured prometheus ports ports: [ 10130 ] prometheus: + # Enable or disable Prometheus monitoring enable: true + # List of ports that Prometheus listens on; these must match the number of rpc.ports to ensure correct monitoring setup ports: [ 20102 ] -#发消息是否需要好友验证 + +# Does sending messages require friend verification friendVerify: false diff --git a/config/openim-rpc-third.yml b/config/openim-rpc-third.yml index e3a847bf2c..bb41c93aee 100644 --- a/config/openim-rpc-third.yml +++ b/config/openim-rpc-third.yml @@ -1,13 +1,20 @@ rpc: + # The IP address where this RPC service registers itself; if left blank, it defaults to the internal network IP registerIP: '' + # IP address that the RPC service listens on; setting to 0.0.0.0 listens on both internal and external IPs. If left blank, it automatically uses the internal network IP listenIP: 0.0.0.0 + # List of ports that the RPC service listens on; configuring multiple ports will launch multiple instances. These must match the number of configured prometheus ports ports: [ 10190 ] prometheus: + # Enable or disable Prometheus monitoring enable: true + # List of ports that Prometheus listens on; these must match the number of rpc.ports to ensure correct monitoring setup ports: [ 20101 ] + object: + # Use MinIO as object storage, or set to "cos", "oss", "kodo", "aws", while also configuring the corresponding settings enable: "minio" cos: bucketURL: https://temp-1252357374.cos.ap-chengdu.myqcloud.com diff --git a/go.mod b/go.mod index 891125a899..1279a12857 100644 --- a/go.mod +++ b/go.mod @@ -1,6 +1,6 @@ module github.com/openimsdk/open-im-server/v3 -go 1.20 +go 1.21.2 require ( firebase.google.com/go v3.13.0+incompatible diff --git a/go.sum b/go.sum index 5dc0cc524c..9b31342d84 100644 --- a/go.sum +++ b/go.sum @@ -22,10 +22,13 @@ github.com/QcloudApi/qcloud_sign_golang v0.0.0-20141224014652-e4130a326409/go.mo github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible h1:8psS8a+wKfiLt1iVDX79F7Y6wUM49Lcha2FMXt4UM8g= github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible/go.mod h1:T/Aws4fEfogEE9v+HPhhw+CntffsBHJ8nXQCwKr0/g8= github.com/benbjohnson/clock v1.1.0 h1:Q92kusRqC1XV2MjkWETPvjJVqKetz1OzxZB7mHJLju8= +github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs= +github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c= github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA= +github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0= github.com/bytedance/sonic v1.5.0/go.mod h1:ED5hyg4y6t3/9Ku1R6dU/4KyJ48DZ4jPhfY1O2AihPM= github.com/bytedance/sonic v1.9.1 h1:6iJ6NqdoxCDr6mbY8h18oSO+cShGSMRGCEo7F2h0x8s= github.com/bytedance/sonic v1.9.1/go.mod h1:i736AoUSYt75HyZLoJW9ERYxcy6eaN6h4BZXU064P/U= @@ -43,6 +46,7 @@ github.com/clbanning/mxj v1.8.4/go.mod h1:BVjHeAH+rl9rs6f+QIpeRl0tfu10SXn1pUSa5P github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= github.com/cncf/xds/go v0.0.0-20231128003011-0fa0005c9caa h1:jQCWAUqqlij9Pgj2i/PB79y4KOPYVyFYdROxgaCwdTQ= +github.com/cncf/xds/go v0.0.0-20231128003011-0fa0005c9caa/go.mod h1:x/1Gn8zydmfq8dk6e9PdstVsDgu9RuyIIJqAaF//0IM= github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= @@ -66,12 +70,15 @@ github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.m github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98= github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/envoyproxy/protoc-gen-validate v1.0.4 h1:gVPz/FMfvh57HdSJQyvBtF00j8JU4zdyUgIUNhlgg0A= +github.com/envoyproxy/protoc-gen-validate v1.0.4/go.mod h1:qys6tmnRsYrQqIhm2bvKZH4Blx/1gTIZ2UKVY1M+Yew= github.com/fatih/color v1.14.1 h1:qfhVLaG5s+nCROl1zJsZRxFeYrHLqWroPOQ8BWiNb4w= github.com/fatih/color v1.14.1/go.mod h1:2oHN61fhTpgcxD3TSWCgKDiH1+x4OiDVVGH8WlgGZGg= github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg= github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= github.com/fortytw2/leaktest v1.3.0 h1:u8491cBMTQ8ft8aeV+adlcytMZylmA5nnwwkRZjI8vw= +github.com/fortytw2/leaktest v1.3.0/go.mod h1:jDsjWgpAGjm2CA7WthBh/CdZYEPF31XHquHwclZch5g= github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8= +github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0= github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= github.com/fsnotify/fsnotify v1.7.0 h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA= @@ -90,6 +97,7 @@ github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre github.com/go-ole/go-ole v1.2.6 h1:/Fpf6oFPoeFik9ty7siob0G6Ke8QvQEuVcuChpwXzpY= github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0= github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s= +github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4= github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA= github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY= github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY= @@ -137,11 +145,13 @@ github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/ github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= +github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck= github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8= github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/martian/v3 v3.3.2 h1:IqNFLAmvJOgVlpdEBiQbDc2EwKW77amAycfTuWKdfvw= +github.com/google/martian/v3 v3.3.2/go.mod h1:oBOf6HBosgwRXnUGWUB05QECsc6uvmMiJ3+6W4l/CUk= github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/s2a-go v0.1.7 h1:60BLSyTrOV4/haCDW4zb1guZItoSq8foHCXrAnjBo/o= github.com/google/s2a-go v0.1.7/go.mod h1:50CgR4k1jNlWBu4UfS4AcfhVe1r6pdZPygJ3R8F0Qdw= @@ -194,6 +204,7 @@ github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkr github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ= github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8= github.com/jonboulle/clockwork v0.4.0 h1:p4Cf1aMWXnXAUh8lVfewRBx1zaTSYKrKMF2g3ST4RZ4= +github.com/jonboulle/clockwork v0.4.0/go.mod h1:xgRqUGwRcjKCO1vbZUEtSLrqKoPSsUpK7fnezOII0kc= github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= github.com/kelindar/bitmap v1.5.2 h1:XwX7CTvJtetQZ64zrOkApoZZHBJRkjE23NfqUALA/HE= @@ -209,7 +220,9 @@ github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa02 github.com/klauspost/cpuid/v2 v2.2.6 h1:ndNyv040zDGIDh8thGkXYjnFtiN02M1PVVF+JE/48xc= github.com/klauspost/cpuid/v2 v2.2.6/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ= github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI= github.com/lestrrat-go/envload v0.0.0-20180220234015-a3eb8ddeffcc h1:RKf14vYWi2ttpEmkA4aQ3j4u9dStX2t4M8UM6qqNsG8= @@ -282,6 +295,7 @@ github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZN github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U= github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/prashantv/gostub v1.1.0 h1:BTyx3RfQjRHnUWaGF9oQos79AlQ5k8WNktv7VGvVH4g= +github.com/prashantv/gostub v1.1.0/go.mod h1:A5zLQHz7ieHGG7is6LLXLz7I8+3LZzsrV0P1IAHhP5U= github.com/prometheus/client_golang v1.18.0 h1:HzFfmkOzH5Q8L8G+kSJKUx5dtG87sewO+FoDDqP5Tbk= github.com/prometheus/client_golang v1.18.0/go.mod h1:T+GXkCk5wSJyOqMIzVgvvjFDlkOQntgjkJWKrN5txjA= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= @@ -298,6 +312,7 @@ github.com/redis/go-redis/v9 v9.4.0/go.mod h1:hdY0cQFCN4fnSYT6TkisLufl/4W5UIXyv0 github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs= github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro= github.com/rogpeppe/go-internal v1.11.0 h1:cWPaGQEPrBb5/AsnsZesgZZ9yb1OQ+GOISoDNXVBh4M= +github.com/rogpeppe/go-internal v1.11.0/go.mod h1:ddIwULY96R17DhadqLgMfk9H9tvdUzkipdSkR5nkCZA= github.com/rs/xid v1.5.0 h1:mKX4bl4iPYJtEIxp6CYiUuLQ/8DYMoz0PUdtGgMFRVc= github.com/rs/xid v1.5.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= @@ -376,6 +391,7 @@ go.opentelemetry.io/otel v1.23.0/go.mod h1:YCycw9ZeKhcJFrb34iVSkyT0iczq/zYDtZYFu go.opentelemetry.io/otel/metric v1.23.0 h1:pazkx7ss4LFVVYSxYew7L5I6qvLXHA0Ap2pwV+9Cnpo= go.opentelemetry.io/otel/metric v1.23.0/go.mod h1:MqUW2X2a6Q8RN96E2/nqNoT+z9BSms20Jb7Bbp+HiTo= go.opentelemetry.io/otel/sdk v1.21.0 h1:FTt8qirL1EysG6sTQRZ5TokkU8d0ugCj8htOgThZXQ8= +go.opentelemetry.io/otel/sdk v1.21.0/go.mod h1:Nna6Yv7PWTdgJHVRD9hIYywQBRx7pbox6nwBnZIxl/E= go.opentelemetry.io/otel/trace v1.23.0 h1:37Ik5Ib7xfYVb4V1UtnT97T1jI+AoIYkJyPkuL4iJgI= go.opentelemetry.io/otel/trace v1.23.0/go.mod h1:GSGTbIClEsuZrGIzoEHqsVfxgn5UkggkflQwDScNUsk= go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE= @@ -383,6 +399,7 @@ go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/automaxprocs v1.5.3 h1:kWazyxZUrS3Gs4qUpbwo5kEIMGe/DAvi5Z4tl2NW4j8= go.uber.org/automaxprocs v1.5.3/go.mod h1:eRbA25aqJrxAbsLO0xy5jVwPt7FQnRgjW+efnwa1WM0= go.uber.org/goleak v1.1.11 h1:wy28qYRKZgnJTxGxvye5/wgWr1EKjmUDGYox5mGlRlI= +go.uber.org/goleak v1.1.11/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= go.uber.org/zap v1.24.0 h1:FiJd5l1UOLj0wCgbSE0rwwXHzEdAZS6hiiSnxJN/D60= @@ -489,6 +506,7 @@ golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8T golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 h1:H2TDz8ibqkAF6YGhCdN3jS9O0/s90v0rJh3X/OLHEUk= +golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2/go.mod h1:K8+ghG5WaK9qNqU5K3HdILfMLy1f3aNYFI/wnl100a8= google.golang.org/api v0.165.0 h1:zd5d4JIIIaYYsfVy1HzoXYZ9rWCSBxxAglbczzo7Bgc= google.golang.org/api v0.165.0/go.mod h1:2OatzO7ZDQsoS7IFf3rvsE17/TldiU3F/zxFHeqUB5o= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= @@ -526,6 +544,7 @@ google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGm google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA= gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= diff --git a/internal/push/offlinepush/offlinepusher.go b/internal/push/offlinepush/offlinepusher.go index 0706be64a4..d4fcae434a 100644 --- a/internal/push/offlinepush/offlinepusher.go +++ b/internal/push/offlinepush/offlinepusher.go @@ -26,7 +26,7 @@ import ( ) const ( - geTUI = "getui" + geTUI = "geTui" firebase = "fcm" jPush = "jpush" ) diff --git a/internal/push/push_handler.go b/internal/push/push_handler.go index e09b0b2ec3..3a9a696f64 100644 --- a/internal/push/push_handler.go +++ b/internal/push/push_handler.go @@ -259,7 +259,7 @@ func (c *ConsumerHandler) groupMessagesHandler(ctx context.Context, groupID stri *pushToUserIDs = append(*pushToUserIDs, kickedUsers...) case constant.GroupDismissedNotification: - if msgprocessor.IsNotification(msgprocessor.GetConversationIDByMsg(msg)) { // 消息先到,通知后到 + if msgprocessor.IsNotification(msgprocessor.GetConversationIDByMsg(msg)) { var tips sdkws.GroupDismissedTips if unmarshalNotificationElem(msg.Content, &tips) != nil { return err diff --git a/start-config.yml b/start-config.yml index cd9663c98b..a9c412b33f 100644 --- a/start-config.yml +++ b/start-config.yml @@ -4,7 +4,7 @@ serviceBinaries: openim-rpc-user: 1 openim-msggateway: 1 openim-push: 1 - openim-msgtransfer: 1 + openim-msgtransfer: 4 openim-rpc-conversation: 1 openim-rpc-auth: 1 openim-rpc-group: 1 @@ -12,8 +12,6 @@ serviceBinaries: openim-rpc-msg: 1 openim-rpc-third: 1 toolBinaries: - - ncpu - check-free-memory - - versionchecker - check-component maxFileDescriptors: 10000 From e11b928f4f6f73c5208a1b31c93ef41678a93f33 Mon Sep 17 00:00:00 2001 From: skiffer-git <72860476+skiffer-git@users.noreply.github.com> Date: Thu, 25 Apr 2024 14:17:17 +0800 Subject: [PATCH 3/6] Update the document (#2221) * Update the document * Update the document --- .github/workflows/e2e-test.yml | 2 - .github/workflows/golangci-lint.yml | 58 ------------- .github/workflows/openimci.yml | 1 + README.md | 128 +++++++++------------------- README_zh_CN.md | 87 ++++++------------- docs/CODEOWNERS | 38 ++++----- 6 files changed, 88 insertions(+), 226 deletions(-) delete mode 100644 .github/workflows/golangci-lint.yml diff --git a/.github/workflows/e2e-test.yml b/.github/workflows/e2e-test.yml index 62df69ed5c..6231697c25 100644 --- a/.github/workflows/e2e-test.yml +++ b/.github/workflows/e2e-test.yml @@ -80,7 +80,6 @@ jobs: - name: Module Operations run: | - sudo go mod tidy echo "===========> Verifying go-gitlint is installed" if [ ! -f ./_output/tools/go-gitlint ]; then export GOBIN=$(pwd)/_output/tools @@ -89,7 +88,6 @@ jobs: echo "===========> go-gitlint is installed in /home/ubuntu/DF/open-im-server/_output/tools/go-gitlint" fi - - name: Build, Start(make build && make start) run: | sudo ./scripts/install/install.sh -i diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml deleted file mode 100644 index 64bd498c54..0000000000 --- a/.github/workflows/golangci-lint.yml +++ /dev/null @@ -1,58 +0,0 @@ -# Copyright © 2023 OpenIM open source community. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - - -name: OpenIM golangci-lint -on: - push: - branches: [main] - pull_request: -jobs: - golangci: - name: lint - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-go@v5 - with: - go-version: '1.21' - cache: false - - name: OpenIM Scripts Verification(make verify) - run: | - cd scripts - for script in verify-*; do - if [ -x "$script" ]; then - ./"$script" - fi - done - - name: golangci-lint - uses: golangci/golangci-lint-action@v4.0.0 - with: - # Require: version of golangci-lint to use in form of v1.2 or v1.2.3 or `latest` to use the latest version - version: v1.54 - - # Optional: working directory, useful for monorepos - # working-directory: server - - # Optional: golangci-lint command line arguments. - # - # Note: by default the `.golangci.yml` file should be at the root of the repository. - # The location of the configuration file can be changed by using `--config=` - # args: --timeout=30m --config=/scripts/.golangci.yml --issues-exit-code=0 - - # Optional: show only new issues if it's a pull request. The default value is `false`. - only-new-issues: true - - # Optional:The mode to install golangci-lint. It can be 'binary' or 'goinstall'. - # install-mode: "goinstall" diff --git a/.github/workflows/openimci.yml b/.github/workflows/openimci.yml index 033868a2ed..83d495a0e4 100644 --- a/.github/workflows/openimci.yml +++ b/.github/workflows/openimci.yml @@ -36,6 +36,7 @@ on: - "CONTRIBUTING/**" - "**.md" - "docs/**" + workflow_dispatch: jobs: diff --git a/README.md b/README.md index 175db3326b..a80bd14320 100644 --- a/README.md +++ b/README.md @@ -51,137 +51,89 @@

-## :busts_in_silhouette: Community +## :busts_in_silhouette: Join Our Community -+ 💬 [Follow our Twitter account](https://twitter.com/founder_im63606) -+ 👫 [Join our Reddit](https://www.reddit.com/r/OpenIMessaging) -+ 🚀 [Join our Slack community](https://join.slack.com/t/openimsdk/shared_invite/zt-22720d66b-o_FvKxMTGXtcnnnHiMqe9Q) -+ :eyes: [Join our wechat (微信群)](https://openim-1253691595.cos.ap-nanjing.myqcloud.com/WechatIMG20.jpeg) -+ 📚 [OpenIM Community](https://github.com/OpenIMSDK/community) -+ 💕 [OpenIM Interest Group](https://github.com/Openim-sigs) ++ 💬 [Follow us on Twitter](https://twitter.com/founder_im63606) ++ 🚀 [Join our Slack](https://join.slack.com/t/openimsdk/shared_invite/zt-22720d66b-o_FvKxMTGXtcnnnHiMqe9Q) ++ :eyes: [Join our WeChat Group](https://openim-1253691595.cos.ap-nanjing.myqcloud.com/WechatIMG20.jpeg) ## Ⓜ️ About OpenIM -OpenIM is a service platform specifically designed for integrating chat, audio-video calls, notifications, and AI chatbots into applications. It provides a range of powerful APIs and Webhooks, enabling developers to easily incorporate these interactive features into their applications. OpenIM is not a standalone chat application, but rather serves as a platform to support other applications in achieving rich communication functionalities. The following diagram illustrates the interaction between AppServer, AppClient, OpenIMServer, and OpenIMSDK to explain in detail. +Unlike standalone chat applications such as Telegram, Signal, and Rocket.Chat, OpenIM offers an open-source instant messaging solution designed specifically for developers rather than as a directly installable standalone chat app. Comprising OpenIM SDK and OpenIM Server, it provides developers with a complete set of tools and services to integrate instant messaging functions into their applications, including message sending and receiving, user management, and group management. Overall, OpenIM aims to provide developers with the necessary tools and framework to implement efficient instant messaging solutions in their applications. ![App-OpenIM Relationship](./docs/images/oepnim-design.png) -## 🚀 About OpenIMSDK +## 🚀 Introduction to OpenIMSDK -**OpenIMSDK** is an IM SDK designed for **OpenIMServer**, created specifically for embedding in client applications. Its main features and modules are as follows: +**OpenIMSDK**, designed for **OpenIMServer**, is an IM SDK created specifically for integration into client applications. It supports various functionalities and modules: + 🌟 Main Features: - - - 📦 Local storage - - 🔔 Listener callbacks - - 🛡️ API wrapping - - 🌐 Connection management + - 📦 Local Storage + - 🔔 Listener Callbacks + - 🛡️ API Wrapping + - 🌐 Connection Management + 📚 Main Modules: - 1. 🚀 Initialization and Login 2. 👤 User Management - 3. 👫 Friend Management + 3. 👫 Friends Management 4. 🤖 Group Functions - 5. 💬 Conversation Handling - -It is built using Golang and supports cross-platform deployment, ensuring a consistent access experience across all platforms. - -👉 **[Explore GO SDK](https://github.com/openimsdk/openim-sdk-core)** - -## 🌐 About OpenIMServer + 5. 💬 Session Handling -+ **OpenIMServer** has the following characteristics: - - 🌐 Microservice architecture: Supports cluster mode, including a gateway and multiple rpc services. - - 🚀 Diverse deployment methods: Supports deployment via source code, Kubernetes, or Docker. - - Support for massive user base: Super large groups with hundreds of thousands of users, tens of millions of users, and billions of messages. +Built with Golang and supports cross-platform deployment to ensure a consistent integration experience across all platforms. -### Enhanced Business Functionality: +👉 **[Explore the GO SDK](https://github.com/openimsdk/openim-sdk-core)** -+ **REST API**: OpenIMServer offers REST APIs for business systems, aimed at empowering businesses with more functionalities, such as creating groups and sending push messages through backend interfaces. -+ **Webhooks**: OpenIMServer provides callback capabilities to extend more business forms. A callback means that OpenIMServer sends a request to the business server before or after a certain event, like callbacks before or after sending a message. +## 🌐 Introduction to OpenIMServer -👉 **[Learn more](https://docs.openim.io/guides/introduction/product)** ++ **OpenIMServer** features include: + - 🌐 Microservices Architecture: Supports cluster mode, including a gateway and multiple rpc services. + - 🚀 Diverse Deployment Options: Supports source code, Kubernetes, or Docker deployment. + - Massive User Support: Supports large-scale groups with hundreds of thousands, millions of users, and billions of messages. -## :building_construction: Overall Architecture +### Enhanced Business Functions: -Delve into the heart of Open-IM-Server's functionality with our architecture diagram. ++ **REST API**: Provides a REST API for business systems to enhance functionality, such as group creation and message pushing through backend interfaces. -![Overall Architecture](./docs/images/architecture-layers.png) ++ **Webhooks**: Expands business forms through callbacks, sending requests to business servers before or after certain events. + ![Overall Architecture](./docs/images/architecture-layers.png) ## :rocket: Quick Start -We support many platforms. Here are the addresses for quick experience on the web side： +Experience online for iOS/Android/H5/PC/Web: -👉 **[OpenIM online web demo](https://web-enterprise.rentsoft.cn/)** +👉 **[OpenIM Online Demo](https://www.openim.io/en/commercial)** -🤲 To facilitate user experience, we offer various deployment solutions. You can choose your deployment method from the list below: +To facilitate user experience, we offer various deployment solutions. You can choose your preferred deployment method from the list below: + **[Source Code Deployment Guide](https://docs.openim.io/guides/gettingStarted/imSourceCodeDeployment)** + **[Docker Deployment Guide](https://docs.openim.io/guides/gettingStarted/dockerCompose)** -+ **[Kubernetes Deployment Guide](https://docs.openim.io/guides/gettingStarted/k8s-deployment)** -+ **[Mac Developer Deployment Guide](https://docs.openim.io/guides/gettingstarted/mac-deployment-guide)** - -## :hammer_and_wrench: To Start Developing OpenIM - -[![Open in Dev Container](https://img.shields.io/static/v1?label=Dev%20Container&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/github/openimsdk/open-im-server) - -[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/openimsdk/open-im-server) - -OpenIM Our goal is to build a top-level open source community. We have a set of standards, in the [Community repository](https://github.com/OpenIMSDK/community). -If you'd like to contribute to this Open-IM-Server repository, please read our [contributor documentation](https://github.com/openimsdk/open-im-server/blob/main/CONTRIBUTING.md). +## System Support -Before you start, please make sure your changes are in demand. The best for that is to create a [new discussion](https://github.com/openimsdk/open-im-server/discussions/new/choose) OR [Slack Communication](https://join.slack.com/t/openimsdk/shared_invite/zt-22720d66b-o_FvKxMTGXtcnnnHiMqe9Q), or if you find an issue, [report it](https://github.com/openimsdk/open-im-server/issues/new/choose) first. +Supports Linux, Windows, Mac systems, and ARM and AMD CPU architectures. -- [OpenIM API Reference](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/api.md) -- [OpenIM Bash Logging](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/bash-log.md) -- [OpenIM CI/CD Actions](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/cicd-actions.md) -- [OpenIM Code Conventions](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/code-conventions.md) -- [OpenIM Commit Guidelines](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/commit.md) -- [OpenIM Development Guide](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/development.md) -- [OpenIM Directory Structure](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/directory.md) -- [OpenIM Environment Setup](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/environment.md) -- [OpenIM Error Code Reference](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/error-code.md) -- [OpenIM Git Workflow](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/git-workflow.md) -- [OpenIM Git Cherry Pick Guide](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/gitcherry-pick.md) -- [OpenIM GitHub Workflow](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/github-workflow.md) -- [OpenIM Go Code Standards](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/go-code.md) -- [OpenIM Image Guidelines](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/images.md) -- [OpenIM Initial Configuration](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/init-config.md) -- [OpenIM Docker Installation Guide](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/install-docker.md) -- [OpenIM OpenIM Linux System Installation](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/install-openim-linux-system.md) -- [OpenIM Linux Development Guide](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/linux-development.md) -- [OpenIM Local Actions Guide](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/local-actions.md) -- [OpenIM Logging Conventions](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/logging.md) -- [OpenIM Offline Deployment](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/offline-deployment.md) -- [OpenIM Protoc Tools](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/protoc-tools.md) -- [OpenIM Testing Guide](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/test.md) -- [OpenIM Utility Go](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/util-go.md) -- [OpenIM Makefile Utilities](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/util-makefile.md) -- [OpenIM Script Utilities](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/util-scripts.md) -- [OpenIM Versioning](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/version.md) -- [Manage backend and monitor deployment](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/prometheus-grafana.md) -- [Mac Developer Deployment Guide for OpenIM](https://github.com/openimsdk/open-im-server/tree/main/docs/contrib/mac-developer-deployment-guide.md) +## :link: Links -## :calendar: Community Meetings + + **[Developer Manual](https://docs.openim.io/)** + + **[Changelog](https://github.com/openimsdk/open-im-server/blob/main/CHANGELOG.md)** -We want anyone to get involved in our community and contributing code, we offer gifts and rewards, and we welcome you to join us every Thursday night. +## :writing_hand: How to Contribute -Our conference is in the [OpenIM Slack](https://join.slack.com/t/openimsdk/shared_invite/zt-22720d66b-o_FvKxMTGXtcnnnHiMqe9Q) 🎯, then you can search the Open-IM-Server pipeline to join +We welcome contributions of any kind! Please make sure to read our [Contributor Documentation](https://github.com/openimsdk/open-im-server/blob/main/CONTRIBUTING.md) before submitting a Pull Request. -We take notes of each [biweekly meeting](https://github.com/orgs/OpenIMSDK/discussions/categories/meeting) in [GitHub discussions](https://github.com/openimsdk/open-im-server/discussions/categories/meeting), Our historical meeting notes, as well as replays of the meetings are available at [Google Docs :bookmark_tabs:](https://docs.google.com/document/d/1nx8MDpuG74NASx081JcCpxPgDITNTpIIos0DS6Vr9GU/edit?usp=sharing). + + **[Report a Bug](https://github.com/openimsdk/open-im-server/issues/new?assignees=&labels=bug&template=bug_report.md&title=)** + + **[Suggest a Feature](https://github.com/openimsdk/open-im-server/issues/new?assignees=&labels=enhancement&template=feature_request.md&title=)** + + **[Submit a Pull Request](https://github.com/openimsdk/open-im-server/pulls)** -## :eyes: Who Are Using OpenIM +Thank you for contributing to building a powerful instant messaging solution! -Check out our [user case studies](https://github.com/OpenIMSDK/community/blob/main/ADOPTERS.md) page for a list of the project users. Don't hesitate to leave a [📝comment](https://github.com/openimsdk/open-im-server/issues/379) and share your use case. +## :closed_book: License -## :page_facing_up: License +OpenIMSDK is available under the Apache License 2.0. See the [LICENSE file](https://github.com/openimsdk/open-im-server/blob/main/LICENSE) for more information. -OpenIM is licensed under the Apache 2.0 license. See [LICENSE](https://github.com/openimsdk/open-im-server/tree/main/LICENSE) for the full license text. -The OpenIM logo, including its variations and animated versions, displayed in this repository [OpenIM](https://github.com/openimsdk/open-im-server) under the [assets/logo](./assets/logo) and [assets/logo-gif](assets/logo-gif) directories, are protected by copyright laws. ## 🔮 Thanks to our contributors! diff --git a/README_zh_CN.md b/README_zh_CN.md index f42031165f..e9e56994fb 100644 --- a/README_zh_CN.md +++ b/README_zh_CN.md @@ -51,31 +51,29 @@

-## 🟢 扫描微信进群交流 - +## :busts_in_silhouette: 加入我们的社区 ++ 💬 [关注我们的 Twitter](https://twitter.com/founder_im63606) ++ 🚀 [加入我们的 Slack](https://join.slack.com/t/openimsdk/shared_invite/zt-22720d66b-o_FvKxMTGXtcnnnHiMqe9Q) ++ :eyes: [加入我们的微信群](https://openim-1253691595.cos.ap-nanjing.myqcloud.com/WechatIMG20.jpeg) ## Ⓜ️ 关于 OpenIM -OpenIM 是一个专门设计用于在应用程序中集成聊天、音视频通话、通知以及AI聊天机器人等通信功能的服务平台。它通过提供一系列强大的API和Webhooks，使开发者可以轻松地在他们的应用中加入这些交互特性。OpenIM 本身并不是一个独立运行的聊天应用，而是作为一个平台，为其他应用提供支持，实现丰富的通信功能。下图展示 AppServer、AppClient、OpenIMServer 和 OpenIMSDK 之间的交互关系来具体说明。 - - +与Telegram、Signal、Rocket.Chat等独立聊天应用不同，OpenIM提供了专为开发者设计的开源即时通讯解决方案，而不是直接安装使用的独立聊天应用。OpenIM由OpenIM SDK和OpenIM Server两大部分组成，为开发者提供了一整套集成即时通讯功能的工具和服务，包括消息发送接收、用户管理和群组管理等。总体来说，OpenIM旨在为开发者提供必要的工具和框架，帮助他们在自己的应用中实现高效的即时通讯解决方案。 ![App-OpenIM 关系](./docs/images/oepnim-design.png) -## 🚀 关于 OpenIMSDK +## 🚀 OpenIMSDK 介绍 -**OpenIMSDK** 是为 **OpenIMServer** 设计的IM SDK，专为嵌入客户端应用而生。其主要功能及模块如下： +**OpenIMSDK** 是为 **OpenIMServer** 设计的IM SDK，专为集成到客户端应用而生。它支持多种功能和模块： + 🌟 主要功能： - - 📦 本地存储 - 🔔 监听器回调 - 🛡️ API封装 - 🌐 连接管理 - ## 📚 主要模块： - ++ 📚 主要模块： 1. 🚀 初始化及登录 2. 👤 用户管理 3. 👫 好友管理 @@ -86,81 +84,52 @@ OpenIM 是一个专门设计用于在应用程序中集成聊天、音视频通 👉 **[探索 GO SDK](https://github.com/openimsdk/openim-sdk-core)** -## 🌐 关于 OpenIMServer +## 🌐 OpenIMServer 介绍 -+ **OpenIMServer** 具有以下特点： ++ **OpenIMServer** 的特点包括： - 🌐 微服务架构：支持集群模式，包括网关(gateway)和多个rpc服务。 - - 🚀 部署方式多样：支持源代码、kubernetes或docker部署。 - - 海量用户支持：十万超级大群，千万用户，及百亿消息 + - 🚀 多样的部署方式：支持源代码、Kubernetes或Docker部署。 + - 海量用户支持：支持十万级超大群组，千万级用户和百亿级消息。 ### 增强的业务功能： -+ **REST API**：OpenIMServer 提供了REST API供业务系统使用，旨在赋予业务更多功能，例如通过后台接口建立群组、发送推送消息等。 -+ **Webhooks**：OpenIMServer提供了回调能力以扩展更多的业务形态，所谓回调，即OpenIMServer会在某一事件发生之前或者之后，向业务服务器发送请求，如发送消息之前或之后的回调。 ++ **REST API**：为业务系统提供REST API，增加群组创建、消息推送等后台接口功能。 + ++ **Webhooks**：通过事件前后的回调，向业务服务器发送请求，扩展更多的业务形态。 -👉 **[了解更多](https://docs.openim.io/guides/introduction/product)** + ![整体架构](./docs/images/architecture-layers.png) -## :rocket: 快速开始 + + +## :rocket: 快速入门 在线体验iOS/Android/H5/PC/Web： -👉 **[OpenIM online demo](https://www.openim.io/zh/commercial)** +👉 **[OpenIM在线演示](https://www.openim.io/en/commercial)** -🤲 为了方便用户体验，我们提供了多种部署解决方案，您可以根据下面的列表选择自己的部署方法： +为了便于用户体验，我们提供了多种部署解决方案，您可以根据以下列表选择适合您的部署方式： + **[源代码部署指南](https://docs.openim.io/guides/gettingStarted/imSourceCodeDeployment)** + **[Docker 部署指南](https://docs.openim.io/guides/gettingStarted/dockerCompose)** -+ **[Kubernetes 部署指南](https://docs.openim.io/guides/gettingStarted/k8s-deployment)** - -## :hammer_and_wrench: 开始开发 OpenIM - -OpenIM 我们的目标是建立一个顶级的开源社区。我们有一套标准，在[社区仓库](https://github.com/OpenIMSDK/community)中。 - -如果你想为这个 Open-IM-Server 仓库做贡献，请阅读我们的[贡献者文档](https://github.com/openimsdk/open-im-server/blob/main/CONTRIBUTING.md)。 - -在开始之前，请确保你的更改是有需求的。最好的方法是创建一个[新的讨论](https://github.com/openimsdk/open-im-server/discussions/new/choose) 或 [Slack 通信](https://join.slack.com/t/openimsdk/shared_invite/zt-22720d66b-o_FvKxMTGXtcnnnHiMqe9Q)，或者如果你发现一个问题，首先[报告它](https://github.com/openimsdk/open-im-server/issues/new/choose)。 -+ [代码标准](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/go-code.md) +## 系统支持 -+ [Docker 镜像标准](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/images.md) +支持 Linux、Windows、Mac 系统以及 ARM 和 AMD CPU 架构。 -+ [目录标准](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/directory.md) +## :link: 相关链接 -+ [提交标准](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/commit.md) - -+ [版本控制标准](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/version.md) - -+ [接口标准](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/interface.md) - -+ [OpenIM配置和环境变量设置](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md) - -> **Note** -> 针对中国的用户，阅读我们的 [Docker 镜像标准](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/images.md) 以便使用国内 aliyun 的镜像地址。OpenIM 也有针对中国的 gitee 同步仓库，你可以在 [gitee.com](https://gitee.com/openimsdk) 上找到它。 - -## :link: 链接 - - + **[完整文档](https://doc.rentsoft.cn/)** + + **[开发手册](https://docs.openim.io/)** + **[更新日志](https://github.com/openimsdk/open-im-server/blob/main/CHANGELOG.md)** - + **[FAQ](https://github.com/openimsdk/open-im-server/blob/main/FAQ.md)** - + **[代码示例](https://github.com/openimsdk/open-im-server/blob/main/examples)** - -## :handshake: 社区 - - + **[GitHub Discussions](https://github.com/openimsdk/open-im-server/discussions)** - + **[Slack 通信](https://join.slack.com/t/openimsdk/shared_invite/zt-22720d66b-o_FvKxMTGXtcnnnHiMqe9Q)** - + **[GitHub Issues](https://github.com/openimsdk/open-im-server/issues)** - - 您可以加入这些平台，讨论问题，提出建议，或分享您的成功故事！ -## :writing_hand: 贡献 +## :writing_hand: 如何贡献 - 我们欢迎任何形式的贡献！请确保在提交 Pull Request 之前阅读我们的[贡献者文档](https://github.com/openimsdk/open-im-server/blob/main/CONTRIBUTING.md)。 +我们欢迎任何形式的贡献！在提交 Pull Request 之前，请确保阅读我们的[贡献者文档](https://github.com/openimsdk/open-im-server/blob/main/CONTRIBUTING.md) + **[报告 Bug](https://github.com/openimsdk/open-im-server/issues/new?assignees=&labels=bug&template=bug_report.md&title=)** + **[提出新特性](https://github.com/openimsdk/open-im-server/issues/new?assignees=&labels=enhancement&template=feature_request.md&title=)** + **[提交 Pull Request](https://github.com/openimsdk/open-im-server/pulls)** - 感谢您的贡献，我们一起打造一个强大的即时通信解决方案！ +感谢您的贡献，一起来打造强大的即时通讯解决方案！ ## :closed_book: 许可证 diff --git a/docs/CODEOWNERS b/docs/CODEOWNERS index d1119eb611..68e459f9dd 100644 --- a/docs/CODEOWNERS +++ b/docs/CODEOWNERS @@ -2,28 +2,28 @@ # Each line is a file pattern followed by one or more owners. # README files -README.md @openimsdk/openim @cubxxw @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @rfyiamcool @withchao +README.md @openimsdk/openim @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao # Contributing guidelines -CONTRIBUTING.md @cubxxw @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @rfyiamcool @withchao +CONTRIBUTING.md @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao # License files -LICENSE @cubxxw @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @rfyiamcool @withchao +LICENSE @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao # Makefile -Makefile @cubxxw @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @rfyiamcool @withchao +Makefile @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao # These owners will be the default owners for everything in # the repo. Unless a later match takes precedence, -# @cubxxw and @openimsdk/bot will be requested for +# and @openimsdk/bot will be requested for # review when someone opens a pull request. -* @openimsdk/openim @cubxxw @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @rfyiamcool @withchao +* @openimsdk/openim @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao # Order is important; the last matching pattern takes the most # precedence. When someone opens a pull request that only # modifies JS files, only @js-owner and not the global # owner(s) will be requested for a review. -*.js @cubxxw @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @rfyiamcool @withchao +*.js @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao # You can also use email addresses if you prefer. They'll be # used to look up users just like we do for commit author @@ -35,7 +35,7 @@ Makefile @cubxxw @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @rfyiam # be identified in the format @org/team-name. Teams must have # explicit write access to the repository. In this example, # the OpenIMSDK team in the github organization owns all .txt files. -*.txt @cubxxw @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @rfyiamcool @withchao +*.txt @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao # The `docs/*` pattern will match files like # `docs/getting-started.md` but not further nested files like @@ -44,13 +44,13 @@ docs/* 3293172751nss@gmail.com @openimsdk/bot @skiffer-git # In this example, @octocat owns any file in an apps directory # anywhere in your repository. -api/ @openimsdk/openim @cubxxw @skiffer-git +api/ @openimsdk/openim @skiffer-git # This is a comment. # Each line is a file pattern followed by one or more owners. # CHANGELOG file -CHANGELOG/* @cubxxw @skiffer-git +CHANGELOG/* @skiffer-git # _output directory _output/* @skiffer-git @@ -67,21 +67,21 @@ config/* @skiffer-git # internal directory internal/ @openimsdk/openim @skiffer-git @FGadvancer -tools @openimsdk/openim @openimsdk/bot @cubxxw @skiffer-git @FGadvancer +tools @openimsdk/openim @openimsdk/bot @skiffer-git @FGadvancer # logs directory logs/* @skiffer-git @FGadvancer # pkg directory -pkg/a2r @openimsdk/openim @skiffer-git @cubxxw @openimsdk/bot +pkg/a2r @openimsdk/openim @skiffer-git @openimsdk/bot # scripts directory -scripts/template/* @openimsdk/openim @cubxxw @skiffer-git @FGadvancer -scripts/enterprise/* @openimsdk/openim @FGadvancer @cubxxw @skiffer-git @openimsdk/bot -scripts/githooks/* @openimsdk/openim @cubxxw @skiffer-git @FGadvancer -scripts/lib/* @openimsdk/openim @FGadvancer @cubxxw @skiffer-git @openimsdk/bot -scripts/make-rules/* @openimsdk/openim @FGadvancer @cubxxw @skiffer-git @openimsdk/bot +scripts/template/* @openimsdk/openim @skiffer-git @FGadvancer +scripts/enterprise/* @openimsdk/openim @FGadvancer @skiffer-git @openimsdk/bot +scripts/githooks/* @openimsdk/openim @skiffer-git @FGadvancer +scripts/lib/* @openimsdk/openim @FGadvancer @skiffer-git @openimsdk/bot +scripts/make-rules/* @openimsdk/openim @FGadvancer @skiffer-git @openimsdk/bot # test directory -test/mongo @FGadvancer @cubxxw @skiffer-git @openimsdk/bot -test/mysql @FGadvancer @cubxxw @skiffer-git @openimsdk/bot +test/mongo @FGadvancer @skiffer-git @openimsdk/bot +test/mysql @FGadvancer @skiffer-git @openimsdk/bot From 357b73e8cc846872bc979758b0850ca5883a626a Mon Sep 17 00:00:00 2001 From: skiffer-git <44203734@qq.com> Date: Thu, 25 Apr 2024 14:31:32 +0800 Subject: [PATCH 4/6] Clean up unnecessary scripts and templates --- docs/CODEOWNERS | 87 -- scripts/README.md | 311 ------ scripts/build-all-service.sh | 72 -- scripts/check-all.sh | 113 -- scripts/docker-start-all.sh | 26 - scripts/install-im-server.sh | 86 -- scripts/install/README.md | 116 -- scripts/install/dependency.sh | 89 -- scripts/install/install-protobuf.sh | 118 -- scripts/install/install.sh | 155 --- scripts/install/openim-api.sh | 169 --- scripts/install/openim-crontask.sh | 126 --- scripts/install/openim-msggateway.sh | 141 --- scripts/install/openim-msgtransfer.sh | 186 ---- scripts/install/openim-push.sh | 150 --- scripts/install/openim-rpc.sh | 254 ----- scripts/install/openim-tools.sh | 179 --- scripts/install/test.sh | 1466 ------------------------- scripts/lib/logging.sh | 271 ----- scripts/make-rules/common-versions.mk | 58 - scripts/make-rules/common.mk | 196 ---- scripts/make-rules/copyright.mk | 63 -- scripts/make-rules/dependencies.mk | 41 - scripts/make-rules/gen.mk | 100 -- scripts/make-rules/golang.mk | 272 ----- scripts/make-rules/image.mk | 183 --- scripts/make-rules/release.mk | 42 - scripts/make-rules/swagger.mk | 19 - scripts/make-rules/tools.mk | 280 ----- scripts/release.sh | 150 --- scripts/start-all.sh | 109 -- scripts/stop-all.sh | 60 - 32 files changed, 5688 deletions(-) delete mode 100644 scripts/README.md delete mode 100755 scripts/build-all-service.sh delete mode 100755 scripts/check-all.sh delete mode 100755 scripts/docker-start-all.sh delete mode 100755 scripts/install-im-server.sh delete mode 100644 scripts/install/README.md delete mode 100755 scripts/install/dependency.sh delete mode 100755 scripts/install/install-protobuf.sh delete mode 100755 scripts/install/install.sh delete mode 100755 scripts/install/openim-api.sh delete mode 100755 scripts/install/openim-crontask.sh delete mode 100755 scripts/install/openim-msggateway.sh delete mode 100755 scripts/install/openim-msgtransfer.sh delete mode 100755 scripts/install/openim-push.sh delete mode 100755 scripts/install/openim-rpc.sh delete mode 100755 scripts/install/openim-tools.sh delete mode 100755 scripts/install/test.sh delete mode 100755 scripts/lib/logging.sh delete mode 100644 scripts/make-rules/common-versions.mk delete mode 100644 scripts/make-rules/common.mk delete mode 100644 scripts/make-rules/copyright.mk delete mode 100644 scripts/make-rules/dependencies.mk delete mode 100644 scripts/make-rules/gen.mk delete mode 100644 scripts/make-rules/golang.mk delete mode 100644 scripts/make-rules/image.mk delete mode 100644 scripts/make-rules/release.mk delete mode 100644 scripts/make-rules/swagger.mk delete mode 100644 scripts/make-rules/tools.mk delete mode 100755 scripts/release.sh delete mode 100755 scripts/start-all.sh delete mode 100755 scripts/stop-all.sh diff --git a/docs/CODEOWNERS b/docs/CODEOWNERS index 68e459f9dd..e69de29bb2 100644 --- a/docs/CODEOWNERS +++ b/docs/CODEOWNERS @@ -1,87 +0,0 @@ -# This is a comment. -# Each line is a file pattern followed by one or more owners. - -# README files -README.md @openimsdk/openim @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao - -# Contributing guidelines -CONTRIBUTING.md @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao - -# License files -LICENSE @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao - -# Makefile -Makefile @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao - -# These owners will be the default owners for everything in -# the repo. Unless a later match takes precedence, -# and @openimsdk/bot will be requested for -# review when someone opens a pull request. -* @openimsdk/openim @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao - -# Order is important; the last matching pattern takes the most -# precedence. When someone opens a pull request that only -# modifies JS files, only @js-owner and not the global -# owner(s) will be requested for a review. -*.js @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao - -# You can also use email addresses if you prefer. They'll be -# used to look up users just like we do for commit author -# emails. -*.go 3293172751nss@gmail.com -*.py 3293172751nss@gmail.com - -# Teams can be specified as code owners as well. Teams should -# be identified in the format @org/team-name. Teams must have -# explicit write access to the repository. In this example, -# the OpenIMSDK team in the github organization owns all .txt files. -*.txt @openimsdk/bot @Bloomingg @FGadvancer @skiffer-git @withchao - -# The `docs/*` pattern will match files like -# `docs/getting-started.md` but not further nested files like -# `docs/build-app/troubleshooting.md`. -docs/* 3293172751nss@gmail.com @openimsdk/bot @skiffer-git - -# In this example, @octocat owns any file in an apps directory -# anywhere in your repository. -api/ @openimsdk/openim @skiffer-git - -# This is a comment. -# Each line is a file pattern followed by one or more owners. - -# CHANGELOG file -CHANGELOG/* @skiffer-git - -# _output directory -_output/* @skiffer-git - -# bin directory -bin/* @skiffer-git @FGadvancer - -# cmd directory -cmd/* - -# config directory -config/* @skiffer-git - -# internal directory -internal/ @openimsdk/openim @skiffer-git @FGadvancer - -tools @openimsdk/openim @openimsdk/bot @skiffer-git @FGadvancer - -# logs directory -logs/* @skiffer-git @FGadvancer - -# pkg directory -pkg/a2r @openimsdk/openim @skiffer-git @openimsdk/bot - -# scripts directory -scripts/template/* @openimsdk/openim @skiffer-git @FGadvancer -scripts/enterprise/* @openimsdk/openim @FGadvancer @skiffer-git @openimsdk/bot -scripts/githooks/* @openimsdk/openim @skiffer-git @FGadvancer -scripts/lib/* @openimsdk/openim @FGadvancer @skiffer-git @openimsdk/bot -scripts/make-rules/* @openimsdk/openim @FGadvancer @skiffer-git @openimsdk/bot - -# test directory -test/mongo @FGadvancer @skiffer-git @openimsdk/bot -test/mysql @FGadvancer @skiffer-git @openimsdk/bot diff --git a/scripts/README.md b/scripts/README.md deleted file mode 100644 index b55049a7b5..0000000000 --- a/scripts/README.md +++ /dev/null @@ -1,311 +0,0 @@ -# OpenIM Scripts Directory Structure - -- [OpenIM Scripts Directory Structure](#openim-scripts-directory-structure) - - [log directory](#log-directory) - - [Supported platforms](#supported-platforms) - - [Get started quickly - demo.sh](#get-started-quickly---demosh) - - [Guide: Using and Understanding OpenIM Utility Functions](#guide-using-and-understanding-openim-utility-functions) - - [Table of Contents](#table-of-contents) - - [1. Checking the Status of Services by Ports](#1-checking-the-status-of-services-by-ports) - - [Function: `openim::util::check_ports`](#function-openimutilcheck_ports) - - [Example:](#example) - - [2. Checking the Status of Services by Process Names](#2-checking-the-status-of-services-by-process-names) - - [Function: `openim::util::check_process_names`](#function-openimutilcheck_process_names) - - [Example:](#example-1) - - [3. Stopping Services by Ports](#3-stopping-services-by-ports) - - [Function: `openim::util::stop_services_on_ports`](#function-openimutilstop_services_on_ports) - - [Example:](#example-2) - - [4. Stopping Services by Process Names](#4-stopping-services-by-process-names) - - [Function: `openim::util::stop_services_with_name`](#function-openimutilstop_services_with_name) - - [Example:](#example-3) - - [system management and installation of openim via Linux system](#system-management-and-installation-of-openim-via-linux-system) - - [examples](#examples) - - -This document outlines the directory structure for scripts in the OpenIM Server project. These scripts play a critical role in various areas like building, deploying, running and managing the services of OpenIM. - -```bash -scripts/ -├── README.md # Documentation for the scripts directory. -├── advertise.sh # Script to advertise or broadcast services. -├── batch_start_all.sh # Batch script to start all services. -├── build-all-service.sh # Script to build all services. -├── build.cmd # Command script for building (usually for Windows). -├── check-all.sh # Check script for all components or services. -├── cherry-pick.sh # Helper script for git cherry-pick operations. -├── common.sh # Common utilities and shared functions. -├── coverage.awk # AWK script for processing code coverage data. -├── coverage.sh # Script to gather and report code coverage. -├── demo.sh # Demonstration or example script. -├── docker-check-service.sh # Docker script to check services' status. -├── docker-start-all.sh # Docker script to start all containers/services. -├── ensure-tag.sh # Ensure correct tags or labeling. -├── env_check.sh # Environment verification and checking. -├── gen-swagger-docs.sh # Script to generate Swagger documentation. -├── genconfig.sh # Generate configuration files. -├── gendoc.sh # General documentation generation script. -├── githooks # Directory containing git hooks. -│ ├── commit-msg # Git hook for commit messages. -│ ├── pre-commit # Pre-commit git hook. -│ └── pre-push # Pre-push git hook. -├── golangci.yml # Configuration for GolangCI linting. -├── init-config.sh # Initialize configurations. -├── init-env.sh # Initialize the environment. -├── init-pwd.sh # Initialize or set password. -├── install # Installation scripts directory. -│ ├── README.md # Installation documentation. -│ ├── common.sh # Common utilities for installation. -│ ├── dependency.sh # Script to install dependencies. -│ ├── environment.sh # Set up the environment during installation. -│ ├── install-protobuf.sh # Install Protocol Buffers. -│ ├── install.sh # Main installation script. -│ ├── openim-api.sh # Install OpenIM API. -│ ├── openim-crontask.sh # Install OpenIM crontask. -│ ├── openim-man.sh # Install OpenIM management tool. -│ ├── openim-msggateway.sh # Install OpenIM message gateway. -│ ├── openim-msgtransfer.sh # Install OpenIM message transfer. -│ ├── openim-push.sh # Install OpenIM push service. -│ ├── openim-rpc.sh # Install OpenIM RPC. -│ ├── openim-tools.sh # Install OpenIM tools. -│ ├── test.sh # Installation testing script. -│ └── vimrc # Vim configuration file. -├── install-im-server.sh # Script to install the OpenIM server. -├── install_im_compose.sh # Install OpenIM using Docker Compose. -├── lib # Library or utility scripts directory. -│ ├── chat.sh # Utilities related to chat. -│ ├── color.sh # Color-related utilities. -│ ├── golang.sh # Golang utilities. -│ ├── init.sh # Initialization utilities. -│ ├── logging.sh # Logging utilities. -│ ├── release.sh # Release related utilities. -│ ├── util.sh # General utility functions. -│ └── version.sh # Version management utilities. -├── list-feature-tests.sh # Script to list feature tests. -├── make-rules # Makefile rule templates. -│ ├── common.mk # Common make rules. -│ ├── copyright.mk # Copyright related make rules. -│ ├── dependencies.mk # Dependency management rules. -│ ├── gen.mk # Generic or general rules. -│ ├── golang.mk # Golang-specific make rules. -│ ├── image.mk # Image or container-related rules. -│ ├── release.mk # Release specific rules. -│ ├── swagger.mk # Swagger documentation rules. -│ └── tools.mk # Tooling-related make rules. -├── mongo-init.sh # MongoDB initialization script. -├── release.sh # Script for releasing or deployment. -├── run-in-gopath.sh # Script to run commands within GOPATH. -├── start-all.sh # Script to start all services. -├── start.bat # Batch file to start services (usually for Windows). -├── stop-all.sh # Script to stop all services. -├── template # Directory containing template files. -│ ├── LICENSE # License template. -│ ├── LICENSE_TEMPLATES # Collection of license templates. -│ ├── boilerplate.txt # Boilerplate template. -│ ├── footer.md.tmpl # Footer template for markdown. -│ ├── head.md.tmpl # Header template for markdown. -│ └── project_README.md # Project README template. -├── update-generated-docs.sh # Update generated documentation. -├── update-yamlfmt.sh # Update YAML formatting. -├── verify-pkg-names.sh # Verify package names. -├── verify-shellcheck.sh # Shell script linting verification. -├── verify-spelling.sh # Spelling verification script. -├── verify-typecheck.sh # Type checking verification. -├── verify-yamlfmt.sh # Verify YAML format. -└── wait-for-it.sh # Script to wait for a condition or service to be ready. -``` - -The purpose of having a structured scripts directory like this is to make the operations of OpenIM Server clear and easy to manage. Each script has its own responsibility, making it easier to maintain and update. It's also helpful for newcomers who can easily understand what each part of the system is doing by just looking at this directory structure. - -Each directory and script in the structure should be understood as a part of a larger whole. All scripts work together to ensure the smooth operation and maintenance of the OpenIM Server. - - -## log directory - -**PATH:** `scripts/lib/logging.sh` - -+ [log details](../docs/contrib/bash-log.md) - -## Supported platforms - -- Linux x86_64 (linux_amd64) : 64-bit Linux for most desktop and server systems. - -- Windows x86_64 (windows_amd64) : 64-bit version for most Windows operating systems. - -- macOS x86_64 (darwin_amd64) : 64-bit version for Apple Macintosh computers. - -- Linux ARM64 (linux_arm64) : For ARM-based 64-bit Linux systems such as Raspberry Pi 4 and Jetson Nano. - -- Linux s390x (linux_s390x) : 64-bit Linux for IBM System z hosts. - -- Linux MIPS64 (linux_mips64) : 64-bit Linux for MIPS architecture. - -- Linux MIPS64LE (linux_mips64le) : Suitable for 64-bit Linux systems with little endian MIPS architecture. - -## Get started quickly - demo.sh - -Is the `demo.sh` script teaching you how to quickly get started with OpenIM development and use - - -Steps to run demo: - -```sh -$ make demo -``` - -More about `make` read: - -+ [makefile](../docs/contrib/go-code.md) - -Instructions for producing the demo movie: - -```bash -# Create temporary directory -mkdir /tmp/kb-demo -cd /tmp/kb-demo - -asciinema rec -/scripts/demo/run.sh - - to terminate the script - to terminate the asciinema recording - to save the recording locally - -# Edit the recorded file by editing the controller-gen path -# Once you are happy with the recording, use svg-term program to generate the svg - -svg-term --cast= --out _output/demo.svg --window -``` - -Here you will learn how to test a script, We take the four functions for starting and checking a service as an example. - -## Guide: Using and Understanding OpenIM Utility Functions - -This document provides an overview of the four utility functions designed for managing processes and services. These functions can check the status of services based on ports and process names, as well as stop services based on the same criteria. - -### Table of Contents -- [1. Checking the Status of Services by Ports](#checking-the-status-of-services-by-ports) -- [2. Checking the Status of Services by Process Names](#checking-the-status-of-services-by-process-names) -- [3. Stopping Services by Ports](#stopping-services-by-ports) -- [4. Stopping Services by Process Names](#stopping-services-by-process-names) - -### 1. Checking the Status of Services by Ports - -#### Function: `openim::util::check_ports` - -This function checks the status of services running on specified ports. - -**Usage**: - -```bash -$ openim::util::check_ports ... -``` - -**Design**: - -- The function iterates through each provided port. -- It uses the `lsof` command to identify if there is a service running on the specified port. -- If a service is running, it logs the command, PID, and start time of the service. -- If a service is not running, it logs that the port is not started. -- If any service is not running, the function returns a status of 1. - -#### Example: - -```bash -$ openim::util::check_ports 8080 8081 8082 -``` - -### 2. Checking the Status of Services by Process Names - -#### Function: `openim::util::check_process_names` - -This function checks the status of services based on their process names. - -**Usage**: - -```bash -$ openim::util::check_process_names ... -``` - -**Design**: - -- The function uses `pgrep` to find process IDs associated with the given process names. -- If processes are found, it logs the command, PID, associated port, and start time. -- If no processes are found for a name, it logs that the process is not started. -- If any process is not running, the function returns a status of 1. - -#### Example: - -```bash -$ openim::util::check_process_names nginx mysql redis -``` - -### 3. Stopping Services by Ports - -#### Function: `openim::util::stop_services_on_ports` - -This function attempts to stop services running on the specified ports. - -**Usage**: - -```bash -$ openim::util::stop_services_on_ports ... -``` - -**Design**: - -- The function uses the `lsof` command to identify services running on the specified ports. -- If a service is running on a port, it tries to terminate the associated process using the `kill` command. -- It logs successful terminations and any failures. -- If any service couldn't be stopped, the function returns a status of 1. - -#### Example: - -```bash -$ openim::util::stop_services_on_ports 8080 8081 8082 -``` - -### 4. Stopping Services by Process Names - -#### Function: `openim::util::stop_services_with_name` - -This function attempts to stop services based on their process names. - -**Usage**: - -```bash -$ openim::util::stop_services_with_name ... -``` - -**Design**: - -- The function uses `pgrep` to identify processes associated with the specified names. -- If processes are found, it tries to terminate them using the `kill` command. -- It logs successful terminations and any failures. -- If any service couldn't be stopped, the function returns a status of 1. - -#### Example: - -```bash -$ openim::util::stop_services_with_name nginx apache -``` - -### system management and installation of openim via Linux system - -```bash -$ ./scripts/install/install.sh -``` - -## examples -Scripts to perform various build, install, analysis, etc operations. - -The script directory design of OpenIM and the writing of scripts and tools refer to many excellent open source projects, such as helm, iam, kubernetes, docker, etc. - -Maybe they'll give you inspiration for later maintenance... - -These scripts keep the root level Makefile small and simple. - -Examples: - -* https://github.com/kubernetes/helm/tree/master/scripts -* https://github.com/cockroachdb/cockroach/tree/master/scripts -* https://github.com/hashicorp/terraform/tree/master/scripts \ No newline at end of file diff --git a/scripts/build-all-service.sh b/scripts/build-all-service.sh deleted file mode 100755 index eea380b4fd..0000000000 --- a/scripts/build-all-service.sh +++ /dev/null @@ -1,72 +0,0 @@ -#!/usr/bin/env bash -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -# This script runs `make build` command. -# The command compiles all Makefile configs. -# Args: -# WHAT: Directory names to build. If any of these directories has a 'main' -# package, the build will produce executable files under $(OUT_DIR)/bin/platforms OR $(OUT_DIR)/bin—tools/platforms. -# If not specified, "everything" will be built. -# Usage: `scripts/build-all-service.sh`. -# Example: `scripts/build-go.sh WHAT=cmd/kubelet`. - - -OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/.. -source "${OPENIM_ROOT}/scripts/lib/init.sh" - -# CPU core number -pushd "${OPENIM_ROOT}/tools/ncpu" >/dev/null -cpu_count=$(go run .) -popd >/dev/null - -openim::color::echo ${GREEN_PREFIX} "======> cpu_count=$cpu_count" - -openim::log::info "Building OpenIM, Parallel compilation compile=$cpu_count" -compile_count=$((cpu_count / 2)) - -# For help output -ARGHELP="" -if [[ "$#" -gt 0 ]]; then - ARGHELP="'$*'" -fi - -openim::color::echo $COLOR_CYAN "NOTE: $0 has been replaced by 'make multiarch' or 'make build'" -echo -echo "The equivalent of this invocation is: " -echo " make build ${ARGHELP}" -echo " ./scripts/build-all-service.sh ${ARGHELP}" -echo -echo " Example: " -echo " Print a single binary:" -echo " make build BINS=openim-api" -echo " ./scripts/build-all-service.sh BINS=openim-api" -echo " Print : Enable debugging and logging" -echo " make build BINS=openim-api V=1 DEBUG=1" -echo " ./scripts/build-all-service.sh BINS=openim-api V=1 DEBUG=1" -echo - -if [ -z "$*" ]; then - openim::log::info "no args, build all service" - make --no-print-directory -C "${OPENIM_ROOT}" -j$compile_count build -else - openim::log::info "build service: $*" - make --no-print-directory -C "${OPENIM_ROOT}" -j$compile_count build "$*" -fi - -if [ $? -eq 0 ]; then - openim::log::success "all service build success, run 'make start' or './scripts/start-all.sh'" -else - openim::log::error "make build Error, script exits" -fi diff --git a/scripts/check-all.sh b/scripts/check-all.sh deleted file mode 100755 index 54296b22d5..0000000000 --- a/scripts/check-all.sh +++ /dev/null @@ -1,113 +0,0 @@ -#!/usr/bin/env bash -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# This script is check openim service is running normally -# -# Usage: `scripts/check-all.sh`. -# Encapsulated as: `make check`. -# READ: https://github.com/openimsdk/open-im-server/tree/main/scripts/install/environment.sh - - -OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/.. -source "${OPENIM_ROOT}/scripts/install/common.sh" - -if openim::util::is_running_in_container; then - exec >> ${DOCKER_LOG_FILE} 2>&1 -fi - - -OPENIM_VERBOSE=4 - -openim::log::info "\n# Begin to check all OpenIM service" - -openim::log::status "Check all dependent service ports" -# Elegant printing function -# Elegant printing function -print_services_and_ports() { - local service_names=("$@") - local half_length=$((${#service_names[@]} / 2)) - local service_ports=("${service_names[@]:half_length}") - - echo "+-------------------------+----------+" - echo "| Service Name | Port |" - echo "+-------------------------+----------+" - - for ((index=0; index < half_length; index++)); do - printf "| %-23s | %-8s |\n" "${service_names[$index]}" "${service_ports[$index]}" - done - - echo "+-------------------------+----------+" -} - - -# Assuming OPENIM_SERVER_NAME_TARGETS and OPENIM_SERVER_PORT_TARGETS are defined -# Similarly for OPENIM_DEPENDENCY_TARGETS and OPENIM_DEPENDENCY_PORT_TARGETS - -# Print out services and their ports -print_services_and_ports "${OPENIM_SERVER_NAME_TARGETS[@]}" "${OPENIM_SERVER_PORT_TARGETS[@]}" - -# Print out dependencies and their ports -print_services_and_ports "${OPENIM_DEPENDENCY_TARGETS[@]}" "${OPENIM_DEPENDENCY_PORT_TARGETS[@]}" - -# OpenIM check -#echo "++ The port being checked: ${OPENIM_SERVER_PORT_LISTARIES[@]}" -openim::log::info "\n## Check all dependent components service ports" -#echo "++ The port being checked: ${OPENIM_DEPENDENCY_PORT_LISTARIES[@]}" - - -# Later, after discarding Docker, the Docker keyword is unreliable, and Kubepods is used -if grep -qE 'docker|kubepods' /proc/1/cgroup || [ -f /.dockerenv ]; then - openim::color::echo ${COLOR_CYAN} "Environment in the interior of the container" -else - openim::color::echo ${COLOR_CYAN}"The environment is outside the container" - openim::util::check_ports ${OPENIM_DEPENDENCY_PORT_LISTARIES[@]} -fi - -if [[ $? -ne 0 ]]; then - openim::log::error_exit "The service does not start properly, please check the port, query variable definition!" - echo "+++ https://github.com/openimsdk/open-im-server/tree/main/scripts/install/environment.sh +++" -else - openim::log::success "All components depended on by OpenIM are running normally! " -fi - - -openim::log::status "Check OpenIM service:" -openim::log::colorless "${OPENIM_OUTPUT_HOSTBIN}/openim-msgtransfer" -result=$(. $(dirname ${BASH_SOURCE})/install/openim-msgtransfer.sh openim::msgtransfer::check) -if [[ $? -ne 0 ]]; then - #echo "+++ cat openim log file >>> ${LOG_FILE}" - - openim::log::error "The service is not running properly, please check the logs $result" -fi - - -openim::log::status "Check OpenIM service:" -for item in "${OPENIM_ALL_SERVICE_LIBRARIES_NO_TRANSFER[@]}"; do - openim::log::colorless "$item" -done - - -result=$(openim::util::check_process_names ${OPENIM_ALL_SERVICE_LIBRARIES_NO_TRANSFER[@]}) -if [[ $? -ne 0 ]]; then - #echo "+++ cat OpenIM log file >>> ${LOG_FILE}" - openim::log::error "The service is not running properly, please check the logs " - echo "$result" - exit 1 -else - openim::log::status "List the ports listened to by the OpenIM service:" - openim::util::find_ports_for_all_services ${OPENIM_ALL_SERVICE_LIBRARIES_NO_TRANSFER[@]} - openim::util::find_ports_for_all_services ${OPENIM_MSGTRANSFER_BINARY[@]} - openim::log::success "All OpenIM services are running normally! " -fi diff --git a/scripts/docker-start-all.sh b/scripts/docker-start-all.sh deleted file mode 100755 index 116038b536..0000000000 --- a/scripts/docker-start-all.sh +++ /dev/null @@ -1,26 +0,0 @@ -#!/usr/bin/env bash - -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - - - - - -#fixme This scripts is the total startup scripts -#fixme The full name of the shell scripts that needs to be started is placed in the need_to_start_server_shell array - -cd /openim/openim-server -mage start -tail -f /dev/null diff --git a/scripts/install-im-server.sh b/scripts/install-im-server.sh deleted file mode 100755 index d11a49dc8d..0000000000 --- a/scripts/install-im-server.sh +++ /dev/null @@ -1,86 +0,0 @@ -#!/usr/bin/env bash -# Copyright © 2024 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# -# OpenIM Docker Deployment Script -# -# This script automates the process of building the OpenIM server image -# and deploying it using Docker Compose. -# -# Variables: -# - SERVER_IMAGE_VERSION: Version of the server image (default: test) -# - IMAGE_REGISTRY: Docker image registry (default: openim) -# - DOCKER_COMPOSE_FILE_URL: URL to the docker-compose.yml file -# -# Usage: -# SERVER_IMAGE_VERSION=latest IMAGE_REGISTRY=myregistry ./this_script.sh - - - - - -OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/.. -source "${OPENIM_ROOT}/scripts/lib/init.sh" - -trap 'openim::util::onCtrlC' INT - -chmod +x "${OPENIM_ROOT}"/scripts/*.sh - -openim::util::ensure_docker_daemon_connectivity - -# Default values for variables -: ${SERVER_IMAGE_VERSION:=test} -: ${IMAGE_REGISTRY:=openim} -: ${DOCKER_COMPOSE_FILE_URL:="https://raw.githubusercontent.com/openimsdk/openim-docker/main/docker-compose.yaml"} - -DOCKER_COMPOSE_COMMAND= -# Check if docker-compose command is available -openim::util::check_docker_and_compose_versions -if command -v docker compose &> /dev/null; then - openim::log::info "docker compose command is available" - DOCKER_COMPOSE_COMMAND="docker compose" -else - DOCKER_COMPOSE_COMMAND="docker-compose" -fi - -export SERVER_IMAGE_VERSION -export IMAGE_REGISTRY -"${OPENIM_ROOT}"/scripts/init-config.sh - -pushd "${OPENIM_ROOT}" -docker build -t "${IMAGE_REGISTRY}/openim-server:${SERVER_IMAGE_VERSION}" . -${DOCKER_COMPOSE_COMMAND} stop -curl "${DOCKER_COMPOSE_FILE_URL}" -o docker-compose.yml -${DOCKER_COMPOSE_COMMAND} up -d - -# Function to check container status -check_containers() { - if ! ${DOCKER_COMPOSE_COMMAND} ps | grep -q 'Up'; then - echo "Error: One or more docker containers failed to start." - ${DOCKER_COMPOSE_COMMAND} logs openim-server - ${DOCKER_COMPOSE_COMMAND} logs openim-chat - return 1 - fi - return 0 -} - -# Wait for a short period to allow containers to initialize -sleep 100 - -${DOCKER_COMPOSE_COMMAND} ps - -check_containers - -popd \ No newline at end of file diff --git a/scripts/install/README.md b/scripts/install/README.md deleted file mode 100644 index 8e710add8d..0000000000 --- a/scripts/install/README.md +++ /dev/null @@ -1,116 +0,0 @@ -# OpenIM Suite Scripts - -The OpenIM Suite represents a comprehensive collection of scripts, each tailored to manage and operate specific services within the OpenIM ecosystem. These scripts offer consistent, reliable, and efficient tools for initializing, controlling, and managing various OpenIM services on a Linux platform. - -## Features - -- **Robustness:** Built with Bash's error handling mechanisms (`errexit`, `nounset`, and `pipefail`), ensuring scripts fail fast and provide relevant error messages. -- **Modularity:** Each script is dedicated to a particular service, promoting clarity and ease of maintenance. -- **Comprehensive Logging:** Integrated logging utilities offer real-time insights into operations, enhancing transparency and debuggability. -- **Systemd Integration:** Where applicable, scripts integrate with the systemd service manager, offering standardized service controls like start, stop, restart, and status checks. - -## Scripts Overview - -1. **openim-api:** Control interface for the OpenIM API service. -2. **openim-cmdutils:** Utility toolkit for common OpenIM command-line operations. -3. **openim-crontask:** Manages the OpenIM CronTask service, with both direct and systemctl installation methods. -4. **openim-msggateway:** Script to operate the OpenIM Message Gateway service. -5. **openim-msgtransfer:** Manages the OpenIM Message Transfer functionalities. -6. **openim-push:** Interface for controlling the OpenIM Push Notification service. -7. **openim-rpc-auth:** Script dedicated to the OpenIM RPC Authentication service. -8. **openim-rpc-conversation:** Manages operations related to the OpenIM RPC Conversation service. -9. **openim-rpc-friend:** Control interface for the OpenIM RPC Friend functionalities. -10. **openim-rpc-group:** Script for managing the OpenIM RPC Group service. -11. **openim-rpc-msg:** Operates the OpenIM RPC Messaging service. -12. **openim-rpc-third:** Script dedicated to third-party integrations with OpenIM RPC. -13. **openim-rpc-user:** Control interface for OpenIM RPC User operations. - -## OpenIM Server Installation Script Usage - -The scripts within the OpenIM Suite generally adhere to two primary execution methodologies. To illustrate these methodologies, we'll use `openim-crontask` as a representative example. - -1. **Direct Script Execution:** Running the script directly, typically for straightforward start/stop operations. - - ```bash - ./[script-name].sh - ``` - -2. **Function-based Execution:** Invoking specific functions within the script for more specialized operations (e.g., install, uninstall). - - ```bash - ./scripts/install/install.sh [options] - ``` - -**Description:** -This script is designed to handle the installation, uninstallation, and status checking of OpenIM components on the server. OpenIM is a presumed communication or messaging platform. - -### Commands: -- **-i, --install**: - Initiate the installation of all OpenIM components. - -- **-u, --uninstall**: - Uninstall or remove all OpenIM components from the server. - -- **-s, --status**: - Check and report the current operational status of the installed OpenIM components. - -- **-h, --help**: - Display the help menu for available commands. - -### Example Usage: -To install all OpenIM components: -```bash -./scripts/install/install.sh -i -``` -or -```bash -./scripts/install/install.sh --install -``` -> **Note**: -> Ensure you have the necessary privileges to execute installation or uninstallation operations. It's generally recommended to take a backup before making major changes. - - -### 1. Direct Script Execution - -This method involves invoking the script directly, initiating its default behavior. For instance, with `openim-crontask`, direct execution will start the OpenIM CronTask as a background process. - -**Example:** - -```bash -./openim-crontask.sh -``` - -Upon execution, the script will source any necessary configurations, log the start of the CronTask, and finally run the CronTask in the background. The log messages will provide feedback about the process, ensuring the user is informed of the task's progress. - -### 2. Function-based Execution - -This approach is more specialized, enabling users to call specific functions defined within the script. This is particularly useful for tasks like installation, uninstallation, and status checks. - -For the `openim-crontask` script: - -- **Installation**: It includes building the service, generating configuration files, setting up systemd services, and starting the service. - - ```bash - ./openim-crontask.sh openim::crontask::install - ``` - -- **Uninstallation**: Stops the service, removes associated binaries, configuration files, and systemd service files. - - ```bash - ./openim-crontask.sh openim::crontask::uninstall - ``` - -- **Status Check**: Verifies the running status of the service, checking for active processes and listening ports. - - ```bash - ./openim-crontask.sh openim::crontask::status - ``` - -It's crucial to familiarize oneself with the available functions within each script. This ensures optimal utilization of the provided tools and a deeper understanding of the underlying operations. - - - -## Notes - -- Always ensure you have the correct permissions before executing any script. -- Environment variables may need to be set or sourced depending on your installation and configuration. \ No newline at end of file diff --git a/scripts/install/dependency.sh b/scripts/install/dependency.sh deleted file mode 100755 index bad1cb6f92..0000000000 --- a/scripts/install/dependency.sh +++ /dev/null @@ -1,89 +0,0 @@ -#!/usr/bin/env bash -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# This script will install the dependencies required for openim - - - - - -OPENIM_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")"/../.. && pwd -P) -[[ -z ${COMMON_SOURCED} ]] && source "${OPENIM_ROOT}"/scripts/install/common.sh - -# Start MongoDB service -docker run -d \ ---name mongo \ --p 37017:27017 \ --v "${DATA_DIR}/components/mongodb/data/db:/data/db" \ --v "${DATA_DIR}/components/mongodb/data/logs:/data/logs" \ --v "${DATA_DIR}/components/mongodb/data/conf:/etc/mongo" \ --v "./scripts/mongo-init.sh:/docker-entrypoint-initdb.d/mongo-init.sh:ro" \ --e TZ=Asia/Shanghai \ --e wiredTigerCacheSizeGB=1 \ --e MONGO_INITDB_ROOT_USERNAME=${OPENIM_USER} \ --e MONGO_INITDB_ROOT_PASSWORD=${PASSWORD} \ --e MONGO_INITDB_DATABASE=openim_v3 \ --e MONGO_OPENIM_USERNAME=${OPENIM_USER} \ --e MONGO_OPENIM_PASSWORD=${PASSWORD} \ ---restart always \ -mongo:6.0.2 --wiredTigerCacheSizeGB 1 --auth - -# Start Redis service -docker run -d \ ---name redis \ --p 16379:6379 \ --v "${DATA_DIR}/components/redis/data:/data" \ --v "${DATA_DIR}/components/redis/config/redis.conf:/usr/local/redis/config/redis.conf" \ --e TZ=Asia/Shanghai \ ---sysctl net.core.somaxconn=1024 \ ---restart always \ -redis:7.0.0 redis-server --requirepass ${PASSWORD} --appendonly yes - -# Start Zookeeper service -docker run -d \ ---name zookeeper \ --p 2181:2181 \ --v "/etc/localtime:/etc/localtime" \ --e TZ=Asia/Shanghai \ ---restart always \ -wurstmeister/zookeeper - -# Start Kafka service -docker run -d \ ---name kafka \ --p 9092:9092 \ --e TZ=Asia/Shanghai \ --e KAFKA_BROKER_ID=0 \ --e KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 \ --e KAFKA_CREATE_TOPICS="latestMsgToRedis:8:1,msgToPush:8:1,offlineMsgToMongoMysql:8:1" \ --e KAFKA_ADVERTISED_LISTENERS="INSIDE://127.0.0.1:9092,OUTSIDE://103.116.45.174:9092" \ --e KAFKA_LISTENERS="INSIDE://:9092,OUTSIDE://:9093" \ --e KAFKA_LISTENER_SECURITY_PROTOCOL_MAP="INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT" \ --e KAFKA_INTER_BROKER_LISTENER_NAME=INSIDE \ ---restart always \ ---link zookeeper \ -wurstmeister/kafka - -# Start MinIO service -docker run -d \ ---name minio \ --p 10005:9000 \ --p 9090:9090 \ --v "/mnt/data:/data" \ --v "/mnt/config:/root/.minio" \ --e MINIO_ROOT_USER=${OPENIM_USER} \ --e MINIO_ROOT_PASSWORD=${PASSWORD} \ ---restart always \ -minio/minio server /data --console-address ':9090' diff --git a/scripts/install/install-protobuf.sh b/scripts/install/install-protobuf.sh deleted file mode 100755 index 838b390b50..0000000000 --- a/scripts/install/install-protobuf.sh +++ /dev/null @@ -1,118 +0,0 @@ -#!/usr/bin/env bash -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - - -# -------------------------------------------------------------- -# OpenIM Protoc Tool v1.0.0 -# -------------------------------------------------------------- -# OpenIM has released its custom Protoc tool version v1.0.0. -# This tool is customized to meet the specific needs of OpenIM and resides in its separate repository. -# It can be downloaded from the following link: -# https://github.com/OpenIMSDK/Open-IM-Protoc/releases/tag/v1.0.0 -# -# About the tool: -# https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/protoc-tools.md -# Download link (Windows): https://github.com/OpenIMSDK/Open-IM-Protoc/releases/download/v1.0.0/windows.zip -# Download link (Linux): https://github.com/OpenIMSDK/Open-IM-Protoc/releases/download/v1.0.0/linux.zip -# -# Installation steps (taking Windows as an example): -# 1. Visit the above link and download the version suitable for Windows. -# 2. Extract the downloaded file. -# 3. Add the extracted tool to your PATH environment variable so that it can be run directly from the command line. -# -# Note: The specific installation and usage instructions may vary based on the tool's actual implementation. It's advised to refer to official documentation. -# -------------------------------------------------------------- - -PROTOC_DOWNLOAD_URL="https://github.com/OpenIMSDK/Open-IM-Protoc/releases/download/v1.0.0/linux.zip" -DOWNLOAD_DIR="/tmp/openim-protoc" -INSTALL_DIR="/usr/local/bin" - -function help_message { - echo "Usage: ./install-protobuf.sh [option]" - echo "Options:" - echo "-i, --install Install the OpenIM Protoc tool." - echo "-u, --uninstall Uninstall the OpenIM Protoc tool." - echo "-r, --reinstall Reinstall the OpenIM Protoc tool." - echo "-c, --check Check if the OpenIM Protoc tool is installed." - echo "-h, --help Display this help message." -} - -function install_protobuf { - echo "Installing OpenIM Protoc tool..." - - # Create temporary directory and download the zip file - mkdir -p $DOWNLOAD_DIR - wget $PROTOC_DOWNLOAD_URL -O $DOWNLOAD_DIR/linux.zip - - # Unzip the file - unzip -o $DOWNLOAD_DIR/linux.zip -d $DOWNLOAD_DIR - - # Move binaries to the install directory and make them executable - sudo cp $DOWNLOAD_DIR/linux/protoc $INSTALL_DIR/ - sudo cp $DOWNLOAD_DIR/linux/protoc-gen-go $INSTALL_DIR/ - sudo chmod +x $INSTALL_DIR/protoc - sudo chmod +x $INSTALL_DIR/protoc-gen-go - - # Clean up - rm -rf $DOWNLOAD_DIR - - echo "OpenIM Protoc tool installed successfully!" -} - -function uninstall_protobuf { - echo "Uninstalling OpenIM Protoc tool..." - - # Removing binaries from the install directory - sudo rm -f $INSTALL_DIR/protoc - sudo rm -f $INSTALL_DIR/protoc-gen-go - - echo "OpenIM Protoc tool uninstalled successfully!" -} - -function reinstall_protobuf { - echo "Reinstalling OpenIM Protoc tool..." - uninstall_protobuf - install_protobuf -} - -function check_protobuf { - echo "Checking for OpenIM Protoc tool installation..." - - which protoc > /dev/null 2>&1 - if [ $? -eq 0 ]; then - echo "OpenIM Protoc tool is installed." - else - echo "OpenIM Protoc tool is not installed." - fi -} - -while [ "$1" != "" ]; do - case $1 in - -i | --install ) install_protobuf - ;; - -u | --uninstall ) uninstall_protobuf - ;; - -r | --reinstall ) reinstall_protobuf - ;; - -c | --check ) check_protobuf - ;; - -h | --help ) help_message - exit - ;; - * ) help_message - exit 1 - esac - shift -done diff --git a/scripts/install/install.sh b/scripts/install/install.sh deleted file mode 100755 index bb09675bfd..0000000000 --- a/scripts/install/install.sh +++ /dev/null @@ -1,155 +0,0 @@ -#!/usr/bin/env bash -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -# OpenIM Server Installation Script -# -# Description: -# This script is designed to handle the installation, Is a deployment solution -# that uses the Linux systen extension. uninstallation, and -# status checking of OpenIM components on the server. OpenIM is a presumed -# communication or messaging platform based on the context. -# -# Usage: -# To utilize this script, you need to invoke it with specific commands -# and options as detailed below. -# -# Commands: -# -i, --install : Use this command to initiate the installation of all -# OpenIM components. -# -u, --uninstall : Use this command to uninstall or remove all -# OpenIM components from the server. -# -s, --status : This command can be used to check and report the -# current operational status of the installed OpenIM components. -# -h, --help : For any assistance or to view the available commands, -# use this command to display the help menu. -# -# Example Usage: -# To install all OpenIM components: -# ./scripts/install/install.sh -i -# or -# ./scripts/install/install.sh --install -# -# Note: -# Ensure you have the necessary privileges to execute installation or -# uninstallation operations. It's generally recommended to take a backup -# before making major changes. -# -############################################################################### - -OPENIM_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")"/../.. && pwd -P) -[[ -z ${COMMON_SOURCED} ]] && source "${OPENIM_ROOT}"/scripts/install/common.sh - -${OPENIM_ROOT}/scripts/install/openim-man.sh -${OPENIM_ROOT}/scripts/install/openim-tools.sh -${OPENIM_ROOT}/scripts/install/test.sh - -# Detailed help function -function openim::install::show_help() { - echo "OpenIM Installer" - echo "Usage: $0 [options]" - echo "" - echo "Commands:" - echo " -i, --install Install all OpenIM components." - echo " -u, --uninstall Remove all OpenIM components." - echo " -s, --status Check the current status of OpenIM components." - echo " -h, --help Show this help menu." - echo "" - echo "Example: " - echo " $0 -i Will install all OpenIM components." - echo " $0 --install Same as above." -} - -function openim::install::install_openim() { - openim::common::sudo "mkdir -p ${OPENIM_DATA_DIR} ${OPENIM_INSTALL_DIR} ${OPENIM_CONFIG_DIR} ${OPENIM_LOG_DIR}" - openim::log::info "check openim dependency" - openim::common::sudo "cp -r ${OPENIM_ROOT}/config/* ${OPENIM_CONFIG_DIR}/" - - ${OPENIM_ROOT}/scripts/genconfig.sh ${ENV_FILE} ${OPENIM_ROOT}/deployments/templates/config.yaml > ${OPENIM_CONFIG_DIR}/config.yaml - ${OPENIM_ROOT}/scripts/genconfig.sh ${ENV_FILE} ${OPENIM_ROOT}/deployments/templates/prometheus.yml > ${OPENIM_CONFIG_DIR}/prometheus.yml - - openim::util::check_ports ${OPENIM_DEPENDENCY_PORT_LISTARIES[@]} - - ${OPENIM_ROOT}/scripts/install/openim-msggateway.sh openim::msggateway::install || return 1 - ${OPENIM_ROOT}/scripts/install/openim-msgtransfer.sh openim::msgtransfer::install || return 1 - ${OPENIM_ROOT}/scripts/install/openim-push.sh openim::push::install || return 1 - ${OPENIM_ROOT}/scripts/install/openim-crontask.sh openim::crontask::install || return 1 - ${OPENIM_ROOT}/scripts/install/openim-rpc.sh openim::rpc::install || return 1 - ${OPENIM_ROOT}/scripts/install/openim-api.sh openim::api::install || return 1 - - openim::common::sudo "cp -r ${OPENIM_ROOT}/deployments/templates/openim.target /etc/systemd/system/openim.target" - openim::common::sudo "systemctl daemon-reload" - openim::common::sudo "systemctl restart openim.target" - openim::common::sudo "systemctl enable openim.target" - openim::log::success "openim install success" -} - -function openim::uninstall::uninstall_openim() { - openim::log::info "uninstall openim" - - ${OPENIM_ROOT}/scripts/install/openim-msggateway.sh openim::msggateway::uninstall || return 1 - ${OPENIM_ROOT}/scripts/install/openim-msgtransfer.sh openim::msgtransfer::uninstall || return 1 - ${OPENIM_ROOT}/scripts/install/openim-push.sh openim::push::uninstall || return 1 - ${OPENIM_ROOT}/scripts/install/openim-crontask.sh openim::crontask::uninstall || return 1 - ${OPENIM_ROOT}/scripts/install/openim-rpc.sh openim::rpc::uninstall || return 1 - ${OPENIM_ROOT}/scripts/install/openim-api.sh openim::api::uninstall || return 1 - - set +o errexit - openim::common::sudo "systemctl stop openim.target" - openim::common::sudo "systemctl disable openim.target" - openim::common::sudo "rm -f /etc/systemd/system/openim.target" - - openim::log::success "openim uninstall success" -} - -function openim::install::status() { - openim::log::info "check openim status" - - ${OPENIM_ROOT}/scripts/install/openim-msggateway.sh openim::msggateway::status || return 1 - ${OPENIM_ROOT}/scripts/install/openim-msgtransfer.sh openim::msgtransfer::status || return 1 - ${OPENIM_ROOT}/scripts/install/openim-push.sh openim::push::status || return 1 - ${OPENIM_ROOT}/scripts/install/openim-crontask.sh openim::crontask::status || return 1 - ${OPENIM_ROOT}/scripts/install/openim-rpc.sh openim::rpc::status || return 1 - ${OPENIM_ROOT}/scripts/install/openim-api.sh openim::api::status || return 1 - - openim::log::success "openim status success" -} - -# If no arguments are provided, show help -if [[ $# -eq 0 ]]; then - openim::install::show_help - exit 0 -fi - -# Argument parsing to call functions based on user input -while (( "$#" )); do - case "$1" in - -i|--install) - openim::install::install_openim - shift - ;; - -u|--uninstall) - openim::uninstall::uninstall_openim - shift - ;; - -s|--status) - openim::install::status - shift - ;; - -h|--help|*) - openim::install::show_help - exit 0 - ;; - esac -done \ No newline at end of file diff --git a/scripts/install/openim-api.sh b/scripts/install/openim-api.sh deleted file mode 100755 index 8403382e6a..0000000000 --- a/scripts/install/openim-api.sh +++ /dev/null @@ -1,169 +0,0 @@ -#!/usr/bin/env bash - -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - - - - - -OPENIM_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")"/../.. && pwd -P) -[[ -z ${COMMON_SOURCED} ]] && source "${OPENIM_ROOT}"/scripts/install/common.sh - -SERVER_NAME="openim-api" - -readonly OPENIM_API_PORT_TARGETS=( - ${API_OPENIM_PORT} -) -readonly OPENIM_API_PORT_LISTARIES=("${OPENIM_API_PORT_TARGETS[@]##*/}") - -readonly OPENIM_API_SERVICE_TARGETS=( - openim-api -) -readonly OPENIM_API_SERVICE_LISTARIES=("${OPENIM_API_SERVICE_TARGETS[@]##*/}") - -readonly OPENIM_API_PROMETHEUS_PORT_TARGETS=( - ${API_PROM_PORT} -) -readonly OPENIM_API_PROMETHEUS_PORT_LISTARIES=("${OPENIM_API_PROMETHEUS_PORT_TARGETS[@]##*/}") - -function openim::api::start() { - rm -rf "$TMP_LOG_FILE" - - echo "++ OPENIM_API_SERVICE_LISTARIES: ${OPENIM_API_SERVICE_LISTARIES[@]}" - echo "++ OPENIM_API_PORT_LISTARIES: ${OPENIM_API_PORT_LISTARIES[@]}" - echo "++ OpenIM API config path: ${OPENIM_API_CONFIG}" - - openim::log::info "Starting ${SERVER_NAME} ..." - - readonly OPENIM_API_SERVER_LIBRARIES="${OPENIM_OUTPUT_HOSTBIN}/${SERVER_NAME}" - - - printf "+------------------------+--------------+\n" - printf "| Service Name | Port |\n" - printf "+------------------------+--------------+\n" - - - local length=${#OPENIM_API_SERVICE_LISTARIES[@]} - for ((i=0; i> "${LOG_FILE}" 2> >(tee -a "${STDERR_LOG_FILE}" "$TMP_LOG_FILE" >&2) & - nohup ${cmd} >> "${LOG_FILE}" 2> >(tee -a "$TMP_LOG_FILE" | while read line; do echo -e "\e[31m${line}\e[0m"; done >&2) >> "${LOG_FILE}" 2>&1 & - if [ $? -ne 0 ]; then - openim::log::error_exit "Failed to start ${binary_name} on port ${service_port}." - return 1 - fi - return 0 -} - -###################################### Linux Systemd ###################################### -SYSTEM_FILE_PATH="/etc/systemd/system/${SERVER_NAME}.service" - -# Print the necessary information after installation -function openim::api::info() { -cat << EOF -openim-api listen on: ${OPENIM_API_HOST}:${API_OPENIM_PORT} -EOF -} - -# install openim-api -function openim::api::install() { - openim::log::info "Installing ${SERVER_NAME} ..." - - pushd "${OPENIM_ROOT}" - - # 1. Build openim-api - make build BINS=${SERVER_NAME} - openim::common::sudo "cp -r ${OPENIM_OUTPUT_HOSTBIN}/${SERVER_NAME} ${OPENIM_INSTALL_DIR}/${SERVER_NAME}" - openim::log::status "${SERVER_NAME} binary: ${OPENIM_INSTALL_DIR}/${SERVER_NAME}/${SERVER_NAME}" - - # 2. Generate and install the openim-api configuration file (config) - openim::log::status "${SERVER_NAME} config file: ${OPENIM_CONFIG_DIR}/config.yaml" - - # 3. Create and install the ${SERVER_NAME} systemd unit file - echo ${LINUX_PASSWORD} | sudo -S bash -c \ - "SERVER_NAME=${SERVER_NAME} ./scripts/genconfig.sh ${ENV_FILE} deployments/templates/openim.service > ${SYSTEM_FILE_PATH}" - openim::log::status "${SERVER_NAME} systemd file: ${SYSTEM_FILE_PATH}" - - # 4. Start the openim-api service - openim::common::sudo "systemctl daemon-reload" - openim::common::sudo "systemctl restart ${SERVER_NAME}" - openim::common::sudo "systemctl enable ${SERVER_NAME}" - openim::api::status || return 1 - openim::api::info - - openim::log::info "install ${SERVER_NAME} successfully" - popd -} - -# Unload -function openim::api::uninstall() { - openim::log::info "Uninstalling ${SERVER_NAME} ..." - - set +o errexit - openim::common::sudo "systemctl stop ${SERVER_NAME}" - openim::common::sudo "systemctl disable ${SERVER_NAME}" - openim::common::sudo "rm -f ${OPENIM_INSTALL_DIR}/${SERVER_NAME}" - openim::common::sudo "rm -f ${OPENIM_CONFIG_DIR}/${SERVER_NAME}.yaml" - openim::common::sudo "rm -f /etc/systemd/system/${SERVER_NAME}.service" - - openim::log::info "uninstall ${SERVER_NAME} successfully" -} - -# Status Check -function openim::api::status() { - openim::log::info "Checking ${SERVER_NAME} status ..." - - # Check the running status of the ${SERVER_NAME}. If active (running) is displayed, the ${SERVER_NAME} is started successfully. - systemctl status ${SERVER_NAME}|grep -q 'active' || { - openim::log::error "${SERVER_NAME} failed to start, maybe not installed properly" - return 1 - } - - openim::util::check_ports ${OPENIM_API_PORT_LISTARIES[@]} -} - -if [[ "$*" =~ openim::api:: ]];then - eval $* -fi diff --git a/scripts/install/openim-crontask.sh b/scripts/install/openim-crontask.sh deleted file mode 100755 index d785b37a4e..0000000000 --- a/scripts/install/openim-crontask.sh +++ /dev/null @@ -1,126 +0,0 @@ -#!/usr/bin/env bash - -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -# OpenIM CronTask Control Script -# -# Description: -# This script provides a control interface for the OpenIM CronTask service within a Linux environment. It supports two installation methods: installation via function calls to systemctl, and direct installation through background processes. -# -# Features: -# 1. Robust error handling leveraging Bash built-ins such as 'errexit', 'nounset', and 'pipefail'. -# 2. Capability to source common utility functions and configurations, ensuring environmental consistency. -# 3. Comprehensive logging tools, offering clear operational insights. -# 4. Support for creating, managing, and interacting with Linux systemd services. -# 5. Mechanisms to verify the successful running of the service. -# -# Usage: -# 1. Direct Script Execution: -# This will start the OpenIM CronTask directly through a background process. -# Example: ./openim-crontask.sh openim::crontask::start -# -# 2. Controlling through Functions for systemctl operations: -# Specific operations like installation, uninstallation, and status check can be executed by passing the respective function name as an argument to the script. -# Example: ./openim-crontask.sh openim::crontask::install -# -# Note: Ensure that the appropriate permissions and environmental variables are set prior to script execution. -# - -OPENIM_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")"/../.. && pwd -P) -[[ -z ${COMMON_SOURCED} ]] && source "${OPENIM_ROOT}"/scripts/install/common.sh - -SERVER_NAME="openim-crontask" - -function openim::crontask::start() { - - rm -rf "$TMP_LOG_FILE" - - openim::log::info "Start OpenIM Cron, binary root: ${SERVER_NAME}" - openim::log::status "Start OpenIM Cron, path: ${OPENIM_CRONTASK_BINARY}" - - openim::log::status "start cron_task process, path: ${OPENIM_CRONTASK_BINARY}" - #nohup ${OPENIM_CRONTASK_BINARY} -c ${OPENIM_PUSH_CONFIG} >> ${LOG_FILE} 2> >(tee -a "${STDERR_LOG_FILE}" "$TMP_LOG_FILE" >&2) & - cmd="${OPENIM_CRONTASK_BINARY} -c ${OPENIM_PUSH_CONFIG}" - nohup ${cmd} >> "${LOG_FILE}" 2> >(tee -a "$TMP_LOG_FILE" | while read line; do echo -e "\e[31m${line}\e[0m"; done >&2) >> "${LOG_FILE}" 2>&1 & - return 0 - -} - -###################################### Linux Systemd ###################################### -SYSTEM_FILE_PATH="/etc/systemd/system/${SERVER_NAME}.service" - -# Print the necessary information after installation -function openim::crontask::info() { -cat << EOF -openim-crontask listen on: ${OPENIM_CRONTASK_HOST} -EOF -} - -# install openim-crontask -function openim::crontask::install() { - pushd "${OPENIM_ROOT}" - - # 1. Build openim-crontask - make build BINS=${SERVER_NAME} - - openim::common::sudo "cp -r ${OPENIM_OUTPUT_HOSTBIN}/${SERVER_NAME} ${OPENIM_INSTALL_DIR}/${SERVER_NAME}" - openim::log::status "${SERVER_NAME} binary: ${OPENIM_INSTALL_DIR}/${SERVER_NAME}/${SERVER_NAME}" - - # 2. Generate and install the openim-crontask configuration file (openim-crontask.yaml) - openim::log::status "${SERVER_NAME} config file: ${OPENIM_CONFIG_DIR}/config.yaml" - - # 3. Create and install the ${SERVER_NAME} systemd unit file - echo ${LINUX_PASSWORD} | sudo -S bash -c \ - "SERVER_NAME=${SERVER_NAME} ./scripts/genconfig.sh ${ENV_FILE} deployments/templates/openim.service > ${SYSTEM_FILE_PATH}" - openim::log::status "${SERVER_NAME} systemd file: ${SYSTEM_FILE_PATH}" - - # 4. Start the openim-crontask service - openim::common::sudo "systemctl daemon-reload" - openim::common::sudo "systemctl restart ${SERVER_NAME}" - openim::common::sudo "systemctl enable ${SERVER_NAME}" - openim::crontask::status || return 1 - openim::crontask::info - - openim::log::info "install ${SERVER_NAME} successfully" - popd -} - - -# Unload -function openim::crontask::uninstall() { - set +o errexit - openim::common::sudo "systemctl stop ${SERVER_NAME}" - openim::common::sudo "systemctl disable ${SERVER_NAME}" - openim::common::sudo "rm -f ${OPENIM_INSTALL_DIR}/${SERVER_NAME}" - openim::common::sudo "rm -f ${OPENIM_CONFIG_DIR}/${SERVER_NAME}.yaml" - openim::common::sudo "rm -f /etc/systemd/system/${SERVER_NAME}.service" - - openim::log::info "uninstall ${SERVER_NAME} successfully" -} - -# Status Check -function openim::crontask::status() { - # Check the running status of the ${SERVER_NAME}. If active (running) is displayed, the ${SERVER_NAME} is started successfully. - if systemctl is-active --quiet "${SERVER_NAME}"; then - openim::log::info "${SERVER_NAME} is running successfully." - else - openim::log::error "${SERVER_NAME} failed to start, maybe not installed properly" - return 1 - fi -} - -if [[ "$*" =~ openim::crontask:: ]];then - eval $* -fi diff --git a/scripts/install/openim-msggateway.sh b/scripts/install/openim-msggateway.sh deleted file mode 100755 index ed79108055..0000000000 --- a/scripts/install/openim-msggateway.sh +++ /dev/null @@ -1,141 +0,0 @@ -#!/usr/bin/env bash -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# Common utilities, variables and checks for all build scripts. - - - - -OPENIM_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")"/../.. && pwd -P) -[[ -z ${COMMON_SOURCED} ]] && source "${OPENIM_ROOT}"/scripts/install/common.sh - - -SERVER_NAME="openim-msggateway" - -function openim::msggateway::start() { - - rm -rf "$TMP_LOG_FILE" - - openim::log::info "Start OpenIM Msggateway, binary root: ${SERVER_NAME}" - openim::log::status "Start OpenIM Msggateway, path: ${OPENIM_MSGGATEWAY_BINARY}" - - - # OpenIM message gateway service port - OPENIM_MESSAGE_GATEWAY_PORTS=$(openim::util::list-to-string ${OPENIM_MESSAGE_GATEWAY_PORT} ) - read -a OPENIM_MSGGATEWAY_PORTS_ARRAY <<< ${OPENIM_MESSAGE_GATEWAY_PORTS} - openim::util::stop_services_on_ports ${OPENIM_MSGGATEWAY_PORTS_ARRAY[*]} - # OpenIM WS port - OPENIM_WS_PORTS=$(openim::util::list-to-string ${OPENIM_WS_PORT} ) - read -a OPENIM_WS_PORTS_ARRAY <<< ${OPENIM_WS_PORTS} - - # Message Gateway Prometheus port of the service - MSG_GATEWAY_PROM_PORTS=$(openim::util::list-to-string ${MSG_GATEWAY_PROM_PORT} ) - read -a MSG_GATEWAY_PROM_PORTS_ARRAY <<< ${MSG_GATEWAY_PROM_PORTS} - - openim::log::status "OpenIM Mssage Getway ports: ${OPENIM_MESSAGE_GATEWAY_PORTS[*]}" - openim::log::status "OpenIM WS ports: ${OPENIM_WS_PORTS[*]}" - openim::log::status "OpenIM Prometheus ports: ${MSG_GATEWAY_PROM_PORTS[*]}" - - openim::log::status "OpenIM Msggateway config path: ${OPENIM_MSGGATEWAY_CONFIG}" - - if [ ${#OPENIM_MSGGATEWAY_PORTS_ARRAY[@]} -ne ${#OPENIM_WS_PORTS_ARRAY[@]} ]; then - openim::log::error_exit "ws_ports does not match push_rpc_ports or prome_ports in quantity!!!" - fi - - for ((i = 0; i < ${#OPENIM_WS_PORTS_ARRAY[@]}; i++)); do - openim::log::info "start push process, port: ${OPENIM_MSGGATEWAY_PORTS_ARRAY[$i]}, prometheus port: ${MSG_GATEWAY_PROM_PORTS_ARRAY[$i]}" - - PROMETHEUS_PORT_OPTION="" - if [[ -n "${MSG_GATEWAY_PROM_PORTS_ARRAY[$i]}" ]]; then - PROMETHEUS_PORT_OPTION="--prometheus_port ${MSG_GATEWAY_PROM_PORTS_ARRAY[$i]}" - fi - cmd="${OPENIM_MSGGATEWAY_BINARY} --port ${OPENIM_MSGGATEWAY_PORTS_ARRAY[$i]} --ws_port ${OPENIM_WS_PORTS_ARRAY[$i]} $PROMETHEUS_PORT_OPTION -c ${OPENIM_MSGGATEWAY_CONFIG}" - nohup ${cmd} >> "${LOG_FILE}" 2> >(tee -a "$TMP_LOG_FILE" | while read line; do echo -e "\e[31m${line}\e[0m"; done >&2) >> "${LOG_FILE}" 2>&1 & - # nohup ${OPENIM_MSGGATEWAY_BINARY} --port ${OPENIM_MSGGATEWAY_PORTS_ARRAY[$i]} --ws_port ${OPENIM_WS_PORTS_ARRAY[$i]} $PROMETHEUS_PORT_OPTION -c ${OPENIM_MSGGATEWAY_CONFIG} >> ${LOG_FILE} 2> >(tee -a "${STDERR_LOG_FILE}" "$TMP_LOG_FILE" >&2) & - - done - return 0 -} - -###################################### Linux Systemd ###################################### -SYSTEM_FILE_PATH="/etc/systemd/system/${SERVER_NAME}.service" - -# Print the necessary information after installation -function openim::msggateway::info() { -cat << EOF -openim-msggateway listen on: ${OPENIM_MSGGATEWAY_HOST}:${OPENIM_MESSAGE_GATEWAY_PORT} -EOF -} - -# install openim-msggateway -function openim::msggateway::install() { - pushd "${OPENIM_ROOT}" - - # 1. Build openim-msggateway - make build BINS=${SERVER_NAME} - openim::common::sudo "cp -r ${OPENIM_OUTPUT_HOSTBIN}/${SERVER_NAME} ${OPENIM_INSTALL_DIR}/${SERVER_NAME}" - openim::log::status "${SERVER_NAME} binary: ${OPENIM_INSTALL_DIR}/${SERVER_NAME}" - - # 2. Generate and install the openim-msggateway configuration file (openim-msggateway.yaml) - # nono - - # 3. Create and install the ${SERVER_NAME} systemd unit file - echo ${LINUX_PASSWORD} | sudo -S bash -c \ - "SERVER_NAME=${SERVER_NAME} ./scripts/genconfig.sh ${ENV_FILE} deployments/templates/openim.service > ${SYSTEM_FILE_PATH}" - openim::log::status "${SERVER_NAME} systemd file: ${SYSTEM_FILE_PATH}" - - # 4. Start the openim-msggateway service - openim::common::sudo "systemctl daemon-reload" - openim::common::sudo "systemctl restart ${SERVER_NAME}" - openim::common::sudo "systemctl enable ${SERVER_NAME}" - openim::msggateway::status || return 1 - openim::msggateway::info - - openim::log::info "install ${SERVER_NAME} successfully" - popd -} - - -# Unload -function openim::msggateway::uninstall() { - set +o errexit - openim::common::sudo "systemctl stop ${SERVER_NAME}" - openim::common::sudo "systemctl disable ${SERVER_NAME}" - openim::common::sudo "rm -f ${OPENIM_INSTALL_DIR}/${SERVER_NAME}" - openim::common::sudo "rm -f ${OPENIM_CONFIG_DIR}/${SERVER_NAME}.yaml" - openim::common::sudo "rm -f /etc/systemd/system/${SERVER_NAME}.service" - - openim::log::info "uninstall ${SERVER_NAME} successfully" -} - -# Status Check -function openim::msggateway::status() { - # Check the running status of the ${SERVER_NAME}. If active (running) is displayed, the ${SERVER_NAME} is started successfully. - systemctl status ${SERVER_NAME}|grep -q 'active' || { - openim::log::error "${SERVER_NAME} failed to start, maybe not installed properly" - - return 1 - } - - # The listening port is hardcode in the configuration file - if echo | telnet ${OPENIM_MSGGATEWAY_HOST} ${OPENIM_MESSAGE_GATEWAY_PORT} 2>&1|grep refused &>/dev/null;then - openim::log::error "cannot access health check port, ${SERVER_NAME} maybe not startup" - return 1 - fi -} - -if [[ "$*" =~ openim::msggateway:: ]];then - eval $* -fi diff --git a/scripts/install/openim-msgtransfer.sh b/scripts/install/openim-msgtransfer.sh deleted file mode 100755 index 8403d1cfcf..0000000000 --- a/scripts/install/openim-msgtransfer.sh +++ /dev/null @@ -1,186 +0,0 @@ -#!/usr/bin/env bash -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# Use: -# ./scripts/install/openim-msgtransfer.sh openim::msgtransfer::start - -# Common utilities, variables and checks for all build scripts. - - - - -OPENIM_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")"/../.. && pwd -P) -[[ -z ${COMMON_SOURCED} ]] && source "${OPENIM_ROOT}"/scripts/install/common.sh - -SERVER_NAME="openim-msgtransfer" - -function openim::msgtransfer::start() { - - rm -rf "$TMP_LOG_FILE" - - openim::log::info "Start OpenIM Msggateway, binary root: ${SERVER_NAME}" - openim::log::status "Start OpenIM Msggateway, path: ${OPENIM_MSGTRANSFER_BINARY}" - - # Message Transfer Prometheus port list - MSG_TRANSFER_PROM_PORTS=(openim::util::list-to-string ${MSG_TRANSFER_PROM_PORT} ) - - openim::log::status "OpenIM Prometheus ports: ${MSG_TRANSFER_PROM_PORTS[*]}" - - openim::log::status "OpenIM Msggateway config path: ${OPENIM_MSGTRANSFER_CONFIG}" - - openim::log::info "openim maggateway num: ${OPENIM_MSGGATEWAY_NUM}" - - if [ "${OPENIM_MSGGATEWAY_NUM}" -lt 1 ]; then - opeim::log::error "OPENIM_MSGGATEWAY_NUM must be greater than 0" - fi - - if [ ${OPENIM_MSGGATEWAY_NUM} -ne $((${#MSG_TRANSFER_PROM_PORTS[@]} - 1)) ]; then - openim::log::error "OPENIM_MSGGATEWAY_NUM must be equal to the number of MSG_TRANSFER_PROM_PORTS" - fi - - for (( i=0; i<$OPENIM_MSGGATEWAY_NUM; i++ )) do - openim::log::info "prometheus port: ${MSG_TRANSFER_PROM_PORTS[$i]}" - PROMETHEUS_PORT_OPTION="" - if [[ -n "${MSG_TRANSFER_PROM_PORTS[$i+1]}" ]]; then - PROMETHEUS_MSG_TRANSFER_PORT="${MSG_TRANSFER_PROM_PORTS[$i+1]%,}" - openim::util::stop_services_on_ports ${PROMETHEUS_MSG_TRANSFER_PORT} - PROMETHEUS_PORT_OPTION="--prometheus_port ${PROMETHEUS_MSG_TRANSFER_PORT}" - fi - cmd="${OPENIM_MSGTRANSFER_BINARY} ${PROMETHEUS_PORT_OPTION} -c ${OPENIM_MSGTRANSFER_CONFIG} -n ${i}" - #nohup ${cmd} >> "${LOG_FILE}" 2> >(tee -a "$TMP_LOG_FILE" | while read line; do echo -e "\e[31m${line}\e[0m"; done >&2) >/dev/null & - nohup ${cmd} >> "${LOG_FILE}" 2> >(tee -a "$TMP_LOG_FILE" | while read line; do echo -e "\e[31m${line}\e[0m"; done >&2) >> "${LOG_FILE}" 2>&1 & - #nohup ${OPENIM_MSGTRANSFER_BINARY} ${PROMETHEUS_PORT_OPTION} -c ${OPENIM_MSGTRANSFER_CONFIG} -n ${i} >> ${LOG_FILE} 2> >(tee -a "${STDERR_LOG_FILE}" "$TMP_LOG_FILE" >&2) & - done - return 0 -} - -function openim::msgtransfer::check() { - PIDS=$(pgrep -f "${OPENIM_OUTPUT_HOSTBIN}/openim-msgtransfer") - if [ -z "$PIDS" ]; then - NUM_PROCESSES=0 - else - NUM_PROCESSES=$(echo "$PIDS" | wc -l) - fi - if [ "$NUM_PROCESSES" -eq "$OPENIM_MSGGATEWAY_NUM" ]; then - for PID in $PIDS; do - if [[ "$OSTYPE" == "linux-gnu"* ]]; then - ps -p $PID -o pid,cmd - elif [[ "$OSTYPE" == "darwin"* ]]; then - ps -p $PID -o pid,comm - else - openim::log::error "Unsupported OS type: $OSTYPE" - fi - done - else - openim::log::error "Expected $OPENIM_MSGGATEWAY_NUM OpenIM msgtransfer processes, but found $NUM_PROCESSES msgtransfer processes running" - return 1 - fi - return 0 -} - -function openim::msgtransfer::check_for_stop() { - PIDS=$(pgrep -f "${OPENIM_OUTPUT_HOSTBIN}/openim-msgtransfer") || PIDS="0" - if [ "$PIDS" = "0" ]; then - return 0 - fi - - NUM_PROCESSES=$(echo "$PIDS" | wc -l | xargs) - - if [ "$NUM_PROCESSES" -gt 0 ]; then - openim::log::error "Found $NUM_PROCESSES processes for $OPENIM_OUTPUT_HOSTBIN/openim-msgtransfer" - for PID in $PIDS; do - if [[ "$OSTYPE" == "linux-gnu"* ]]; then - echo -e "\033[31m$(ps -p $PID -o pid,cmd)\033[0m" - elif [[ "$OSTYPE" == "darwin"* ]]; then - echo -e "\033[31m$(ps -p $PID -o pid,comm)\033[0m" - else - openim::log::error "Unsupported OS type: $OSTYPE" - fi - done - openim::log::error "Processes have not been stopped properly." - else - openim::log::success "All openim-msgtransfer processes have been stopped properly." - fi - return 0 -} - - -###################################### Linux Systemd ###################################### -SYSTEM_FILE_PATH="/etc/systemd/system/${SERVER_NAME}.service" - -# Print the necessary information after installation -function openim::msgtransfer::info() { -cat << EOF -openim-msgtransfer listen on: ${OPENIM_MSGTRANSFER_HOST} -EOF -} - -# install openim-msgtransfer -function openim::msgtransfer::install() { - pushd "${OPENIM_ROOT}" - - # 1. Build openim-msgtransfer - make build BINS=${SERVER_NAME} - - openim::common::sudo "cp -r ${OPENIM_OUTPUT_HOSTBIN}/${SERVER_NAME} ${OPENIM_INSTALL_DIR}/${SERVER_NAME}" - openim::log::status "${SERVER_NAME} binary: ${OPENIM_INSTALL_DIR}/${SERVER_NAME}/${SERVER_NAME}" - - openim::log::status "${SERVER_NAME} binary: ${OPENIM_INSTALL_DIR}/bin/${SERVER_NAME}" - - # 2. Generate and install the openim-msgtransfer configuration file (openim-msgtransfer.yaml) - # nono - - # 3. Create and install the ${SERVER_NAME} systemd unit file - echo ${LINUX_PASSWORD} | sudo -S bash -c \ - "SERVER_NAME=${SERVER_NAME} ./scripts/genconfig.sh ${ENV_FILE} deployments/templates/openim.service > ${SYSTEM_FILE_PATH}" - openim::log::status "${SERVER_NAME} systemd file: ${SYSTEM_FILE_PATH}" - - # 4. Start the openim-msgtransfer service - openim::common::sudo "systemctl daemon-reload" - openim::common::sudo "systemctl restart ${SERVER_NAME}" - openim::common::sudo "systemctl enable ${SERVER_NAME}" - openim::msgtransfer::status || return 1 - openim::msgtransfer::info - - openim::log::info "install ${SERVER_NAME} successfully" - popd -} - - -# Unload -function openim::msgtransfer::uninstall() { - set +o errexit - openim::common::sudo "systemctl stop ${SERVER_NAME}" - openim::common::sudo "systemctl disable ${SERVER_NAME}" - openim::common::sudo "rm -f ${OPENIM_INSTALL_DIR}/${SERVER_NAME}" - openim::common::sudo "rm -f ${OPENIM_CONFIG_DIR}/${SERVER_NAME}.yaml" - openim::common::sudo "rm -f /etc/systemd/system/${SERVER_NAME}.service" - - openim::log::info "uninstall ${SERVER_NAME} successfully" -} - -# Status Check -function openim::msgtransfer::status() { - # Check the running status of the ${SERVER_NAME}. If active (running) is displayed, the ${SERVER_NAME} is started successfully. - if systemctl is-active --quiet "${SERVER_NAME}"; then - openim::log::info "${SERVER_NAME} is running successfully." - else - openim::log::error "${SERVER_NAME} failed to start, maybe not installed properly" - return 1 - fi -} - -if [[ "$*" =~ openim::msgtransfer:: ]];then - eval $* -fi diff --git a/scripts/install/openim-push.sh b/scripts/install/openim-push.sh deleted file mode 100755 index aafb24c7b0..0000000000 --- a/scripts/install/openim-push.sh +++ /dev/null @@ -1,150 +0,0 @@ -#!/usr/bin/env bash -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -# OpenIM Push Control Script -# -# Description: -# This script provides a control interface for the OpenIM Push service within a Linux environment. It supports two installation methods: installation via function calls to systemctl, and direct installation through background processes. -# -# Features: -# 1. Robust error handling leveraging Bash built-ins such as 'errexit', 'nounset', and 'pipefail'. -# 2. Capability to source common utility functions and configurations, ensuring environmental consistency. -# 3. Comprehensive logging tools, offering clear operational insights. -# 4. Support for creating, managing, and interacting with Linux systemd services. -# 5. Mechanisms to verify the successful running of the service. -# -# Usage: -# 1. Direct Script Execution: -# This will start the OpenIM push directly through a background process. -# Example: ./openim-push.sh -# -# 2. Controlling through Functions for systemctl operations: -# Specific operations like installation, uninstallation, and status check can be executed by passing the respective function name as an argument to the script. -# Example: ./openim-push.sh openim::push::install -# -# ENVIRONMENT VARIABLES: -# export OPENIM_PUSH_BINARY="8080 8081 8082" -# export OPENIM_PUSH_PORT="9090 9091 9092" -# -# Note: Ensure that the appropriate permissions and environmental variables are set prior to script execution. -# - - - - -OPENIM_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")"/../.. && pwd -P) -[[ -z ${COMMON_SOURCED} ]] && source "${OPENIM_ROOT}"/scripts/install/common.sh - -SERVER_NAME="openim-push" - -function openim::push::start() { - - rm -rf "$TMP_LOG_FILE" - - openim::log::status "Start OpenIM Push, binary root: ${SERVER_NAME}" - openim::log::info "Start OpenIM Push, path: ${OPENIM_PUSH_BINARY}" - - openim::log::status "prepare start push process, path: ${OPENIM_PUSH_BINARY}" - openim::log::status "prepare start push process, port: ${OPENIM_PUSH_PORT}, prometheus port: ${PUSH_PROM_PORT}" - - OPENIM_PUSH_PORTS_ARRAY=$(openim::util::list-to-string ${OPENIM_PUSH_PORT} ) - PUSH_PROM_PORTS_ARRAY=$(openim::util::list-to-string ${PUSH_PROM_PORT} ) - - openim::log::status "push port list: ${OPENIM_PUSH_PORTS_ARRAY[@]}" - openim::log::status "prometheus port list: ${PUSH_PROM_PORTS_ARRAY[@]}" - - if [ ${#OPENIM_PUSH_PORTS_ARRAY[@]} -ne ${#PUSH_PROM_PORTS_ARRAY[@]} ]; then - openim::log::error "The length of the two port lists is different!" - fi - - for (( i=0; i<${#OPENIM_PUSH_PORTS_ARRAY[@]}; i++ )); do - openim::log::info "start push process, port: ${OPENIM_PUSH_PORTS_ARRAY[$i]}, prometheus port: ${PUSH_PROM_PORTS_ARRAY[$i]}" - cmd="${OPENIM_PUSH_BINARY} --port ${OPENIM_PUSH_PORTS_ARRAY[$i]} -c ${OPENIM_PUSH_CONFIG} --prometheus_port ${PUSH_PROM_PORTS_ARRAY[$i]}" - #nohup ${cmd} >> "${LOG_FILE}" 2> >(tee -a "$TMP_LOG_FILE" | while read line; do echo -e "\e[31m${line}\e[0m"; done >&2) >/dev/null & - nohup ${cmd} >> "${LOG_FILE}" 2> >(tee -a "$TMP_LOG_FILE" | while read line; do echo -e "\e[31m${line}\e[0m"; done >&2) >> "${LOG_FILE}" 2>&1 & - #nohup ${OPENIM_PUSH_BINARY} --port ${OPENIM_PUSH_PORTS_ARRAY[$i]} -c ${OPENIM_PUSH_CONFIG} --prometheus_port ${PUSH_PROM_PORTS_ARRAY[$i]} >> ${LOG_FILE} 2> >(tee -a "${STDERR_LOG_FILE}" "$TMP_LOG_FILE" >&2) & - done - return 0 - -} - -###################################### Linux Systemd ###################################### -SYSTEM_FILE_PATH="/etc/systemd/system/${SERVER_NAME}.service" - -# Print the necessary information after installation -function openim::push::info() { -cat << EOF -openim-push listen on: ${OPENIM_PUSH_HOST}:${OPENIM_PUSH_PORT} -EOF -} - -# install openim-push -function openim::push::install() { - pushd "${OPENIM_ROOT}" - - # 1. Build openim-push - make build BINS=${SERVER_NAME} - openim::common::sudo "cp -r ${OPENIM_OUTPUT_HOSTBIN}/${SERVER_NAME} ${OPENIM_INSTALL_DIR}/${SERVER_NAME}" - openim::log::status "${SERVER_NAME} binary: ${OPENIM_INSTALL_DIR}/${SERVER_NAME}/${SERVER_NAME}" - - # 2. Generate and install the openim-push configuration file (config) - openim::log::status "${SERVER_NAME} config file: ${OPENIM_CONFIG_DIR}/config.yaml" - - # 3. Create and install the ${SERVER_NAME} systemd unit file - echo ${LINUX_PASSWORD} | sudo -S bash -c \ - "SERVER_NAME=${SERVER_NAME} ./scripts/genconfig.sh ${ENV_FILE} deployments/templates/openim.service > ${SYSTEM_FILE_PATH}" - openim::log::status "${SERVER_NAME} systemd file: ${SYSTEM_FILE_PATH}" - - # 4. Start the openim-push service - openim::common::sudo "systemctl daemon-reload" - openim::common::sudo "systemctl restart ${SERVER_NAME}" - openim::common::sudo "systemctl enable ${SERVER_NAME}" - openim::push::status || return 1 - openim::push::info - - openim::log::info "install ${SERVER_NAME} successfully" - popd -} - -# Unload -function openim::push::uninstall() { - set +o errexit - openim::common::sudo "systemctl stop ${SERVER_NAME}" - openim::common::sudo "systemctl disable ${SERVER_NAME}" - openim::common::sudo "rm -f ${OPENIM_INSTALL_DIR}/${SERVER_NAME}" - openim::common::sudo "rm -f ${OPENIM_CONFIG_DIR}/${SERVER_NAME}.yaml" - openim::common::sudo "rm -f /etc/systemd/system/${SERVER_NAME}.service" - - openim::log::info "uninstall ${SERVER_NAME} successfully" -} - -# Status Check -function openim::push::status() { - # Check the running status of the ${SERVER_NAME}. If active (running) is displayed, the ${SERVER_NAME} is started successfully. - systemctl status ${SERVER_NAME}|grep -q 'active' || { - openim::log::error "${SERVER_NAME} failed to start, maybe not installed properly" - return 1 - } - - # The listening port is hardcode in the configuration file - if echo | telnet ${OPENIM_MSGGATEWAY_HOST} ${OPENIM_PUSH_PORT} 2>&1|grep refused &>/dev/null;then # Assuming a different port for push - openim::log::error "cannot access health check port, ${SERVER_NAME} maybe not startup" - return 1 - fi -} - -if [[ "$*" =~ openim::push:: ]];then - eval $* -fi diff --git a/scripts/install/openim-rpc.sh b/scripts/install/openim-rpc.sh deleted file mode 100755 index 1265a11c67..0000000000 --- a/scripts/install/openim-rpc.sh +++ /dev/null @@ -1,254 +0,0 @@ -#!/usr/bin/env bash - -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -# OpenIM RPC Service Control Script -# -# Description: -# This script provides a control interface for the OpenIM RPC service within a Linux environment. It offers functionalities to start multiple RPC services, each denoted by their respective names under openim::rpc::service_name. -# -# Features: -# 1. Robust error handling using Bash built-ins like 'errexit', 'nounset', and 'pipefail'. -# 2. The capability to source common utility functions and configurations to ensure uniform environmental settings. -# 3. Comprehensive logging functionalities, providing a detailed understanding of operational processes. -# 4. Provision for declaring and managing a set of RPC services, each associated with its unique name and corresponding ports. -# 5. The ability to define and associate Prometheus ports for service monitoring purposes. -# 6. Functionalities to start each RPC service, along with its designated ports, in a sequence. -# -# Usage: -# 1. Direct Script Execution: -# This initiates all the RPC services declared under the function openim::rpc::service_name. -# Example: ./openim-rpc-{rpc-name}.sh openim::rpc::start -# 2. Controlling through Functions for systemctl operations: -# Specific operations like installation, uninstallation, and status check can be executed by passing the respective function name as an argument to the script. -# Example: ./openim-rpc-{rpc-name}.sh openim::rpc::install -# -# Note: Before executing this script, ensure that the necessary permissions are granted and relevant environmental variables are set. -# - -OPENIM_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")"/../.. && pwd -P) -[[ -z ${COMMON_SOURCED} ]] && source "${OPENIM_ROOT}"/scripts/install/common.sh - -SERVER_NAME="openim-rpc" -readonly OPENIM_RPC_CONFIG="${OPENIM_ROOT}"/config - -openim::rpc::service_name() { - local targets=( - openim-rpc-user - openim-rpc-friend - openim-rpc-msg - openim-rpc-group - openim-rpc-auth - openim-rpc-conversation - openim-rpc-third - ) - echo "${targets[@]}" -} -IFS=" " read -ra OPENIM_RPC_SERVICE_TARGETS <<< "$(openim::rpc::service_name)" -readonly OPENIM_RPC_SERVICE_TARGETS -readonly OPENIM_RPC_SERVICE_LISTARIES=("${OPENIM_RPC_SERVICE_TARGETS[@]##*/}") - - -OPENIM_ALL_RPC_FULL_PATH=() -for target in openim::rpc::service_name; do - OPENIM_ALL_RPC_FULL_PATH+=("${OPENIM_OUTPUT_HOSTBIN}/${target}") -done -readonly OPENIM_ALL_RPC_FULL_PATH - - - - -# Make sure the environment is only called via common to avoid too much nesting -openim::rpc::service_port() { - local targets=( - ${OPENIM_USER_PORT} # User service 10110 - ${OPENIM_FRIEND_PORT} # Friend service 10120 - ${OPENIM_MESSAGE_PORT} # Message service 10130 - # ${OPENIM_MESSAGE_GATEWAY_PORT} # Message gateway 10140 - ${OPENIM_GROUP_PORT} # Group service 10150 - ${OPENIM_AUTH_PORT} # Authorization service 10160 - # ${OPENIM_PUSH_PORT} # Push service 10170 - ${OPENIM_CONVERSATION_PORT} # Conversation service 10180 - ${OPENIM_THIRD_PORT} # Third-party service 10190 - ) - echo "${targets[@]}" -} -IFS=" " read -ra OPENIM_RPC_PORT_TARGETS <<< "$(openim::rpc::service_port)" -readonly OPENIM_RPC_PORT_TARGETS -readonly OPENIM_RPC_PORT_LISTARIES=("${OPENIM_RPC_PORT_TARGETS[@]##*/}") - -openim::rpc::prometheus_port() { - # Declare an array to hold all the Prometheus ports for different services - local targets=( - ${USER_PROM_PORT} # Prometheus port for user service - ${FRIEND_PROM_PORT} # Prometheus port for friend service - ${MESSAGE_PROM_PORT} # Prometheus port for message service - ${GROUP_PROM_PORT} # Prometheus port for group service - ${AUTH_PROM_PORT} # Prometheus port for authentication service - ${CONVERSATION_PROM_PORT} # Prometheus port for conversation service - ${THIRD_PROM_PORT} # Prometheus port for third-party integrations service - ) - # Print the list of ports - echo "${targets[@]}" -} -IFS=" " read -ra OPENIM_RPC_PROM_PORT_TARGETS <<< "$(openim::rpc::prometheus_port)" -readonly OPENIM_RPC_PROM_PORT_TARGETS -readonly OPENIM_RPC_PROM_PORT_LISTARIES=("${OPENIM_RPC_PROM_PORT_TARGETS[@]##*/}") - -function openim::rpc::start() { - rm -rf "$TMP_LOG_FILE" - - echo "OPENIM_RPC_SERVICE_LISTARIES: ${OPENIM_RPC_SERVICE_LISTARIES[@]}" - echo "OPENIM_RPC_PROM_PORT_LISTARIES: ${OPENIM_RPC_PROM_PORT_LISTARIES[@]}" - echo "OPENIM_RPC_PORT_LISTARIES: ${OPENIM_RPC_PORT_LISTARIES[@]}" - - openim::log::info "Starting ${SERVER_NAME} ..." - - printf "+------------------------+-------+-----------------+\n" - printf "| Service Name | Port | Prometheus Port |\n" - printf "+------------------------+-------+-----------------+\n" - - length=${#OPENIM_RPC_SERVICE_LISTARIES[@]} - - for ((i=0; i<$length; i++)); do - printf "| %-22s | %-5s | %-15s |\n" "${OPENIM_RPC_SERVICE_LISTARIES[$i]}" "${OPENIM_RPC_PORT_LISTARIES[$i]}" "${OPENIM_RPC_PROM_PORT_LISTARIES[$i]}" - printf "+------------------------+-------+-----------------+\n" - done - - - # start all rpc services - for ((i = 0; i < ${#OPENIM_RPC_SERVICE_LISTARIES[*]}; i++)); do - - openim::log::info "OpenIM ${OPENIM_RPC_SERVICE_LISTARIES[$i]} config path: ${OPENIM_RPC_CONFIG}" - # Get the service and Prometheus ports. - OPENIM_RPC_SERVICE_PORTS=( $(openim::util::list-to-string ${OPENIM_RPC_PORT_LISTARIES[$i]}) ) - read -a OPENIM_RPC_SERVICE_PORTS_ARRAY <<< ${OPENIM_RPC_SERVICE_PORTS} - - OPENIM_RPC_PROM_PORTS=( $(openim::util::list-to-string ${OPENIM_RPC_PROM_PORT_LISTARIES[$i]}) ) - read -a OPENIM_RPC_PROM_PORTS_ARRAY <<< ${OPENIM_RPC_PROM_PORTS} - - for ((j = 0; j < ${#OPENIM_RPC_SERVICE_PORTS_ARRAY[@]}; j++)); do - openim::log::info "Starting ${OPENIM_RPC_SERVICE_LISTARIES[$i]} service, port: ${OPENIM_RPC_SERVICE_PORTS[j]}, prometheus port: ${OPENIM_RPC_PROM_PORTS[j]}, binary root: ${OPENIM_OUTPUT_HOSTBIN}/${OPENIM_RPC_SERVICE_LISTARIES[$i]}" - result=$(openim::rpc::start_service "${OPENIM_RPC_SERVICE_LISTARIES[$i]}" "${OPENIM_RPC_SERVICE_PORTS[j]}" "${OPENIM_RPC_PROM_PORTS[j]}") - if [[ $? -ne 0 ]]; then - openim::log::error "start ${SERVER_NAME} failed" - else - openim::log::info "$result" - fi - done - done - - - return 0 -} - -function openim::rpc::start_service() { - local binary_name="$1" - local service_port="$2" - local prometheus_port="$3" - - local cmd="${OPENIM_OUTPUT_HOSTBIN}/${binary_name} --port ${service_port} -c ${OPENIM_RPC_CONFIG}" - - if [ -n "${prometheus_port}" ]; then - printf "Specifying prometheus port: %s\n" "${prometheus_port}" - cmd="${cmd} --prometheus_port ${prometheus_port}" - fi - #nohup ${cmd} >> "${LOG_FILE}" 2> >(tee -a "${STDERR_LOG_FILE}" "$TMP_LOG_FILE" >&2) & - #nohup ${cmd} >> "${LOG_FILE}" 2> >(tee -a "$TMP_LOG_FILE" | while read line; do echo -e "\e[31m${line}\e[0m"; done >&2) >/dev/null & - nohup ${cmd} >> "${LOG_FILE}" 2> >(tee -a "$TMP_LOG_FILE" | while read line; do echo -e "\e[31m${line}\e[0m"; done >&2) >> "${LOG_FILE}" 2>&1 & - return 0 -} - -###################################### Linux Systemd ###################################### -declare -A SYSTEM_FILE_PATHS -for service in "${OPENIM_RPC_SERVICE_LISTARIES[@]}"; do - SYSTEM_FILE_PATHS["$service"]="/etc/systemd/system/${service}.service" -done - -# Print the necessary information after installation -function openim::rpc::info() { - for service in "${OPENIM_RPC_SERVICE_LISTARIES[@]}"; do - echo "${service} listen on: ${OPENIM_RPC_PORT_LISTARIES[@]}" - done -} - -# install openim-rpc -function openim::rpc::install() { - pushd "${OPENIM_ROOT}" - - # 1. Build openim-rpc - for service in "${OPENIM_RPC_SERVICE_LISTARIES[@]}"; do - make build BINS=${service} - openim::common::sudo "cp -r ${OPENIM_OUTPUT_HOSTBIN}/${service} ${OPENIM_INSTALL_DIR}/${service}" - openim::log::status "${service} binary: ${OPENIM_INSTALL_DIR}/${service}/${service}" - done - - # 2. Generate and install the openim-rpc configuration file (config) - openim::log::status "openim-rpc config file: ${OPENIM_CONFIG_DIR}/config.yaml" - - # 3. Create and install the systemd unit files - for service in "${OPENIM_RPC_SERVICE_LISTARIES[@]}"; do - echo ${LINUX_PASSWORD} | sudo -S bash -c \ - "SERVER_NAME=${service} ./scripts/genconfig.sh ${ENV_FILE} deployments/templates/openim.service > ${SYSTEM_FILE_PATHS[$service]}" - openim::log::status "${service} systemd file: ${SYSTEM_FILE_PATHS[$service]}" - done - - # 4. Start the openim-rpc services - openim::common::sudo "systemctl daemon-reload" - for service in "${OPENIM_RPC_SERVICE_LISTARIES[@]}"; do - openim::common::sudo "systemctl restart ${service}" - openim::common::sudo "systemctl enable ${service}" - done - openim::rpc::status || return 1 - openim::rpc::info - - openim::log::info "install openim-rpc successfully" - popd -} - -# Unload -function openim::rpc::uninstall() { - set +o errexit - for service in "${OPENIM_RPC_SERVICE_LISTARIES[@]}"; do - openim::common::sudo "systemctl stop ${service}" - openim::common::sudo "systemctl disable ${service}" - openim::common::sudo "rm -f ${OPENIM_INSTALL_DIR}/${service}" - openim::common::sudo "rm -f ${OPENIM_CONFIG_DIR}/${service}.yaml" - openim::common::sudo "rm -f ${SYSTEM_FILE_PATHS[$service]}" - done - - openim::log::info "uninstall openim-rpc successfully" -} - -# Status Check -function openim::rpc::status() { - for service in "${OPENIM_RPC_SERVICE_LISTARIES[@]}"; do - # Check the running status of the ${service}. If active (running) is displayed, the ${service} is started successfully. - systemctl status ${service}|grep -q 'active' || { - openim::log::error "${service} failed to start, maybe not installed properly" - return 1 - } - - # The listening port is hardcoded in the configuration file - if echo | telnet ${OPENIM_MSGGATEWAY_HOST} ${OPENIM_RPC_PORT_LISTARIES[@]} 2>&1|grep refused &>/dev/null;then - openim::log::error "cannot access health check port, ${service} maybe not startup" - return 1 - fi - done -} - -if [[ "$*" =~ openim::rpc:: ]];then - eval $* -fi diff --git a/scripts/install/openim-tools.sh b/scripts/install/openim-tools.sh deleted file mode 100755 index 57de9772d3..0000000000 --- a/scripts/install/openim-tools.sh +++ /dev/null @@ -1,179 +0,0 @@ -#!/usr/bin/env bash - -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -# OpenIM Tools Control Script -# -# Description: -# This script is responsible for managing the lifecycle of OpenIM tools, which include starting, stopping, -# and handling pre and post operations. It's designed to be modular and extensible, ensuring that the -# individual operations can be managed separately, and integrated seamlessly with Linux systemd. -# -# Features: -# 1. Robust error handling using Bash built-ins like 'errexit', 'nounset', and 'pipefail'. -# 2. The capability to source common utility functions and configurations to ensure uniform environmental settings. -# 3. Comprehensive logging functionalities, providing a detailed understanding of operational processes. -# 4. Provision for declaring and managing a set of OpenIM tools, each associated with its unique name and corresponding ports. -# 5. The ability to define and associate Prometheus ports for service monitoring purposes. -# 6. Functionalities to start each OpenIM tool, along with its designated ports, in a sequence. -# -# Usage: -# 1. Direct Script Execution: -# This initiates all the OpenIM tools declared under the function openim::tools::service_name. -# Example: ./openim-tools.sh openim::tools::start -# 2. Controlling through Functions for systemctl operations: -# Specific operations like installation, uninstallation, and status check can be executed by passing the respective function name as an argument to the script. -# Example: ./openim-tools.sh openim::tools::install -# - - -OPENIM_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")"/../.. && pwd -P) -[[ -z ${COMMON_SOURCED} ]] && source "${OPENIM_ROOT}"/scripts/install/common.sh - -SERVER_NAME="openim-tools" - -openim::tools::start_name() { - local targets=( - imctl - ) - echo "${targets[@]}" -} -IFS=" " read -ra OPENIM_TOOLS_NAME_TARGETS <<< "$(openim::tools::start_name)" -readonly OPENIM_TOOLS_NAME_TARGETS -readonly OPENIM_TOOLS_NAME_LISTARIES=("${OPENIM_TOOLS_NAME_TARGETS[@]##*/}") - -openim::tools::pre_start_name() { - local targets=( - ncpu - component - ) - echo "${targets[@]}" -} -IFS=" " read -ra OPENIM_TOOLS_PRE_START_NAME_TARGETS <<< "$(openim::tools::pre_start_name)" -readonly OPENIM_TOOLS_PRE_START_NAME_TARGETS -readonly OPENIM_TOOLS_PRE_START_NAME_LISTARIES=("${OPENIM_TOOLS_PRE_START_NAME_TARGETS[@]##*/}") - -openim::tools::post_start_name() { - local targets=( - infra - versionchecker - ) - echo "${targets[@]}" -} -IFS=" " read -ra OPENIM_TOOLS_POST_START_NAME_TARGETS <<< "$(openim::tools::post_start_name)" -readonly OPENIM_TOOLS_POST_START_NAME_TARGETS -readonly OPENIM_TOOLS_POST_START_NAME_LISTARIES=("${OPENIM_TOOLS_POST_START_NAME_TARGETS[@]##*/}") - -function openim::tools::start_service() { - local binary_name="$1" - local config="$2" - local service_port="$3" - local prometheus_port="$4" - - local cmd="${OPENIM_OUTPUT_HOSTBIN_TOOLS}/${binary_name}" - #openim::log::info "Starting PATH: ${OPENIM_OUTPUT_HOSTBIN_TOOLS}/${binary_name}..." - - if [ -n "${config}" ]; then - # printf "Specifying config: %s\n" "${config}" - cmd="${cmd} -c ${config}/config.yaml" - fi - - if [ -n "${service_port}" ]; then - #printf "Specifying service port: %s\n" "${service_port}" - cmd="${cmd} --port ${service_port}" - fi - - if [ -n "${prometheus_port}" ]; then - #printf "Specifying prometheus port: %s\n" "${prometheus_port}" - cmd="${cmd} --prometheus_port ${prometheus_port}" - fi - #openim::log::status "Starting binary ${binary_name}..." - - - # openim::log::info "cmd: $cmd" - ${cmd} - - - local status=$? - if [ $status -eq 0 ]; then - openim::log::colorless "Service ${binary_name} started successfully." - return 0 - else - openim::log::error "Failed to start service ${binary_name}." - return 1 - fi -} - -function openim::tools::start() { - openim::log::info "Starting OpenIM Tools..." - for tool in "${OPENIM_TOOLS_NAME_LISTARIES[@]}"; do - openim::log::colorless "Starting tool ${tool}..." - # openim::tools::start_service ${tool} - #sleep 0.2 - done -} - - -function openim::tools::pre-start() { - #openim::log::info "Preparing to start OpenIM Tools..." - for tool in "${OPENIM_TOOLS_PRE_START_NAME_LISTARIES[@]}"; do - openim::log::colorless "Starting tool: ${tool}" - if ! openim::tools::start_service ${tool} ${OPNEIM_CONFIG}; then - openim::log::error "Failed to start ${tool}, aborting..." - return 1 - fi - done - #openim::log::info "All tools started successfully." -} - -function openim::tools::post-start() { - #openim::log::info "Post-start actions for OpenIM Tools..." - for tool in "${OPENIM_TOOLS_POST_START_NAME_LISTARIES[@]}"; do - openim::log::colorless "Starting tool: ${tool}" - openim::tools::start_service ${tool} - done -} - -function openim::tools::stop() { - openim::log::info "Stopping OpenIM Tools..." - for tool in "${OPENIM_TOOLS_NAME_LISTARIES[@]}"; do - openim::log::info "Stopping ${tool}..." - # Similarly, place the actual command to stop the tool here. - echo "Stopping service for ${tool}" - sleep 0.2 - done -} - -function openim::tools::pre-stop() { - openim::log::info "Preparing to stop OpenIM Tools..." - for tool in "${OPENIM_TOOLS_PRE_START_NAME_LISTARIES[@]}"; do - openim::log::info "Setting up pre-stop for ${tool}..." - echo "Pre-stop actions for ${tool}" - sleep 0.2 - done -} - -function openim::tools::post-stop() { - openim::log::info "Post-stop actions for OpenIM Tools..." - for tool in "${OPENIM_TOOLS_POST_START_NAME_LISTARIES[@]}"; do - openim::log::info "Executing post-stop for ${tool}..." - echo "Post-stop cleanup for ${tool}" - sleep 0.2 - done -} - -if [[ "$*" =~ openim::tools:: ]];then - eval $* -fi diff --git a/scripts/install/test.sh b/scripts/install/test.sh deleted file mode 100755 index 51d541ecea..0000000000 --- a/scripts/install/test.sh +++ /dev/null @@ -1,1466 +0,0 @@ -#!/usr/bin/env bash - -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -# OpenIM RPC Service Test Control Script -# -# This control script is designed to conduct various tests on the OpenIM RPC services. -# It includes functions to perform smoke tests, API tests, and comprehensive service tests. -# The script is intended to be used in a Linux environment with appropriate permissions and -# environmental variables set. -# -# It provides robust error handling and logging to facilitate debugging and service monitoring. -# Functions within the script can be called directly or passed as arguments to perform -# systematic testing, ensuring the integrity of the RPC services. -# -# Test Functions: -# - openim::test::smoke: Runs basic tests to ensure the fundamental functionality of the service. -# - openim::test::api: Executes a series of API tests covering authentication, user, friend, -# group, and message functionalities. -# - openim::test::test: Performs a complete test suite, invoking utility checks and all defined -# test cases, and reports on their success. -# - -# The root of the build/dist directory -OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/../.. -[[ -z ${COMMON_SOURCED} ]] && source ${OPENIM_ROOT}/scripts/install/common.sh - -# API Server API Address:Port -INSECURE_OPENIMAPI="http://${OPENIM_API_HOST}:${API_OPENIM_PORT}" -INSECURE_OPENIMAUTO=${OPENIM_RPC_AUTH_HOST}:${OPENIM_AUTH_PORT} -CCURL="curl -f -s -XPOST" # Create -UCURL="curl -f -s -XPUT" # Update -RCURL="curl -f -s -XGET" # Retrieve -DCURL="curl -f -s -XDELETE" # Delete - -openim::test::check_error() { - local response=$1 - local err_code=$(echo "$response" | jq '.errCode') - openim::log::status "Response from user registration: $response" - if [[ "$err_code" != "0" ]]; then - openim::log::error_exit "Error occurred: $response, You can read the error code in the API documentation https://docs.openim.io/restapi/errcode" - else - openim::log::success "Operation was successful." - fi -} - -# The `openim::test::auth` function serves as a test suite for authentication-related operations. -function openim::test::auth() { - # 1. Retrieve and set the authentication token. - openim::test::get_token - - # 2. Force logout the test user from a specific platform. - openim::test::force_logout - - # Log the completion of the auth test suite. - openim::log::success "Auth test suite completed successfully." -} - -#################################### Auth Module #################################### - -# Define a function to get a token for a specific user -openim::test::get_token() { - local user_id="${1:-imAdmin}" # Default user ID if not provided - token_response=$( - ${CCURL} "${OperationID}" "${Header}" ${INSECURE_OPENIMAPI}/auth/user_token \ - -d'{"secret": "'"$SECRET"'","platformID": 1,"userID": "'$user_id'"}' - ) - token=$(echo $token_response | grep -Po 'token[" :]+\K[^"]+') - echo "$token" -} - -Header="-HContent-Type: application/json" -OperationID="-HoperationID: 1646445464564" -Token="-Htoken: $(openim::test::get_token)" - -# Forces a user to log out from the specified platform by user ID. -openim::test::force_logout() { - local request_body=$( - cat <&2 - local i - for ((i=1 ; i <= ${#FUNCNAME[@]} - stack_skip ; i++)) - do - local frame_no=$((i - 1 + stack_skip)) - local source_file=${BASH_SOURCE[${frame_no}]} - local source_lineno=${BASH_LINENO[$((frame_no - 1))]} - local funcname=${FUNCNAME[${frame_no}]} - echo_log " ${i}: ${source_file}:${source_lineno} ${funcname}(...)" >&2 - done - fi -} - -# Log an error and exit. -# Args: -# $1 Message to log with the error -# $2 The error code to return -# $3 The number of stack frames to skip when printing. -openim::log::error_exit() { - local message="${1:-}" - local code="${2:-1}" - local stack_skip="${3:-0}" - stack_skip=$((stack_skip + 1)) - - if [[ ${OPENIM_VERBOSE} -ge 4 ]]; then - local source_file=${BASH_SOURCE[${stack_skip}]} - local source_line=${BASH_LINENO[$((stack_skip - 1))]} - echo_log -e "${COLOR_RED}!!! Error in ${source_file}:${source_line} ${COLOR_SUFFIX}" >&2 - [[ -z ${1-} ]] || { - echo_log " ${1}" >&2 - } - - openim::log::stack ${stack_skip} - - echo_log "Exiting with status ${code}" >&2 - fi - - exit "${code}" -} - -# Log an error but keep going. Don't dump the stack or exit. -openim::log::error() { - # Define red color - red='\033[0;31m' - # No color (reset) - nc='\033[0m' # No Color - - timestamp=$(date +"[%Y-%m-%d %H:%M:%S %Z]") - # Apply red color for error message - echo_log "${red}!!! ${timestamp} ${1-}${nc}" >&2 - shift - for message; do - # Apply red color for subsequent lines of the error message - echo_log "${red} ${message}${nc}" >&2 - done -} - - -# Print an usage message to stderr. The arguments are printed directly. -openim::log::usage() { - echo_log >&2 - local message - for message; do - echo_log "${message}" >&2 - done - echo_log >&2 -} - -openim::log::usage_from_stdin() { - local messages=() - while read -r line; do - messages+=("${line}") - done - - openim::log::usage "${messages[@]}" -} - -# Print out some info that isn't a top level status line -openim::log::info() { - local V="${V:-0}" - if [[ ${OPENIM_VERBOSE} < ${V} ]]; then - return - fi - - for message; do - echo_log "${message}" - done -} - -# Just like openim::log::info, but no \n, so you can make a progress bar -openim::log::progress() { - for message; do - echo_log -e -n "${message}" - done -} - -# Print out some info that isn't a top level status line -openim::log::info_from_stdin() { - local messages=() - while read -r line; do - messages+=("${line}") - done - - openim::log::info "${messages[@]}" -} - -# Print a status line. Formatted to show up in a stream of output. -openim::log::status() { - local V="${V:-0}" - if [[ ${OPENIM_VERBOSE} < ${V} ]]; then - return - fi - - local COLOR_BLUE="\033[0;34m" - local COLOR_RESET="\033[0m" - - local timestamp=$(date +"[%Y-%m-%d %H:%M:%S %Z]") - - echo_log() { - echo -e "$@" - } - - - echo_log "${COLOR_BLUE}${timestamp} ${1}${COLOR_RESET}" - shift - for message; do - echo_log "${COLOR_BLUE}${message}${COLOR_RESET}" - done -} - - - -openim::log::success() { - local V="${V:-0}" - if [[ ${OPENIM_VERBOSE} < ${V} ]]; then - return - fi - local timestamp=$(date +"%m%d %H:%M:%S") - local reset_color='\033[0m' - echo_log -e "${COLOR_GREEN}[success ${timestamp}]${COLOR_SUFFIX}==> ${COLOR_GREEN}$@${reset_color}" -} - - - -function openim::log::test_log() { - echo_log "test log" - openim::log::info "openim::log::info" - openim::log::progress "openim::log::progress" - openim::log::status "openim::log::status" - openim::log::success "openim::log::success" - openim::log::error "openim::log::error" - openim::log::error_exit "openim::log::error_exit" -} - -# openim::log::test_log - -function openim::log::print_blue() { - echo -e "\033[0;36m$1\033[0m" -} - - -openim::log::colorless() { - local V="${V:-0}" - if [[ ${OPENIM_VERBOSE} < ${V} ]]; then - return - fi - timestamp=$(date +"[%Y-%m-%d %H:%M:%S %Z]") - echo_log -e "${timestamp} ${1} " -} - - diff --git a/scripts/make-rules/common-versions.mk b/scripts/make-rules/common-versions.mk deleted file mode 100644 index 572585fcee..0000000000 --- a/scripts/make-rules/common-versions.mk +++ /dev/null @@ -1,58 +0,0 @@ -# Copyright © 2023 OpenIMSDK. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# ============================================================================== -# OpenIM Makefile Versions used -# -# Define the latest version for each tool to ensure consistent versioning across installations -GOLANGCI_LINT_VERSION ?= latest -GOIMPORTS_VERSION ?= latest -ADDLICENSE_VERSION ?= latest -DEEPCOPY_GEN_VERSION ?= latest -CONVERSION_GEN_VERSION ?= latest -GINKGO_VERSION ?= v1.16.2 -GO_GITLINT_VERSION ?= latest -GO_JUNIT_REPORT_VERSION ?= latest -GOTESTS_VERSION ?= latest -SWAGGER_VERSION ?= latest -KUBE_SCORE_VERSION ?= latest -KUBECONFORM_VERSION ?= latest -GSEMVER_VERSION ?= latest -GIT_CHGLOG_VERSION ?= latest -KO_VERSION ?= latest -GITHUB_RELEASE_VERSION ?= latest -COSCLI_VERSION ?= v0.19.0-beta -MINIO_VERSION ?= latest -DELVE_VERSION ?= latest -AIR_VERSION ?= latest -GOLINES_VERSION ?= latest -GO_MOD_OUTDATED_VERSION ?= latest -CFSSL_VERSION ?= latest -DEPTH_VERSION ?= latest -GO_CALLVIS_VERSION ?= latest -MISSPELL_VERSION ?= latest -GOTHANKS_VERSION ?= latest -RICHGO_VERSION ?= latest -RTS_VERSION ?= latest -TYPECHECK_VERSION ?= latest -COMMENT_LANG_DETECTOR_VERSION ?= latest -STANDARDIZER_VERSION ?= latest -GO_TESTS_VERSION ?= v1.6.0 -GO_APIDIFF_VERSION ?= v0.8.2 -KAFKACTL_VERSION ?= latest -GOTESTSUM_VERSION ?= latest - -WIRE_VERSION ?= latest -# WIRE_VERSION ?= $(call get_go_version,github.com/google/wire) -MOCKGEN_VERSION ?= $(call get_go_version,github.com/golang/mock) -PROTOC_GEN_GO_VERSION ?= $(call get_go_version,github.com/golang/protobuf/protoc-gen-go) \ No newline at end of file diff --git a/scripts/make-rules/common.mk b/scripts/make-rules/common.mk deleted file mode 100644 index ffbb69a55d..0000000000 --- a/scripts/make-rules/common.mk +++ /dev/null @@ -1,196 +0,0 @@ -# Copyright © 2023 OpenIMSDK. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# ============================================================================== -# Makefile helper functions for common tasks -# - -SHELL := /bin/bash -GO:=go -DIRS=$(shell ls) -DEBUG ?= 0 -GIT_TAG := $(shell git describe --exact-match --tags --abbrev=0 2> /dev/null || echo untagged) -GIT_COMMIT ?= $(shell git rev-parse --short HEAD || echo "0.0.0") -BUILD_DATE ?=$(shell date -u +'%Y-%m-%dT%H:%M:%SZ') # Blank error: date '+%FT %T %z':"buildDate":"2023-03-31T 20:05:43 +0800" - -# include the common makefile -COMMON_SELF_DIR := $(dir $(lastword $(MAKEFILE_LIST))) - -SRC = $(shell find . -type f -name '*.go' -not -path "./vendor/*") - -# ROOT_DIR: root directory of the code base -ifeq ($(origin ROOT_DIR),undefined) -ROOT_DIR := $(abspath $(shell cd $(COMMON_SELF_DIR)/../.. && pwd -P)) -endif - -# OUTPUT_DIR: The directory where the build output is stored. -ifeq ($(origin OUTPUT_DIR),undefined) -OUTPUT_DIR := $(ROOT_DIR)/_output -$(shell mkdir -p $(OUTPUT_DIR)) -endif - -# BIN_DIR: Directory where executable files are stored. -ifeq ($(origin BIN_DIR),undefined) -BIN_DIR := $(OUTPUT_DIR)/bin -$(shell mkdir -p $(BIN_DIR)) -endif - -# BIN_TOOLS_DIR: Directory where executable files are stored. -ifeq ($(origin BIN_TOOLS_DIR),undefined) -BIN_TOOLS_DIR := $(BIN_DIR)/tools -$(shell mkdir -p $(BIN_TOOLS_DIR)) -endif - -# LOGS_DIR: Directory where log files are stored. -ifeq ($(origin LOGS_DIR),undefined) -LOGS_DIR := $(OUTPUT_DIR)/logs -$(shell mkdir -p $(LOGS_DIR)) -endif - -# TOOLS_DIR: The directory where tools are stored for build and testing. -ifeq ($(origin TOOLS_DIR),undefined) -TOOLS_DIR := $(OUTPUT_DIR)/tools -$(shell mkdir -p $(TOOLS_DIR)) -endif - -# TMP_DIR: directory where temporary files are stored. -ifeq ($(origin TMP_DIR),undefined) -TMP_DIR := $(OUTPUT_DIR)/tmp -$(shell mkdir -p $(TMP_DIR)) -endif - -ifeq ($(origin VERSION), undefined) -# VERSION := $(shell git describe --tags --always --match='v*') -# git describe --tags --always --match="v*" --dirty -# VERSION := $(shell git describe --tags --always --match="v*" --dirty | sed 's/-/./g') #v2.3.3.631.g00abdc9b.dirty -VERSION := $(shell git describe --tags --always --match='v*') -# v2.3.3: git tag -endif - -# Helper function to get dependency version from go.mod -get_gomod_version = $(shell go list -m $1 | awk '{print $$2}') -define go_install -$(info ===========> Installing $(1)@$(2)) -$(GO) install $(1)@$(2) -endef - -# Check if the tree is dirty. default to dirty(maybe u should commit?) -GIT_TREE_STATE:="dirty" -ifeq (, $(shell git status --porcelain 2>/dev/null)) - GIT_TREE_STATE="clean" -endif -GIT_COMMIT:=$(shell git rev-parse HEAD) - -# Minimum test coverage -# can u use make cover COVERAGE=90 -ifeq ($(origin COVERAGE),undefined) -COVERAGE := 60 -endif - -# Get the currently used golang install path (in GOPATH/bin, unless GOBIN is set) -ifeq (,$(shell go env GOBIN)) -GOBIN=$(shell go env GOPATH)/bin -else -GOBIN=$(shell go env GOBIN) -endif - -# The OS must be linux when building docker images -# PLATFORMS ?= linux_amd64 linux_arm64 -# The OS can be linux/windows/darwin when building binaries -PLATFORMS ?= linux_s390x linux_mips64 linux_mips64le darwin_amd64 darwin_arm64 windows_amd64 linux_amd64 linux_arm64 linux_ppc64le # wasip1_wasm - -# set a specific PLATFORM, defaults to the host platform -ifeq ($(origin PLATFORM), undefined) - ifeq ($(origin GOARCH), undefined) - GOARCH := $(shell go env GOARCH) - endif - # Determine the host OS - GOOS := $(shell go env GOOS) - PLATFORM := $(GOOS)_$(GOARCH) - # Use the host OS and GOARCH as the default when building images - IMAGE_PLAT := $(PLATFORM) -else - # Parse the PLATFORM variable - GOOS := $(word 1, $(subst _, ,$(PLATFORM))) - GOARCH := $(word 2, $(subst _, ,$(PLATFORM))) - IMAGE_PLAT := $(PLATFORM) -endif - - -# Protobuf file storage path -APIROOT=$(ROOT_DIR)/pkg/proto - -# Linux command settings -# TODO: Whether you need to join utils? -FIND := find . ! -path './utils/*' ! -path './vendor/*' ! -path './third_party/*' ! -path './components/*' ! -path './logs/*' -XARGS := xargs -r --no-run-if-empty - -# Linux command settings-CODE DIRS Copyright -CODE_DIRS := $(ROOT_DIR)/pkg $(ROOT_DIR)/cmd $(ROOT_DIR)/config $(ROOT_DIR)/internal $(ROOT_DIR)/scripts $(ROOT_DIR)/test $(ROOT_DIR)/.github $(ROOT_DIR)/build $(ROOT_DIR)/tools $(ROOT_DIR)/deployments -FINDS := find $(CODE_DIRS) - -# Makefile settings: Select different behaviors by determining whether V option is set -ifndef V -MAKEFLAGS += --no-print-directory -endif - -# COMMA: Concatenate multiple strings to form a list of strings -COMMA := , -# SPACE: Used to separate strings -SPACE := -# SPACE: Replace multiple consecutive Spaces with a single space -SPACE += - -# ============================================================================== -# Makefile helper functions for common tasks - -# Help information for the makefile package -define makehelp - @printf "\n\033[1mUsage: make ...\033[0m\n\n\\033[1mTargets:\\033[0m\n\n" - @sed -n 's/^##//p' $< | awk -F':' '{printf "\033[36m%-28s\033[0m %s\n", $$1, $$2}' | sed -e 's/^/ /' - @printf "\n\033[1m$$USAGE_OPTIONS\033[0m\n" -endef - -# Here are some examples of builds -define MAKEFILE_EXAMPLE -# make build BINS=openim-api Only a single openim-api binary is built. -# make -j (nproc) all Run tidy gen add-copyright format lint cover build concurrently. -# make gen Generate all necessary files. -# make release Build release binaries for all platforms. -# make verify-copyright Verify the license headers for all files. -# make install-deepcopy-gen Install deepcopy-gen tools if the license is missing. -# make build BINS=openim-api V=1 DEBUG=1 Build debug binaries for only openim-api. -# make multiarch -j PLATFORMS="linux_arm64 linux_amd64" V=1 Build binaries for both platforms. -# make image -endef -export MAKEFILE_EXAMPLE - -# Define all help functions @printf "\n\033[1mCurrent openim-api version information: $(shell openim-api version):\033[0m\n\n" -define makeallhelp - @printf "\n\033[1mMake example:\033[0m\n\n" - $(call MAKEFILE_EXAMPLE) - @printf "\n\033[1mAriables:\033[0m\n\n" - @echo " DEBUG: $(DEBUG)" - @echo " BINS: $(BINS)" - @echo " PLATFORMS: $(PLATFORMS)" - @echo " V: $(V)" -endef - -# Help information for other makefile packages -CUT_OFF?="---------------------------------------------------------------------------------" -HELP_NAME:=$(shell basename $(MAKEFILE_LIST)) -define smallhelp - @sed -n 's/^##//p' $< | awk -F':' '{printf "\033[36m%-35s\033[0m %s\n", $$1, $$2}' | sed -e 's/^/ /' - @echo $(CUT_OFF) -endef \ No newline at end of file diff --git a/scripts/make-rules/copyright.mk b/scripts/make-rules/copyright.mk deleted file mode 100644 index 18b4dd7097..0000000000 --- a/scripts/make-rules/copyright.mk +++ /dev/null @@ -1,63 +0,0 @@ -# Copyright © 2023 OpenIMSDK. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -# ============================================================================== -# wget https://github.com/google/addlicense/releases/download/v1.0.0/addlicense_1.0.0_Linux_x86_64.tar.gz -# Makefile helper functions for copyright -# - -LICENSE_TEMPLATE ?= $(ROOT_DIR)/scripts/template/LICENSE_TEMPLATES - -## copyright.verify: Validate boilerplate headers for assign files -.PHONY: copyright.verify -copyright.verify: tools.verify.addlicense - @echo "===========> Validate boilerplate headers for assign files starting in the $(ROOT_DIR) directory" - @$(TOOLS_DIR)/addlicense -v -check -ignore **/test/** -ignore **pb** -f $(LICENSE_TEMPLATE) $(CODE_DIRS) - @echo "===========> End of boilerplate headers check..." - -## copyright.add: Add the boilerplate headers for all files -.PHONY: copyright.add -copyright.add: tools.verify.addlicense - @echo "===========> Adding $(LICENSE_TEMPLATE) the boilerplate headers for all files" - @$(TOOLS_DIR)/addlicense -y $(shell date +"%Y") -ignore **pb** -v -c "OpenIM." -f $(LICENSE_TEMPLATE) $(CODE_DIRS) - @echo "===========> End the copyright is added..." - -# Addlicense Flags: -# -c string -# copyright holder (default "Google LLC") -# -check -# check only mode: verify presence of license headers and exit with non-zero code if missing -# -f string -# license file -# -ignore value -# file patterns to ignore, for example: -ignore **/*.go -ignore vendor/** -# -l string -# license type: apache, bsd, mit, mpl (default "apache") -# -s Include SPDX identifier in license header. Set -s=only to only include SPDX identifier. -# -skip value -# [deprecated: see -ignore] file extensions to skip, for example: -skip rb -skip go -# -v verbose mode: print the name of the files that are modified or were skipped -# -y string -# copyright year(s) (default "2023") - -## copyright.advertise: Advertise the license of the project -.PHONY: copyright.advertise -copyright.advertise: - @chmod +x $(ROOT_DIR)/scripts/advertise.sh - @$(ROOT_DIR)/scripts/advertise.sh - -## copyright.help: Show copyright help -.PHONY: copyright.help -copyright.help: scripts/make-rules/copyright.mk - $(call smallhelp) \ No newline at end of file diff --git a/scripts/make-rules/dependencies.mk b/scripts/make-rules/dependencies.mk deleted file mode 100644 index 1a896f1ff5..0000000000 --- a/scripts/make-rules/dependencies.mk +++ /dev/null @@ -1,41 +0,0 @@ -# Copyright © 2023 OpenIMSDK. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# ============================================================================== -# Makefile helper functions for dependencies -# - -.PHONY: dependencies.run -dependencies.run: dependencies.packages dependencies.tools - -.PHONY: dependencies.packages -dependencies.packages: - @$(GO) mod tidy - -.PHONY: dependencies.tools -dependencies.tools: dependencies.tools.blocker dependencies.tools.critical - -.PHONY: dependencies.tools.blocker -dependencies.tools.blocker: go.build.verify $(addprefix tools.verify., $(BLOCKER_TOOLS)) - -.PHONY: dependencies.tools.critical -dependencies.tools.critical: $(addprefix tools.verify., $(CRITICAL_TOOLS)) - -.PHONY: dependencies.tools.trivial -dependencies.tools.trivial: $(addprefix tools.verify., $(TRIVIAL_TOOLS)) - -## dependencies.help: Print help for dependencies targets -.PHONY: dependencies.help -dependencies.help: scripts/make-rules/dependencies.mk - $(call smallhelp) \ No newline at end of file diff --git a/scripts/make-rules/gen.mk b/scripts/make-rules/gen.mk deleted file mode 100644 index fba0132ff5..0000000000 --- a/scripts/make-rules/gen.mk +++ /dev/null @@ -1,100 +0,0 @@ -# Copyright © 2023 OpenIMSDK. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# ============================================================================== -# Makefile helper functions for generate necessary files and docs -# https://cloud.redhat.com/blog/kubernetes-deep-dive-code-generation-customresources -# ! The stock of code generated by `make gen` should be idempotent -# -# Questions about go mod instead of go path: https://github.com/kubernetes/kubernetes/issues/117181 -# ============================================================================== -# Makefile helper functions for generate necessary files -# - -## gen.init: Initialize openim server project ✨ -.PHONY: gen.init -gen.init: - @echo "===========> Initializing openim server project" - @${ROOT_DIR}/scripts/init-config.sh - -## gen.init-githooks: Initialize git hooks ✨ -.PHONY: gen.init-githooks -gen.init-githooks: - @echo "===========> Initializing git hooks" - @${ROOT_DIR}/scripts/init-githooks.sh - -## gen.run: Generate necessary files and docs ✨ -.PHONY: gen.run -#gen.run: gen.errcode gen.docgo -gen.run: gen.clean gen.errcode gen.docgo.doc - -## gen.errcode: Generate necessary files and docs ✨ -.PHONY: gen.errcode -gen.errcode: gen.errcode.code gen.errcode.doc - -## gen.errcode.code: Generate openim error code go source files ✨ -.PHONY: gen.errcode.code -gen.errcode.code: tools.verify.codegen - @echo "===========> Generating openim error code go source files" - @codegen -type=int ${ROOT_DIR}/internal/pkg/code - -## gen.errcode.doc: Generate openim error code markdown documentation ✨ -.PHONY: gen.errcode.doc -gen.errcode.doc: tools.verify.codegen - @echo "===========> Generating error code markdown documentation" - @codegen -type=int -doc \ - -output ${ROOT_DIR}/docs/guide/zh-CN/api/error_code_generated.md ${ROOT_DIR}/internal/pkg/code - -## gen.docgo: Generate missing doc.go for go packages ✨ -.PHONY: gen.ca.% -gen.ca.%: - $(eval CA := $(word 1,$(subst ., ,$*))) - @echo "===========> Generating CA files for $(CA)" - @${ROOT_DIR}/scripts/gencerts.sh generate-openim-cert $(OUTPUT_DIR)/cert $(CA) - -## gen.ca: Generate CA files for all certificates ✨ -.PHONY: gen.ca -gen.ca: $(addprefix gen.ca., $(CERTIFICATES)) - -## gen.docgo: Generate missing doc.go for go packages ✨ -.PHONY: gen.docgo.doc -gen.docgo.doc: - @echo "===========> Generating missing doc.go for go packages" - @${ROOT_DIR}/scripts/gendoc.sh - -## gen.docgo.check: Check if there are untracked doc.go files ✨ -.PHONY: gen.docgo.check -gen.docgo.check: gen.docgo.doc - @n="$$(git ls-files --others '*/doc.go' | wc -l)"; \ - if test "$$n" -gt 0; then \ - git ls-files --others '*/doc.go' | sed -e 's/^/ /'; \ - echo "$@: untracked doc.go file(s) exist in working directory" >&2 ; \ - false ; \ - fi - -## gen.docgo.add: Add untracked doc.go files to git index ✨ -.PHONY: gen.docgo.add -gen.docgo.add: - @git ls-files --others '*/doc.go' | $(XARGS) -- git add - -## gen.docgo: Generate missing doc.go for go packages ✨ -.PHONY: gen.clean -gen.clean: - @rm -rf ./api/client/{clientset,informers,listers} - @$(FIND) -type f -name '*_generated.go' -delete - -## gen.help: show help for gen -.PHONY: gen.help -gen.help: scripts/make-rules/gen.m - $(call smallhelp) \ No newline at end of file diff --git a/scripts/make-rules/golang.mk b/scripts/make-rules/golang.mk deleted file mode 100644 index cdc5a5fc95..0000000000 --- a/scripts/make-rules/golang.mk +++ /dev/null @@ -1,272 +0,0 @@ -# Copyright © 2023 OpenIMSDK. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# ============================================================================== -# Build management helpers. These functions help to set, save and load the -# - -GO := go -GO_MINIMUM_VERSION ?= 1.19 - -GO_LDFLAGS += -X $(VERSION_PACKAGE).gitVersion=$(GIT_TAG) \ - -X $(VERSION_PACKAGE).gitCommit=$(GIT_COMMIT) \ - -X $(VERSION_PACKAGE).gitTreeState=$(GIT_TREE_STATE) \ - -X $(VERSION_PACKAGE).buildDate=$(BUILD_DATE) \ - -s -w # -s -w deletes debugging information and symbol tables -ifeq ($(DEBUG), 1) - GO_BUILD_FLAGS += -gcflags "all=-N -l" - GO_LDFLAGS= -endif - -GO_BUILD_FLAGS += -ldflags "$(GO_LDFLAGS)" - -ifeq ($(GOOS),windows) - GO_OUT_EXT := .exe -endif - -ifeq ($(ROOT_PACKAGE),) - $(error the variable ROOT_PACKAGE must be set prior to including golang.mk, ->/Makefile) -endif - -GOPATH ?= $(shell go env GOPATH) -ifeq ($(origin GOBIN), undefined) - GOBIN := $(GOPATH)/bin -endif - -# COMMANDS is Specify all files under ${ROOT_DIR}/cmd/ and ${ROOT_DIR}/tools/ except those ending in.md -COMMANDS ?= $(filter-out %.md, $(wildcard ${ROOT_DIR}/cmd/* ${ROOT_DIR}/tools/* ${ROOT_DIR}/tools/data-conversion/chat/cmd/* ${ROOT_DIR}/tools/data-conversion/openim/cmd/* ${ROOT_DIR}/cmd/openim-rpc/*)) -ifeq (${COMMANDS},) - $(error Could not determine COMMANDS, set ROOT_DIR or run in source dir) -endif - -# BINS is the name of each file in ${COMMANDS}, excluding the directory path -# If there are no files in ${COMMANDS}, or if all files end in.md, ${BINS} will be empty -BINS ?= $(foreach cmd,${COMMANDS},$(notdir ${cmd})) -ifeq (${BINS},) - $(error Could not determine BINS, set ROOT_DIR or run in source dir) -endif - -ifeq ($(OS),Windows_NT) - NULL := - SPACE := $(NULL) $(NULL) - ROOT_DIR := $(subst $(SPACE),\$(SPACE),$(shell cd)) -else - ROOT_DIR := $(shell pwd) -endif - -ifeq ($(strip $(COMMANDS)),) - $(error Could not determine COMMANDS, set ROOT_DIR or run in source dir) -endif -ifeq ($(strip $(BINS)),) - $(error Could not determine BINS, set ROOT_DIR or run in source dir) -endif - -# TODO: EXCLUDE_TESTS variable, which contains the name of the package to be excluded from the test -EXCLUDE_TESTS=github.com/openimsdk/open-im-server/test github.com/openimsdk/open-im-server/v3/pkg/log github.com/openimsdk/open-im-server/db github.com/openimsdk/open-im-server/scripts github.com/openimsdk/open-im-server/config - -# ============================================================================== -# ❯ tree -L 1 cmd -# cmd -# ├── openim-sdk-core/ - main.go -# ├── openim-api -# ├── openim_cms_api -# ├── openim-crontask -# ├── openim_demo -# ├── openim-rpc-msg_gateway -# ├── openim-msgtransfer -# ├── openim-push -# ├── rpc/openim_admin_cms/ - main.go -# └── test/ - main.go -# COMMAND=openim -# PLATFORM=linux_amd64 -# OS=linux -# ARCH=amd64 -# BINS=openim-api openim_cms_api openim-crontask openim_demo openim-rpc-msg_gateway openim-msgtransfer openim-push -# BIN_DIR=/root/workspaces/OpenIM/_output/bin -# ============================================================================== - -## go.build: Build binaries -.PHONY: go.build -go.build: go.build.verify $(addprefix go.build., $(addprefix $(PLATFORM)., $(BINS))) - @echo "===========> Building binary $(BINS) $(VERSION) for $(PLATFORM)" - -## go.start: Start openim -.PHONY: go.start -go.start: - @echo "=========================> Starting OpenIM <=========================" - @$(ROOT_DIR)/scripts/start-all.sh - -## go.stop: Stop openim -.PHONY: go.stop -go.stop: - @echo "=========================> Stopping OpenIM <=========================" - @$(ROOT_DIR)/scripts/stop-all.sh - -## go.check: Check openim -.PHONY: go.check -go.check: - @echo "=========================> Checking OpenIM <=========================" - @$(ROOT_DIR)/scripts/check-all.sh - -## go.check-component: Check openim component -.PHONY: go.check-component -go.check-component: - @echo "=========================> Checking OpenIM component <=========================" - @$(ROOT_DIR)/scripts/install/openim-tools.sh openim::tools::pre-start - -## go.versionchecker: Design, detect some environment variables and versions -go.versionchecker: - @$(ROOT_DIR)/scripts/install/openim-tools.sh openim::tools::post-start - -## go.build.verify: Verify that a suitable version of Go exists -.PHONY: go.build.verify -go.build.verify: -ifneq ($(shell $(GO) version|awk -v min=$(GO_MINIMUM_VERSION) '{gsub(/go/,"",$$3);if($$3 >= min){print 0}else{print 1}}'), 0) - $(error unsupported go version. Please install a go version which is greater than or equal to '$(GO_MINIMUM_VERSION)') -endif - -## go.build.%: Build binaries for a specific platform -# CGO_ENABLED=0 https://wiki.musl-libc.org/functional-differences-from-glibc.html -.PHONY: go.build.% -go.build.%: - $(eval COMMAND := $(word 2,$(subst ., ,$*))) - $(eval PLATFORM := $(word 1,$(subst ., ,$*))) - $(eval OS := $(word 1,$(subst _, ,$(PLATFORM)))) - $(eval ARCH := $(word 2,$(subst _, ,$(PLATFORM)))) - @echo "=====> COMMAND=$(COMMAND)" - @echo "=====> PLATFORM=$(PLATFORM)" - @echo "===========> Building binary $(COMMAND) $(VERSION) for $(OS)_$(ARCH)" - @mkdir -p $(BIN_DIR)/platforms/$(OS)/$(ARCH) - @if [ "$(COMMAND)" == "openim-sdk-core" ]; then \ - echo "===========> DEBUG: OpenIM-SDK-Core It is no longer supported for openim-server $(COMMAND)"; \ - elif [ -d $(ROOT_DIR)/cmd/openim-rpc/$(COMMAND) ]; then \ - CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) $(GO) build $(GO_BUILD_FLAGS) -o \ - $(BIN_DIR)/platforms/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT) $(ROOT_DIR)/cmd/openim-rpc/$(COMMAND)/main.go; \ - else \ - if [ -f $(ROOT_DIR)/cmd/$(COMMAND)/main.go ]; then \ - CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) $(GO) build $(GO_BUILD_FLAGS) -o \ - $(BIN_DIR)/platforms/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT) $(ROOT_DIR)/cmd/$(COMMAND)/main.go; \ - elif [ -f $(ROOT_DIR)/tools/$(COMMAND)/$(COMMAND).go ]; then \ - CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) $(GO) build $(GO_BUILD_FLAGS) -o \ - $(BIN_TOOLS_DIR)/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT) $(ROOT_DIR)/tools/$(COMMAND)/$(COMMAND).go; \ - chmod +x $(BIN_TOOLS_DIR)/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT); \ - elif [ -f $(ROOT_DIR)/tools/data-conversion/openim/cmd/$(COMMAND)/$(COMMAND).go ]; then \ - CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) $(GO) build $(GO_BUILD_FLAGS) -o \ - $(BIN_TOOLS_DIR)/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT) $(ROOT_DIR)/tools/data-conversion/openim/cmd/$(COMMAND)/$(COMMAND).go; \ - chmod +x $(BIN_TOOLS_DIR)/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT); \ - elif [ -f $(ROOT_DIR)/tools/data-conversion/chat/cmd/$(COMMAND)/$(COMMAND).go ]; then \ - CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) $(GO) build $(GO_BUILD_FLAGS) -o \ - $(BIN_TOOLS_DIR)/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT) $(ROOT_DIR)/tools/data-conversion/chat/cmd/$(COMMAND)/$(COMMAND).go; \ - chmod +x $(BIN_TOOLS_DIR)/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT); \ - fi \ - fi - -## go.install: Install deployment openim -.PHONY: go.install -go.install: - @echo "===========> Installing deployment openim" - @$(ROOT_DIR)/scripts/install-im-server.sh - -## go.multiarch: Build multi-arch binaries -.PHONY: go.build.multiarch -go.build.multiarch: go.build.verify $(foreach p,$(PLATFORMS),$(addprefix go.build., $(addprefix $(p)., $(BINS)))) - -## go.lint: Run golangci to lint source codes -.PHONY: go.lint -go.lint: tools.verify.golangci-lint - @echo "===========> Run golangci to lint source codes" - @$(TOOLS_DIR)/golangci-lint run --color always -c $(ROOT_DIR)/.golangci.yml $(ROOT_DIR)/... - -## go.test: Run unit test -.PHONY: go.test -go.test: - @$(GO) test ./... - -## go.test.api: Run api test -.PHONY: go.test.api -go.test.api: - @echo "===========> Run api test" - @$(ROOT_DIR)/scripts/install/test.sh openim::test::test - -## go.test.e2e: Run e2e test -.PHONY: go.test.e2e -go.test.e2e: tools.verify.ginkgo - @echo "===========> Run e2e test" - @$(TOOLS_DIR)/ginkgo -v $(ROOT_DIR)/test/e2e - -## go.demo: Run demo -.PHONY: go.demo -go.demo: - @echo "===========> Run demo" - @$(ROOT_DIR)/scripts/demo.sh - -## go.test.junit-report: Run unit test -.PHONY: go.test.junit-report -go.test.junit-report: tools.verify.go-junit-report - @touch $(TMP_DIR)/coverage.out - @echo "===========> Run unit test > $(TMP_DIR)/report.xml" -# @$(GO) test -v -coverprofile=$(TMP_DIR)/coverage.out 2>&1 $(GO_BUILD_FLAGS) ./... | $(TOOLS_DIR)/go-junit-report -set-exit-code > $(TMP_DIR)/report.xml - @$(GO) test -v -coverprofile=$(TMP_DIR)/coverage.out 2>&1 ./... | $(TOOLS_DIR)/go-junit-report -set-exit-code > $(TMP_DIR)/report.xml - @sed -i '/mock_.*.go/d' $(TMP_DIR)/coverage.out - @echo "===========> Test coverage of Go code is reported to $(TMP_DIR)/coverage.html by generating HTML" - @$(GO) tool cover -html=$(TMP_DIR)/coverage.out -o $(TMP_DIR)/coverage.html - -## go.test.cover: Run unit test with coverage -.PHONY: go.test.cover -go.test.cover: go.test.junit-report - @$(GO) tool cover -func=$(TMP_DIR)/coverage.out | \ - awk -v target=$(COVERAGE) -f $(ROOT_DIR)/scripts/coverage.awk - -## go.format: Run unit test and format codes -.PHONY: go.format -go.format: tools.verify.golines tools.verify.goimports - @echo "===========> Formatting codes" - @$(FIND) -type f -name '*.go' -not -name '*pb*' | $(XARGS) gofmt -s -w - @$(FIND) -type f -name '*.go' -not -name '*pb*' | $(XARGS) $(TOOLS_DIR)/goimports -w -local $(ROOT_PACKAGE) - @$(FIND) -type f -name '*.go' -not -name '*pb*' | $(XARGS) $(TOOLS_DIR)/golines -w --max-len=200 --reformat-tags --shorten-comments --ignore-generated . - @$(GO) mod edit -fmt - -## go.imports: task to automatically handle import packages in Go files using goimports tool -.PHONY: go.imports -go.imports: tools.verify.goimports - @$(TOOLS_DIR)/goimports -l -w $(SRC) - -## go.verify: execute all verity scripts. -.PHONY: go.verify -go.verify: tools.verify.misspell - @echo "Starting verification..." - @scripts_list=$$(find $(ROOT_DIR)/scripts -type f -name 'verify-*' | sort); \ - for script in $$scripts_list; do \ - echo "Executing $$script..."; \ - $$script || exit 1; \ - echo "$$script completed successfully"; \ - done - @echo "All verification scripts executed successfully." - -## go.updates: Check for updates to go.mod dependencies -.PHONY: go.updates -go.updates: tools.verify.go-mod-outdated - @$(GO) list -u -m -json all | go-mod-outdated -update -direct - -## go.clean: Clean all builds directories and files -.PHONY: go.clean -go.clean: - @echo "===========> Cleaning all builds tmp, bin, logs directories and files" - @-rm -vrf $(TMP_DIR) $(BIN_DIR) $(BIN_TOOLS_DIR) $(LOGS_DIR) - @echo "===========> End clean..." - -## go.help: Show go tools help -.PHONY: go.help -go.help: scripts/make-rules/golang.mk - $(call smallhelp) diff --git a/scripts/make-rules/image.mk b/scripts/make-rules/image.mk deleted file mode 100644 index eaec4a1271..0000000000 --- a/scripts/make-rules/image.mk +++ /dev/null @@ -1,183 +0,0 @@ -# Copyright © 2023 OpenIMSDK. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# ============================================================================== -# Makefile helper functions for docker image -# ============================================================================== -# Path: scripts/make-rules/image.mk -# docker registry: registry.example.com/namespace/image:tag as: registry.hub.docker.com/cubxxw/: -# https://docs.docker.com/build/building/multi-platform/ -# - -DOCKER := docker - -# read: https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/images.md -REGISTRY_PREFIX ?= registry.cn-hangzhou.aliyuncs.com/openimsdk -# REGISTRY_PREFIX ?= ghcr.io/openimsdk - -BASE_IMAGE ?= ghcr.io/openim-sigs/openim-bash-image - -IMAGE_PLAT ?= $(subst $(SPACE),$(COMMA),$(subst _,/,$(PLATFORMS))) - -EXTRA_ARGS ?= --no-cache -_DOCKER_BUILD_EXTRA_ARGS := - -ifdef HTTP_PROXY -_DOCKER_BUILD_EXTRA_ARGS += --build-arg HTTP_PROXY=${HTTP_PROXY} -endif - -ifneq ($(EXTRA_ARGS), ) -_DOCKER_BUILD_EXTRA_ARGS += $(EXTRA_ARGS) -endif - -# Determine image files by looking into build/images/*/Dockerfile -IMAGES_DIR ?= $(wildcard ${ROOT_DIR}/build/images/*) -# Determine images names by stripping out the dir names, and filter out the undesired directories -# IMAGES ?= $(filter-out Dockerfile,$(foreach image,${IMAGES_DIR},$(notdir ${image}))) -IMAGES ?= $(filter-out Dockerfile openim-tools openim-rpc-extend-msg openim-rpc-encryption openim-cmdutils,$(foreach image,${IMAGES_DIR},$(notdir ${image}))) -# IMAGES ?= $(filter-out Dockerfile openim-tools openim-cmdutils,$(foreach image,${IMAGES_DIR},$(notdir ${image}))) # !pro - -ifeq (${IMAGES},) - $(error Could not determine IMAGES, set ROOT_DIR or run in source dir) -endif - -# ============================================================================== -# Image targets -# ============================================================================== - -# PLATFORMS defines the target platforms for the manager image be build to provide support to multiple -# architectures. (i.e. make docker-buildx IMG=myregistry/mypoperator:0.0.1). To use this option you need to: -# - able to use docker buildx . More info: https://docs.docker.com/build/buildx/ -# - have enable BuildKit, More info: https://docs.docker.com/develop/develop-images/build_enhancements/ -# - be able to push the image for your registry (i.e. if you do not inform a valid value via IMG=> then the export will fail) -# To properly provided solutions that supports more than one platform you should use this option. -## image.docker-buildx: Build and push docker image for the manager for cross-platform support -PLATFORMS ?= linux/arm64,linux/amd64,linux/s390x,linux/ppc64le -# copy existing Dockerfile and insert --platform=${BUILDPLATFORM} into Dockerfile.cross, and preserve the original Dockerfile -.PHONY: image.docker-buildx -image.docker-buildx: - sed -e '1 s/\(^FROM\)/FROM --platform=\$$\{BUILDPLATFORM\}/; t' -e ' 1,// s//FROM --platform=\$$\{BUILDPLATFORM\}/' Dockerfile > Dockerfile.cross - - $(CONTAINER_TOOL) buildx create --name project-v3-builder - $(CONTAINER_TOOL) buildx use project-v3-builder - - $(CONTAINER_TOOL) buildx build --push --platform=$(PLATFORMS) --tag ${IMAGES} -f Dockerfile.cross . - - $(CONTAINER_TOOL) buildx rm project-v3-builder - rm Dockerfile.cross - -## image.verify: Verify docker version -.PHONY: image.verify -image.verify: - @$(ROOT_DIR)/scripts/lib/util.sh openim::util::check_docker_and_compose_versions - -## image.daemon.verify: Verify docker daemon experimental features -.PHONY: image.daemon.verify -image.daemon.verify: - @$(ROOT_DIR)/scripts/lib/util.sh openim::util::ensure_docker_daemon_connectivity - @$(ROOT_DIR)/scripts/lib/util.sh openim::util::ensure-docker-buildx - -# If you wish built the manager image targeting other platforms you can use the --platform flag. -# (i.e. docker build --platform linux/arm64 ). However, you must enable docker buildKit for it. -# More info: https://docs.docker.com/develop/develop-images/build_enhancements/ -## image.build: Build docker images -.PHONY: image.build -image.build: image.verify $(addprefix image.build., $(addprefix $(PLATFORM)., $(IMAGES))) - -.PHONY: image.build.multiarch -image.build.multiarch: image.verify $(foreach p,$(PLATFORMS),$(addprefix image.build., $(addprefix $(p)., $(IMAGES)))) - -## image.build.%: Build docker image for a specific platform -.PHONY: image.build.% -image.build.%: go.build.% - $(eval IMAGE := $(COMMAND)) - $(eval IMAGE_PLAT := $(subst _,/,$(PLATFORM))) - $(eval ARCH := $(word 2,$(subst _, ,$(PLATFORM)))) - @echo "===========> Building docker image $(IMAGE) $(VERSION) for $(IMAGE_PLAT)" - @mkdir -p $(TMP_DIR)/$(IMAGE)/$(PLATFORM) - @cat $(ROOT_DIR)/build/images/Dockerfile\ - | sed "s#BASE_IMAGE#$(BASE_IMAGE)#g" \ - | sed "s#BINARY_NAME#$(IMAGE)#g" >$(TMP_DIR)/$(IMAGE)/Dockerfile - @cp $(BIN_DIR)/platforms/$(IMAGE_PLAT)/$(IMAGE) $(TMP_DIR)/$(IMAGE) - $(eval BUILD_SUFFIX := $(_DOCKER_BUILD_EXTRA_ARGS) --pull -t $(REGISTRY_PREFIX)/$(IMAGE)-$(ARCH):$(VERSION) $(TMP_DIR)/$(IMAGE)) - @echo $(DOCKER) build --platform $(IMAGE_PLAT) $(BUILD_SUFFIX) - @if [ $(shell $(GO) env GOARCH) != $(ARCH) ] ; then \ - $(MAKE) image.daemon.verify ;\ - $(DOCKER) build --platform $(IMAGE_PLAT) $(BUILD_SUFFIX) ; \ - else \ - $(DOCKER) build $(BUILD_SUFFIX) ; \ - fi - @rm -rf $(TMP_DIR)/$(IMAGE) - -# https://docs.docker.com/build/building/multi-platform/ -# busybox image supports amd64, arm32v5, arm32v6, arm32v7, arm64v8, i386, ppc64le, and s390x -## image.buildx.%: Build docker images with buildx -.PHONY: image.buildx.% -image.buildx.%: - $(eval IMAGE := $(word 1,$(subst ., ,$*))) - echo "===========> Building docker image $(IMAGE) $(VERSION)" - $(DOCKER) buildx build -f $(ROOT_DIR)/Dockerfile --pull --no-cache --platform=$(PLATFORMS) --push . -t $(REGISTRY_PREFIX)/$(IMAGE)-$(ARCH):$(VERSION) - -## image.push: Push docker images -.PHONY: image.push -image.push: image.verify go.build.verify $(addprefix image.push., $(addprefix $(IMAGE_PLAT)., $(IMAGES))) - -## image.push.multiarch: Push docker images for all platforms -.PHONY: image.push.multiarch -image.push.multiarch: image.verify go.build.verify $(foreach p,$(PLATFORMS),$(addprefix image.push., $(addprefix $(p)., $(IMAGES)))) - -## image.push.%: Push docker image for a specific platform -.PHONY: image.push.% -image.push.%: image.build.% - @echo "===========> Pushing image $(IMAGE) $(VERSION) to $(REGISTRY_PREFIX)" - $(DOCKER) push $(REGISTRY_PREFIX)/$(IMAGE)-$(ARCH):$(VERSION) - -## image.manifest.push: Push manifest list for multi-arch images -.PHONY: image.manifest.push -image.manifest.push: export DOCKER_CLI_EXPERIMENTAL := enabled -image.manifest.push: image.verify go.build.verify \ -$(addprefix image.manifest.push., $(addprefix $(IMAGE_PLAT)., $(IMAGES))) - -## image.manifest.push.%: Push manifest list for multi-arch images for a specific platform -.PHONY: image.manifest.push.% -image.manifest.push.%: image.push.% image.manifest.remove.% - @echo "===========> Pushing manifest $(IMAGE) $(VERSION) to $(REGISTRY_PREFIX) and then remove the local manifest list" - @$(DOCKER) manifest create $(REGISTRY_PREFIX)/$(IMAGE):$(VERSION) \ - $(REGISTRY_PREFIX)/$(IMAGE)-$(ARCH):$(VERSION) - @$(DOCKER) manifest annotate $(REGISTRY_PREFIX)/$(IMAGE):$(VERSION) \ - $(REGISTRY_PREFIX)/$(IMAGE)-$(ARCH):$(VERSION) \ - --os $(OS) --arch ${ARCH} - @$(DOCKER) manifest push --purge $(REGISTRY_PREFIX)/$(IMAGE):$(VERSION) - -# Docker cli has a bug: https://github.com/docker/cli/issues/954 -# If you find your manifests were not updated, -# Please manually delete them in $HOME/.docker/manifests/ -# and re-run. -## image.manifest.remove.%: Remove local manifest list -.PHONY: image.manifest.remove.% -image.manifest.remove.%: - @rm -rf ${HOME}/.docker/manifests/docker.io_$(REGISTRY_PREFIX)_$(IMAGE)-$(VERSION) - -## image.manifest.push.multiarch: Push manifest list for multi-arch images for all platforms -.PHONY: image.manifest.push.multiarch -image.manifest.push.multiarch: image.push.multiarch $(addprefix image.manifest.push.multiarch., $(IMAGES)) - -## image.manifest.push.multiarch.%: Push manifest list for multi-arch images for all platforms for a specific image -.PHONY: image.manifest.push.multiarch.% -image.manifest.push.multiarch.%: - @echo "===========> Pushing manifest $* $(VERSION) to $(REGISTRY_PREFIX) and then remove the local manifest list" - REGISTRY_PREFIX=$(REGISTRY_PREFIX) PLATFORMS="$(PLATFORMS)" IMAGE=$* VERSION=$(VERSION) DOCKER_CLI_EXPERIMENTAL=enabled \ - $(ROOT_DIR)/build/lib/create-manifest.sh - -## image.help: Print help for image targets -.PHONY: image.help -image.help: scripts/make-rules/image.mk - $(call smallhelp) \ No newline at end of file diff --git a/scripts/make-rules/release.mk b/scripts/make-rules/release.mk deleted file mode 100644 index 68bf058981..0000000000 --- a/scripts/make-rules/release.mk +++ /dev/null @@ -1,42 +0,0 @@ -# Copyright © 2023 OpenIMSDK. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# ============================================================================== -# Makefile helper functions for release -# Versions are used after merging -# - -## release.run: release the project -.PHONY: release.run -release.run: release.verify release.ensure-tag - @scripts/release.sh - -## release.verify: Check if a tool is installed and install it -.PHONY: release.verify -release.verify: tools.verify.git-chglog tools.verify.github-release tools.verify.coscmd tools.verify.coscli - -## release.tag: release the project -.PHONY: release.tag -release.tag: tools.verify.gsemver release.ensure-tag - @git push origin `git describe --tags --abbrev=0` - -## release.ensure-tag: ensure tag -.PHONY: release.ensure-tag -release.ensure-tag: tools.verify.gsemver - @scripts/ensure-tag.sh - -## release.help: Display help information about the release package -.PHONY: release.help -release.help: scripts/make-rules/release.mk - $(call smallhelp) diff --git a/scripts/make-rules/swagger.mk b/scripts/make-rules/swagger.mk deleted file mode 100644 index 991de43c63..0000000000 --- a/scripts/make-rules/swagger.mk +++ /dev/null @@ -1,19 +0,0 @@ -# ============================================================================== -# Makefile helper functions for swagger -# - -## swagger.run: Generate swagger document. -.PHONY: swagger.run -swagger.run: tools.verify.swagger - @echo "===========> Generating swagger API docs" - @$(TOOLS_DIR)/swagger generate spec --scan-models -w $(ROOT_DIR)/cmd/genswaggertypedocs -o $(ROOT_DIR)/api/swagger/swagger.yaml - -## swagger.serve: Serve swagger spec and docs. -.PHONY: swagger.serve -swagger.serve: tools.verify.swagger - @$(TOOLS_DIR)/swagger serve -F=redoc --no-open --port 36666 $(ROOT_DIR)/api/swagger/swagger.yaml - -## swagger.help: Display help information about the release package -.PHONY: swagger.help -swagger.help: scripts/make-rules/swagger.mk - $(call smallhelp) \ No newline at end of file diff --git a/scripts/make-rules/tools.mk b/scripts/make-rules/tools.mk deleted file mode 100644 index 5335d094d9..0000000000 --- a/scripts/make-rules/tools.mk +++ /dev/null @@ -1,280 +0,0 @@ -# Copyright © 2023 OpenIMSDK. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# ============================================================================== -# Makefile helper functions for tools(https://github.com/avelino/awesome-go) -> DIR: {TOOT_DIR}/tools | (go >= 1.19) -# Why download to the tools directory, thinking we might often switch Go versions using gvm. -# - -# openim build use BUILD_TOOLS -BUILD_TOOLS ?= golangci-lint goimports addlicense deepcopy-gen conversion-gen ginkgo go-junit-report go-gitlint -# Code analysis tools -ANALYSIS_TOOLS = golangci-lint goimports golines go-callvis kube-score -# Code generation tools -GENERATION_TOOLS = deepcopy-gen conversion-gen protoc-gen-go cfssl rts codegen -# Testing tools -TEST_TOOLS = ginkgo go-junit-report gotests -# tenxun cos tools -COS_TOOLS = coscli coscmd -# Version control tools -VERSION_CONTROL_TOOLS = addlicense go-gitlint git-chglog github-release gsemver -# Utility tools -UTILITY_TOOLS = go-mod-outdated mockgen gothanks richgo kubeconform -# All tools -ALL_TOOLS ?= $(ANALYSIS_TOOLS) $(GENERATION_TOOLS) $(TEST_TOOLS) $(VERSION_CONTROL_TOOLS) $(UTILITY_TOOLS) $(COS_TOOLS) - -## tools.install: Install a must tools -.PHONY: tools.install -tools.install: $(addprefix tools.verify., $(BUILD_TOOLS)) - -## tools.install-all: Install all tools -.PHONY: tools.install-all -tools.install-all: $(addprefix tools.install-all., $(ALL_TOOLS)) - -## tools.install.%: Install a single tool in $GOBIN/ -.PHONY: tools.install.% -tools.install.%: - @echo "===========> Installing $,The default installation path is $(GOBIN)/$*" - @$(MAKE) install.$* - -## tools.install-all.%: Parallelism install a single tool in ./tools/* -.PHONY: tools.install-all.% -tools.install-all.%: - @echo "===========> Installing $,The default installation path is $(TOOLS_DIR)/$*" - @$(MAKE) -j $(nproc) install.$* - -## tools.verify.%: Check if a tool is installed and install it -.PHONY: tools.verify.% -tools.verify.%: - @echo "===========> Verifying $* is installed" - @if [ ! -f $(TOOLS_DIR)/$* ]; then GOBIN=$(TOOLS_DIR) $(MAKE) tools.install.$*; fi - @echo "===========> $* is install in $(TOOLS_DIR)/$*" - -## install.golangci-lint: Install golangci-lint -.PHONY: install.golangci-lint -install.golangci-lint: - @$(GO) install github.com/golangci/golangci-lint/cmd/golangci-lint@$(GOLANGCI_LINT_VERSION) - -## install.goimports: Install goimports, used to format go source files -.PHONY: install.goimports -install.goimports: - @$(GO) install golang.org/x/tools/cmd/goimports@$(GOIMPORTS_VERSION) - -## install.addlicense: Install addlicense, used to add license header to source files -.PHONY: install.addlicense -install.addlicense: - @$(GO) install github.com/google/addlicense@$(ADDLICENSE_VERSION) - -## install.deepcopy-gen: Install deepcopy-gen, used to generate deep copy functions -.PHONY: install.deepcopy-gen -install.deepcopy-gen: - @$(GO) install k8s.io/code-generator/cmd/deepcopy-gen@$(DEEPCOPY_GEN_VERSION) - -## install.conversion-gen: Install conversion-gen, used to generate conversion functions -.PHONY: install.conversion-gen -install.conversion-gen: - @$(GO) install k8s.io/code-generator/cmd/conversion-gen@$(CONVERSION_GEN_VERSION) - -## install.ginkgo: Install ginkgo to run a single test or set of tests -.PHONY: install.ginkgo -install.ginkgo: - @$(GO) install github.com/onsi/ginkgo/ginkgo@$(GINKGO_VERSION) - -## install.go-gitlint: Install go-gitlint, used to check git commit message -.PHONY: install.go-gitlint -install.go-gitlint: - @$(GO) install github.com/marmotedu/go-gitlint/cmd/go-gitlint@$(GO_GITLINT_VERSION) - -## install.go-junit-report: Install go-junit-report, used to convert go test output to junit xml -.PHONY: install.go-junit-report -install.go-junit-report: - @$(GO) install github.com/jstemmer/go-junit-report@$(GO_JUNIT_REPORT_VERSION) - -## install.gotests: Install gotests, used to generate go tests -.PHONY: install.gotests -install.gotests: - @$(GO) install github.com/cweill/gotests/gotests@$(GO_TESTS_VERSION) - -## install.kafkactl: Install kafkactl command line tool. -.PHONY: install.kafkactl -install.kafkactl: - @$(GO) install github.com/deviceinsight/kafkactl@$(KAFKACTL_VERSION) - -## install.go-apidiff: Install go-apidiff, used to check api changes -.PHONY: install.go-apidiff -install.go-apidiff: - @$(GO) install github.com/joelanford/go-apidiff@$(GO_APIDIFF_VERSION) - -## install.swagger: Install swagger, used to generate swagger documentation -.PHONY: install.swagger -install.swagger: - @$(GO) install github.com/go-swagger/go-swagger/cmd/swagger@$(SWAGGER_VERSION) - -# ============================================================================== -# Tools that might be used include go gvm -# - -## install.gotestsum: Install gotestsum, used to run go tests -.PHONY: install.gotestsum -install.gotestsum: - @$(GO) install gotest.tools/gotestsum@$(GOTESTSUM_VERSION) - -## install.kube-score: Install kube-score, used to check kubernetes yaml files -.PHONY: install.kube-score -install.kube-score: - @$(GO) install github.com/zegl/kube-score/cmd/kube-score@$(KUBE_SCORE_VERSION) - -## install.kubeconform: Install kubeconform, used to check kubernetes yaml files -.PHONY: install.kubeconform -install.kubeconform: - @$(GO) install github.com/yannh/kubeconform/cmd/kubeconform@$(KUBECONFORM_VERSION) - -## install.gsemver: Install gsemver, used to generate semver -.PHONY: install.gsemver -install.gsemver: - @$(GO) install github.com/arnaud-deprez/gsemver@$(GSEMVER_VERSION) - -## install.git-chglog: Install git-chglog, used to generate changelog -.PHONY: install.git-chglog -install.git-chglog: - @$(GO) install github.com/git-chglog/git-chglog/cmd/git-chglog@$(GIT_CHGLOG_VERSION) - -## install.ko: Install ko, used to build go program into container images -.PHONY: install.ko -install.ko: - @$(GO) install github.com/google/ko@$(KO_VERSION) - -## install.github-release: Install github-release, used to create github release -.PHONY: install.github-release -install.github-release: - @$(GO) install github.com/github-release/github-release@$(GITHUB_RELEASE_VERSION) - -## install.coscli: Install coscli, used to upload files to cos -# example: ./coscli cp/sync -r /home/off-line/docker-off-line/ cos://openim-1306374445/openim/image/amd/off-line/off-line/ -e cos.ap-guangzhou.myqcloud.com -# https://cloud.tencent.com/document/product/436/71763 -# amd64 -.PHONY: install.coscli -install.coscli: - @wget -q https://github.com/tencentyun/coscli/releases/download/$(COSCLI_VERSION)/coscli-linux -O ${TOOLS_DIR}/coscli - @chmod +x ${TOOLS_DIR}/coscli - -## install.coscmd: Install coscmd, used to upload files to cos -.PHONY: install.coscmd -install.coscmd: - @if which pip &>/dev/null; then pip install coscmd; else pip3 install coscmd; fi - -## install.minio: Install minio, used to upload files to minio -.PHONY: install.minio -install.minio: - @$(GO) install github.com/minio/minio@$(MINIO_VERSION) - -## install.delve: Install delve, used to debug go program -.PHONY: install.delve -install.delve: - @$(GO) install github.com/go-delve/delve/cmd/dlv@$(DELVE_VERSION) - -## install.air: Install air, used to hot reload go program -.PHONY: install.air -install.air: - @$(GO) install github.com/cosmtrek/air@$(AIR_VERSION) - -## install.gvm: Install gvm, gvm is a Go version manager, built on top of the official go tool. -.PHONY: install.gvm -install.gvm: - @echo "===========> Installing gvm, The default installation path is ~/.gvm/scripts/gvm" - @bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer) - @source /root/.gvm/scripts/gvm - -## install.golines: Install golines, used to format long lines -.PHONY: install.golines -install.golines: - @$(GO) install github.com/segmentio/golines@$(GOLINES_VERSION) - -## install.go-mod-outdated: Install go-mod-outdated, used to check outdated dependencies -.PHONY: install.go-mod-outdated -install.go-mod-outdated: - @$(GO) install github.com/psampaz/go-mod-outdated@$(GO_MOD_OUTDATED_VERSION) - -## install.mockgen: Install mockgen, used to generate mock functions -.PHONY: install.mockgen -install.mockgen: - @$(GO) install github.com/golang/mock/mockgen@$(MOCKGEN_VERSION) - -## install.wire: Install wire, used to generate wire files -.PHONY: install.wire -install.wire: - @$(GO) install github.com/google/wire/cmd/wire@$(WIRE_VERSION) - - -## install.protoc-gen-go: Install protoc-gen-go, used to generate go source files from protobuf files -.PHONY: install.protoc-gen-go -install.protoc-gen-go: - @$(GO) install github.com/golang/protobuf/protoc-gen-go@$(PROTOC_GEN_GO_VERSION) - -## install.cfssl: Install cfssl, used to generate certificates -.PHONY: install.cfssl -install.cfssl: - @$(ROOT_DIR)/scripts/install/install.sh openim::install::install_cfssl - -## install.depth: Install depth, used to check dependency tree -.PHONY: install.depth -install.depth: - @$(GO) install github.com/KyleBanks/depth/cmd/depth@$(DEPTH_VERSION) - -## install.go-callvis: Install go-callvis, used to visualize call graph -.PHONY: install.go-callvis -install.go-callvis: - @$(GO) install github.com/ofabry/go-callvis@$(GO_CALLVIS_VERSION) - -## install.misspell: Install misspell -.PHONY: install.misspell -install.misspell: - @$(GO) install github.com/client9/misspell/cmd/misspell@$(MISSPELL_VERSION) - -## install.gothanks: Install gothanks, used to thank go dependencies -.PHONY: install.gothanks -install.gothanks: - @$(GO) install github.com/psampaz/gothanks@$(GOTHANKS_VERSION) - -## install.richgo: Install richgo -.PHONY: install.richgo -install.richgo: - @$(GO) install github.com/kyoh86/richgo@$(RICHGO_VERSION) - -## install.rts: Install rts -.PHONY: install.rts -install.rts: - @$(GO) install github.com/galeone/rts/cmd/rts@$(RTS_VERSION) - -# ================= kubecub openim tools ========================================= -# https://github.com/kubecub -## install.typecheck: Install kubecub typecheck, checks for go code -.PHONY: install.typecheck -install.typecheck: - @$(GO) install github.com/kubecub/typecheck@$(TYPECHECK_VERSION) - -## install.comment-lang-detector: Install kubecub comment-lang-detector, checks for go code comment language -.PHONY: install.comment-lang-detector -install.comment-lang-detector: - @$(GO) install github.com/kubecub/comment-lang-detector/cmd/cld@$(COMMENT_LANG_DETECTOR_VERSION) - -## install.standardizer: Install kubecub standardizer, checks for go code standardization -.PHONY: install.standardizer -install.standardizer: - @$(GO) install github.com/kubecub/standardizer@$(STANDARDIZER_VERSION) - -## tools.help: Display help information about the tools package -.PHONY: tools.help -tools.help: scripts/make-rules/tools.mk - $(call smallhelp) \ No newline at end of file diff --git a/scripts/release.sh b/scripts/release.sh deleted file mode 100755 index eb8b04359f..0000000000 --- a/scripts/release.sh +++ /dev/null @@ -1,150 +0,0 @@ -#!/usr/bin/env bash -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# Description: -# This script automates the process of building and releasing OpenIM, -# including tasks like setting up the environment, verifying prerequisites, -# building commands, packaging tarballs, uploading tarballs, creating GitHub -# releases, and generating changelogs. -# -# Usage: -# ./scripts/release.sh [options] -# Options include: -# -h, --help : Show help message -# -se, --setup-env : Execute setup environment -# -vp, --verify-prereqs : Execute prerequisites verification -# -bc, --build-command : Execute build command -# -bi, --build-image : Execute build image (default: not executed) -# -pt, --package-tarballs : Execute package tarballs -# -ut, --upload-tarballs : Execute upload tarballs -# -gr, --github-release : Execute GitHub release -# -gc, --generate-changelog: Execute generate changelog -# -# This script can also be executed via the 'make release' command as an alternative. -# -# Dependencies: -# This script depends on external scripts found in the 'scripts' directory and -# assumes the presence of necessary tools and permissions for building and -# releasing software. -# -# Note: -# The script uses standard bash script practices with error handling, -# and it defaults to executing all steps if no specific option is provided. -# -# Build a OpenIM release. This will build the binaries, create the Docker -# images and other build artifacts. -# Build a OpenIM release. This script supports various flags for flexible execution control. - - - - -OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/.. -source "${OPENIM_ROOT}/scripts/common.sh" -source "${OPENIM_ROOT}/scripts/lib/release.sh" - -OPENIM_RELEASE_RUN_TESTS=${OPENIM_RELEASE_RUN_TESTS-y} - -# Function to show help message -show_help() { - echo "Usage: $(basename $0) [options]" - echo "Options:" - echo " -h, --help Show this help message" - echo " -se, --setup-env Execute setup environment" - echo " -vp, --verify-prereqs Execute prerequisites verification" - echo " -bc, --build-command Execute build command" - echo " -bi, --build-image Execute build image (default: not executed)" - echo " -pt, --package-tarballs Execute package tarballs" - echo " -ut, --upload-tarballs Execute upload tarballs" - echo " -gr, --github-release Execute GitHub release" - echo " -gc, --generate-changelog Execute generate changelog" -} - -# Initialize all actions to false -perform_setup_env=false -perform_verify_prereqs=false -perform_build_command=false -perform_build_image=false # New flag for build image -perform_package_tarballs=false -perform_upload_tarballs=false -perform_github_release=false -perform_generate_changelog=false - -# Process command-line arguments -while getopts "hsevpbciptutgrgc-" opt; do - case "${opt}" in - h) show_help; exit 0 ;; - se) perform_setup_env=true ;; - vp) perform_verify_prereqs=true ;; - bc) perform_build_command=true ;; - bi) perform_build_image=true ;; # Handling new option - pt) perform_package_tarballs=true ;; - ut) perform_upload_tarballs=true ;; - gr) perform_github_release=true ;; - gc) perform_generate_changelog=true ;; - --) case "${OPTARG}" in - help) show_help; exit 0 ;; - setup-env) perform_setup_env=true ;; - verify-prereqs) perform_verify_prereqs=true ;; - build-command) perform_build_command=true ;; - build-image) perform_build_image=true ;; # Handling new long option - package-tarballs) perform_package_tarballs=true ;; - upload-tarballs) perform_upload_tarballs=true ;; - github-release) perform_github_release=true ;; - generate-changelog) perform_generate_changelog=true ;; - *) echo "Invalid option: --${OPTARG}"; show_help; exit 1 ;; - esac ;; - *) show_help; exit 1 ;; - esac -done - -# Enable all actions by default if no options are provided -if [ "$#" -eq 0 ]; then - perform_setup_env=true - perform_verify_prereqs=true - perform_build_command=true - perform_package_tarballs=true - perform_upload_tarballs=true - perform_github_release=true - perform_generate_changelog=true - # TODO: Not enabling build_image by default - # perform_build_image=true -fi - -# Function to perform actions -perform_action() { - local flag=$1 - local message=$2 - local command=$3 - - if [ "$flag" == true ]; then - openim::log::info "## $message..." - if ! eval "$command"; then - openim::log::errexit "Error in $message" - fi - fi -} - -echo "Starting script execution..." - -perform_action $perform_setup_env "Setting up environment" "openim::golang::setup_env" -perform_action $perform_verify_prereqs "Verifying prerequisites" "openim::build::verify_prereqs && openim::release::verify_prereqs" -perform_action $perform_build_command "Executing build command" "openim::build::build_command" -perform_action $perform_build_image "Building image" "openim::build::build_image" -perform_action $perform_package_tarballs "Packaging tarballs" "openim::release::package_tarballs" -perform_action $perform_upload_tarballs "Uploading tarballs" "openim::release::upload_tarballs" -perform_action $perform_github_release "Creating GitHub release" "openim::release::github_release" -perform_action $perform_generate_changelog "Generating changelog" "openim::release::generate_changelog" - -openim::log::success "OpenIM Relase Script Execution Completed." diff --git a/scripts/start-all.sh b/scripts/start-all.sh deleted file mode 100755 index 6f4a6c5748..0000000000 --- a/scripts/start-all.sh +++ /dev/null @@ -1,109 +0,0 @@ -#!/usr/bin/env bash -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -#FIXME This script is the startup script for multiple servers. -#FIXME The full names of the shell scripts that need to be started are placed in the `need_to_start_server_shell` array. - -OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/.. -source "${OPENIM_ROOT}/scripts/install/common.sh" - -# Function to execute the scripts. -function execute_start_scripts() { - for script_path in "${OPENIM_SERVER_SCRIPT_START_LIST[@]}"; do - # Extract the script name without extension for argument generation. - script_name_with_prefix=$(basename "$script_path" .sh) - - # Remove the "openim-" prefix. - script_name=${script_name_with_prefix#openim-} - - # Construct the argument based on the script name. - arg="openim::${script_name}::start" - - # Check if the script file exists and is executable. - if [[ -x "$script_path" ]]; then - openim::log::colorless "Starting script: ${script_path##*/}" # Log the script name. - # Execute the script with the constructed argument. - result=$("$script_path" "$arg") - if [[ $? -ne 0 ]]; then - openim::log::error "Start script: ${script_path##*/} failed" - openim::log::error "$result" - return 1 - fi - - else - openim::log::errexit "Script ${script_path##*/} is missing or not executable." - return 1 - fi - done -} - -if openim::util::is_running_in_container; then - exec >> ${DOCKER_LOG_FILE} 2>&1 -fi - -openim::golang::check_openim_binaries -if [[ $? -ne 0 ]]; then - openim::log::error "OpenIM binaries are not found. Please run 'make build' to build binaries." - "${OPENIM_ROOT}"/scripts/build-all-service.sh -fi - -"${OPENIM_ROOT}"/scripts/init-config.sh --skip - -#openim::log::print_blue "Execute the following script in sequence: ${OPENIM_SERVER_SCRIPTARIES[@]}" - -# TODO Prelaunch tools, simple for now, can abstract functions later -TOOLS_START_SCRIPTS_PATH=${START_SCRIPTS_PATH}/openim-tools.sh - -openim::log::status "Start the pre-start tools:" - -# if ! ${TOOLS_START_SCRIPTS_PATH} openim::tools::pre-start; then -# openim::log::error "Start the pre-start tools, aborting!" -# exit 1 -# fi - -openim::log::colorless "pre-start has been successfully completed!" - -result=$("${OPENIM_ROOT}"/scripts/stop-all.sh) -if [[ $? -ne 0 ]]; then - openim::log::error "View the error logs from this startup. ${LOG_FILE} \n" - openim::log::error "Some programs have not exited; the start process is aborted .\n $result" - exit 1 -fi - -openim::log::status "Start the OpenIM startup scripts: " -execute_start_scripts -openim::log::status "OpenIM startup scripts have been successfully completed!" - -sleep 2 - -result=$(. $(dirname ${BASH_SOURCE})/install/openim-msgtransfer.sh openim::msgtransfer::check) -if [[ $? -ne 0 ]]; then - openim::log::error "The OpenIM services may fail to start.\n $result" - exit 1 -fi - -result=$(openim::util::check_process_names ${OPENIM_ALL_SERVICE_LIBRARIES_NO_TRANSFER[@]}) -if [[ $? -ne 0 ]]; then - openim::log::error "The OpenIM services may fail to start.\n $result" - exit 1 -fi - -openim::log::status "Start the post-start tools:" -# ${TOOLS_START_SCRIPTS_PATH} openim::tools::post-start -openim::log::status "post-start has been successfully completed!" -openim::util::find_ports_for_all_services ${OPENIM_ALL_SERVICE_LIBRARIES_NO_TRANSFER[@]} -openim::util::find_ports_for_all_services ${OPENIM_MSGTRANSFER_BINARY[@]} - -openim::log::success "All OpenIM services have been successfully started!" \ No newline at end of file diff --git a/scripts/stop-all.sh b/scripts/stop-all.sh deleted file mode 100755 index b2572f7d5e..0000000000 --- a/scripts/stop-all.sh +++ /dev/null @@ -1,60 +0,0 @@ -#!/usr/bin/env bash -# Copyright © 2023 OpenIM. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# This script is stop all openim service -# -# Usage: `scripts/stop.sh`. -# Encapsulated as: `make stop`. - - - - - -OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/.. - -source "${OPENIM_ROOT}/scripts/install/common.sh" - -openim::log::status "Begin to stop all openim service" - -openim::log::status "Stop all processes in the path ${OPENIM_OUTPUT_HOSTBIN}" - -openim::util::stop_services_with_name "${OPENIM_OUTPUT_HOSTBIN}" -# todo OPENIM_ALL_SERVICE_LIBRARIES - - - - -max_retries=15 -attempt=0 - -while [[ $attempt -lt $max_retries ]] -do - result=$(openim::util::check_process_names_for_stop) - - if [[ $? -ne 0 ]]; then - if [[ $attempt -ne 0 ]] ; then - echo "+++ cat openim log file >>> ${LOG_FILE} " $attempt - openim::log::error "stop process failed. continue waiting\n" "${result}" - fi - sleep 1 - ((attempt++)) - else - openim::log::success " All openim processes to be stopped" - exit 0 - fi -done - -openim::log::error "openim processes stopped failed" -exit 1 From 2e99b509d57be303063ff29bff91c71cf2a461ac Mon Sep 17 00:00:00 2001 From: skiffer-git <44203734@qq.com> Date: Thu, 25 Apr 2024 14:37:56 +0800 Subject: [PATCH 5/6] Clean up unnecessary scripts and templates --- docs/CODEOWNERS | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/CODEOWNERS b/docs/CODEOWNERS index e69de29bb2..015466d027 100644 --- a/docs/CODEOWNERS +++ b/docs/CODEOWNERS @@ -0,0 +1 @@ +* @Bloomingg @FGadvancer @skiffer-git @withchao From 15e6c2a987701caa18441af80fe97a7e6e0e3bef Mon Sep 17 00:00:00 2001 From: skiffer-git <44203734@qq.com> Date: Thu, 25 Apr 2024 14:38:46 +0800 Subject: [PATCH 6/6] Clean up unnecessary scripts and templates --- scripts/make-rules/common-versions.mk | 0 scripts/make-rules/common.mk | 196 ++++++++++++++++++ scripts/make-rules/copyright.mk | 63 ++++++ scripts/make-rules/dependencies.mk | 41 ++++ scripts/make-rules/gen.mk | 100 +++++++++ scripts/make-rules/golang.mk | 272 +++++++++++++++++++++++++ scripts/make-rules/image.mk | 183 +++++++++++++++++ scripts/make-rules/release.mk | 42 ++++ scripts/make-rules/swagger.mk | 19 ++ scripts/make-rules/tools.mk | 280 ++++++++++++++++++++++++++ 10 files changed, 1196 insertions(+) create mode 100644 scripts/make-rules/common-versions.mk create mode 100644 scripts/make-rules/common.mk create mode 100644 scripts/make-rules/copyright.mk create mode 100644 scripts/make-rules/dependencies.mk create mode 100644 scripts/make-rules/gen.mk create mode 100644 scripts/make-rules/golang.mk create mode 100644 scripts/make-rules/image.mk create mode 100644 scripts/make-rules/release.mk create mode 100644 scripts/make-rules/swagger.mk create mode 100644 scripts/make-rules/tools.mk diff --git a/scripts/make-rules/common-versions.mk b/scripts/make-rules/common-versions.mk new file mode 100644 index 0000000000..e69de29bb2 diff --git a/scripts/make-rules/common.mk b/scripts/make-rules/common.mk new file mode 100644 index 0000000000..ffbb69a55d --- /dev/null +++ b/scripts/make-rules/common.mk @@ -0,0 +1,196 @@ +# Copyright © 2023 OpenIMSDK. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# ============================================================================== +# Makefile helper functions for common tasks +# + +SHELL := /bin/bash +GO:=go +DIRS=$(shell ls) +DEBUG ?= 0 +GIT_TAG := $(shell git describe --exact-match --tags --abbrev=0 2> /dev/null || echo untagged) +GIT_COMMIT ?= $(shell git rev-parse --short HEAD || echo "0.0.0") +BUILD_DATE ?=$(shell date -u +'%Y-%m-%dT%H:%M:%SZ') # Blank error: date '+%FT %T %z':"buildDate":"2023-03-31T 20:05:43 +0800" + +# include the common makefile +COMMON_SELF_DIR := $(dir $(lastword $(MAKEFILE_LIST))) + +SRC = $(shell find . -type f -name '*.go' -not -path "./vendor/*") + +# ROOT_DIR: root directory of the code base +ifeq ($(origin ROOT_DIR),undefined) +ROOT_DIR := $(abspath $(shell cd $(COMMON_SELF_DIR)/../.. && pwd -P)) +endif + +# OUTPUT_DIR: The directory where the build output is stored. +ifeq ($(origin OUTPUT_DIR),undefined) +OUTPUT_DIR := $(ROOT_DIR)/_output +$(shell mkdir -p $(OUTPUT_DIR)) +endif + +# BIN_DIR: Directory where executable files are stored. +ifeq ($(origin BIN_DIR),undefined) +BIN_DIR := $(OUTPUT_DIR)/bin +$(shell mkdir -p $(BIN_DIR)) +endif + +# BIN_TOOLS_DIR: Directory where executable files are stored. +ifeq ($(origin BIN_TOOLS_DIR),undefined) +BIN_TOOLS_DIR := $(BIN_DIR)/tools +$(shell mkdir -p $(BIN_TOOLS_DIR)) +endif + +# LOGS_DIR: Directory where log files are stored. +ifeq ($(origin LOGS_DIR),undefined) +LOGS_DIR := $(OUTPUT_DIR)/logs +$(shell mkdir -p $(LOGS_DIR)) +endif + +# TOOLS_DIR: The directory where tools are stored for build and testing. +ifeq ($(origin TOOLS_DIR),undefined) +TOOLS_DIR := $(OUTPUT_DIR)/tools +$(shell mkdir -p $(TOOLS_DIR)) +endif + +# TMP_DIR: directory where temporary files are stored. +ifeq ($(origin TMP_DIR),undefined) +TMP_DIR := $(OUTPUT_DIR)/tmp +$(shell mkdir -p $(TMP_DIR)) +endif + +ifeq ($(origin VERSION), undefined) +# VERSION := $(shell git describe --tags --always --match='v*') +# git describe --tags --always --match="v*" --dirty +# VERSION := $(shell git describe --tags --always --match="v*" --dirty | sed 's/-/./g') #v2.3.3.631.g00abdc9b.dirty +VERSION := $(shell git describe --tags --always --match='v*') +# v2.3.3: git tag +endif + +# Helper function to get dependency version from go.mod +get_gomod_version = $(shell go list -m $1 | awk '{print $$2}') +define go_install +$(info ===========> Installing $(1)@$(2)) +$(GO) install $(1)@$(2) +endef + +# Check if the tree is dirty. default to dirty(maybe u should commit?) +GIT_TREE_STATE:="dirty" +ifeq (, $(shell git status --porcelain 2>/dev/null)) + GIT_TREE_STATE="clean" +endif +GIT_COMMIT:=$(shell git rev-parse HEAD) + +# Minimum test coverage +# can u use make cover COVERAGE=90 +ifeq ($(origin COVERAGE),undefined) +COVERAGE := 60 +endif + +# Get the currently used golang install path (in GOPATH/bin, unless GOBIN is set) +ifeq (,$(shell go env GOBIN)) +GOBIN=$(shell go env GOPATH)/bin +else +GOBIN=$(shell go env GOBIN) +endif + +# The OS must be linux when building docker images +# PLATFORMS ?= linux_amd64 linux_arm64 +# The OS can be linux/windows/darwin when building binaries +PLATFORMS ?= linux_s390x linux_mips64 linux_mips64le darwin_amd64 darwin_arm64 windows_amd64 linux_amd64 linux_arm64 linux_ppc64le # wasip1_wasm + +# set a specific PLATFORM, defaults to the host platform +ifeq ($(origin PLATFORM), undefined) + ifeq ($(origin GOARCH), undefined) + GOARCH := $(shell go env GOARCH) + endif + # Determine the host OS + GOOS := $(shell go env GOOS) + PLATFORM := $(GOOS)_$(GOARCH) + # Use the host OS and GOARCH as the default when building images + IMAGE_PLAT := $(PLATFORM) +else + # Parse the PLATFORM variable + GOOS := $(word 1, $(subst _, ,$(PLATFORM))) + GOARCH := $(word 2, $(subst _, ,$(PLATFORM))) + IMAGE_PLAT := $(PLATFORM) +endif + + +# Protobuf file storage path +APIROOT=$(ROOT_DIR)/pkg/proto + +# Linux command settings +# TODO: Whether you need to join utils? +FIND := find . ! -path './utils/*' ! -path './vendor/*' ! -path './third_party/*' ! -path './components/*' ! -path './logs/*' +XARGS := xargs -r --no-run-if-empty + +# Linux command settings-CODE DIRS Copyright +CODE_DIRS := $(ROOT_DIR)/pkg $(ROOT_DIR)/cmd $(ROOT_DIR)/config $(ROOT_DIR)/internal $(ROOT_DIR)/scripts $(ROOT_DIR)/test $(ROOT_DIR)/.github $(ROOT_DIR)/build $(ROOT_DIR)/tools $(ROOT_DIR)/deployments +FINDS := find $(CODE_DIRS) + +# Makefile settings: Select different behaviors by determining whether V option is set +ifndef V +MAKEFLAGS += --no-print-directory +endif + +# COMMA: Concatenate multiple strings to form a list of strings +COMMA := , +# SPACE: Used to separate strings +SPACE := +# SPACE: Replace multiple consecutive Spaces with a single space +SPACE += + +# ============================================================================== +# Makefile helper functions for common tasks + +# Help information for the makefile package +define makehelp + @printf "\n\033[1mUsage: make ...\033[0m\n\n\\033[1mTargets:\\033[0m\n\n" + @sed -n 's/^##//p' $< | awk -F':' '{printf "\033[36m%-28s\033[0m %s\n", $$1, $$2}' | sed -e 's/^/ /' + @printf "\n\033[1m$$USAGE_OPTIONS\033[0m\n" +endef + +# Here are some examples of builds +define MAKEFILE_EXAMPLE +# make build BINS=openim-api Only a single openim-api binary is built. +# make -j (nproc) all Run tidy gen add-copyright format lint cover build concurrently. +# make gen Generate all necessary files. +# make release Build release binaries for all platforms. +# make verify-copyright Verify the license headers for all files. +# make install-deepcopy-gen Install deepcopy-gen tools if the license is missing. +# make build BINS=openim-api V=1 DEBUG=1 Build debug binaries for only openim-api. +# make multiarch -j PLATFORMS="linux_arm64 linux_amd64" V=1 Build binaries for both platforms. +# make image +endef +export MAKEFILE_EXAMPLE + +# Define all help functions @printf "\n\033[1mCurrent openim-api version information: $(shell openim-api version):\033[0m\n\n" +define makeallhelp + @printf "\n\033[1mMake example:\033[0m\n\n" + $(call MAKEFILE_EXAMPLE) + @printf "\n\033[1mAriables:\033[0m\n\n" + @echo " DEBUG: $(DEBUG)" + @echo " BINS: $(BINS)" + @echo " PLATFORMS: $(PLATFORMS)" + @echo " V: $(V)" +endef + +# Help information for other makefile packages +CUT_OFF?="---------------------------------------------------------------------------------" +HELP_NAME:=$(shell basename $(MAKEFILE_LIST)) +define smallhelp + @sed -n 's/^##//p' $< | awk -F':' '{printf "\033[36m%-35s\033[0m %s\n", $$1, $$2}' | sed -e 's/^/ /' + @echo $(CUT_OFF) +endef \ No newline at end of file diff --git a/scripts/make-rules/copyright.mk b/scripts/make-rules/copyright.mk new file mode 100644 index 0000000000..18b4dd7097 --- /dev/null +++ b/scripts/make-rules/copyright.mk @@ -0,0 +1,63 @@ +# Copyright © 2023 OpenIMSDK. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# ============================================================================== +# wget https://github.com/google/addlicense/releases/download/v1.0.0/addlicense_1.0.0_Linux_x86_64.tar.gz +# Makefile helper functions for copyright +# + +LICENSE_TEMPLATE ?= $(ROOT_DIR)/scripts/template/LICENSE_TEMPLATES + +## copyright.verify: Validate boilerplate headers for assign files +.PHONY: copyright.verify +copyright.verify: tools.verify.addlicense + @echo "===========> Validate boilerplate headers for assign files starting in the $(ROOT_DIR) directory" + @$(TOOLS_DIR)/addlicense -v -check -ignore **/test/** -ignore **pb** -f $(LICENSE_TEMPLATE) $(CODE_DIRS) + @echo "===========> End of boilerplate headers check..." + +## copyright.add: Add the boilerplate headers for all files +.PHONY: copyright.add +copyright.add: tools.verify.addlicense + @echo "===========> Adding $(LICENSE_TEMPLATE) the boilerplate headers for all files" + @$(TOOLS_DIR)/addlicense -y $(shell date +"%Y") -ignore **pb** -v -c "OpenIM." -f $(LICENSE_TEMPLATE) $(CODE_DIRS) + @echo "===========> End the copyright is added..." + +# Addlicense Flags: +# -c string +# copyright holder (default "Google LLC") +# -check +# check only mode: verify presence of license headers and exit with non-zero code if missing +# -f string +# license file +# -ignore value +# file patterns to ignore, for example: -ignore **/*.go -ignore vendor/** +# -l string +# license type: apache, bsd, mit, mpl (default "apache") +# -s Include SPDX identifier in license header. Set -s=only to only include SPDX identifier. +# -skip value +# [deprecated: see -ignore] file extensions to skip, for example: -skip rb -skip go +# -v verbose mode: print the name of the files that are modified or were skipped +# -y string +# copyright year(s) (default "2023") + +## copyright.advertise: Advertise the license of the project +.PHONY: copyright.advertise +copyright.advertise: + @chmod +x $(ROOT_DIR)/scripts/advertise.sh + @$(ROOT_DIR)/scripts/advertise.sh + +## copyright.help: Show copyright help +.PHONY: copyright.help +copyright.help: scripts/make-rules/copyright.mk + $(call smallhelp) \ No newline at end of file diff --git a/scripts/make-rules/dependencies.mk b/scripts/make-rules/dependencies.mk new file mode 100644 index 0000000000..1a896f1ff5 --- /dev/null +++ b/scripts/make-rules/dependencies.mk @@ -0,0 +1,41 @@ +# Copyright © 2023 OpenIMSDK. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# ============================================================================== +# Makefile helper functions for dependencies +# + +.PHONY: dependencies.run +dependencies.run: dependencies.packages dependencies.tools + +.PHONY: dependencies.packages +dependencies.packages: + @$(GO) mod tidy + +.PHONY: dependencies.tools +dependencies.tools: dependencies.tools.blocker dependencies.tools.critical + +.PHONY: dependencies.tools.blocker +dependencies.tools.blocker: go.build.verify $(addprefix tools.verify., $(BLOCKER_TOOLS)) + +.PHONY: dependencies.tools.critical +dependencies.tools.critical: $(addprefix tools.verify., $(CRITICAL_TOOLS)) + +.PHONY: dependencies.tools.trivial +dependencies.tools.trivial: $(addprefix tools.verify., $(TRIVIAL_TOOLS)) + +## dependencies.help: Print help for dependencies targets +.PHONY: dependencies.help +dependencies.help: scripts/make-rules/dependencies.mk + $(call smallhelp) \ No newline at end of file diff --git a/scripts/make-rules/gen.mk b/scripts/make-rules/gen.mk new file mode 100644 index 0000000000..fba0132ff5 --- /dev/null +++ b/scripts/make-rules/gen.mk @@ -0,0 +1,100 @@ +# Copyright © 2023 OpenIMSDK. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# ============================================================================== +# Makefile helper functions for generate necessary files and docs +# https://cloud.redhat.com/blog/kubernetes-deep-dive-code-generation-customresources +# ! The stock of code generated by `make gen` should be idempotent +# +# Questions about go mod instead of go path: https://github.com/kubernetes/kubernetes/issues/117181 +# ============================================================================== +# Makefile helper functions for generate necessary files +# + +## gen.init: Initialize openim server project ✨ +.PHONY: gen.init +gen.init: + @echo "===========> Initializing openim server project" + @${ROOT_DIR}/scripts/init-config.sh + +## gen.init-githooks: Initialize git hooks ✨ +.PHONY: gen.init-githooks +gen.init-githooks: + @echo "===========> Initializing git hooks" + @${ROOT_DIR}/scripts/init-githooks.sh + +## gen.run: Generate necessary files and docs ✨ +.PHONY: gen.run +#gen.run: gen.errcode gen.docgo +gen.run: gen.clean gen.errcode gen.docgo.doc + +## gen.errcode: Generate necessary files and docs ✨ +.PHONY: gen.errcode +gen.errcode: gen.errcode.code gen.errcode.doc + +## gen.errcode.code: Generate openim error code go source files ✨ +.PHONY: gen.errcode.code +gen.errcode.code: tools.verify.codegen + @echo "===========> Generating openim error code go source files" + @codegen -type=int ${ROOT_DIR}/internal/pkg/code + +## gen.errcode.doc: Generate openim error code markdown documentation ✨ +.PHONY: gen.errcode.doc +gen.errcode.doc: tools.verify.codegen + @echo "===========> Generating error code markdown documentation" + @codegen -type=int -doc \ + -output ${ROOT_DIR}/docs/guide/zh-CN/api/error_code_generated.md ${ROOT_DIR}/internal/pkg/code + +## gen.docgo: Generate missing doc.go for go packages ✨ +.PHONY: gen.ca.% +gen.ca.%: + $(eval CA := $(word 1,$(subst ., ,$*))) + @echo "===========> Generating CA files for $(CA)" + @${ROOT_DIR}/scripts/gencerts.sh generate-openim-cert $(OUTPUT_DIR)/cert $(CA) + +## gen.ca: Generate CA files for all certificates ✨ +.PHONY: gen.ca +gen.ca: $(addprefix gen.ca., $(CERTIFICATES)) + +## gen.docgo: Generate missing doc.go for go packages ✨ +.PHONY: gen.docgo.doc +gen.docgo.doc: + @echo "===========> Generating missing doc.go for go packages" + @${ROOT_DIR}/scripts/gendoc.sh + +## gen.docgo.check: Check if there are untracked doc.go files ✨ +.PHONY: gen.docgo.check +gen.docgo.check: gen.docgo.doc + @n="$$(git ls-files --others '*/doc.go' | wc -l)"; \ + if test "$$n" -gt 0; then \ + git ls-files --others '*/doc.go' | sed -e 's/^/ /'; \ + echo "$@: untracked doc.go file(s) exist in working directory" >&2 ; \ + false ; \ + fi + +## gen.docgo.add: Add untracked doc.go files to git index ✨ +.PHONY: gen.docgo.add +gen.docgo.add: + @git ls-files --others '*/doc.go' | $(XARGS) -- git add + +## gen.docgo: Generate missing doc.go for go packages ✨ +.PHONY: gen.clean +gen.clean: + @rm -rf ./api/client/{clientset,informers,listers} + @$(FIND) -type f -name '*_generated.go' -delete + +## gen.help: show help for gen +.PHONY: gen.help +gen.help: scripts/make-rules/gen.m + $(call smallhelp) \ No newline at end of file diff --git a/scripts/make-rules/golang.mk b/scripts/make-rules/golang.mk new file mode 100644 index 0000000000..cdc5a5fc95 --- /dev/null +++ b/scripts/make-rules/golang.mk @@ -0,0 +1,272 @@ +# Copyright © 2023 OpenIMSDK. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# ============================================================================== +# Build management helpers. These functions help to set, save and load the +# + +GO := go +GO_MINIMUM_VERSION ?= 1.19 + +GO_LDFLAGS += -X $(VERSION_PACKAGE).gitVersion=$(GIT_TAG) \ + -X $(VERSION_PACKAGE).gitCommit=$(GIT_COMMIT) \ + -X $(VERSION_PACKAGE).gitTreeState=$(GIT_TREE_STATE) \ + -X $(VERSION_PACKAGE).buildDate=$(BUILD_DATE) \ + -s -w # -s -w deletes debugging information and symbol tables +ifeq ($(DEBUG), 1) + GO_BUILD_FLAGS += -gcflags "all=-N -l" + GO_LDFLAGS= +endif + +GO_BUILD_FLAGS += -ldflags "$(GO_LDFLAGS)" + +ifeq ($(GOOS),windows) + GO_OUT_EXT := .exe +endif + +ifeq ($(ROOT_PACKAGE),) + $(error the variable ROOT_PACKAGE must be set prior to including golang.mk, ->/Makefile) +endif + +GOPATH ?= $(shell go env GOPATH) +ifeq ($(origin GOBIN), undefined) + GOBIN := $(GOPATH)/bin +endif + +# COMMANDS is Specify all files under ${ROOT_DIR}/cmd/ and ${ROOT_DIR}/tools/ except those ending in.md +COMMANDS ?= $(filter-out %.md, $(wildcard ${ROOT_DIR}/cmd/* ${ROOT_DIR}/tools/* ${ROOT_DIR}/tools/data-conversion/chat/cmd/* ${ROOT_DIR}/tools/data-conversion/openim/cmd/* ${ROOT_DIR}/cmd/openim-rpc/*)) +ifeq (${COMMANDS},) + $(error Could not determine COMMANDS, set ROOT_DIR or run in source dir) +endif + +# BINS is the name of each file in ${COMMANDS}, excluding the directory path +# If there are no files in ${COMMANDS}, or if all files end in.md, ${BINS} will be empty +BINS ?= $(foreach cmd,${COMMANDS},$(notdir ${cmd})) +ifeq (${BINS},) + $(error Could not determine BINS, set ROOT_DIR or run in source dir) +endif + +ifeq ($(OS),Windows_NT) + NULL := + SPACE := $(NULL) $(NULL) + ROOT_DIR := $(subst $(SPACE),\$(SPACE),$(shell cd)) +else + ROOT_DIR := $(shell pwd) +endif + +ifeq ($(strip $(COMMANDS)),) + $(error Could not determine COMMANDS, set ROOT_DIR or run in source dir) +endif +ifeq ($(strip $(BINS)),) + $(error Could not determine BINS, set ROOT_DIR or run in source dir) +endif + +# TODO: EXCLUDE_TESTS variable, which contains the name of the package to be excluded from the test +EXCLUDE_TESTS=github.com/openimsdk/open-im-server/test github.com/openimsdk/open-im-server/v3/pkg/log github.com/openimsdk/open-im-server/db github.com/openimsdk/open-im-server/scripts github.com/openimsdk/open-im-server/config + +# ============================================================================== +# ❯ tree -L 1 cmd +# cmd +# ├── openim-sdk-core/ - main.go +# ├── openim-api +# ├── openim_cms_api +# ├── openim-crontask +# ├── openim_demo +# ├── openim-rpc-msg_gateway +# ├── openim-msgtransfer +# ├── openim-push +# ├── rpc/openim_admin_cms/ - main.go +# └── test/ - main.go +# COMMAND=openim +# PLATFORM=linux_amd64 +# OS=linux +# ARCH=amd64 +# BINS=openim-api openim_cms_api openim-crontask openim_demo openim-rpc-msg_gateway openim-msgtransfer openim-push +# BIN_DIR=/root/workspaces/OpenIM/_output/bin +# ============================================================================== + +## go.build: Build binaries +.PHONY: go.build +go.build: go.build.verify $(addprefix go.build., $(addprefix $(PLATFORM)., $(BINS))) + @echo "===========> Building binary $(BINS) $(VERSION) for $(PLATFORM)" + +## go.start: Start openim +.PHONY: go.start +go.start: + @echo "=========================> Starting OpenIM <=========================" + @$(ROOT_DIR)/scripts/start-all.sh + +## go.stop: Stop openim +.PHONY: go.stop +go.stop: + @echo "=========================> Stopping OpenIM <=========================" + @$(ROOT_DIR)/scripts/stop-all.sh + +## go.check: Check openim +.PHONY: go.check +go.check: + @echo "=========================> Checking OpenIM <=========================" + @$(ROOT_DIR)/scripts/check-all.sh + +## go.check-component: Check openim component +.PHONY: go.check-component +go.check-component: + @echo "=========================> Checking OpenIM component <=========================" + @$(ROOT_DIR)/scripts/install/openim-tools.sh openim::tools::pre-start + +## go.versionchecker: Design, detect some environment variables and versions +go.versionchecker: + @$(ROOT_DIR)/scripts/install/openim-tools.sh openim::tools::post-start + +## go.build.verify: Verify that a suitable version of Go exists +.PHONY: go.build.verify +go.build.verify: +ifneq ($(shell $(GO) version|awk -v min=$(GO_MINIMUM_VERSION) '{gsub(/go/,"",$$3);if($$3 >= min){print 0}else{print 1}}'), 0) + $(error unsupported go version. Please install a go version which is greater than or equal to '$(GO_MINIMUM_VERSION)') +endif + +## go.build.%: Build binaries for a specific platform +# CGO_ENABLED=0 https://wiki.musl-libc.org/functional-differences-from-glibc.html +.PHONY: go.build.% +go.build.%: + $(eval COMMAND := $(word 2,$(subst ., ,$*))) + $(eval PLATFORM := $(word 1,$(subst ., ,$*))) + $(eval OS := $(word 1,$(subst _, ,$(PLATFORM)))) + $(eval ARCH := $(word 2,$(subst _, ,$(PLATFORM)))) + @echo "=====> COMMAND=$(COMMAND)" + @echo "=====> PLATFORM=$(PLATFORM)" + @echo "===========> Building binary $(COMMAND) $(VERSION) for $(OS)_$(ARCH)" + @mkdir -p $(BIN_DIR)/platforms/$(OS)/$(ARCH) + @if [ "$(COMMAND)" == "openim-sdk-core" ]; then \ + echo "===========> DEBUG: OpenIM-SDK-Core It is no longer supported for openim-server $(COMMAND)"; \ + elif [ -d $(ROOT_DIR)/cmd/openim-rpc/$(COMMAND) ]; then \ + CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) $(GO) build $(GO_BUILD_FLAGS) -o \ + $(BIN_DIR)/platforms/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT) $(ROOT_DIR)/cmd/openim-rpc/$(COMMAND)/main.go; \ + else \ + if [ -f $(ROOT_DIR)/cmd/$(COMMAND)/main.go ]; then \ + CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) $(GO) build $(GO_BUILD_FLAGS) -o \ + $(BIN_DIR)/platforms/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT) $(ROOT_DIR)/cmd/$(COMMAND)/main.go; \ + elif [ -f $(ROOT_DIR)/tools/$(COMMAND)/$(COMMAND).go ]; then \ + CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) $(GO) build $(GO_BUILD_FLAGS) -o \ + $(BIN_TOOLS_DIR)/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT) $(ROOT_DIR)/tools/$(COMMAND)/$(COMMAND).go; \ + chmod +x $(BIN_TOOLS_DIR)/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT); \ + elif [ -f $(ROOT_DIR)/tools/data-conversion/openim/cmd/$(COMMAND)/$(COMMAND).go ]; then \ + CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) $(GO) build $(GO_BUILD_FLAGS) -o \ + $(BIN_TOOLS_DIR)/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT) $(ROOT_DIR)/tools/data-conversion/openim/cmd/$(COMMAND)/$(COMMAND).go; \ + chmod +x $(BIN_TOOLS_DIR)/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT); \ + elif [ -f $(ROOT_DIR)/tools/data-conversion/chat/cmd/$(COMMAND)/$(COMMAND).go ]; then \ + CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) $(GO) build $(GO_BUILD_FLAGS) -o \ + $(BIN_TOOLS_DIR)/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT) $(ROOT_DIR)/tools/data-conversion/chat/cmd/$(COMMAND)/$(COMMAND).go; \ + chmod +x $(BIN_TOOLS_DIR)/$(OS)/$(ARCH)/$(COMMAND)$(GO_OUT_EXT); \ + fi \ + fi + +## go.install: Install deployment openim +.PHONY: go.install +go.install: + @echo "===========> Installing deployment openim" + @$(ROOT_DIR)/scripts/install-im-server.sh + +## go.multiarch: Build multi-arch binaries +.PHONY: go.build.multiarch +go.build.multiarch: go.build.verify $(foreach p,$(PLATFORMS),$(addprefix go.build., $(addprefix $(p)., $(BINS)))) + +## go.lint: Run golangci to lint source codes +.PHONY: go.lint +go.lint: tools.verify.golangci-lint + @echo "===========> Run golangci to lint source codes" + @$(TOOLS_DIR)/golangci-lint run --color always -c $(ROOT_DIR)/.golangci.yml $(ROOT_DIR)/... + +## go.test: Run unit test +.PHONY: go.test +go.test: + @$(GO) test ./... + +## go.test.api: Run api test +.PHONY: go.test.api +go.test.api: + @echo "===========> Run api test" + @$(ROOT_DIR)/scripts/install/test.sh openim::test::test + +## go.test.e2e: Run e2e test +.PHONY: go.test.e2e +go.test.e2e: tools.verify.ginkgo + @echo "===========> Run e2e test" + @$(TOOLS_DIR)/ginkgo -v $(ROOT_DIR)/test/e2e + +## go.demo: Run demo +.PHONY: go.demo +go.demo: + @echo "===========> Run demo" + @$(ROOT_DIR)/scripts/demo.sh + +## go.test.junit-report: Run unit test +.PHONY: go.test.junit-report +go.test.junit-report: tools.verify.go-junit-report + @touch $(TMP_DIR)/coverage.out + @echo "===========> Run unit test > $(TMP_DIR)/report.xml" +# @$(GO) test -v -coverprofile=$(TMP_DIR)/coverage.out 2>&1 $(GO_BUILD_FLAGS) ./... | $(TOOLS_DIR)/go-junit-report -set-exit-code > $(TMP_DIR)/report.xml + @$(GO) test -v -coverprofile=$(TMP_DIR)/coverage.out 2>&1 ./... | $(TOOLS_DIR)/go-junit-report -set-exit-code > $(TMP_DIR)/report.xml + @sed -i '/mock_.*.go/d' $(TMP_DIR)/coverage.out + @echo "===========> Test coverage of Go code is reported to $(TMP_DIR)/coverage.html by generating HTML" + @$(GO) tool cover -html=$(TMP_DIR)/coverage.out -o $(TMP_DIR)/coverage.html + +## go.test.cover: Run unit test with coverage +.PHONY: go.test.cover +go.test.cover: go.test.junit-report + @$(GO) tool cover -func=$(TMP_DIR)/coverage.out | \ + awk -v target=$(COVERAGE) -f $(ROOT_DIR)/scripts/coverage.awk + +## go.format: Run unit test and format codes +.PHONY: go.format +go.format: tools.verify.golines tools.verify.goimports + @echo "===========> Formatting codes" + @$(FIND) -type f -name '*.go' -not -name '*pb*' | $(XARGS) gofmt -s -w + @$(FIND) -type f -name '*.go' -not -name '*pb*' | $(XARGS) $(TOOLS_DIR)/goimports -w -local $(ROOT_PACKAGE) + @$(FIND) -type f -name '*.go' -not -name '*pb*' | $(XARGS) $(TOOLS_DIR)/golines -w --max-len=200 --reformat-tags --shorten-comments --ignore-generated . + @$(GO) mod edit -fmt + +## go.imports: task to automatically handle import packages in Go files using goimports tool +.PHONY: go.imports +go.imports: tools.verify.goimports + @$(TOOLS_DIR)/goimports -l -w $(SRC) + +## go.verify: execute all verity scripts. +.PHONY: go.verify +go.verify: tools.verify.misspell + @echo "Starting verification..." + @scripts_list=$$(find $(ROOT_DIR)/scripts -type f -name 'verify-*' | sort); \ + for script in $$scripts_list; do \ + echo "Executing $$script..."; \ + $$script || exit 1; \ + echo "$$script completed successfully"; \ + done + @echo "All verification scripts executed successfully." + +## go.updates: Check for updates to go.mod dependencies +.PHONY: go.updates +go.updates: tools.verify.go-mod-outdated + @$(GO) list -u -m -json all | go-mod-outdated -update -direct + +## go.clean: Clean all builds directories and files +.PHONY: go.clean +go.clean: + @echo "===========> Cleaning all builds tmp, bin, logs directories and files" + @-rm -vrf $(TMP_DIR) $(BIN_DIR) $(BIN_TOOLS_DIR) $(LOGS_DIR) + @echo "===========> End clean..." + +## go.help: Show go tools help +.PHONY: go.help +go.help: scripts/make-rules/golang.mk + $(call smallhelp) diff --git a/scripts/make-rules/image.mk b/scripts/make-rules/image.mk new file mode 100644 index 0000000000..eaec4a1271 --- /dev/null +++ b/scripts/make-rules/image.mk @@ -0,0 +1,183 @@ +# Copyright © 2023 OpenIMSDK. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# ============================================================================== +# Makefile helper functions for docker image +# ============================================================================== +# Path: scripts/make-rules/image.mk +# docker registry: registry.example.com/namespace/image:tag as: registry.hub.docker.com/cubxxw/: +# https://docs.docker.com/build/building/multi-platform/ +# + +DOCKER := docker + +# read: https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/images.md +REGISTRY_PREFIX ?= registry.cn-hangzhou.aliyuncs.com/openimsdk +# REGISTRY_PREFIX ?= ghcr.io/openimsdk + +BASE_IMAGE ?= ghcr.io/openim-sigs/openim-bash-image + +IMAGE_PLAT ?= $(subst $(SPACE),$(COMMA),$(subst _,/,$(PLATFORMS))) + +EXTRA_ARGS ?= --no-cache +_DOCKER_BUILD_EXTRA_ARGS := + +ifdef HTTP_PROXY +_DOCKER_BUILD_EXTRA_ARGS += --build-arg HTTP_PROXY=${HTTP_PROXY} +endif + +ifneq ($(EXTRA_ARGS), ) +_DOCKER_BUILD_EXTRA_ARGS += $(EXTRA_ARGS) +endif + +# Determine image files by looking into build/images/*/Dockerfile +IMAGES_DIR ?= $(wildcard ${ROOT_DIR}/build/images/*) +# Determine images names by stripping out the dir names, and filter out the undesired directories +# IMAGES ?= $(filter-out Dockerfile,$(foreach image,${IMAGES_DIR},$(notdir ${image}))) +IMAGES ?= $(filter-out Dockerfile openim-tools openim-rpc-extend-msg openim-rpc-encryption openim-cmdutils,$(foreach image,${IMAGES_DIR},$(notdir ${image}))) +# IMAGES ?= $(filter-out Dockerfile openim-tools openim-cmdutils,$(foreach image,${IMAGES_DIR},$(notdir ${image}))) # !pro + +ifeq (${IMAGES},) + $(error Could not determine IMAGES, set ROOT_DIR or run in source dir) +endif + +# ============================================================================== +# Image targets +# ============================================================================== + +# PLATFORMS defines the target platforms for the manager image be build to provide support to multiple +# architectures. (i.e. make docker-buildx IMG=myregistry/mypoperator:0.0.1). To use this option you need to: +# - able to use docker buildx . More info: https://docs.docker.com/build/buildx/ +# - have enable BuildKit, More info: https://docs.docker.com/develop/develop-images/build_enhancements/ +# - be able to push the image for your registry (i.e. if you do not inform a valid value via IMG=> then the export will fail) +# To properly provided solutions that supports more than one platform you should use this option. +## image.docker-buildx: Build and push docker image for the manager for cross-platform support +PLATFORMS ?= linux/arm64,linux/amd64,linux/s390x,linux/ppc64le +# copy existing Dockerfile and insert --platform=${BUILDPLATFORM} into Dockerfile.cross, and preserve the original Dockerfile +.PHONY: image.docker-buildx +image.docker-buildx: + sed -e '1 s/\(^FROM\)/FROM --platform=\$$\{BUILDPLATFORM\}/; t' -e ' 1,// s//FROM --platform=\$$\{BUILDPLATFORM\}/' Dockerfile > Dockerfile.cross + - $(CONTAINER_TOOL) buildx create --name project-v3-builder + $(CONTAINER_TOOL) buildx use project-v3-builder + - $(CONTAINER_TOOL) buildx build --push --platform=$(PLATFORMS) --tag ${IMAGES} -f Dockerfile.cross . + - $(CONTAINER_TOOL) buildx rm project-v3-builder + rm Dockerfile.cross + +## image.verify: Verify docker version +.PHONY: image.verify +image.verify: + @$(ROOT_DIR)/scripts/lib/util.sh openim::util::check_docker_and_compose_versions + +## image.daemon.verify: Verify docker daemon experimental features +.PHONY: image.daemon.verify +image.daemon.verify: + @$(ROOT_DIR)/scripts/lib/util.sh openim::util::ensure_docker_daemon_connectivity + @$(ROOT_DIR)/scripts/lib/util.sh openim::util::ensure-docker-buildx + +# If you wish built the manager image targeting other platforms you can use the --platform flag. +# (i.e. docker build --platform linux/arm64 ). However, you must enable docker buildKit for it. +# More info: https://docs.docker.com/develop/develop-images/build_enhancements/ +## image.build: Build docker images +.PHONY: image.build +image.build: image.verify $(addprefix image.build., $(addprefix $(PLATFORM)., $(IMAGES))) + +.PHONY: image.build.multiarch +image.build.multiarch: image.verify $(foreach p,$(PLATFORMS),$(addprefix image.build., $(addprefix $(p)., $(IMAGES)))) + +## image.build.%: Build docker image for a specific platform +.PHONY: image.build.% +image.build.%: go.build.% + $(eval IMAGE := $(COMMAND)) + $(eval IMAGE_PLAT := $(subst _,/,$(PLATFORM))) + $(eval ARCH := $(word 2,$(subst _, ,$(PLATFORM)))) + @echo "===========> Building docker image $(IMAGE) $(VERSION) for $(IMAGE_PLAT)" + @mkdir -p $(TMP_DIR)/$(IMAGE)/$(PLATFORM) + @cat $(ROOT_DIR)/build/images/Dockerfile\ + | sed "s#BASE_IMAGE#$(BASE_IMAGE)#g" \ + | sed "s#BINARY_NAME#$(IMAGE)#g" >$(TMP_DIR)/$(IMAGE)/Dockerfile + @cp $(BIN_DIR)/platforms/$(IMAGE_PLAT)/$(IMAGE) $(TMP_DIR)/$(IMAGE) + $(eval BUILD_SUFFIX := $(_DOCKER_BUILD_EXTRA_ARGS) --pull -t $(REGISTRY_PREFIX)/$(IMAGE)-$(ARCH):$(VERSION) $(TMP_DIR)/$(IMAGE)) + @echo $(DOCKER) build --platform $(IMAGE_PLAT) $(BUILD_SUFFIX) + @if [ $(shell $(GO) env GOARCH) != $(ARCH) ] ; then \ + $(MAKE) image.daemon.verify ;\ + $(DOCKER) build --platform $(IMAGE_PLAT) $(BUILD_SUFFIX) ; \ + else \ + $(DOCKER) build $(BUILD_SUFFIX) ; \ + fi + @rm -rf $(TMP_DIR)/$(IMAGE) + +# https://docs.docker.com/build/building/multi-platform/ +# busybox image supports amd64, arm32v5, arm32v6, arm32v7, arm64v8, i386, ppc64le, and s390x +## image.buildx.%: Build docker images with buildx +.PHONY: image.buildx.% +image.buildx.%: + $(eval IMAGE := $(word 1,$(subst ., ,$*))) + echo "===========> Building docker image $(IMAGE) $(VERSION)" + $(DOCKER) buildx build -f $(ROOT_DIR)/Dockerfile --pull --no-cache --platform=$(PLATFORMS) --push . -t $(REGISTRY_PREFIX)/$(IMAGE)-$(ARCH):$(VERSION) + +## image.push: Push docker images +.PHONY: image.push +image.push: image.verify go.build.verify $(addprefix image.push., $(addprefix $(IMAGE_PLAT)., $(IMAGES))) + +## image.push.multiarch: Push docker images for all platforms +.PHONY: image.push.multiarch +image.push.multiarch: image.verify go.build.verify $(foreach p,$(PLATFORMS),$(addprefix image.push., $(addprefix $(p)., $(IMAGES)))) + +## image.push.%: Push docker image for a specific platform +.PHONY: image.push.% +image.push.%: image.build.% + @echo "===========> Pushing image $(IMAGE) $(VERSION) to $(REGISTRY_PREFIX)" + $(DOCKER) push $(REGISTRY_PREFIX)/$(IMAGE)-$(ARCH):$(VERSION) + +## image.manifest.push: Push manifest list for multi-arch images +.PHONY: image.manifest.push +image.manifest.push: export DOCKER_CLI_EXPERIMENTAL := enabled +image.manifest.push: image.verify go.build.verify \ +$(addprefix image.manifest.push., $(addprefix $(IMAGE_PLAT)., $(IMAGES))) + +## image.manifest.push.%: Push manifest list for multi-arch images for a specific platform +.PHONY: image.manifest.push.% +image.manifest.push.%: image.push.% image.manifest.remove.% + @echo "===========> Pushing manifest $(IMAGE) $(VERSION) to $(REGISTRY_PREFIX) and then remove the local manifest list" + @$(DOCKER) manifest create $(REGISTRY_PREFIX)/$(IMAGE):$(VERSION) \ + $(REGISTRY_PREFIX)/$(IMAGE)-$(ARCH):$(VERSION) + @$(DOCKER) manifest annotate $(REGISTRY_PREFIX)/$(IMAGE):$(VERSION) \ + $(REGISTRY_PREFIX)/$(IMAGE)-$(ARCH):$(VERSION) \ + --os $(OS) --arch ${ARCH} + @$(DOCKER) manifest push --purge $(REGISTRY_PREFIX)/$(IMAGE):$(VERSION) + +# Docker cli has a bug: https://github.com/docker/cli/issues/954 +# If you find your manifests were not updated, +# Please manually delete them in $HOME/.docker/manifests/ +# and re-run. +## image.manifest.remove.%: Remove local manifest list +.PHONY: image.manifest.remove.% +image.manifest.remove.%: + @rm -rf ${HOME}/.docker/manifests/docker.io_$(REGISTRY_PREFIX)_$(IMAGE)-$(VERSION) + +## image.manifest.push.multiarch: Push manifest list for multi-arch images for all platforms +.PHONY: image.manifest.push.multiarch +image.manifest.push.multiarch: image.push.multiarch $(addprefix image.manifest.push.multiarch., $(IMAGES)) + +## image.manifest.push.multiarch.%: Push manifest list for multi-arch images for all platforms for a specific image +.PHONY: image.manifest.push.multiarch.% +image.manifest.push.multiarch.%: + @echo "===========> Pushing manifest $* $(VERSION) to $(REGISTRY_PREFIX) and then remove the local manifest list" + REGISTRY_PREFIX=$(REGISTRY_PREFIX) PLATFORMS="$(PLATFORMS)" IMAGE=$* VERSION=$(VERSION) DOCKER_CLI_EXPERIMENTAL=enabled \ + $(ROOT_DIR)/build/lib/create-manifest.sh + +## image.help: Print help for image targets +.PHONY: image.help +image.help: scripts/make-rules/image.mk + $(call smallhelp) \ No newline at end of file diff --git a/scripts/make-rules/release.mk b/scripts/make-rules/release.mk new file mode 100644 index 0000000000..68bf058981 --- /dev/null +++ b/scripts/make-rules/release.mk @@ -0,0 +1,42 @@ +# Copyright © 2023 OpenIMSDK. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# ============================================================================== +# Makefile helper functions for release +# Versions are used after merging +# + +## release.run: release the project +.PHONY: release.run +release.run: release.verify release.ensure-tag + @scripts/release.sh + +## release.verify: Check if a tool is installed and install it +.PHONY: release.verify +release.verify: tools.verify.git-chglog tools.verify.github-release tools.verify.coscmd tools.verify.coscli + +## release.tag: release the project +.PHONY: release.tag +release.tag: tools.verify.gsemver release.ensure-tag + @git push origin `git describe --tags --abbrev=0` + +## release.ensure-tag: ensure tag +.PHONY: release.ensure-tag +release.ensure-tag: tools.verify.gsemver + @scripts/ensure-tag.sh + +## release.help: Display help information about the release package +.PHONY: release.help +release.help: scripts/make-rules/release.mk + $(call smallhelp) diff --git a/scripts/make-rules/swagger.mk b/scripts/make-rules/swagger.mk new file mode 100644 index 0000000000..991de43c63 --- /dev/null +++ b/scripts/make-rules/swagger.mk @@ -0,0 +1,19 @@ +# ============================================================================== +# Makefile helper functions for swagger +# + +## swagger.run: Generate swagger document. +.PHONY: swagger.run +swagger.run: tools.verify.swagger + @echo "===========> Generating swagger API docs" + @$(TOOLS_DIR)/swagger generate spec --scan-models -w $(ROOT_DIR)/cmd/genswaggertypedocs -o $(ROOT_DIR)/api/swagger/swagger.yaml + +## swagger.serve: Serve swagger spec and docs. +.PHONY: swagger.serve +swagger.serve: tools.verify.swagger + @$(TOOLS_DIR)/swagger serve -F=redoc --no-open --port 36666 $(ROOT_DIR)/api/swagger/swagger.yaml + +## swagger.help: Display help information about the release package +.PHONY: swagger.help +swagger.help: scripts/make-rules/swagger.mk + $(call smallhelp) \ No newline at end of file diff --git a/scripts/make-rules/tools.mk b/scripts/make-rules/tools.mk new file mode 100644 index 0000000000..5335d094d9 --- /dev/null +++ b/scripts/make-rules/tools.mk @@ -0,0 +1,280 @@ +# Copyright © 2023 OpenIMSDK. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# ============================================================================== +# Makefile helper functions for tools(https://github.com/avelino/awesome-go) -> DIR: {TOOT_DIR}/tools | (go >= 1.19) +# Why download to the tools directory, thinking we might often switch Go versions using gvm. +# + +# openim build use BUILD_TOOLS +BUILD_TOOLS ?= golangci-lint goimports addlicense deepcopy-gen conversion-gen ginkgo go-junit-report go-gitlint +# Code analysis tools +ANALYSIS_TOOLS = golangci-lint goimports golines go-callvis kube-score +# Code generation tools +GENERATION_TOOLS = deepcopy-gen conversion-gen protoc-gen-go cfssl rts codegen +# Testing tools +TEST_TOOLS = ginkgo go-junit-report gotests +# tenxun cos tools +COS_TOOLS = coscli coscmd +# Version control tools +VERSION_CONTROL_TOOLS = addlicense go-gitlint git-chglog github-release gsemver +# Utility tools +UTILITY_TOOLS = go-mod-outdated mockgen gothanks richgo kubeconform +# All tools +ALL_TOOLS ?= $(ANALYSIS_TOOLS) $(GENERATION_TOOLS) $(TEST_TOOLS) $(VERSION_CONTROL_TOOLS) $(UTILITY_TOOLS) $(COS_TOOLS) + +## tools.install: Install a must tools +.PHONY: tools.install +tools.install: $(addprefix tools.verify., $(BUILD_TOOLS)) + +## tools.install-all: Install all tools +.PHONY: tools.install-all +tools.install-all: $(addprefix tools.install-all., $(ALL_TOOLS)) + +## tools.install.%: Install a single tool in $GOBIN/ +.PHONY: tools.install.% +tools.install.%: + @echo "===========> Installing $,The default installation path is $(GOBIN)/$*" + @$(MAKE) install.$* + +## tools.install-all.%: Parallelism install a single tool in ./tools/* +.PHONY: tools.install-all.% +tools.install-all.%: + @echo "===========> Installing $,The default installation path is $(TOOLS_DIR)/$*" + @$(MAKE) -j $(nproc) install.$* + +## tools.verify.%: Check if a tool is installed and install it +.PHONY: tools.verify.% +tools.verify.%: + @echo "===========> Verifying $* is installed" + @if [ ! -f $(TOOLS_DIR)/$* ]; then GOBIN=$(TOOLS_DIR) $(MAKE) tools.install.$*; fi + @echo "===========> $* is install in $(TOOLS_DIR)/$*" + +## install.golangci-lint: Install golangci-lint +.PHONY: install.golangci-lint +install.golangci-lint: + @$(GO) install github.com/golangci/golangci-lint/cmd/golangci-lint@$(GOLANGCI_LINT_VERSION) + +## install.goimports: Install goimports, used to format go source files +.PHONY: install.goimports +install.goimports: + @$(GO) install golang.org/x/tools/cmd/goimports@$(GOIMPORTS_VERSION) + +## install.addlicense: Install addlicense, used to add license header to source files +.PHONY: install.addlicense +install.addlicense: + @$(GO) install github.com/google/addlicense@$(ADDLICENSE_VERSION) + +## install.deepcopy-gen: Install deepcopy-gen, used to generate deep copy functions +.PHONY: install.deepcopy-gen +install.deepcopy-gen: + @$(GO) install k8s.io/code-generator/cmd/deepcopy-gen@$(DEEPCOPY_GEN_VERSION) + +## install.conversion-gen: Install conversion-gen, used to generate conversion functions +.PHONY: install.conversion-gen +install.conversion-gen: + @$(GO) install k8s.io/code-generator/cmd/conversion-gen@$(CONVERSION_GEN_VERSION) + +## install.ginkgo: Install ginkgo to run a single test or set of tests +.PHONY: install.ginkgo +install.ginkgo: + @$(GO) install github.com/onsi/ginkgo/ginkgo@$(GINKGO_VERSION) + +## install.go-gitlint: Install go-gitlint, used to check git commit message +.PHONY: install.go-gitlint +install.go-gitlint: + @$(GO) install github.com/marmotedu/go-gitlint/cmd/go-gitlint@$(GO_GITLINT_VERSION) + +## install.go-junit-report: Install go-junit-report, used to convert go test output to junit xml +.PHONY: install.go-junit-report +install.go-junit-report: + @$(GO) install github.com/jstemmer/go-junit-report@$(GO_JUNIT_REPORT_VERSION) + +## install.gotests: Install gotests, used to generate go tests +.PHONY: install.gotests +install.gotests: + @$(GO) install github.com/cweill/gotests/gotests@$(GO_TESTS_VERSION) + +## install.kafkactl: Install kafkactl command line tool. +.PHONY: install.kafkactl +install.kafkactl: + @$(GO) install github.com/deviceinsight/kafkactl@$(KAFKACTL_VERSION) + +## install.go-apidiff: Install go-apidiff, used to check api changes +.PHONY: install.go-apidiff +install.go-apidiff: + @$(GO) install github.com/joelanford/go-apidiff@$(GO_APIDIFF_VERSION) + +## install.swagger: Install swagger, used to generate swagger documentation +.PHONY: install.swagger +install.swagger: + @$(GO) install github.com/go-swagger/go-swagger/cmd/swagger@$(SWAGGER_VERSION) + +# ============================================================================== +# Tools that might be used include go gvm +# + +## install.gotestsum: Install gotestsum, used to run go tests +.PHONY: install.gotestsum +install.gotestsum: + @$(GO) install gotest.tools/gotestsum@$(GOTESTSUM_VERSION) + +## install.kube-score: Install kube-score, used to check kubernetes yaml files +.PHONY: install.kube-score +install.kube-score: + @$(GO) install github.com/zegl/kube-score/cmd/kube-score@$(KUBE_SCORE_VERSION) + +## install.kubeconform: Install kubeconform, used to check kubernetes yaml files +.PHONY: install.kubeconform +install.kubeconform: + @$(GO) install github.com/yannh/kubeconform/cmd/kubeconform@$(KUBECONFORM_VERSION) + +## install.gsemver: Install gsemver, used to generate semver +.PHONY: install.gsemver +install.gsemver: + @$(GO) install github.com/arnaud-deprez/gsemver@$(GSEMVER_VERSION) + +## install.git-chglog: Install git-chglog, used to generate changelog +.PHONY: install.git-chglog +install.git-chglog: + @$(GO) install github.com/git-chglog/git-chglog/cmd/git-chglog@$(GIT_CHGLOG_VERSION) + +## install.ko: Install ko, used to build go program into container images +.PHONY: install.ko +install.ko: + @$(GO) install github.com/google/ko@$(KO_VERSION) + +## install.github-release: Install github-release, used to create github release +.PHONY: install.github-release +install.github-release: + @$(GO) install github.com/github-release/github-release@$(GITHUB_RELEASE_VERSION) + +## install.coscli: Install coscli, used to upload files to cos +# example: ./coscli cp/sync -r /home/off-line/docker-off-line/ cos://openim-1306374445/openim/image/amd/off-line/off-line/ -e cos.ap-guangzhou.myqcloud.com +# https://cloud.tencent.com/document/product/436/71763 +# amd64 +.PHONY: install.coscli +install.coscli: + @wget -q https://github.com/tencentyun/coscli/releases/download/$(COSCLI_VERSION)/coscli-linux -O ${TOOLS_DIR}/coscli + @chmod +x ${TOOLS_DIR}/coscli + +## install.coscmd: Install coscmd, used to upload files to cos +.PHONY: install.coscmd +install.coscmd: + @if which pip &>/dev/null; then pip install coscmd; else pip3 install coscmd; fi + +## install.minio: Install minio, used to upload files to minio +.PHONY: install.minio +install.minio: + @$(GO) install github.com/minio/minio@$(MINIO_VERSION) + +## install.delve: Install delve, used to debug go program +.PHONY: install.delve +install.delve: + @$(GO) install github.com/go-delve/delve/cmd/dlv@$(DELVE_VERSION) + +## install.air: Install air, used to hot reload go program +.PHONY: install.air +install.air: + @$(GO) install github.com/cosmtrek/air@$(AIR_VERSION) + +## install.gvm: Install gvm, gvm is a Go version manager, built on top of the official go tool. +.PHONY: install.gvm +install.gvm: + @echo "===========> Installing gvm, The default installation path is ~/.gvm/scripts/gvm" + @bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer) + @source /root/.gvm/scripts/gvm + +## install.golines: Install golines, used to format long lines +.PHONY: install.golines +install.golines: + @$(GO) install github.com/segmentio/golines@$(GOLINES_VERSION) + +## install.go-mod-outdated: Install go-mod-outdated, used to check outdated dependencies +.PHONY: install.go-mod-outdated +install.go-mod-outdated: + @$(GO) install github.com/psampaz/go-mod-outdated@$(GO_MOD_OUTDATED_VERSION) + +## install.mockgen: Install mockgen, used to generate mock functions +.PHONY: install.mockgen +install.mockgen: + @$(GO) install github.com/golang/mock/mockgen@$(MOCKGEN_VERSION) + +## install.wire: Install wire, used to generate wire files +.PHONY: install.wire +install.wire: + @$(GO) install github.com/google/wire/cmd/wire@$(WIRE_VERSION) + + +## install.protoc-gen-go: Install protoc-gen-go, used to generate go source files from protobuf files +.PHONY: install.protoc-gen-go +install.protoc-gen-go: + @$(GO) install github.com/golang/protobuf/protoc-gen-go@$(PROTOC_GEN_GO_VERSION) + +## install.cfssl: Install cfssl, used to generate certificates +.PHONY: install.cfssl +install.cfssl: + @$(ROOT_DIR)/scripts/install/install.sh openim::install::install_cfssl + +## install.depth: Install depth, used to check dependency tree +.PHONY: install.depth +install.depth: + @$(GO) install github.com/KyleBanks/depth/cmd/depth@$(DEPTH_VERSION) + +## install.go-callvis: Install go-callvis, used to visualize call graph +.PHONY: install.go-callvis +install.go-callvis: + @$(GO) install github.com/ofabry/go-callvis@$(GO_CALLVIS_VERSION) + +## install.misspell: Install misspell +.PHONY: install.misspell +install.misspell: + @$(GO) install github.com/client9/misspell/cmd/misspell@$(MISSPELL_VERSION) + +## install.gothanks: Install gothanks, used to thank go dependencies +.PHONY: install.gothanks +install.gothanks: + @$(GO) install github.com/psampaz/gothanks@$(GOTHANKS_VERSION) + +## install.richgo: Install richgo +.PHONY: install.richgo +install.richgo: + @$(GO) install github.com/kyoh86/richgo@$(RICHGO_VERSION) + +## install.rts: Install rts +.PHONY: install.rts +install.rts: + @$(GO) install github.com/galeone/rts/cmd/rts@$(RTS_VERSION) + +# ================= kubecub openim tools ========================================= +# https://github.com/kubecub +## install.typecheck: Install kubecub typecheck, checks for go code +.PHONY: install.typecheck +install.typecheck: + @$(GO) install github.com/kubecub/typecheck@$(TYPECHECK_VERSION) + +## install.comment-lang-detector: Install kubecub comment-lang-detector, checks for go code comment language +.PHONY: install.comment-lang-detector +install.comment-lang-detector: + @$(GO) install github.com/kubecub/comment-lang-detector/cmd/cld@$(COMMENT_LANG_DETECTOR_VERSION) + +## install.standardizer: Install kubecub standardizer, checks for go code standardization +.PHONY: install.standardizer +install.standardizer: + @$(GO) install github.com/kubecub/standardizer@$(STANDARDIZER_VERSION) + +## tools.help: Display help information about the tools package +.PHONY: tools.help +tools.help: scripts/make-rules/tools.mk + $(call smallhelp) \ No newline at end of file