Releases: alibaba/MongoShake
Releases · alibaba/MongoShake
release-v2.4.8-20200729
2020-07-29 Alibaba Cloud.
* version: 2.4.8
* BUGFIX: fix bug that checkpoint.storage.db does not work since v2.4.6.
* BUGFIX: fix bug of orphan document filter core. #389.
* IMPROVE: remove listDatabase request when incr_sync.shard_key != auto.
#385.
* IMPROVE: add oplog_max_size and oplog_avg_size metric in restful: "repl/".
* BUGFIX: fix bug of checkpoint duplicate updating when HA switch.
* IMPROVE: increase syncer thread when fetching method is change stream.
* IMPROVE: add readConcern and writeConcern in client to solve the
orphan document problem reading from MongoS. #392
* IMPROVE: fix wrong name usage in mongoshake-stat script, thanks @Neal
Gosalia. #393.
- 修复2.4.6版本引入的，checkpoint db名字不为mongoshake无法正常运行的问题
- 修复过滤孤儿文档出core的bug。 #389.
- 在
incr_sync.shard_key != auto去除listDatabase请求，减少对源mongodb的权限要求。#385. - 优化restful接口，添加
oplog_max_size和oplog_avg_size的统计。 - 修复HA切换后，备mongoshake仍旧刷checkpoint的bug。
- change stream拉取下，增大解析的线程数（4->6）以提高性能。
- 全量阶段过滤mongos上的孤儿文档。#392
- 改进mongoshake-stat中的--help提示。#393
release-v2.4.7-20200630
2020-06-30 Alibaba Cloud.
* version: 2.4.7
* IMPROVE: add exit-point and safe shutdown mechanism. #375, #162
* IMPROVE: add UpdateLookup options in change stream. #350. thanks for
the PR by Raydy.
* IMPROVE: open full_sync.reader.read_document_count parameter to improve
full sync performance when meeting jumbo table.
* IMPROVE: polish log: remove function information in log.
- 添加优雅退出机制；添加退出接口
- 增加UpdateLookup便于用户在异步通道（Kafka，rpc, tcp）获取更新后的整个文档内容，而不仅是局部字段。感谢@Raydy的PR
- 增加
full_sync.reader.read_document_count配置，用于解决单表倾斜全量同步比较慢的情况。 - 优化log：log日志中移除函数路径
关于退出位点和全量并发拉取具体功能请参考wiki: v2.4.7增加退出位点以及全量并发拉取
release-v2.4.6-20200617
2020-06-17 Alibaba Cloud.
* version: 2.4.6
* IMPROVE: when running full sync, change sync source from mongod to
mongos. #360
* IMPROVE: when source type is sharding, change checkpoint position from
config-server to mongos.
* BUGFIX: fix bug of whether the starting time oplog is legal: the oldest
oplog timestamp > checkpoint.start_position
* IMPROVE: convert kafka producer MaxMessageBytes configuration from 1MB
to 16MB. #369.
* IMPROVE: add target delay parameter just like MongoDB primary-secondary
SlaveDelay. #371.
- 源端是sharding全量拉取源从mongod改为mongos。解决：1.全量期间可以不关闭balancer；2.解决全量期间的orphan文档问题
- checkpoint的写入地址从cs改为mongos，解决：1.cs写权限不足的问题 2.对齐副本集，都写源端结构上更清晰
- 修复一个第一次all模式启动，全量和增量位点衔接的bug
- kafka的producer MaxMessageBytes写入配置从1MB调整到18MB
- 添加延迟同步功能：incr_sync.target_delay。类似MongoDB主从的slaveDelay
release-v2.4.5-20200608
2020-06-08 Alibaba Cloud.
* version: 2.4.5
* BUGFIX: fix bug of ignoring error wrong when parsed error list is nil.
release-v2.4.4-20200605
2020-06-05 Alibaba Cloud.
* version: 2.4.4
* BUGFIX: fix bug of "two $set" in update query. see #345.
* BUGFIX: fix bug of source mongodb judgement error. see #343.
* IMPROVE: solve the idempotency problem of oplog synchronization
during the initial sync period. see #345.
release-v2.4.3-20200515
2020-05-14 Alibaba Cloud.
* version: 2.4.3
* BUGFIX: rename `full_sync.collection_exist_no_drop` to
`full_sync.collection_exist_drop`.
release-v2.4.2-20200422
2020-04-22 Alibaba Cloud.
* version: 2.4.2
* BUGFIX: fix bug of applyOps run failed. #329
release-v2.4.1-20200409
2020-04-09 Alibaba Cloud.
* version: 2.4.1
* IMPROVE: add speed limit policy in full sync stage.
* IMPROVE: modify speed limit policy to token bucket algorithm in
incremental sync.
* IMPROVE: add background index build policy, see full_sync.create_index. #316
* IMPROVE: add monitor in full sync stage.
* BUGFIX: fix bug of parsing oplog empty from kafka. #317
* BUGFIX: fix bug of checkpoint flush block. #323
release-v2.0.11-20200409
此版本为2.0的最新小版本，目前最新的大版本请使用2.4。
2020-04-08 Alibaba Cloud.
* version: 2.0.11
* BUGFIX: add checkpoint flush max try times to 10. see #323.
release-v2.0.10-20200327
此版本为2.0的最新小版本，目前最新的大版本请使用2.4。
2020-03-27 Alibaba Cloud.
* version: 2.0.10
* BUGFIX: data may lost when using batch update meets duplicate error.