4.1.0

What's Changed

• Refactored StickyPartitionAssignmentStrategy and implemented task estimation logic in LoadBasedPartitionAssignmentStrategy by @jzakaryan in #835
• Fix flaky test testConsumeFromMultipleTopics by @vmaheshw in #838
• Refactor Optional parameters in the constructor of strategy by @vmaheshw in #836
• Implemented logic to prevent tasks from having more than specified number of partitions by @jzakaryan in #837
• Publishing artifacts to JFrog by @shrinandthakkar in #839
• Make the Throughput based assignment and task estimation based on partition assignment configurable. by @vmaheshw in #841
• Implemented metrics for LoadBasedPartitionAssignmentStrategy by @jzakaryan in #840
• Clear the CallbackStatus entry from the map in FlushlessEventProducerHandler by @vmaheshw in #843
• Handle the new session after session expiry by @vmaheshw in #770
• Fixed issue with config key by @jzakaryan in #847
• Print count of orphaned tasks and orphaned task locks in the log message by @surajkn in #844
• Datastream stop transition redesign by @shrinandthakkar in #842
• Fix flaky stopping simultaneous datastream test by @shrinandthakkar in #849
• Update the DatabaseChunkedReader to take the Connection as input rather than the DataSource by @somandal in #850
• Handle leaking TP exception in handleAssignmentChange by @surajkn in #845
• Migrating from AdminUtils with AdminClient by @surajkn in #848
• Fix running task data structure logic in AbstractKafkaConnector for failing to stop task by @surajkn in #851
• Removing partial Helix ZkClient dependency by @surajkn in #852
• Stop the tasks in parallel in AbstractKafkaConnector by @vmaheshw in #853
• Add additional log in LoadBasedPartitionAssignmentStrategy by @vmaheshw in #856
• Fixing flaky test testCreateDatastreamHappyPathDefaultRetention by @surajkn in #854
• Added toString() override in PartitionThroughputInfo by @jzakaryan in #858
• Add Stats to DatastreamTaskImpl by @vmaheshw in #855
• Make PartitionThroughputProvider metricsAware by @vmaheshw in #859
• Added base strategy metric info in LoadBasedPartitionAssignmentStrategy by @jzakaryan in #857
• Add alert metrics to identify that elastic task configurations require adjustment by @vmaheshw in #860
• Fix restartDeadTask logic when the task thread has died by @surajkn in #861
• Fix metric infos in PartitionThroughputProvider by @vmaheshw in #862
• Fix the metrics deregistration in AbstractKafkaConnector when multiple stop are called by @vmaheshw in #865
• Fix logging in LoadBasedPartitionAssignmentStrategy by @vmaheshw in #866
• Make Default byte in rate and Msg in rate configurable by @vmaheshw in #864
• Metrics are getting emitted in LoadBasedPartitionAssignmentStrategy only when it needs adjustment by @vmaheshw in #867
• Use topic level throughput information when partition level information is unavailable. by @vmaheshw in #871
• Fix compilation error by @vmaheshw in #874
• Loadbased Partition assigner not using topic level metrics to recognize partitions by @vmaheshw in #876
• Flushless producer supporting both comparable and non comparable offsets by @shrinandthakkar in #873
• LiveInstanceProvider subscription should be done only by the leader coordinator by @vmaheshw in #879
• Fixed issue with missing exception message during task initialization by @jzakaryan in #882
• Kafka upgrade by @srinagaraj in #881
• skipping onPartitionsRevoked during consumer.close() call by @srinagaraj in #886
• Scala 2.12 upgrade by @srinagaraj in #895
• Upgrade avro and move jackson from codehaus to fasterxml by @vmaheshw in #894
• Fix topic deletion when multiple duplicate streams expire at the same time by @surajkn in #897
• Use 2.4.1.57 kafka version by @shrinandthakkar in #901
• tests for min/max partitions per task metrics and minor code quality improvements by @ryannedolan in #887
• fix rebalancing-tasks bug and added tests by @shrinandthakkar in #900
• Refactor Stopping Tasks On Assignment Change of Tasks by @shrinandthakkar in #868
• Change python dependency in commit-msg git hook by @surajkn in #904
• Remove Scala Dependencies by @shrinandthakkar in #905
• Introduce broadcast API to TransportProvider by @surajkn in #903
• Dedupe tasks on LeaderDoAssignment by @jogrogan in #906
• Fix Stopping Logic and Maintain Stopping Latch Counter by @shrinandthakkar in #877
• Fixing test OnAssignmentChangeMultipleReassignments by @shrinandthakkar in #908
• Update kafka version by @jogrogan in #910
• Replace 101tec ZkClient with Helix ZkClient by @surajkn in #909
• Add retry to query retention time for destination topic by @surajkn in #863
• Upgrade Zookeeper version to 3.6.3 by @jogrogan in #913

New Contributors

• @surajkn made their first contribution in #844
• @srinagaraj made their first contribution in #881
• @ryannedolan made their first contribution in #887
• @jogrogan made their first contribution in #906

Full Changelog: 1.1.0...4.1.0