4.5.4

Hazelcast Jet 4.5.4 is based on IMDG version 4.2.6.

1. Fixes

[core] Fixed an issue where a Jet job was stuck when there is a Near Cache configured at the target map.
[dependencies] Updated Hazelcast IMDG to 4.2.6
[dependencies] Updated jackson to 2.14.1
[dependencies] Updated snakeyaml to 1.33
[dependencies] Updated hadoop to 3.3.4
[dependencies] Updated parquet to 1.12.3
[dependencies] Updated jetty to 9.4.49.v20220914
[dependencies] Updated hazelcast-aws to 3.4
[dependencies] Updated AWS SDK to 1.12.352
[dependencies] Updated hazelcast-kubernetes to 2.2.3
[dependencies] Updated hazelcast-azure to 2.1.2
[dependencies] Updated grpc to 1.48.2
[dependencies] Updated protobuf to 3.19.6
[dependencies] Updated postgresql driver to 42.5.1
[dependencies] Updated jackson-mapper-asl to 1.9.13
[dependencies] Updated google oauth client to 1.34.1
[dependencies] Updated lucene-queryparser to 6.6.5 (in elasticsearch-5 module)
[distribution] Removed the hazelcast-jet-cdc-mysql module from the Jet Enterprise distribution

4.5.3

Hazelcast Jet 4.5.3 is based on IMDG version 4.2.4 .

1. Fixes

[dependencies] Update log4j to 2.17.0

4.5.2

Hazelcast Jet 4.5.2 is based on IMDG version 4.2.3 .

1. Fixes

[dependencies] Update log4j to 2.16.0
[dependencies] Update slf4j to 1.7.32
[dependencies] Update jackson to 2.11.4
[dependencies] Update snakeyaml to 1.29
[dependencies] Update netty to 4.1.72.Final
[dependencies] Update mysql-connector-java to 8.0.25
[dependencies] Update aws-sdk to 1.12.128
[dependencies] Update avro to 1.11.0

4.5.1

Hazelcast Jet 4.5.1 is based on IMDG version 4.2.2 .

1. Fixes

[core] Fix rebalance() nonfunctional before fused stages (#3051)
[hadoop] Update Apache Commons Compress to 1.21
[hadoop] Update JSON Small and Fast Parser to 2.4.7
[hadoop] Update Jetty to 9.4.43
[hadoop] Apache Commons IO to 2.7
[metrics] Update Prometheus JMX Exporter - Java Agent to 0.16.1
[elasticsearch] Update Elasticsearch6 to 6.8.17

4.5

• @hhromic added an option to configure the batch size for JDBC Sink.
  A new batchLimit() method added to the JdbcSinkBuilder and the
  corresponding constructor arguments to the underlying implementation
  class and meta-suppliers.
• Limit parallel partition reads: In Map/Cache source, Jet tries to
  read all the partitions in parallel, which may lead to high memory
  usage. Now the max number of parallel reads is limited.
• Improve AggregateOperations.pickAny() by adding deduct
• Security updates: updated several libraries with identified security
  vulnerabilities.
• SQL: pre-existing ClassDefinition is no longer required for
  Portable serialization - if it does not exist, it's derived from the
  mapping columns.

4.3.1

1. Fixes

[avro] Update Apache Commons Compress to 1.20 (#2953)
[avro] Update avro version to 1.10.1 (#2937)
[cdc] Update MySQL Connector to 8.0.20 (#2953)
[core] Update Jackson version to 2.11.4 (#2938)
[elasticsearch] Update ElasticSearch-6 to 6.8.14 (#2946)
[elasticsearch] Update ElasticSearch-7 to 7.10.2 (#2945, #2956)
[elasticsearch] Update Snakeyaml version to 1.26 (#2949)
[hadoop] Update Apache Httpclient to 4.5.13 (#2953)
[hadoop] Update Guava to 30.1-jre (#2953)
[kafka] Update Kafka to 2.2.2 (#2948)
[s3] Update Netty version to 4.1.59.Final (#2943)

4.4

• @hhromic improved the naming of source and sink stages across different
  connectors, bringing them all in line with the same convention xSource
  / xSink
• @TomaszGaweda added the pipeline.isEmpty() method that tells whether
  it contains any stage
• SQL Beta: submit jobs to Jet from the command-line SQL shell.
• Unified API to create sources and sinks from file-like resources: local
  filesystem, Amazon S3, Azure Blob Storage and Data Lake Storage, Google
  Cloud Storage
• Amazon Kinesis connector
• Prevent event reordering: by default Jet reorders data for performance,
  now you can disable this to get strict event order where you need it.
• @TomaszGaweda added the jet.imdg.version.mismatch.check.disabled
  config property that disables the enforcement of the exact IMDG
  dependency version. This allows adding IMDG quick fixes to the existing
  Jet release.
• New packaging: download either the full package with all the extensions
  enabled, or the minimal package and separately download the extensions
  you want.
• Improved the behavior of jet submit: now it waits for the job to
  start and prints a message about it.
• Improved the error message when using a Python function but Python
  is not installed.
• Improved the performance of the Kafka source by fine-tuning some
  timeouts.

4.3

• @MohamedMandouh implemented distributed sorting: BatchStage.sort().
• @guenter-hesse added a high-throughput source with precisely controlled
  timing, to be used to benchmark Jet's throughput and latency.
• We added the ability for Jet to suspend an unbounded, fault-tolerant job
  when it fails due to an exception in the pipeline code. This allows you
  to diagnose the problem, remove its cause, and resume the pipeline without
  data loss.
• We have improved the consistency of reconnect behaviour across CDC sources and
  introduced a uniform API to configure the reconnect strategy.
• We improved the behavior of streaming pipelines based on ingestion time: a
  low event rate in any partition of the source used to stall the progress,
  now the watermark advances even without events.
• We moved the Pipeline to DAG transformation from the client to the server,
  in preparation for the upcoming feature of preventing event reordering in the
  pipeline.

4.2

• We added Change Data Capture (CDC) support for PostgreSQL and MySql. New
  connectors use Debezium to provide a high-level API for processing the
  stream of changes from relational databases. A new map sink can be used to
  automatically keep the map in-sync with changes from the database.
• We added new source and sink connectors for Elasticsearch 5, 6 and 7
  with support for slicing and co-located reading for maximum read performance
  and extended Elasticsearch's client retry mechanism for reliable writing.
• We improved JSON support through a new file source which can read JSON files
  directly and also added built-in Jet methods for parsing and mapping JSON strings.
• We introduced .relabance() operator which can be used for distributing items
  across the cluster.
• We made some improvements to Jet startup scripts:
  • You can now use the PROMETHEUS_PORT to start Jet with prometheus metrics enabled.
  • You can use the jvm.options file to edit the startup JVM parameters.
  • You can now use the JET_MODULES environment variable to add modules when using docker
    or jet-start script.
• We did several rounds of improvements to the documentation:
  • Improvements to Docker and Kubernetes instructions
  • New guidance on Garbage Collector choice and settings
  • New and improved tutorials for CDC
  • New documentation for Python module and early results support in windowing

4.1.1

1. Enhancements

[docker] Update Java version to 11 in Docker image

2. Fixes

[core] JmsSink should not throw when it receives a Watermark