Skip to content

Commit

Permalink
Add documentation for Partitions initial offsets (#996)
Browse files Browse the repository at this point in the history
* Add documentation for Partitions initial offsets

---------

Co-authored-by: rebekah-lawrence <[email protected]>
  • Loading branch information
k-jamroz and rebekah-lawrence authored Feb 22, 2024
1 parent de387ca commit ee160fb
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion docs/modules/integrate/pages/kafka-connector.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,12 @@ committing using `enable.auto.commit` and configuring
link:https://kafka.apache.org/22/documentation.html[Kafka documentation]
for the descriptions of these properties.

You can also explicitly specify exact initial offsets for the Kafka source using https://docs.hazelcast.org/docs/{full-version}/javadoc/com/hazelcast/jet/kafka/KafkaSources.html#kafka(java.util.Properties,com.hazelcast.function.FunctionEx,com.hazelcast.jet.kafka.TopicsConfig)[`TopicsConfig` parameter^].
Note that initial offsets provided in `topicConfig` will always have priority over offsets stored in Kafka or associated with a given consumer group.
Those offsets are used only when the job is started for the first time after submission.
Afterwards, the regular fault tolerance mechanism described above is used.
This option is not supported when processing guarantees are disabled.

== Transactional Guarantees

As a sink, the Kafka connector provides exactly-once guarantees at the cost of using
Expand Down Expand Up @@ -149,4 +155,4 @@ This connector supports heterogeneous messages. For example, say you have these

If you map the column `petName`, it will have the value `null` for the
entry with `key=1`. This scenario is supported. Similar behavior works
with Avro format.
with Avro format.

0 comments on commit ee160fb

Please sign in to comment.