Kafka demo

Docs

• Installation

• Theory

• Producer

• Consumer

• Cluster

• Recovery

• Security

  • Security-SSL-Zookeeper

• Kubernetes

Projects

• Library-consumer and Library-producer Request: Library.http

• Docker compose files:

  • kraft.yaml - basic Kafka (3.4.0) example
  • kraft-kouncil.yaml - (UPDATED) Kafka (3.7.0) and Kouncil (1.7) - additional configuration descriptions inside the docker compose file
  • kraft-kouncil-cluster.yaml - three Kafka (3.4.0) instances and Kouncil (1.4)
  • kraft-kouncil-ssl.yaml - single Kafka (3.4.0) instance with ssl and Kouncil (1.4)
  • kraft-kouncil-traefik.yaml - single Kafka (3.4.0) instance with Traefik proxy (v3.0) and Kouncil (1.4)

---

Create topic

./kafka-topics.sh --create \
  --topic quickstart-events \
  --bootstrap-server localhost:9192

or

./kafka-topics.sh --create \
  --topic quickstart-events \
  --replication-factor 1 \
  --partitions 3 \
  --bootstrap-server localhost:9192

---

Produce Log/Message

./kafka-console-producer.sh \
    --broker-list localhost:9192 \
    --topic quickstart-events

With key:

./kafka-console-producer.bat \
  --broker-list localhost:9192 \
  --topic quickstart-events \
  --property "key.separator=-" \
  --property "parse.key=true"

Input e.g. <KEY>-<VALUE>

---

Consume Log/Message

./kafka-console-consumer.sh \
  --bootstrap-server localhost:9192 \
  --topic quickstart-events \
  --from-beginning

With key:

./kafka-console-consumer.sh \
  --bootstrap-server localhost:9192 \
  --topic quickstart-events \
  --from-beginning \
  --property "key.separator=-" \
  --property "print.key=true"

With Consumer Group:

./kafka-console-consumer.sh \
  --bootstrap-server localhost:9192 \
  --topic quickstart-events \
  --group <group-name>

---

List the topics in a cluster

./kafka-topics.sh --bootstrap-server localhost:9192 --list

---

List the consumer groups

./kafka-consumer-groups.sh --bootstrap-server localhost:9192 --list