Skip to content
This repository has been archived by the owner on Feb 21, 2024. It is now read-only.

Latest commit

 

History

History

Chapter9

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 

CHAPTER 9: Building Data Pipelines

The main value Kafka provides to data pipelines is its ability to serve as a very large, reliable buffer between various stages in the pipeline.

Run Kafka Connect

bin/connect-distributed.sh config/connect-distributed.properties

The Debezium Project provides a collection of high-quality, open source, change capture connectors for a variety of databases.

Kafka can be looked at as a platform that can handle data integration (with Connect), application integration (with producers and consumers), and stream processing. Kafka could be a viable replacement for an ETL tool that only integrates data stores.