README.md

Event Stream Processing on Azure with Open Source Software

Reach Out to Use Regarding Your Use Case(s)

If you have a use case for Event Stream Processing and are currently using or would like to deploy your workloads on Microsoft Azure, please use this form to reach out to us with some details and we will get back to you as soon as possible. Thanks.

https://forms.office.com/r/dgVQtcX9E0

Deployment Options for Apache Kafka on Azure

Confluent Cloud Integration with Microsoft Azure

https://azure.microsoft.com/en-us/blog/introducing-seamless-integration-between-microsoft-azure-and-confluent-cloud/

Azure Event Hubs for Apache Kafka

https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-quickstart-kafka-enabled-event-hubs

Azure HDInsights for Apache Kafka

https://docs.microsoft.com/en-us/azure/hdinsight/kafka/apache-kafka-introduction

Integrations with Apache Kafka

• https://www.confluent.io/hub/#azure
• https://www.confluent.io/blog/real-time-search-and-analytics-with-confluent-cloud-azure-redis-spring-cloud/
• https://www.confluent.io/blog/consume-avro-data-from-kafka-topics-and-secured-schema-registry-with-databricks-confluent-cloud-on-azure/
• https://www.confluent.io/blog/integrate-azure-and-confluent-to-stream-data-into-cosmos-db/
• https://developer.confluent.io/podcast/connecting-azure-cosmos-db-with-apache-kafka-better-together-ft-ryan-crawcour
• https://developer.confluent.io/podcast/building-real-time-data-pipelines-with-microsoft-azure-databricks-and-confluent

Integrations with Apache Flink

Apache Flink integrates with Azure Cosmos DB via the Cassandra and MongoDB APIs. It also integrates with Azure Storage. Additional integrations are forth-coming. https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/connectors/datastream/overview/

Integrations with Apache Beam

Apache Beam integrates with Event Hubs via the Kafka protocol and also integrates with Azure Cosmos DB via the Cassandra and Mongo DB APIs. Additional integrations are forth-coming. There is integration with Azure Redis Cache and the ElasticSearch offering from Elasic on Azure. Furthermore, Apache Beam also integrates with any of the Kafka deployment options on Azure (via Confluent, Azure Event Hubs or HDInsights). https://beam.apache.org/documentation/io/built-in/

Integrations with Apache Pulsar

Apache Pulsar integrates with Cosmos DB via the MongoDB and Cassandra API. You can also integrate with the Azure DB for PostgreSQL and MySQL via the Debezium connectors. There is integration with Azure Redis Cache and the ElasticSearch offering from Elasic on Azure. Furthermore, Apache Pulsar also integrates with any of the Kafka deployment options on Azure (via Confluent, Azure Event Hubs or HDInsights). https://pulsar.apache.org/docs/en/io-connectors/

Upcoming Integrations with Flink, Kafka and Apache Beam

https://github.com/projectAcetylcholine/projects