-
Notifications
You must be signed in to change notification settings - Fork 231
Streaming Platforms Comparision
lyx edited this page Jan 17, 2025
·
1 revision
Feature | AutoMQ | Apache Kafka | Confluent | Apache Pulsar | Redpanda | Warpstream |
---|---|---|---|---|---|---|
Apache Kafka Compatibility[1] | Native Kafka | Native Kafka | Native Kafka | Non-Kafka | Kafka Protocol | Kafka Protocol |
Source Code Availability | Yes | Yes | No | Yes | Yes | No |
Stateless Broker | Yes | No | No | Yes | No | Yes |
P99 Latency | Single-digit ms latency |
> 400ms | ||||
Continuous Self-Balancing | Yes | No | Yes | Yes | Yes | Yes |
Scale in/out | In seconds | In hours/days | In hours | In hours (scale-in); In seconds (scale-out) |
In hours/In seconds (Enterprise Only) | In seconds |
Spot Instance Support | Yes | No | No | No | No | Yes |
Partition Reassignment | In seconds | In hours/days | In hours | In seconds | In hours/In seconds (Enterprise Only) | In seconds |
Component | Broker | Broker Zookeeper (Non-KRaft) |
Broker Zookeeper Bookkeeper Proxy(Optional) |
Broker | Agent MetadataServer |
|
Durability | Guaranteed by S3/EBS[2] | Guaranteed by ISR | Guaranteed by Bookkeeper | Guaranteed by Raft | Guaranteed by S3 | |
Inter-AZ Networking Fees | No | Yes | No |
[1] Apache Kafka Compatibility's definition is coming from this blog.
[2] EBS Durability: On Azure, GCP, and Alibaba Cloud, Regional EBS replicas span multiple AZs. On AWS, ensure durability by double writing to EBS and S3 Express One Zone in different AZs.
- What is automq: Overview
- Difference with Apache Kafka
- Difference with WarpStream
- Difference with Tiered Storage
- Compatibility with Apache Kafka
- Licensing
- Deploy Locally
- Cluster Deployment on Linux
- Cluster Deployment on Kubernetes
- Example: Produce & Consume Message
- Example: Simple Benchmark
- Example: Partition Reassignment in Seconds
- Example: Self Balancing when Cluster Nodes Change
- Example: Continuous Data Self Balancing
-
S3stream shared streaming storage
-
Technical advantage
- Deployment: Overview
- Runs on Cloud
- Runs on CEPH
- Runs on CubeFS
- Runs on MinIO
- Runs on HDFS
- Configuration
-
Data analysis
-
Object storage
-
Kafka ui
-
Observability
-
Data integration