-
Notifications
You must be signed in to change notification settings - Fork 231
Technical advantage: Overview
lyx edited this page Jan 17, 2025
·
1 revision
AutoMQ adopts a storage-compute separation approach, evolving Apache Kafka® from a Shared Nothing architecture to a Shared Storage architecture. This shift enables full utilization of cloud-native technology and cost advantages. Technically, the shared storage architecture offers inherent benefits in addressing operational challenges like partition reassignment and automatic scaling. AutoMQ leverages these benefits through product capabilities, as shown in the table below.
Advantages | AutoMQ | Apache Kafka® |
---|---|---|
- | - | - |
Native support for Kafka protocol | ✅ Natively supported, reuses protocol layer code | ✅ Natively supported |
Second-level partition reassignment | ✅ Shared storage, no need to copy data during reassignment | ❌ Reassigns local data, typically takes hours per partition |
Second-level smooth scaling | ✅ Achieves second-level scaling by bulk reassigning partitions | ❌ Due to data copying during partition reassignment, cannot support fast scaling, typically takes tens of hours |
Continuous self-balancing | ✅ Built-in Auto Balancing component, dynamically adjusts traffic to eliminate hotspots | ❌ Prone to local hotspots, affecting stability |
Stateless Broker | ✅ Data offloaded to shared storage, Brokers are stateless and can use Spot instances | ❌ Local disk state, requires extensive data transfer for decommissioning |
Cold-hot isolation, 5x cold read efficiency | ✅ Out-of-the-box cold-hot isolation capability, supports multi-tenant business scenarios | ❌ Cold and hot data affect each other, exacerbating system degradation, cannot support multi-tenant scenarios |
- What is automq: Overview
- Difference with Apache Kafka
- Difference with WarpStream
- Difference with Tiered Storage
- Compatibility with Apache Kafka
- Licensing
- Deploy Locally
- Cluster Deployment on Linux
- Cluster Deployment on Kubernetes
- Example: Produce & Consume Message
- Example: Simple Benchmark
- Example: Partition Reassignment in Seconds
- Example: Self Balancing when Cluster Nodes Change
- Example: Continuous Data Self Balancing
-
S3stream shared streaming storage
-
Technical advantage
- Deployment: Overview
- Runs on Cloud
- Runs on CEPH
- Runs on CubeFS
- Runs on MinIO
- Runs on HDFS
- Configuration
-
Data analysis
-
Object storage
-
Kafka ui
-
Observability
-
Data integration