-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
4ed4f20
commit 89fdd13
Showing
1 changed file
with
80 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,80 @@ | ||
= Introduction to {zdm-product} | ||
:page-tag: migration,zdm,zero-downtime,zdm-proxy,introduction | ||
//ifdef::env-github,env-browser,env-vscode[:imagesprefix: ../images/] | ||
//ifndef::env-github,env-browser,env-vscode[:imagesprefix: ] | ||
|
||
Enterprises today want to reliably migrate mission-critical client applications and data to cloud environments with zero downtime or near zero downtime during the migration. | ||
|
||
A migration is a workflow that encompasses the lifecycle of uploadng and importing your data to the selected keyspaces. | ||
At {company}, we've developed a set of thoroughly-tested self-service tools to walk you through well-defined migration options: | ||
|
||
* Zero Downtime Migration | ||
* Cassandra Data Migrations | ||
* DSBulk Migration | ||
include::partial$migration-scenarios.adoc[] | ||
|
||
First, a couple of key terms to understand for using our migration tools: | ||
|
||
* **Origin:** This cluster is your existing Cassandra-based environment, whether it's open-source Apache Cassandra, DSE, or {astra_db} Classic. | ||
* **Target:** This cluster is the new environment to which you want to migrate client applications and data. | ||
For additional terms, see the xref:glossary.adoc[glossary]. | ||
|
||
//// | ||
[TIP] | ||
==== | ||
An important migration prerequisite is that you already have the matching schema on Target. | ||
A CQL statement that your client application sends to {zdm-proxy} must be able to succeed on both Origin and Target clusters. | ||
This means that any keyspace that your client application uses must exist on both Origin and Target with the same name. | ||
Table names must also match. For more, see xref:feasibility-checklists.adoc#_schemakeyspace_compatibility[Schema/keyspace compatibility]. | ||
==== | ||
|
||
Your migration project occurs through a sequence of phases. | ||
Here is an example of these phases with ZDM using ZDM proxy, which is a simple and lightweight proxy that handles all the real-time requests generated by your client applications. | ||
For more, see http:addlinkhere.adoc[ZDM proxy] | ||
|
||
image::{imagesprefix}zdm-migration-all.png[Migration phases from start to finish] | ||
|
||
Before walking through illustrations of each phase, look at a pre-migration, high-level view. At this point, your client applications are performing read/write operations with an existing CQL-compatible database. That is, Apache Cassandra, DSE, or Astra DB. | ||
|
||
image::{imagesprefix}zdm-migration-before-starting.png[Diagram shows existing CQL-compatible environment before migration starts.] | ||
|
||
Before any migration begins, complete prerequisites, prepare your environment, and set up the recommended infrastructure. | ||
//// | ||
|
||
== With and without zero downtime migration examples | ||
|
||
Data migrations with downtime are those where the data being migrated is critical for ongoing operations, and the migration process itself requires systems to be taken offline temporarily. Downtime may be required for: | ||
* Migration for large-scale systems: these systems may require downtime for completely smooth transition. | ||
* Database migration: large database migrations or database software upgrades may need downtime for data consistency and integrity. | ||
* Storage system migration: migrating from on-prem to cloud may need downtime to reconfigure systems. | ||
* Network infrastructure changes: these systems may require downtime for completely smooth transition. | ||
|
||
A migration with no downtime is seamless and ideal, but doesn't always happen or a viable option. | ||
Downtime may not be required for: | ||
* Continuous data replication: When real-time data replication is implemented, changes made to the source data are continuously mirrored to the target system. | ||
This means both systems remain synchronized without the need for downtime. | ||
* Online database migration: Some database migration allows for the migration to happen while the database remains online and accessible. Technologies like database replication and synchronization tools can also facilitate seamless migration with minimal disruption. | ||
* Incremental data migration:Transfers data in smaller batches over time, which allows for minimal disruption to ongoing operations since only a portion of the data is being migrated at any given time. | ||
This option can be used for large-scale migrations where downtime is not feasible or acceptable. | ||
* Load balancing and redundancy: In environments with redundant systems or load balancing capabilities, data migrations can be performed without downtime by routing traffic to the target system while the migration is in progress. This allows users to access services without interruption, as the load is distributed between the source and target systems. | ||
* Cloud-based data migration: This often provide tools and services that enable live data migration with minimal downtime. | ||
|
||
== Data migration tools | ||
|
||
For modest data volumes where temporary downtime is tolerable, Datastax recommends two open-source data migration tools to help with brief outages: | ||
* Cassandra Data Migrator: The best choice to migrate large data quantities, and where detailed logging, data verifications, table column renaming (if needed), and reconciliation options are provided. | ||
* DSBulk Migrator: This leverages DataStax Bulk Loader (DSBulk) to perform the data migration, and provides new commands specific to migrations. DSBulk Migrator is ideal for simple migration of smaller data quantities, and where data validation (other than post-migration row counts) is not necessary. | ||
|
||
These tools provide sophisticated features that help you migrate your data from any Cassandra Origin (Apache Cassandra®, {company} Enterprise (DSE), {company} {astra_db}) to any Cassandra Target (Apache Cassandra, DSE, {company} {astra_db}). | ||
|
||
== Zero downtime migration | ||
|
||
For large datasets, it's best to use the zero downtime migration tool: Zero Downtime Migration (ZDM). | ||
ZDM provides a simple and reliable way for you to migrate applications from any CQL-based cluster (Apache Cassandra®, {company} Enterprise (DSE), {astra_db}, or any type of CQL-based database) to any other CQL-based cluster, without any interruption of service to the client applications and data. | ||
|
||
* You can move your application to {astra_db}, DSE, or Cassandra with no downtime and with minimal configuration changes. | ||
* Your clusters will be kept in sync at all times by a dual-write logic configuration. | ||
* You can rollback at any point, for complete peace of mind. |