Skip to content

Commit

Permalink
new index page
Browse files Browse the repository at this point in the history
  • Loading branch information
beajohnson committed Apr 12, 2024
1 parent 4ed4f20 commit 89fdd13
Showing 1 changed file with 80 additions and 0 deletions.
80 changes: 80 additions & 0 deletions docs-src/zdm-core/modules/migrate/pages/index.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
= Introduction to {zdm-product}
:page-tag: migration,zdm,zero-downtime,zdm-proxy,introduction
//ifdef::env-github,env-browser,env-vscode[:imagesprefix: ../images/]
//ifndef::env-github,env-browser,env-vscode[:imagesprefix: ]

Enterprises today want to reliably migrate mission-critical client applications and data to cloud environments with zero downtime or near zero downtime during the migration.

A migration is a workflow that encompasses the lifecycle of uploadng and importing your data to the selected keyspaces.
At {company}, we've developed a set of thoroughly-tested self-service tools to walk you through well-defined migration options:

* Zero Downtime Migration
* Cassandra Data Migrations
* DSBulk Migration
include::partial$migration-scenarios.adoc[]

First, a couple of key terms to understand for using our migration tools:

* **Origin:** This cluster is your existing Cassandra-based environment, whether it's open-source Apache Cassandra, DSE, or {astra_db} Classic.
* **Target:** This cluster is the new environment to which you want to migrate client applications and data.
For additional terms, see the xref:glossary.adoc[glossary].

////
[TIP]
====
An important migration prerequisite is that you already have the matching schema on Target.
A CQL statement that your client application sends to {zdm-proxy} must be able to succeed on both Origin and Target clusters.
This means that any keyspace that your client application uses must exist on both Origin and Target with the same name.
Table names must also match. For more, see xref:feasibility-checklists.adoc#_schemakeyspace_compatibility[Schema/keyspace compatibility].
====

Your migration project occurs through a sequence of phases.
Here is an example of these phases with ZDM using ZDM proxy, which is a simple and lightweight proxy that handles all the real-time requests generated by your client applications.
For more, see http:addlinkhere.adoc[ZDM proxy]

image::{imagesprefix}zdm-migration-all.png[Migration phases from start to finish]

Before walking through illustrations of each phase, look at a pre-migration, high-level view. At this point, your client applications are performing read/write operations with an existing CQL-compatible database. That is, Apache Cassandra, DSE, or Astra DB.

image::{imagesprefix}zdm-migration-before-starting.png[Diagram shows existing CQL-compatible environment before migration starts.]

Before any migration begins, complete prerequisites, prepare your environment, and set up the recommended infrastructure.
////

== With and without zero downtime migration examples

Data migrations with downtime are those where the data being migrated is critical for ongoing operations, and the migration process itself requires systems to be taken offline temporarily. Downtime may be required for:
* Migration for large-scale systems: these systems may require downtime for completely smooth transition.
* Database migration: large database migrations or database software upgrades may need downtime for data consistency and integrity.
* Storage system migration: migrating from on-prem to cloud may need downtime to reconfigure systems.
* Network infrastructure changes: these systems may require downtime for completely smooth transition.

A migration with no downtime is seamless and ideal, but doesn't always happen or a viable option.
Downtime may not be required for:
* Continuous data replication: When real-time data replication is implemented, changes made to the source data are continuously mirrored to the target system.
This means both systems remain synchronized without the need for downtime.
* Online database migration: Some database migration allows for the migration to happen while the database remains online and accessible. Technologies like database replication and synchronization tools can also facilitate seamless migration with minimal disruption.
* Incremental data migration:Transfers data in smaller batches over time, which allows for minimal disruption to ongoing operations since only a portion of the data is being migrated at any given time.
This option can be used for large-scale migrations where downtime is not feasible or acceptable.
* Load balancing and redundancy: In environments with redundant systems or load balancing capabilities, data migrations can be performed without downtime by routing traffic to the target system while the migration is in progress. This allows users to access services without interruption, as the load is distributed between the source and target systems.
* Cloud-based data migration: This often provide tools and services that enable live data migration with minimal downtime.

== Data migration tools

For modest data volumes where temporary downtime is tolerable, Datastax recommends two open-source data migration tools to help with brief outages:
* Cassandra Data Migrator: The best choice to migrate large data quantities, and where detailed logging, data verifications, table column renaming (if needed), and reconciliation options are provided.
* DSBulk Migrator: This leverages DataStax Bulk Loader (DSBulk) to perform the data migration, and provides new commands specific to migrations. DSBulk Migrator is ideal for simple migration of smaller data quantities, and where data validation (other than post-migration row counts) is not necessary.

These tools provide sophisticated features that help you migrate your data from any Cassandra Origin (Apache Cassandra®, {company} Enterprise (DSE), {company} {astra_db}) to any Cassandra Target (Apache Cassandra, DSE, {company} {astra_db}).

== Zero downtime migration

For large datasets, it's best to use the zero downtime migration tool: Zero Downtime Migration (ZDM).
ZDM provides a simple and reliable way for you to migrate applications from any CQL-based cluster (Apache Cassandra®, {company} Enterprise (DSE), {astra_db}, or any type of CQL-based database) to any other CQL-based cluster, without any interruption of service to the client applications and data.

* You can move your application to {astra_db}, DSE, or Cassandra with no downtime and with minimal configuration changes.
* Your clusters will be kept in sync at all times by a dual-write logic configuration.
* You can rollback at any point, for complete peace of mind.

0 comments on commit 89fdd13

Please sign in to comment.