Migrate is used to start and manage VReplication workflows for copying keyspaces and/or tables from a source Vitess cluster, to a target Vitess cluster.
This command is built off of MoveTables but has been extended to work with independent source and target topology services. It should be
utilized when moving Keyspaces or Tables between two separate Vitess environments. Migrate is an advantageous strategy for large sharded environments
for a few reasons:
Data can be migrated while the source Vitess cluster, typically the production environment, continues to serve traffic.
Shard mapping between Source and Target Vitess clusters is handled automatically by Migrate.
Similar to MoveTables, you may have different shard counts between the source and target Vitess clusters.
VDiffs and read-only SQL can be performed to verify data integrity before the Migration completes.
Migrate works as a copy of data not a move, source data remains once the Migrate completes.
Could be used for configuring lower environments with production data.
Please note the Migrate command works with an externally mounted source cluster. See the related Mount command for more information
on working with external Vitess clusters.
NOTE: there is no reverse vreplication flow with Migrate. After the Migrate complete command is given; no writes will be replicated between the Source and Target Vitess clusters. They are essentially two identical Vitess clusters running in two different environments. Once writing resumes on one of the clusters they will begin to drift apart.
Mount the source Vitess cluster using Mount. Mount register --name ext1 --topo-type etcd2 --topo-server localhost:12379 --topo-root /vitess/global
Apply source vSchema to the Target's Keyspace. ApplyVSchema --vschema-file commerceVschema.json commerce
Initiate the migration using create. Migrate --workflow import --target-keyspace customer create --source-keyspace commerce --mount-name ext1 --tablet-types replica
Monitor the workflow using show and status. Migrate --workflow import --target-keyspace customer showMigrate --workflow import --target-keyspace customer status
Confirm that data has been copied over correctly using VDiff.
Stop the application from writing to the source Vitess cluster.
Confirm again the data has been copied over correctly using VDiff.
Cleanup vreplication artifacts and source tables with complete. Migrate --workflow import --target-keyspace customer complete
Start the application pointed to the target Vitess Cluster.
Unmount the source cluster. Mount unregister --name ext1
The options for the supported commands are the same as MoveTables, with the exception of --enable-reverse-replication as setting
up the reverse vreplication streams requires modifying the source cluster's _vt sidecar database which we cannot do as that database is
specific to a single Vitess cluster and these streams belong to a different one (the target cluster).
A common option to give if migrating all of the tables from a source keyspace is the --all-tables option.
For Migrate to function properly, you will need to ensure communication is possible between the target Vitess cluster and the source Vitess cluster. At a minimum the following network concerns must be implemented:
Target vtctld/vttablet (PRIMARY) processes must reach the Source topo service.
Target vtctld/vttablet (PRIMARY) processes must reach EACH source vttablet's grpc port.
You can limit your source vttablet's to just the replicas by using the --tablet-types option when creating the migration.
If you're migrating a keyspace from a production system, you may want to target a replica to reduce your load on the primary vttablets. This will also assist you in reducing the number of network considerations you need to make.