VReplication is a core component of Vitess that can be used to compose
many features. It can be used for the following use cases:
Data Migrations: Use MoveTables to migrate tables into
Vitess or across Keyspaces with online revertable workflows.
Resharding: Use Reshard to scale Keyspaces
up or down as needed with automated online revertable workflows.
Materialized Views: Use Materialize to create
a view of the source table in a target keyspace. This materialization
can use a different primary vindex than the source.
It can also materialize a subset of the source columns, or add new expressions from
the source. This view will be kept up-to-date in real time. One can also materialize
reference tables onto all shards for improved data locality, allowing
Vitess to perform efficient local joins with those materialized tables.
Realtime Rollups: Use Materialize with aggregation
expressions in which case Vitess will create a rolled up version of the
source table which can be used for realtime analytics.
Change Notifications (CDC): The VStream
component of VReplication can be used for the application or a systems
operator to subscribe to change notifications and use it to keep downstream
systems up-to-date with the source.
Job Queues: VReplication can also be used to provide a job queue for
asynchronous processing of data using the Messaging
VReplication works as a stream or set of streams.
Each stream establishes replication from a source keyspace/shard to a
A given stream can replicate multiple tables. For each table, you can
specify a SELECT statement that represents both the transformation
rule and the filtering rule. The SELECT expressions specify the
transformation, and the WHERE clause specifies the filtering.
The SELECT expressions can be any non-aggregate MySQL expression, or
they can also be COUNT or SUM as aggregate expressions. Aggregate
expressions combined with the corresponding GROUP BY clauses will
allow you to materialize real-time rollups of the source table, which
can be used for analytics. The target table can have a different name
from the source.
For a sharded system like Vitess, multiple VReplication streams
may be needed to achieve the objective. This is because there
can be multiple source shards and multiple destination shards, and
the relationship between them may not be one to one.
VReplication performs the following essential functions:
from the source to the destination table in a consistent
fashion. For a large table, this copy can be long-running. It can be
interrupted and resumed. If interrupted, VReplication can keep
the copied portion up-to-date with respect to the source, and it can
resume the copy process at a point that is consistent with the
current replication position.
After copying is finished, it can continuously replicate
the data from the source to destination.
The copying rule can be expressed as a SELECT statement. The
statement should be simple enough that the materialized table can
be kept up-to-date from the data coming from the binlog. For
example, joins in the SELECT statement are not supported today.
Correctness verification: VReplication supports the VDiff command
which verifies that the target table is an exact representation of
the SELECT statement from the source by capturing consistent
snapshots of the source and target and comparing them against each
Journaling: If there is any kind of traffic cut-over where we
start writing to a different table than we used
to before, VReplication will save the current binlog positions
into a journal table. This can be used by other streams to resume
replication from the new source.
Routing rules: Although this feature is itself not a direct
functionality of VReplication, it works hand in hand with it. It
automatically manages sophisticated rules about where to route queries
depending on the type of workflow being performed. For example,
it is used to control the cut-over during MoveTables.
VReplication has the capability to batch transactions if the send rate of the source
exceeds the replay rate of the destination. This allows it to catch up very quickly
when there is a backlog. Load tests have shown a 3-20X improvement over traditional
MySQL replication depending on the workload.
The source VTTablet sends its current time along with every event. This allows the
target to correct for clock skew while estimating replication lag. Additionally,
the source starts sending heartbeats if there is nothing to send. If the target
receives no events from the source at all, it knows that it's definitely lagged
and starts reporting itself accordingly.
VReplication allows you to set the source keyspace/shard to be the same as the target.
This is especially useful for performing schema rollouts: you can create the target
table with the intended schema and vreplicate from the source table to the new
target. Once caught up, you can cutover to write to the target table.
In this situation, an apply on
the target generates a binlog event that will be picked up by the source and
sent to the target. Typically, it will be an empty transaction. In such cases,
the target does not generally apply these transactions, because such an application
will generate yet another event. However, there are situations where one needs
to apply empty transactions, especially if it's a required stopping point.
VReplication can differentiate between these situations and apply events
only as needed.
It is possible that multiple streams can conflict with each other and cause
deadlocks or lock waits. When such things happen, VReplication silently retries
such transactions without reporting an error. It does increment a counter so
that the frequency of such occurrences can be tracked.
The MoveTables and Reshard commands allow you to
specify a value for on-ddl. This allows you to specify what to do with DDL SQL
statements when they are encountered
in the replication stream from the source. The values can be as follows:
IGNORE: Ignore all DDLs (this is also the default, if a value for on-ddl
is not provided).
STOP: Stop when DDL is encountered. This allows you to make any necessary
changes to the target. Once changes are made, updating the state to Running
will cause VReplication to continue from just after the point where it
encountered the DDL.
EXEC: Apply the DDL, but stop if an error is encountered while applying it.
EXEC_IGNORE: Apply the DDL, but ignore any errors and continue replicating.
We caution against against using EXEC or EXEC_IGNORE for the following reasons:
You may want a different schema on the target
You may want to apply the DDL in a different way on the target
The DDL may take a long time to apply on the target and may disrupt replication, performance, and query execution while it is being applied (if serving traffic from the target)