The default backup implementation is builtin, however we strongly recommend using the xtrabackup engine as it is more robust and allows for non-blocking backups. Restores will always be done with whichever engine was used to create the backup.
When a tablet starts, Vitess checks the value of the -restore_from_backup command-line flag to determine whether to restore a backup to that tablet.
If the flag is present, Vitess tries to restore the most recent backup from the BackupStorage system when starting the tablet or if the --restore_from_backup_ts flag (Vitess 12.0+) is also set then using the latest backup taken at or before this timestamp instead. Example: '2021-04-29.133050'
If the flag is absent, Vitess does not try to restore a backup to the tablet. This is the equivalent of starting a new tablet in a new shard.
As noted in the Configuration section, the flag is generally enabled all of the time for all of the tablets in a shard. By default, if Vitess cannot find a backup in the Backup Storage system, the tablet will start up empty. This behavior allows you to bootstrap a new shard before any backups exist.
If the --wait_for_backup_interval flag is set to a value greater than zero, the tablet will instead keep checking for a backup to appear at that interval. This can be used to ensure tablets launched concurrently while an initial backup is being seeded for the shard (e.g. uploaded from cold storage or created by another tablet) will wait until the proper time and then pull the new backup when it's ready.
You can also confirm your backup finished by viewing the files in your configured --<engine>_backup_storage_root location. You will still need to test and verify these backups for completeness. Note that backups are stored by keyspace and shard under --<engine>_backup_storage_root. For example, when using --file_backup_storage_root=/vt/vtdataroot/backups:
Bootstrapping a new tablet is almost identical to restoring an existing tablet. The only thing you need to be cautious about is that the tablet specifies its keyspace, shard and tablet type when it registers itself in the topology. Specifically, make sure that the following additional vttablet parameters are set:
E0310 08:15:45.336083 197442 main.go:72] remote error: rpc error: code = Unknown desc = TabletManager.Backup on zone1-0000000102 error: xtrabackupUser must be specified.: xtrabackupUser must be specified
Fix: Set the vtctld and vttablet flag --xtrabackup_user
E0310 08:22:22.260044 200147 main.go:72] remote error: rpc error: code = Unknown desc = TabletManager.Backup on zone1-0000000102 error: unable to start backup: exec: "xtrabackup": executable file not found in $PATH: unable to start backup: exec: "xtrabackup": executable file not found in $PATH
* Ensure the xtrabackup binary is in the $PATH for the $USER running vttablet
* Alternatively, set --xtrabackup_root_path on vttablet provide path to xtrabackup/xbstream binaries via vtctld and vttablet flags
I0310 12:34:47.900363 211809 backup.go:163] I0310 20:34:47.900004 xtrabackupengine.go:310] xtrabackup stderr: Invalid --stream argument: tar
Streaming in tar format is no longer supported in 8.0; use xbstream instead
Fix: Set the --xtrabackup_stream_mode flag to to xbstream on vttablets and vtctlds
I0310 12:49:32.279729 215835 backup.go:163] I0310 20:49:32.279435 xtrabackupengine.go:310] xtrabackup stderr: Error: Unsupported server version 8.0.23-0ubuntu0.20.04.1.
I0310 12:49:32.279773 215835 backup.go:163] I0310 20:49:32.279485 xtrabackupengine.go:310] xtrabackup stderr: Please upgrade PXB, if a new version is available. To continue with risk, use the option --no-server-version-check.
To continue with risk: Set --xtrabackup_backup_flags=--no-server-version-check. Note this occurs when your MySQL server version is technically unsupported by xtrabackup.