Trove/incremental-backups

Incremental Backups
Adding support for incremental backups and restores. The main difference between this and the current manual backups will be that these will be deleted when the instance is deleted. This is intended to be run as a backend scheduled task.

https://wiki.openstack.org/wiki/Trove/scheduled-tasks

Storing Metadata
In order to create incremental backups we need to store a little extra information in full backups and incremental ones.


 * Add 'parent_id' field to the backups table in trove:

ALTER TABLE `backups` ADD COLUMN `parent_id` varchar(36) DEFAULT NULL;


 * Use swift metadata http://docs.openstack.org/api/openstack-object-storage/1.0/content/object-metadata.html to store any extra metadata for the backup.

curl –X POST -i \ -H "X-Auth-Token: my-token" \ -H "X-Object-Meta-Parent: https://storage.com/v1/XXX/path_to_parent_backup"" \           -H "X-Object-Meta-LSN: 123456789" \            https://storage.com/v1/XXX/path_to_incremental_backup

Incremental Workflow
API:

POST /backups {'name': 'incremental-backup-name', 'instance_id': 'instance_id', 'parent_id': ''} backup_info = { 'backup_type': 'incremental', 'backup_location': , 'parent': { 'location': , 'checksum': }      }
 * Another full or incremental backup must be made by the user or an automated system. (This backup ID is used as the parent ID)
 * An api call is made to run an incremental backup (one new optional field: 'parent_id'):
 * A entry in the backup table is created.
 * The backup handler looks up the information on the parent backup (error if it doesn't exist)
 * A message is sent to the guest to run the backup

GUEST: /usr/bin/innobackupex --stream --ibbackup=xtrabackup \ --incremental --incremental-lsn=%(lsn)s /var/lib/mysql
 * Backup agent will be changed to accept a type passed in by the api.
 * A new backup type of increamental uses the meta data to preform the restore. For xtrabackup this requires passing a 'lsn' number to the backup command:
 * After a successful backup record meta data along with other backup info (location, checksum, etc) For xtrabackup that will include the last LSN number of the backup.

Full backup changes (innobackupex)
Change the current backup process to add the meta data from the backup process.

def check_process: # check for 'completed OK!' # parse out the LSN number # update meta data property on backup return True

Restore Workflow (metadata in db)
During the create instance call if a backup ID is passed in restorePoint the api will need to change slightly:

API: a. Check that parent exists/checksum matches and add to the backup_info object b. Check if the parent backup has a parent and repeat
 * Check that the backup file exists and checksum matches.
 * Check if the backup has a parent
 * Send the backup_info to the guest to do the restore.

GUEST:
 * Read the backup_info and select the correct restore logic by backup_type
 * Preform restore on all parents if they exist.
 * Preform restore on backup
 * Finish guest install