Karbor

Karbor Mission Statement
To protect the Data and Meta-Data that comprises an OpenStack-deployed Application against loss/damage (e.g. backup, replication) by providing a standard framework of APIs and services that allows vendors to provide plugins through a unified interface

What is Karbor?
Karbor deals with protecting the Data and Meta-Data that comprises an OpenStack-deployed Application (what is referred to as "Project" in Keystone terminology) against loss/damage (e.g. backup, replication) - not to be confused with Application Security or DLP. It does that by providing a standard framework of APIs and services that enables vendors to introduce various data protection services into a coherent and unified flow for the user.

The koala is an arboreal herbivorous marsupial native to Australia, and Karbor is koala's indigenous name. Karbor has a natural pocket to protect their children. Our Karbor is is designed to give a simple and user-friendly experience to protect the user's data.

The main concept behind Karbor is to provide protection of an entire OpenStack project, across OpenStack sites (or with a single local site).

Glossary

 * Protectable: a type of element which Karbor can protect. Most prominently OpenStack resources (volume, project, server, etc.) The actual instance of Protectable is named Resource
 * Protection Plugin: a plugin, responsible for the implementation of protecting and restoring a Protectable or multiple Protectables
 * Bank Plugin: a plugin, responsible for persisting data between protect and restore operations, and between different sites
 * Protection Provider: a composition of Protection Plugins covering supported resource types, and a Bank Plugin for persisting the data
 * Checkpoint: the state, data, and metadata of protected resources, in a point of time
 * Protection Plan: a Protection Provider applied to a set of resources, with user parameters

Installation
For devstack installation, see: Karbor/Devstack

Sample Case
Lets take a typical 3-tier cloud app:

In order to fully protect such a deployment (e.g. Disaster Recovery), we would have to protect many resources, which have some dependency between them.

The following diagram shows how such a dependency tree might look:

In Karbor, we defined a plugin engine that loads a protection plugin for each resource type. Then, we let the user create a Protection Plan, which consists of all the resources she wants to protect.

These resources can be divided into groups, each of which is handled by a different plugin in Karbor:
 * Volume - Typically, a block of data that is mapped/attached to the VM and used for reading/writing
 * VM - A deployed workload unit, usually comprised of some metadata (configuration, preferences) and connected resources (dependencies)
 * Virtual Network - The virtual network overlay where the VM runs
 * Project - A group of VMs and their shared resources (e.g. networks, volumes, images, etc.)
 * Image - A software distribution package that is used to launch a VM

Open Architecture
Vendors create plugins that implement Protection mechanisms for different OpenStack resources.

User perspective: Protect Application Deployment
Users configure and manage custom protection plans on the deployed resources (topology, VMs, volumes, images, …). The user selects a "Protection Provider" from a selection of available Protection Providers, which is maintained and managed by the admin.

Admin perspective: Configure Protection Providers
The Admin defines which Protection Providers are available to the users. A "Protection Provider" is basically a bundle of per-resource protection plugins and a bank, which are curated from the total available protection plugins and bank plugins. In addition, the Admin configures a Bank Account for each user (tenant).



Resource (Protectable) API
Enables the Karbor user to access information about which resource types are protectable (i.e. can be protected by Karbor). In addition, enables the user to get additional information on each resource type, such as a list of actual instances and their dependencies.

Plan API
Enables the Karbor user to create or edit Protection Plans using a selected Protection Provider, as well as access all the parameters of the plan.

Provider API
Enables the Karbor user to list available providers and get parameters and result schema super-set for all plugins of a specific Provider.

Checkpoints API
Enables the Karbor user to access and manage Protection Checkpoints, as well as listing and querying of the existing Checkpoints in a Provider. In addition, provides Checkpoint Read Access to the Restore API, when recovering a protected application's data. Calling the Checkpoint Create (POST) API will start a protection process that will create a Vault in the user's Bank Account, on the Bank that is assigned to the Provider. The process will then pass the Vault on a call to the Protect action on each of the Protection Plugins assigned to the Provider, so each will write its metadata into the Vault. It is left up to the Plugin implementation to decide where to store the actual data (i.e. in the Vault or somewhere else).

Schedule Operation API
Enables the Karbor user to create a mapping between a Trigger and Operation(s) definitions.

Restore API
Allows the invocation and tracking of restore operations. Restore operation will take a checkpoint in the bank and provision it in a cloud.

Karbor Architecture
We defined three services for Karbor:

Karbor API service
These top-level north-bound APIs expose Application Data Protection services to the Karbor user. The purpose of the services is to maximize flexibility and accommodate for (hopefully) any kind of protection for any type of resource, whether it is a basic OpenStack resource (such as a VM, Volume, Image, etc.) or some ancillary resource within an application system that is not managed in OpenStack (such as a hardware device, an external database, etc.).

Karbor Schedule Service
This subsystem is responsible for scheduling and orchestrating the execution of Protection Plans. The implementation can be replaced by any other external solution. All actual Protection-related activities are managed via the Operation northbound APIs, in order to support:
 * Record maintaining of all operations in the Karbor database (to drive Operation Status APIs)
 * Decoupling the implementation of the Scheduler from the implementation of the Protection Service

Karbor Protection Service
This subsystem is responsible for handling the following tasks:
 * Operation Execution
 * Protectable (resources) plugin management
 * Protection provider management
 * Protection Plugin management
 * Bank Plugin management
 * Bank checkpoints sub-service

Available Protectables
As you might have noticed from it's flexible design. Karbor has the ability to add any new resource type. That being said most users will probably want to just use the default protectables provided with Karbor.

For version 1.0 we are aiming to have these protectables supported:

This is the dependency between protectables in the default distribution of Karbor. The arrows, much like inheritance, point to the parent since Protectables define what types might depend on them so this reflects who is responsible for the connection.