ReddwarfAppliesForIncubation

Project codename:

reddwarf

Summary (one sentence abstract of the project):

A control plane for managing multiple database technologies.

Detailed Description:

Reddwarf is Database as a Service for Openstack. It's designed to run entirely on OpenStack, with the goal of allowing users to quickly and easily utilize the features of a database without the burden of handling complex administrative tasks. Cloud users and database administrators can provision and manage multiple database instances as needed. Initially, the service will focus on providing resource isolation at high performance while automating complex administrative tasks including deployment, configuration, patching, backups, restores, and monitoring. The future of Reddwarf will consume multiple database technologies, including relational and non relational technologies. The vision is for Reddwarf to control any datastore that requires ongoing maintanence, data integrity, and replication.

Reddwarf is a consumer of openstack. It is built on and leverages core openstack infrastructure services. It allows a service provider the flexibility of deployment and configuration of the core openstack components, and uses those components as the building blocks of its service. This includes Keystone for auth, nova for instance creation, cinder for volume support, glance for images, swift to store backups, and quantum for networking. We hope to eventually consume cielometer for metrics and heat for initial image creation, and potentially bare metal to spin up reddwarf instances for the infastructure database(s) within a nova on nova install.

One may ask how is this different from Heat. We will consume heat to set up a initial mysql datastore, which fits perfectly into our vision for using openstack services. The main differenciator for datastores is that they require specific maintainence to back up, restore, and keep continued maintainence of replication, slaves, and yet to be determined management tasks. A datastore is not simply a fire and forget resource. It needs constant monitoring and we want to use this as the differenciator between reddwarf and a service for spinning up fire and forget resources. As heat continues to evolve, and some maintanence tasks can be done through it, such as upgrades and restarts, we will consume those services. Our goal is to focus on the specifics of keeping a datastore online and use other projects as much as possible for the smaller tasks.

Basic roadmap for the project:

Reddwarf can currrently deploy multiple mysql implementations. The API is evolving with new features, but as it stands the API is very stable in its implementation for the basic tasks. In the codebase as it stands today, both stock mysql and percona mysql are supported. The system is built such that multiple technolgies can be leveraged, from the datastore implementation itself, to the tools used to keep maintiannce ongoing. for instance, the backup feature, which is inflight, allows the cloud deployer to choose mysqldump or xtrabackup to backup the datastore.

From a datastore perspective, Reddwarf's functionality currently is the ability to spin up secured mysql databases, create/destroy schemas and users, and enable a root user. From a openstack project perspective, Reddwarf also has quota and limit support and consumes Oslo to build out the architecture of the api.

Reddwarf has a healthy backlog of features in order to fufil mysql features. Some of the inflight features are configuration (my.cnf) edits, manual backups and restores, notifications. Others that are not yet started include automated backups, including point in time restores, API types/versions, full featured user modification, and replication. this does not encompass all features. The current roadmap can be viewed as blueprints on launchpad.

There are also many housekeeping blueprints to maintain parity with other openstack projects. Reddwarf has strived to keep in line with the ecosystem, but there are some things that are necessary, such as completing integration with devstack. As it stands, reddwarf is integrated as much as possible today, and uses local.sh to instrument the reddwarf infrastructure. It also currently uses vm-gating and runs functional tests on every review.

Location of project source code:

Reddwarf operates like any other openstack project. blueprints are registered and approved in the same fashion as the other projects, release cycles are followed, and code is gated through unit tess and functional tests. The codebase is in stackforge. Reviews are in openstack gerrit.

Programming language, required technology dependencies:

Python. pypi dependencies are listed in pip-requires and test-requires.

Is project currently open sourced? What license?:

Yes. Apache 2.0.

Level of maturity of software and team:

The team has followed the openstack ecosystem for a long time. They are well versed in understanding the ecosystem and follow it as closely as possible. Weekly meetings are held, a presence is always in irc, reviews are thuroughly picked through, and they hold a high standard for all code that lands in master. All features require unit and functional tests, and gating is done on these thru openstack jenkins and a external jenkins for said functional tests. The software is run in production currently by rackspace on thousands of nodes.

Proposed project technical lead and qualifications:

Michael Basnight (hub_cap on irc). Michael has lead the project since its inception at rackspace. He was the lead of the internal project, but has shifted his role within rackspace to be fully dedicated to lead the public project. He is focused on the quality of the API, infrastructure, and testability of the project. He follows the openstack community closely and has contributed greatly to the project. He currently manages blueprints, runs the weekly meetings, and evanglises reddwarf in the community. He has spoken at many of the openstack conferences and has attended almost all of the conferences since reddwarf began within rackspace.

Other project developers and qualifications:

Currently HP and rackspace contribute to the reddwarf project. They operate as a joint team within the open. They are listed below.

GUYS::::: INSERT YOUR QUALS HERE :::::::

Infrastructure requirements (testing, etc):

Reddwarf has built its own testing environment. Its currently run as a gating job within by jenkins. Tox runs pep8 and unit tests on each commit through openstack gerrit. The reddwarf developers understand that, over time, they will adopt the openstack testing frameworks for all new functional tests. The core of reddwarf is run on openstack and installed within devstack. Reddwarf-integration, a project on stackforge, controls the installation and setup of reddwarf for development and testing. Tests are run in a openstack/kvm environment.

Have all current contributors agreed to the OpenStack CLA?

Yes.