Self-Organized Distributed OpenStack
Contents
Overview
A no SPOF self-organized solution to make 100 OpenStack clusters collaborate
We propose to handle multiple OpenStack as a unified one. Our solution uses a consensus algorithm to manage a highly-available distributed OpenStack to optimize geo-distributed resources.
Our proposal is fully integrated with current OpenStack APIs and services, and has no impact neither on the api specifications nor the service functionnalities.
Added values of such an architecture enable relevant features: smart orchestration of appliances inside a distributed OpenStack, high availability of project appliances, community cloud creation, simulation of outage, disaster recovery, reduction of costs and energy footprint...
Motivation
What's up in datacenters OpenStack area?
The data center market is seen as a growing market, where new leaders will emerge. The number of data centers in academia, industry and public sector is now significant (>3600 colocation data centers worldwide [1], 32 OpenStack public clouds and 29 OpenStack hosted private clouds according to OpenStack marketplace [2])
This market is segmented, depending of the usage. The main topics covered in data centers are colocation, hosting (shared, dedicated, managed), datacenter facilities (power and cooling, eco-efficiency), CDN, utility computing.
The market is also starting to rationalize in order to get companies with critical size. (Orange bought CloudWatt...)
Generally, the actors can take two different directions to handle the scalability of their infrastructures:
- Vertically - the infrastructures are gathered in a huge centralized data centers (Google, Apple, Amazon...)
- Horizontally - the infrastructures are distributed over multi geo-distributed and independent data centers (Comcast...).
Why do we need multi data center?
Improve performance: Multi data centers are used to replicate a same service. For users concern, it helps to reduce the latency and improves throughput.
Improve resiliency: They are also involved to meet the requirement of disaster tolerance through multiple copies. Thus, it is possible to have continuity in the services on a backup site when primary site is unavailable.
Improve maintenance process: They want to management of certain emergency situations such as maintenance on a complete cluster.
Enable location requirements: They help to have local affinity: host content or services to comply with juridical requirements and geographical constraints.
A less obvious scenario involves computation or distributed services across multiple data center. This can be related to network of computers.
What are the needs of cloud providers with multi data center?
What already exist in OpenStack?
Our proposal
Inspiration
We work with a telco that owns his own network. One way to improve the network is by providing computation and storage capabilities closer to its customer. Our client aspires to open medium size data centers on his geographically distributed nodes and of course all the above requirements.
Our team has a strong expertise on distributed system, through academic research (Inria/Myriads), publications (Cambridge & Scholars Publishing, IEEE Internet Computing, 5th IEEE International Conference on Cloud Computing (CLOUD))and participation in numerous projects such as Contrail, Virtual Execution Platform, ConPaaS, Snooze. We propose our telco client to work on a generic architecture that can also fulfill their needs.
Use cases
Functional architecture
Separation of roles
Our architecture is component-based and implements a separation of roles according to the principle for autonomic computing of MAPE-K feedback loop.
We defined 3 specific roles : Exec, KnowMon and AnalysePlan. Thereafter, we will apply the MAPE-K concept to each component.
- Exec, Execute role of MAPE-K, manages monitoring information.
- KnowMon, Knowledge and Monitor roles of MAPE-K, store the knowledge and monitoring information (from Exec and the infrastructure itself). KnowMon also emits alerts caused by failures, overloads...
- AnalysePlan, Analyze and Plan roles of MAPE-K, analyses the knowledge, schedule applications' placements and deals with alerts from KnowMon involving eg rescheduling...
How to distribute OpenStack
key word of a good distribution
Functional components
- nodes management
- Identity and Services Management
- Images' distribution management
- Monitoring management
- Orchestrator for distributed system
