Jump to: navigation, search

Difference between revisions of "OpenStack cascading solution"

(Last update)
(Last update)
Line 190: Line 190:
  
 
= Last update =
 
= Last update =
Aug 23, 2014
+
Aug 29, 2014
--[[User:Chaoyi Huang|joehuang]] ([[User talk:Chaoyi Huang|talk]]) 13:15, 29 August 2014 (UTC)
 

Revision as of 13:16, 29 August 2014

Overview

OpenStack cascading solution is designed to make the OpenStack being able to integrate distributed OpenStack cloud into one cloud, for example, a cloud with million level VMs distributed in many data centers.

  • The parent OpenStack expose standard OpenStack API
  • The parent OpenStack manage many child OpenStacks by using standard OpenStack API
  • Each child OpenStack functions as a Amazon like available zone and is hidden by the parent OpenStack

Cascading01.png

  • Cascading OpenStack: the parent OpenStack, providing API and scheduling and orchestration of Cascaded OpenStacks
  • Cascaded OpenStack: the child OpenStack, provisioning the VM, Volume and virtual Networking resources


OpenStack cascading mainly concentrate on API aggregation, merging multiple cascaded OpenStacks to work as a single Openstack in the tenant access API interface(Nova/Cinder/Neutron/Glance/Ceilometer API). For tenants, it is a single region, and multiple child OpenStacks are not visible which had already been integrated and hidden by the parent OpenStack. The tenant can see multiple "available zones", and each child OpenStack exactly work internally as Amazon like availability zone.

Motivation

To build large scale OpenStack based cloud, for example, the cloud includes 1 million VMs or 100k hosts. There are big challenges

Naturally, there are two ways to do that:

1. scale up a single monolithic OpenStack region, but

  • It’s a big challenge for a single OpenStack to manage scale for example 1 million VMs or 100K hosts.
  • Can not obtain real fault isolation area like EC2’s available zone, all of the cloud are tighten up into one OpenStack because of shareing RPC message bus and database.
  • Single huge monolithic system bring high risk with O&M & trouble shooting, and big challenge for even the most skilled Op team to handle SW rolling upgrade and configuration changes, etc.
  • Time consumption for heterogeneous vendor’s infrastructure integration, multi-vendor's infrastructure co-existence is high demand for large scale cloud


2. setup hundreds of OpenStack Regions with discrete API endpoint, but

  • Have to buy or develop his own cloud management platform to integrate the discrete cloud into one cloud, and also, OpenStack API ecosystem is lost.
  • Or, leave the cloud with splitted resource island without any association…

Inspiration

It's not reinventing the wheel. The OpenStack cascading solution is inspired from:

  • remote clustered hypvisors, like vCenter, Ironic running under Nova. This makes Nova can scale to larger scale.
  • plugable driver/agent architecture of Nova/Cinder/Neutron, and the magic FRACTAL mathematics, which has character of recursive self-similar and growth to scale. Please refer to OpenStack cascading and fractal for more information.

From the inspiration, the conclusion comes:

  • Handling cascaded Nova/Cinder similar as what has been done over vCenter and Ironic
  • Handling cascaded Neutron similar as the L2 OVS agent / L3 DVR agent. The challenge is to make cross cascaded OpenStack L2/L3 networking like that inside one OpenStack


Now, the cascaded OpenStack simply like a huge computer.

Architecture

For detailed architecture design, please refer to following links.
Please leave your comments in the Google Doc or send mail to PoC team member ( Listed in PoC team section) .

(release date: July 25, 2014)


The OpenStack cascading includes cascading of Nova, Cinder, Neutron, Glance and Ceilometer. KeyStone will be global service shared by cascading and cascaded OpenStacks, and Heat will consume cascading OpenStack API. Therefore no cascading is required for KeyStone and Heat.

Cascading02.png


  • Nova-Proxy: Similar role like Nova-Compute. Transfer the VM operation to cascaded Nova. Also responsible for attach volume and network to the VM in the cascaded OpenStack.
  • Cinder-Proxy: Similar role like Cinder-Volume. Transfer the volume operation to cascaded Cinder.
  • L2-Proxy: Similar role like OVS-Agent. Finish L2-networking in the cascaded OpenStack, including cross OpenStack networking.
  • L3-Proxy: Similar role like L3-Agent. Finish L3-networking in the cascaded OpenStack, including cross OpenStack networking.
  • FW-Proxy: Similar role like FWaaS-Agent. TBD
  • LB-Proxy: Similar role like LBaaS-Agent. TBD
  • VPN-Proxy: Similar role like VPNaaS-Agent. TBD


For Glance, it's not a must to have cascading. It's up to back-end store and deployment decision. Both global glance or glance with cascading can work for OpenStack cascading solution.
For Ceilometer, the Gnocchi may leads to API and design change, so the cascading has not been taken into consideration yet.

Cascading03.png

  • Sync-Manager: Synchronize image among the cascading and policy determined Cascaded OpenStacks.
  • Ceilometer-Proxy: Transfer the request to destined Ceilometer or collect information from several Ceilometer. Implementation not started

Value to cloud admin

See more detail information in the document linked in the section "Architecture of OpenStack cascading". The major advantage of the architecture is listed here:

Cascading04.png
Cascading OpenStack aggregate many OpenStack cloud via standard OpenStack API into one cloud, and expose one OpenStack API endpoint by cascading OpenStack.

Cascading05.png
if one Cascaded OpenStack failed, other part of the cloud can still work and accessible. This makes one cascaded OpenStack can act as Amazon like Availability Zone. If Cascading OpenStack failed, all Cascaded OpenStacks are still manageable via OpenStack API independently. In phase I, the provisioning is not allowed for consistency consideration between cascading and Cascaded OpenStack. In phase II, after the consistency issue is solved, the provisioning can be allowed even if Cascading OpenStack is failed.

Cascading06.png
Cascading OpenStack manages Cascaded OpenStacks via standard OpenStack API. OpenStack api is restful API with backward compatibility and multi-version API in parallel.
Each Cascaded OpenStack and Cascading OpenStack can be managed independently as standalone OpenStack. Therefore the upgrade or operation and maintenance can be done in availability zone granularity.

Cascading07.png
Relied on the standard OpenStack API managed by Cascading OpenStack, a new vendor’s physical resources with OpenStack built-in could be integrated into the cloud via plug & play model, just like USB device plugged into PC. This benefit makes OpenStack API as the soft defined “PCI” bus in Cloud era.

Cascading08.png
Scalability not only in one Cascaded OpenStack, but also for multi-vendors’s Cascaded OpenStack spread into many datacenters. Because OpenStack API is restful API, one Cascading OpenStack to manage multiple data centers across WAN or LAN is feasible.

Value to end user

  • Isolated virtual network across different physical data centers.

OpenStack cascading can provide virtual network across different physical data centers to the end user through the standard OpenStack api.

  • Virtual machine / volume migration from one data center to another:

OpenStack cascading can move virtual from one data center to another one, currently cold migration supported.

  • High availability application across different physical data center.

With the aid of virtual network across data centers and image synchronization function, application backup/disaster recovery/load balance is easy to implement in the distributed cloud.

Deployment Scenario

  • Deployment Scenario 1:

The cloud admin wants to provide one cloud to tenants with one OpenStack api. The cloud is very large scale, and the cloud will be one standalone region in one big data center. The cloud will grow with capacity expansion gradually, to avoid vendor-lock in, multiple vendors to build the cloud is also required.

  • Deployment Scenario 2:

The cloud admin wants to provide one cloud to tenants with one OpenStack api. The cloud is very large scale, and the cloud will include resources distributed in multiple data centers. The cloud will grow with capacity expansion gradually, to avoid vendor-lock in, multiple vendors to build the cloud is also required.


Cascading10.png

Blueprints

To start an incubation project or register blueprint under different project with one umbrella blueprint, it's a tough decision. None of current contribution methodology can work for OpenStack cascading: many projects like Nova/Cinder/Neutron/Glance/Ceilometer are involved, they work together to make cascading work. After the OpenStack cascading is introduced, the CI is quite different from the current single OpenStack instance CI environment, the CI enrionment for OpenStack cascading needs at least one cascading OpenStack and two cascaded OpenStack.

Your comments are welcome.

The blueprints are:

[tbd]

Proof of Concept

The PoC team welcome discussion on any topic of OpenStack cascading solution. Please send mail to all PoC team member for any topic.

Member Chinese Name E-Mail Role
Chaoyi Huang 黄朝意 joehuang@huawei.com inventor, designer of Nova/Cinder/Glance cascading, code reviewer
Hongning Wu 吴红宁 wuhongning@huawei.com inventor, designer of Neutron cascading, code reviewer
Fan Qin 覃帆 qinfan@huawei.com developer of Nova cascading
Chi Zhang 张驰 z.zhangchi@huawei.com developer of Cinder cascading
Haojie Jia 贾豪杰 jiahaojie@huawei.com developer of Neutron cascading
Dong Jia 贾栋 jiadong.jia@huawei.com developer of Glance cascading

Hands on lab

coming soon, hopefully in August, 2014
delayed to Sept. 2014
[tbd]

Source Code Repository

coming soon, the source code will be available from Github, hopefully in August, 2014
delayed to Sept. 2014
[tbd]

CI Environment

Ask for help. Warmly welcome CI guru to help build the CI environment for OpenStack cascading solution.

How to Play

Your comments are welcome. [tbd]

Mail-list

The OpenStack cascading solution PoC team has no special mail-list currently. You can also use OpenStack dev list to reach us, please include [openstack-dev] [cascading] in the mail title.

Meetings

[tbd]

Technology Hub

Last update

Aug 29, 2014