Jump to: navigation, search

Difference between revisions of "CaaS"

(Related Project)
 
(6 intermediate revisions by 3 users not shown)
Line 1: Line 1:
__NOTOC__
 
 
* '''Created''': yoko
 
* '''Created''': yoko
 
* '''Contributors''': nii
 
* '''Contributors''': nii
 +
* '''Blueprint''': https://blueprints.launchpad.net/nova/+spec/cluster-as-a-service
  
 
= Cluster as a Service: Managing multiple clusters for openstack clouds and other diverse frameworks =
 
= Cluster as a Service: Managing multiple clusters for openstack clouds and other diverse frameworks =
 
<<[[TableOfContents]]()>>
 
  
 
== Overview ==
 
== Overview ==
  
Cluster computing has become popular in academia and industry. Clusters of commodity servers are used for variety of distributed applications like simulation, data analysis, web services and so on. None of frameworks can fit to every distributed application. Even openstack cloud framework cannot, because, some researchers want to deploy hadoop clusters on bear metal servers for performance issues, for example.
+
Cluster computing has become popular in academia and industry. Clusters of commodity servers are used for a variety of distributed applications like simulation, data analysis, web services, and so on. No single framework can fit every distributed application. Even the openstack cloud framework cannot, because some researchers want to deploy hadoop clusters on bear metal servers, for example. One solution to this problem is to assign a layer below the IaaS/PaaS layer the job of handling cluster deployment. This idea is called the Cluster-as-a-Service layer for allocating servers.
One solution for this problem is to assign one layer below IaaS/PaaS layer to handle cluster deployment. That is Cluster as a Service layer which is used for allocating several or many servers for frameworks.  
 
  
 
== User stories ==
 
== User stories ==
  
In an academic institution, researchers had been using their own computing resources for each purpose. That was fine until the power shortage became real. The director of the institution asked a cloud research group to develop a research cloud on which every researchers can move their computer clusters.
+
In an academic institution, researchers had been using their own computing resources for their individual research projects. That was fine until power shortages became a problem. The director of the institution asked a cloud research group to develop a research cloud onto which every researcher could move their computer clusters. Some research groups use HPC clusters while others use Hadoop clusters. Quite a few groups provide web services to the public. The usage patterns of the clusters are different from each other and vary according to the time of the day, day of the week, etc.  
Some of the research group use HPC clusters and others use Hadoop clusters. Quite a few groups has providing web services to public. The usage pattern of each cluster is different for each other and varies according to the time of the day, the day of the week etc.  
+
Thus, the resource has to be dynamically allocated to each cluster when power consumption critically matters.    
 
 
So, the resource allocation to each cluster has to be dynamic if the power consumption critically matters.  
 
  
 
== Design ==
 
== Design ==
Line 24: Line 19:
 
** Dynamic resource allocation for each cluster
 
** Dynamic resource allocation for each cluster
 
** Secure isolation among clusters
 
** Secure isolation among clusters
** Enough performance and capacity for each cluster
+
** Sufficient performance and capacity for each cluster
 
* Design sketch
 
* Design sketch
 
** Separation between IaaS/PaaS layer and CaaS layer
 
** Separation between IaaS/PaaS layer and CaaS layer
** IaaS/PaaS layer can scale on the fly
+
** IaaS/PaaS layer can be scaled on the fly
** CaaS layer machine images can be handled efficiently by container
+
** CaaS layer machine images can be efficiently handled by using containers
 
** CaaS layer is a web service  
 
** CaaS layer is a web service  
** Network isolation by taggedVLAN
+
** Network isolation by using tagged VLAN
 
** Automatic package installation
 
** Automatic package installation
  
 
== Implementation ==
 
== Implementation ==
  
Implementation example is the following:
+
The following is an implementation example:
  
[[Image:CaaS$implementation.gif]]
+
[[Image:implementation.gif]]
  
 
== Test/Demo Plan ==
 
== Test/Demo Plan ==
Line 50: Line 45:
 
== Unresolved issues ==
 
== Unresolved issues ==
  
This project is quite related to [[MultiClusterZones]] but have not contacted  
+
This project is related to the [[MultiClusterZones]] project but we have not contacted members of that project yet.  
to the project.
 
  
 
http://wiki.openstack.org/MultiClusterZones
 
http://wiki.openstack.org/MultiClusterZones
 +
 +
== Related Project ==
 +
 +
An Openstack project Senlin has been launched to support cluster as a service.  Its mission is to support the management of groups of homogeneous objects exposed by other OpenStack services, like Nova server, or Heat stack.
 +
 +
https://wiki.openstack.org/wiki/Senlin
 +
 +
http://git.openstack.org/cgit/openstack/senlin
  
 
----
 
----
 
[[Category:Spec]]
 
[[Category:Spec]]

Latest revision as of 07:53, 18 November 2015

Cluster as a Service: Managing multiple clusters for openstack clouds and other diverse frameworks

Overview

Cluster computing has become popular in academia and industry. Clusters of commodity servers are used for a variety of distributed applications like simulation, data analysis, web services, and so on. No single framework can fit every distributed application. Even the openstack cloud framework cannot, because some researchers want to deploy hadoop clusters on bear metal servers, for example. One solution to this problem is to assign a layer below the IaaS/PaaS layer the job of handling cluster deployment. This idea is called the Cluster-as-a-Service layer for allocating servers.

User stories

In an academic institution, researchers had been using their own computing resources for their individual research projects. That was fine until power shortages became a problem. The director of the institution asked a cloud research group to develop a research cloud onto which every researcher could move their computer clusters. Some research groups use HPC clusters while others use Hadoop clusters. Quite a few groups provide web services to the public. The usage patterns of the clusters are different from each other and vary according to the time of the day, day of the week, etc. Thus, the resource has to be dynamically allocated to each cluster when power consumption critically matters.

Design

  • Requirements
    • Dynamic resource allocation for each cluster
    • Secure isolation among clusters
    • Sufficient performance and capacity for each cluster
  • Design sketch
    • Separation between IaaS/PaaS layer and CaaS layer
    • IaaS/PaaS layer can be scaled on the fly
    • CaaS layer machine images can be efficiently handled by using containers
    • CaaS layer is a web service
    • Network isolation by using tagged VLAN
    • Automatic package installation

Implementation

The following is an implementation example:

Implementation.gif

Test/Demo Plan

The current demonstration video can be seen at http://wiki.openstack.org/CaaS_demo

Discussion

http://etherpad.openstack.org/cluster-as-a-service-discussion

Unresolved issues

This project is related to the MultiClusterZones project but we have not contacted members of that project yet.

http://wiki.openstack.org/MultiClusterZones

Related Project

An Openstack project Senlin has been launched to support cluster as a service. Its mission is to support the management of groups of homogeneous objects exposed by other OpenStack services, like Nova server, or Heat stack.

https://wiki.openstack.org/wiki/Senlin

http://git.openstack.org/cgit/openstack/senlin