Jump to: navigation, search


Cluster as a Service: Managing multiple clusters for openstack clouds and other diverse frameworks


Cluster computing has become popular in academia and industry. Clusters of commodity servers are used for a variety of distributed applications like simulation, data analysis, web services, and so on. No single framework can fit every distributed application. Even the openstack cloud framework cannot, because some researchers want to deploy hadoop clusters on bear metal servers, for example. One solution to this problem is to assign a layer below the IaaS/PaaS layer the job of handling cluster deployment. This idea is called the Cluster-as-a-Service layer for allocating servers.

User stories

In an academic institution, researchers had been using their own computing resources for their individual research projects. That was fine until power shortages became a problem. The director of the institution asked a cloud research group to develop a research cloud onto which every researcher could move their computer clusters. Some research groups use HPC clusters while others use Hadoop clusters. Quite a few groups provide web services to the public. The usage patterns of the clusters are different from each other and vary according to the time of the day, day of the week, etc. Thus, the resource has to be dynamically allocated to each cluster when power consumption critically matters.


  • Requirements
    • Dynamic resource allocation for each cluster
    • Secure isolation among clusters
    • Sufficient performance and capacity for each cluster
  • Design sketch
    • Separation between IaaS/PaaS layer and CaaS layer
    • IaaS/PaaS layer can be scaled on the fly
    • CaaS layer machine images can be efficiently handled by using containers
    • CaaS layer is a web service
    • Network isolation by using tagged VLAN
    • Automatic package installation


The following is an implementation example:


Test/Demo Plan

The current demonstration video can be seen at http://wiki.openstack.org/CaaS_demo



Unresolved issues

This project is related to the MultiClusterZones project but we have not contacted members of that project yet.


Related Project

An Openstack project Senlin has been launched to support cluster as a service. Its mission is to support the management of groups of homogeneous objects exposed by other OpenStack services, like Nova server, or Heat stack.