Jump to: navigation, search

CaaS

Cluster as a Service: Managing multiple clusters for openstack clouds and other diverse frameworks

Overview

Cluster computing has become popular in academia and industry. Clusters of commodity servers are used for a variety of distributed applications like simulation, data analysis, web services, and so on. No single framework can fit every distributed application. Even the openstack cloud framework cannot, because some researchers want to deploy hadoop clusters on bear metal servers, for example. One solution to this problem is to assign a layer below the IaaS/PaaS layer the job of handling cluster deployment. This idea is called the Cluster-as-a-Service layer for allocating servers.

User stories

In an academic institution, researchers had been using their own computing resources for their individual research projects. That was fine until power shortages became a problem. The director of the institution asked a cloud research group to develop a research cloud onto which every researcher could move their computer clusters. Some research groups use HPC clusters while others use Hadoop clusters. Quite a few groups provide web services to the public. The usage patterns of the clusters are different from each other and vary according to the time of the day, day of the week, etc. Thus, the resource has to be dynamically allocated to each cluster when power consumption critically matters.

Design

  • Requirements
    • Dynamic resource allocation for each cluster
    • Secure isolation among clusters
    • Sufficient performance and capacity for each cluster
  • Design sketch
    • Separation between IaaS/PaaS layer and CaaS layer
    • IaaS/PaaS layer can be scaled on the fly
    • CaaS layer machine images can be efficiently handled by using containers
    • CaaS layer is a web service
    • Network isolation by using tagged VLAN
    • Automatic package installation

Implementation

The following is an implementation example:

Implementation.gif

Test/Demo Plan

The current demonstration video can be seen at http://wiki.openstack.org/CaaS_demo

Discussion

http://etherpad.openstack.org/cluster-as-a-service-discussion

Unresolved issues

This project is related to the MultiClusterZones project but we have not contacted members of that project yet.

http://wiki.openstack.org/MultiClusterZones