Sahara/Scaling

Overview

Now we are considering 2 variants for scaling:

1. Existing node groups scaling. And here we have an ability to remove instances in existing node groups.
2. Addition of new node groups to cluster. It will allow user to add new node groups with configurable params as it happens during cluster creation.

Limitations:

We are considering only datanode and tasktracker adding/removing within node groups. So, we will work with 3 types of node groups with following processes' lists:

1. [datanode]
2. [tasktracker]
3. [datanode, tasktracker]

First Approach

REST API

POST /v1.0/{tenant_id}/clusters/{cluster_id}/scale Request format:

   {  
    "nodes": [
           {
               "name": "node_group_name",
               "amount": "+1" {amount of instance to be added or deleted (-1)}
           }
      ]
   }

SPI

It is needed to add 2 methods to plugin:

   validate_scaling(node_groups):

Should validate is it possible to remove or add instances

   scale_existing_node_groups(cluster, instances)

Prepares instances to run specified processes.

Instances, as such, will be up and down by savanna_core using existing methods.