• ConfigureInstanceTypesDynamically

• Launchpad Entry: configure-instance-types-dynamically

• Created: Ken Pepple

• Contributors:

Summary

Nova administrators should be able to create their own INSTANCE_TYPES (standard cpu, memory, storage configs such as m1.medium in the EC2 API) without resorting to hacking the source code.

Throughout this spec, instance_types will be used, although we could also use the Rackspace API "flavor" term.

An etherpad for discussion of this blueprint is available at http://etherpad.openstack.org/dynamicflavors

Release Note

nova-manage has been extended with an addition command, instance_types, that allows admins to list, create and delete instance_types in nova.

Rationale

The current implementation hard codes the instance_types in the source code. This is sub-optimal as it assumes that nova admins know both python as well as the nova code base. By exposing this through the nova-manage command, we gain both a familiar CLI for admin, as well as the ability to extend the openstack-dashboard to support modifying instance_types through the web GUI.

Risk

Although the change is simple (replacing the constant with a database query and adding three db queries to maintain it), this change is very low in the stack and in the code path to launch instances. Problems with this change (if done incorrectly) could prevent users from launching instances. Having said all that, leaving these constants in the code will make it increasingly difficult to remove them. At current count, this constant is used over 25 times in trunk (revno 650).

User stories

Jim needs a high memory, low processing (16GB RAM, 2 vCPU) instance_type for his memcache nodes. None of the hard coded values fit his needs. Jim would like to create a instance_type to offer to his users.

Jim also prohibits the use of high vCPU instance_types as he has very low grade machines (1 physical CPU). Launching high vCPU or high memory KVM instances simply fail, infuriating his users and making Jim grumpy. He would like to delete the m1.xlarge instance_type to prevent this.

Assumptions

There is an implicit assumption that the EC2 API and Rackspace APIs are not tightly coupled to particular instance_types. It is also assumed that once a instance_type is referenced for starting an instance that that instance_type definition is not needed again.

Design

Instead of hard coding the instance_types in the instance_types.py file, create a table in the database for them. Add instance_types command to nova-manage to do simple CRD actions (no update).

The update action has been omitted to preserve a history for billing/accounting.

The delete action will not actually delete the instance_type but simply mark it inactive in the database. Again, this is to preserve the record for post billing/accounting runs.

Ideally, there should be an option in nova-manage which would allow INSTANCE_TYPES to added or deleted in the nova DB. A typical use of the command might look something like:

    # nova-manage instance_type list
    m1.medium: Memory: 4096MB, VCPUS: 2, Storage: 40GB, FlavorID: 3, Swap: 0GB, RXTX Quota: 0GB, RXTX Cap: 0MB
    m1.large: Memory: 8192MB, VCPUS: 4, Storage: 80GB, FlavorID: 4, Swap: 0GB, RXTX Quota: 0GB, RXTX Cap: 0MB
    m1.tiny: Memory: 512MB, VCPUS: 1, Storage: 0GB, FlavorID: 1, Swap: 0GB, RXTX Quota: 0GB, RXTX Cap: 0MB 
    m1.xlarge: Memory: 16384MB, VCPUS: 8, Storage: 160GB, FlavorID: 5, Swap: 0GB, RXTX Quota: 0GB, RXTX Cap: 0MB
    m1.small: Memory: 2048MB, VCPUS: 1, Storage: 20GB, FlavorID: 2, Swap: 0GB, RXTX Quota: 0GB, RXTX Cap: 0MB

    # nova-manage instance_type create m1.xxlarge 32768 16 320 0 0 0
    m1.xxlarge created

    # nova-manage instance_type list m1.xxlarge
    m1.xxlarge: Memory: 32768MB, VCPUS: 16, Storage: 320GB, FlavorID: 5, Swap: 0GB, RXTX Quota: 0GB, RXTX Cap: 0MB

    # nova-manage instance_type delete m1.xxlarge
    m1.xxlarge deleted

Implementation

This would probably entail modifying:

• nova/compute/instance_types.py: adding create_instancetype and delete_instancetype, then modifying get_by_type and get_by_flavor
  • create_instancetype will create a row in the instance_type table and set it's "active" flag to true
  • delete_instancetype will set a row's "active" flag to false
  • list_instancetype will list all "active" rows
• Nova DB schema: adding instance_types table and creating a migration
• bin/nova-manage: adding instance_types class with create, delete and list methods
• db/api.py and db/sqlalchemy/api.py: adding appropriate database queries
• tests for both instance_types and nova-manage

Migration

A new table will be added to the database (instance_types or flavors) and seeded with the hard coded data from instance_types.py (INSTANCE_TYPES).

This table will need to take into account not only the immediate needs of this spec, but also look at what may be needed for billing (if "premium" instance_types metadata may need to be stored), security (some instance_types may not be able to used by all users) and scheduling (some instance_types might not be suitable for some nova-compute nodes). This spec does not intend to take implement any of these needs, but does not want to preclude any future improvements with it's design.

Test/Demo Plan

New tests for adding this feature should cover:

• instance_type migration should create instance_types table in DB
• default_instance_type flag should be valid
• nova-manage should create a new instance_type with correct arguments
• nova-manage should list specific instance_type with correct arguments
• nova-manage should list all instance_types
• nova-manage delete should set delete=1 for instance_types specified
• nova-manage create should fail when give incomplete arguments
• nova-manage create should fail when give invalid arguments
• nova-manage delete should fail gracefully when given non-existent instance_type name
• nova-manage create should fail when duplicate name or flavorid are specified

Unresolved issues

1. [SOLVED: implemented with --all] Should there be way in nova-manage to list ALL (even inactive instance_types) ? Perhaps with:

     $ nova-manage instance_type list --all
     'm1.tiny', memory_mb=512, vcpus=1, local_gb=0, flavorid=1
     'm1.small', memory_mb=2048, vcpus=1, local_gb=20, flavorid=2
     'm1.medium', memory_mb=4096, vcpus=2, local_gb=40, flavorid=3
     'm1.large', memory_mb=8192, vcpus=4, local_gb=80, flavorid=4
     'm1.xlarge', memory_mb=16384, vcpus=8, local_gb=160, flavorid=5
     'm1.xxlarge', memory_mb=32768, vcpus=16, local_gb=320, flavorid=6, inactive

1. [SOLVED: added --purge option] Should there be a way for nova-manage to remove an instance_type from the database (i.e. purge) ? Perhaps with a warning ...

    $ nova-manage instance_type purge m1.xxlarge
    *THIS WILL REMOVE THIS INSTANCE TYPE FROM THE DATABASE FOREVER*
    Are you sure you want to do this [y/N]? 
    'm1.xxlarge' delete.

1. [SOLVED: aliased flavors to instance_types] I have used instance_type as the nomenclature for this feature. Should we be using flavors ? We could easily alias flavors for instance_types in nova-manage so that we could use either.

BoF agenda and discussion

http://etherpad.openstack.org/dynamicflavors

CategorySpec