Jump to: navigation, search

Difference between revisions of "Blazar"

(Climate)
(Climate)
Line 3: Line 3:
 
Welcome to Climate, OpenStack reservation service.
 
Welcome to Climate, OpenStack reservation service.
 
At the moment all docs are placed here, they’ll be moved to readthedocs.org soon.
 
At the moment all docs are placed here, they’ll be moved to readthedocs.org soon.
 +
  
 
==== Release Notes ====
 
==== Release Notes ====
  
 
[https://wiki.openstack.org/wiki/Climate/Release_Notes Release Notes]
 
[https://wiki.openstack.org/wiki/Climate/Release_Notes Release Notes]
 +
  
 
==== Code & Bugs & Blueprints ====
 
==== Code & Bugs & Blueprints ====
Line 21: Line 23:
 
* [https://review.openstack.org/#/q/status:open+project:stackforge/python-climateclient,n,z Climate client changes]
 
* [https://review.openstack.org/#/q/status:open+project:stackforge/python-climateclient,n,z Climate client changes]
 
* [https://review.openstack.org/#/q/status:open+project:stackforge/climate-nova,n,z Climate Nova related filters and extensions]
 
* [https://review.openstack.org/#/q/status:open+project:stackforge/climate-nova,n,z Climate Nova related filters and extensions]
 
  
  

Revision as of 10:50, 5 February 2014

Climate

Welcome to Climate, OpenStack reservation service. At the moment all docs are placed here, they’ll be moved to readthedocs.org soon.


Release Notes

Release Notes


Code & Bugs & Blueprints


Open changes


Old Wiki resources

Installation

See Climate installation without Devstack

Introduction

Idea of creating Climate originated with two different use cases:

  • Compute host reservation (when user with admin privileges can reserve hardware resources that are dedicated to the sole use of a tenant)
  • Virtual machine (instance) reservation (when user may ask reservation service to provide him working VM not necessary now, but also in the future)


Now these ideas have been transformed to more general view: with Climate user can request the resources of cloud environment to be provided (“leased”) to his project for specific amount on time, immediately or in future.

Both virtual (Instances, Volumes, Networks) and hardware (full hosts with specific characteristics of RAM, CPU and etc) resources can be allocated via “lease”.

In terms of benefits added, Resource Reservation Service will:

  • improve visibility of cloud resources consumption (current and planned for future);
  • enable cloud resource planning based on current and future demand from end users;
  • automate the processes of resource allocation and reclaiming;
  • provide energy efficiency for physical hosts (both compute and storage ones);
  • potentially provide leases as billable items for which customers can be charged a flat fee or a premium price depending on amount of reserved cloud resources and their usage

Glossary of terms

Reservation is an allocation of certain cloud resource (Nova instance, Cinder volume, compute host, etc.) to particular project. Speaking about virtual reservations, we may have not only simple, solid ones (like already mentioned instances and volumes), but also complex ones - like Heat stacks and Savanna clusters. Reservation is characterized by status, resource type and identifier and lease it belongs to.

Lease is a negotiation agreement between the provider (Climate, using OpenStack resources) and the consumer (user) where the former agrees to make some kind of resources (both virtual and physical) available to latter, based on a set of lease terms presented by the consumer. Here lease may be described as contract between user and reservation service about cloud resources to be provided right now or later. Technically speaking, lease is a group of reservations granted to particular project upon request. Lease is characterized by start time, end time, set of individual reservations and associated events.

Event is simply something that may happen to lease. In most simple case event might describe lease start and lease end. Also it might be notification to user (e.g. about soon lease expiration) and some extra actions.

Rationale

Climate is created to:

  • manage cloud resources not only right now, but also in the future;
  • have dedicated recourses on certain amount of time;
  • prepare for the peak loads and perform capacity planning;
  • optimise energy consumption.

Service Architecture Design

Climate design can be described by following diagram:


Climate architecture.png


climate-client - provides the opportunity to communicate with Climate via REST API (climate-api service).

climate-api - waits for the REST calls from the outside world to redirect them to the manager. climate-api communicates with climate-manager via RPC. Runs as a separated process.

climate-manager - implements all logic and operations with leases, reservations and events. Communicates with Climate DB and stores there data structure of connected leases, reservations (both physical and virtual) and events. climate-manager service is responsible for running events created for lease and process all actions that should be done this moment. Manager uses resource-plugins to work with concrete resources (instances, volumes, compute hosts). climate-manager uses Keystone trusts to commit actions on behalf of user who has created lease before.

resource-plugin - responsible for exact actions to do with reserved resources (VMs, volumes, etc.) When working knows only about resource ID and token to use. All resource plugins work in the same process as climate-manager.

Virtual instance reservation

Virtual instance reservation mostly looks like usual instance booting for user - he/she only passes special hints to Nova containing information about future lease - lease start and end dates, its name, etc. Special Nova API extensions parse these parameter and use them to call Climate, passing to it ID of just created instance. If there is a need to reserve all instances in cloud (like in developer labs to automate process of resource reclaiming), default reservation extension might be used. By default it starts lease at the moment of request and gives it one month of lifetime.

During the time lease has not started yet, instance will be in shelved.

Compute host reservation

Now process of compute hosts reserving contains two steps:

  • admin marks hosts from common pool as possible to be reserved. That is implemented by moving these hosts to special aggregate.
  • user asks for reserving of host with specified characteristics like:
    • the region
    • the availability zone
    • the host capabilities extra specs (scoped and non-scoped format should be accepted)
    • the number of CPU cores
    • the amount of free RAM
    • the amount of free disk space
    • the number of hosts


Technically speaking, resource ID here will be not host ID, because there might be many of them wanted. Resource here will be new aggregate containing reserved hosts. The time lease starts, user may use reserved compute capacity to run his/her instances on it passing special scheduler hint to Nova. When host is reserved, it’s not used for usual instance running, it might be used only when lease starts and only by passing reservation ID to Nova.

Lease types (concepts)

  • Immediate reservation. Resources are provisioned immediately (like VM boot or moving host to reserved user aggregate) or not at all. If request can be fulfilled, lease is created and success status is returned. Lease should be marked as active or to_be_started. Otherwise (if request resource cannot be provisioned right now) failure status for this request should be returned.
  • Reservation with retries. Mostly looks like previous variant, but in case of failure, user may want to have several (configurable number) retries to process lease creation action. In this case request will be processed till that will be possible to create lease but not more than set in configuration file number of times.
  • Best-effort reservation. Also might have place if lease creation request cannot be fulfilled immediately. Best-effort mechanism starts something like scavenger hunt trying to find resources for reservations. For compute hosts reservation that makes much sense, because in case there are instances belonging to other tenant on eligible hosts, and without them there will be possible to reserve these hosts, Climate may start instances migration. This operation can be timely and fairly complex and so different strategies may be applied depending on heuristic factors such as the number, type and state of the instances to be migrated. Also Climate should assert that there are at least enough potential candidates for the migration prior to starting the actual migration. If Climate decides to start migration, it returns success state and marks lease as 'in_progress, otherwise - failure. If this 'hunting' ends successfully before configurable timeout has passed, lease should be marked as active, otherwise its status is set to timedout.
  • Delayed resource acquiring or scheduled reservation. In this reservation type lease is created successfully if Climate thinks there will be enough resources to process provisioning later (otherwise this request returns failure status). Lease is marked as inactive till all resources will be actually provisioned. That works pretty nice and predictable speaking about compute hosts reservation (because hosts as resources are got not from common cloud pool, but from admin defined pool). So Climate is possible to predict these physical resources usage and use that information during lease creation. If we speak about virtual reservations, here situation is more complicated, because all resources are got from common cloud resources pool, and Climate cannot guarantee there will be enough resources to provision them. In this failure case lease state will be marked as error with appropriate explanation.

REST API

See Climate REST API.