Jump to: navigation, search

Difference between revisions of "ExtensibleResourceTracking"

m (Resource requirements in instances)
m (Resource requirements in instances)
Line 102: Line 102:
  
 
The standard resource requirements are already present and will remain unchanged for backward compatibility. Extended resource requirements will be supported by adding an additional attribute to the instance object called '''resources'''. This will contain a dictionary of resource requirements.
 
The standard resource requirements are already present and will remain unchanged for backward compatibility. Extended resource requirements will be supported by adding an additional attribute to the instance object called '''resources'''. This will contain a dictionary of resource requirements.
 +
 +
 +
== Changes for the database ==
 +
 +
The database is currently used to record the state of compute nodes and instances. This also provides a path for information to pass from the compute nodes to the scheduler. In the future this is likely to change.
 +
 +
There will be a database schema change to support extensible resource requirements for the compute_nodes table and the instances table. It is becoming common practice in Nova to represent extensible data structures as JSON strings rather than additional attribute value tables for reason of performance. The same will be done here.
 +
 +
=== Extended resources at compute_nodes ===
 +
 +
The compute_nodes table already contains columns for the standard resource types. An additional column called '''resources''' will be added to hold a JSON string serialising the capacity information for extended attributes at the node.
 +
 +
:"{my_resource:5, another_resource:10}"
 +
 +
=== Extended resource requirements in instances ===
 +
 +
The instances table already contains columns for the standard resource requirements. An additional column called '''resources''' will be added to hold a JSON string serialising the resource requirements for extended resources:
 +
 +
:"{my_resource:{total:100, used:5}, another_resource:{total:2000, used:100}}"

Revision as of 05:10, 29 November 2013

Summary - (in progress)

There are a number of defined resources in Nova, including disk, memory and number of vCPU. The purpose of this specification is to support an extensible set of resources so that new resources can be defined for compute nodes and resource requirements for instances. These will be made available to the scheduler and allocated by the compute manager.

The resources of a compute node have limited capacity. Most, including disk, memory and vCPU, map to a physical counterpart, typically via a virtualisation technology, but it is also possible to create entirely abstract resource types that do not. As an example, a provider may chose that only one instances of a certain type should run on any compute node. This trivial example can be supported by defining a new resource for compute nodes and associating a requirement for that resource with the instance type.

Each instance has resource requirements specified as the quantity of each type of resource it needs. These are used by the scheduler to match it to compute nodes with sufficient available resources.

The compute manager on a host is aware of the resources available at the compute node and is responsible for their allocation and management. Because it has definitive knowledge of what resources are actually in use, the compute manager has the final say as to whether a compute node is able to host an instance.

We will need:

  • a model of a resource
  • a model of a resource requirement
  • a naming scheme to allow resource requirements to be related to resources
  • changes for compute nodes
    • resource configuration - to define extended resources
    • resource_tracker - to track extended resources
    • claims - to test requirements against available extended resources
  • changes for instances
    • resource requirements in flavors
    • resource requirements in requests
    • resource requirements in instances
  • changes for the database
    • extended resources at compute_nodes
    • extended resource requirements in instances
  • changes for scheduler (filter scheduler)
    • host manager - to access extended resources
    • filters - examples using extended resources
    • weighers - examples using extended resources

Resource Models and Naming

Resource Model

A resource has the following attributes:

  • name - used to identify this resource
  • type - a resource type name for human viewing
  • units - a resource unit name for human viewing
  • description - a short description for human viewing


Each resource at a compute node is represented by its total amount and the amount used - this is recorded by the resource tracker and used in scheduling and allocation decisions. This information is held at the resource tracker and passed via the database to the host_manager at the scheduler.

Resource capacity:

  • resource total = value (configured or discovered - total amount of resource)
  • resource used = value (tracked - amount of resource currently used)


Each resource required for a particular instance is represented by a single value. This information is defined for a flavor, passed in create requests to be used at the scheduler and the resource_tracker/claims at a compute manager, and is inherited by an instance.

Resource requirement:

  • resource required = value (configured - amount required)


A final quantity associated with resources is the limit, the total amount that can be committed. This can be different to the total capacity as it reflects the policy of over or under committing the resource. The limit is calculated per compute node at the scheduler according to a function of other attributes and policy implemented in the filters. The limit is communicated with a create request and used by the resource_tracker claims.

Resource limit:

  • resource limit = value (calculated - the total amount of resource that can be allocated)

Naming

The names of resources should be unique in a given OpenStack deployment. In all cases a resource is known by its name and this is used to perform comparisons between information from different sources at the scheduler and the resource tracker.

In some cases (such as flavors, where extra specs will be used to hold the information) a prefix is required. In this case the name resource will be used as a prefix for individual resources.

Changes for compute nodes

Resource configuration

The resource tracker at a compute manager needs to know the resources and their quantities. The way that the quantity is obtained may vary by resource type, some being configured, others being discovered programmatically. So we will use a plug-in framework for resources.

A resource class will have methods that return the properties of a resource and a method that implements the claims test against resources. A base class will be implemented that new resource types can extend. These classes will be loaded at run time as they are discovered in the configuration file. This follows the model of filter/weight classes at the filter scheduler.

The standard memory, disk and vCPU resources can be implemented given an implementation in this form.

Resource tracker

The resource tracker will load the resource classes. These will be used to obtain the local capacity.

Claims

The claims class will be modified to use test methods provided by the resource classes.

Changes for instances

Resource requirements in flavors

Flavors already have attributes defined for standard resource types. These will continue to be used for backward compatibility. The extra specs extension will be used to incorporate extended resource requirements.

Flavors have a field called extra_specs that can contain arbitrary attributes. The prefix resource will be used to identify attributes that are resource requirements:

"extra_specs" : { "resource:my_resource" : 5, "resource:another_resource" : 10}

Resource requirements in requests

Resource requirements are communicated in requests. The standard resource requirements are already present and will remain unchanged for backward compatibility. Extended resource requirements will be by supported by an additional attribute in filter properties called resources holding a list of resource requirements:

"resources" : {"my_resource": 5, "another_resource": 10}

Filter properties is populated by the filter scheduler code.

Resource requirements in instances

Instances are represented by the instance object. Resource requirements (or alternately, resources allocated, where an instance is created) are held in the instance object as attributes.

The standard resource requirements are already present and will remain unchanged for backward compatibility. Extended resource requirements will be supported by adding an additional attribute to the instance object called resources. This will contain a dictionary of resource requirements.


Changes for the database

The database is currently used to record the state of compute nodes and instances. This also provides a path for information to pass from the compute nodes to the scheduler. In the future this is likely to change.

There will be a database schema change to support extensible resource requirements for the compute_nodes table and the instances table. It is becoming common practice in Nova to represent extensible data structures as JSON strings rather than additional attribute value tables for reason of performance. The same will be done here.

Extended resources at compute_nodes

The compute_nodes table already contains columns for the standard resource types. An additional column called resources will be added to hold a JSON string serialising the capacity information for extended attributes at the node.

"{my_resource:5, another_resource:10}"

Extended resource requirements in instances

The instances table already contains columns for the standard resource requirements. An additional column called resources will be added to hold a JSON string serialising the resource requirements for extended resources:

"{my_resource:{total:100, used:5}, another_resource:{total:2000, used:100}}"