Jump to: navigation, search

Heat/PolicyExtension

< Heat
Revision as of 07:07, 27 September 2013 by Mike Spreitzer (talk | contribs) (How Policies Combine)

Policy Extension to Heat Templates

This page describes a proposed extension to Heat templates allowing expression of various sorts of policies relevant to infrastructure. First is a description of the information model, which focuses on what is being said without getting into details of concrete syntax. Next are some use cases. Next is a proposal for concrete syntax. Lastly there are some examples using the concrete syntax.

A Word About Networking

You will see relatively little concern here for network resources. The focus here is on declaring what is needed about and between endpoints, with the expectation that the template processor will invent network resources as appropriate to get those jobs done. However, external (to the template) network resources can be explicitly mentioned.

Information Model

Generalities

This extension introduces three concepts in templates: groups, policies, and relationships.

This extension also introduces a meta-model for the datacenters in which templates will be instantiated, to support the semantics of some particular types of policies as well as to support smart placement in general.

Datacenter Meta-Model

A datacenter is modeled as a tree whose leaves are the physical resources that can host the virtual resources prescribed by templates. The tree need not be of uniform depth. Each vertex in the tree is labeled with a level, drawn from a vocabulary established by the datacenter or some more central authority. Open question: who defines these vocabularies? The levels stand in a partial order; we say some levels are more fine grained or more coarse grained than other levels. The usage of levels in the tree must be consistent with this partial order. Thus, the root might be labeled with level=room while a leaf might be labeled with level=PM. The ordering is only partial because some levels might be incommensurate. For example, level=PM can host VMs while level=StorageBox can host storage volumes.

To support placement with an eye toward avoiding overloads, each physical resource that can host virtual resources is characterized by a capacity vector. Each virtual resource that can be hosted on such a physical resource has a demand vector of the same length as the host's capacity vector; this demand vector is essentially the allocation of capacity that is planned when the virtual resource is placed. The non-overload constraint is that the sum, of all the virtual resources placed in a host, of their demand vectors cannot exceed the host's capacity vector; this test must pass at each position in the vectors. The particulars of the demand and capacity vectors used in the core placement algorithm can be synthetic things invented to encode various surface aspects of the placement problem. The exposed and implicit components of these vectors are defined by the datacenter or a more central authority. Open question: who?

To support network overload avoidance, the datacenter metamodel includes a network whose links correspond to the edges in the physical resource tree. This is admittedly an approximation. It is hoped that this is close enough to serve our needs. Each network link has a bandwidth capacity.

To support license types that need it, the model of a physical machine identifies the type/configuration of the machine in enough detail to allow correct calculation of license charges for software on that machine.

Groups, Policies, and Relationships

Groups are used to group the resources of a template together for concise application of policies and relationships. The resources of the template are partitioned among some groups, and those groups are members of other groups, and so on. In a given template, the groups are arranged into a tree and the leaf groups contain the resources. The tree need not be of uniform depth.

There are different types of policies, each type with a specific meaning and possibly taking type-specific parameters.

Each policy is either hard or soft --- that is, a requirement or a preference.

A policy type is either primitive or synthetic. A synthetic policy is a shorthand for a type-specific assembly of other policies.

A primitive policy is something that, in its essence, is about one or two resources. A primitive policy type is either monadic, meaning its essential statement is about a single resource, or dyadic, meaning its essential statement is a relationship between two resources. For a monadic example, consider a type of policy that takes an identifier of a licensed product as a parameter and declares the presence of that product in the resource. For a dyadic example, an anti-collocation policy might take a physical hierarchy level parameter and be applied to two VMs to mean that their locations should differ at the given level. Some dyadic types of policies are symmetric, others are not. For example, an anti-collocation policy is symmetric; an endpoint accessibility policy (i.e., firewall rule) is asymmetric.

While a primitive policy's essence is about a resource or two, the actual applications of policies can be at higher levels of organization. Each higher level application of a primitive policy is a shorthand for many lower level application of the policy. There are two high level ways to apply a policy. One is to apply a policy to a group; you would not want to do this for an asymmetric primitive policy. The other, which makes sense only for dyadic policies, is to apply a policy to a pair of groups.

Open Question: who groups exterrnal resources? A dyadic policy in a template can relate one of the template's resources with a pre-existing resource outside the template. What about groups of external resources? Do we allow a template to define groups of external resources, or allow access to groups defined in existing stacks?

Applying a monadic primitive policy to a group is shorthand for applying the policy to every member of the group. Thus, after recursion, this amounts to applying the policy's essential statement to every resource covered by the group.

Applying a symmetric dyadic primitive policy to a group is a shorthand for applying the policy to every distinct unordered pair of members of the group. For example, applying a dyadic policy to a group of the resources {A, B, C} is shorthand for applying that policy to {A, B}, to {B, C}, and to {C, A}. For a more complicated example, consider a tree of three groups: GP has members GL and GR, while GL has members {R1, R2} and GR has members {R3, R4}. Applying a dyadic policy to group GP is shorthand for applying that policy to the unordered pair {GL, GR}.

Applying a symmetric dyadic primitive policy to an unordered pair of groups {A, B} is equivalent to applying that policy to the ordered pair (A, B) and also equivalent to applying it to (B, A).

Applying a dyadic primitive policy to an ordered pair (A, B), where A and B are either groups or resources, is shorthand for applying that policy to every ordered pair of (X, Y) where (1) X is either A (if A is a resource) or a member of A (if A is a group) and (2) Y is either B (if B is a resource) or a member of B (if B is a group). For example, if group GL's members are resources R1 and R2 while group GR's members are resources R3 and R4, applying a dyadic primitive policy to (GL, GR) amounts to applying the policy to (R1, R3), to (R1, R4), to (R2, R3), and to (R2, R4).

How Policies Combine

Policies combine in the natural way for their semantics. Orthogonal policies combine by conjunction. Policies that allow networking combine by disjunction. Allowing connectivity implicitly denies related connectivity that is not explicitly allowed. That is, when a policy allows some network connectivity from resource A in stack S to resource B in stack T, all the networking from S to T is denied except that which is explicitly allowed by some policy.

Catalog of Policy Types

Licensed Product Presence

A policy of this monadic primitive type takes two string parameters. The first identifies a catalog of licensed products, and the second is a product identifier drawn from that catalog. Note that a given catalog can contain products from many vendors. A vendor of management software typically has a catalog of products from many sources that can all be managed by the management software. For this reason and others, it is not expected that all the products in a catalog use the same type of license; the catalog identifies the applicable license type for each product.

This type of policy is relevant to placement decision-making because, for some types of license, the license charges for a collection of VMs can depend on the degree to which those VMs share physical resources.

Collocation

A policy of this symmetric primitive type takes one parameter, a level L in the physical hierarchy. When applied to resources {A, B} such a policy says that A's location and B's location should be the same at level L (and thus, implicitly, every coarser grain level); whether they agree at finer grain levels is not specified. For example, when L=chassis this means that A and B should be placed in the same chassis. Note that hard collocation constraints at fine grained levels make live migration problematic, users will probably not want to use them.

Anti-Collocation

A policy of this symmetric primitive type takes one parameter, a level L in the physical hierarchy. When applied to resources {A, B} such a policy says that A's location and B's location should differ at level L (and thus, implicitly, every relevant finer grain level); whether they differ at coarser grain levels is not specified. For example, when L=PM this means that A and B should be hosted by different PMs and it does not matter how close together those PMs are.

LLMN Anti-Collocation

This is the only synthetic policy type, and it is monadic. A policy of this type is applied to a group of M interchangeable members to make a concise statement of multiple levels of anti-collocation. There are three parameters: L1 and L2 are physical hierarchy levels, and N is a positive integer. The meaning of such a policy is that the M members should be spread among at least N different physical hierarchy vertices at level L1, with no more than ceiling(M/N) covered by any one such vertex, and no two covered by the same level L2 vertex. For example, when applied to a group of 7 VMs, with L1=rack, L2=PM, and N=2, this means that the 7 VMs should be spread out across at least 2 racks, with no more than 4 VMs on any one rack and no two VMs on the same PM. The implementation is allowed to introduce inhomogeneous treatment of the members, for example to partition them into particular subgroups for placement purposes.

Network Reachability

This symmetric dyadic primitive type of policy takes no parameters. When applied to two resources it means that they can communicate on all protocols and ports.

Network Endpoint Accessibility

This asymmetric dyadic primitive type of policy takes two parameters, a set P of protocols and a set E of (protocol, port) pairs. Applying such a policy to the resource pair (A, B) means that A can initiate communication over any protocol in P and also to any endpoint in E.

Network Bandwidth

This symmetric dyadic primitive type of policy takes one parameter, a number of bytes per second. Applying such a policy to resources A and B declares a network bandwidth need in each direction between A and B.

Network Hop-Count Limit

This symmetric dyadic primitive type of policy takes one parameter, a non-negative integer N. When applied to resource pair {A,B} it means that the length (in hops) of the path between A and B in the datacenter model is supposed to not exceed N.

Use Cases

Multiple HA Clusters of Appservers

Consider a template that specifies a collection of 3 clusters of 4 VMs each (for a total of 12 VMs), where each VM is running an application server and all the VMs in each cluster are interchangeable. We want each cluster to be highly available. We deem a cluster to be available if at least one member is.

Suppose the physical hierarchy is rooted at the room level, with division into rack and then into individual PM.

Suppose the template includes VM instance resources named M11, ... M14, M21, ... M24, M31, ... M34.

For each cluster C, create group G${C} containing the cluster's VM instances. Put all of those groups together into one more group that is the root; call it GRoot.

For each cluster C, apply the LLMN anti-collocation constraint to group G${C} with parameters L1=rack, L2=PM, and N=2; M=4 in these cases.

Hadoop with Direct Attached Storage

A net hop count limit of zero between a VM instance and its associated storage volume means that the volume should be hosted on directly attached storage.

Concrete Syntax

Concrete Examples