Nova/PlacementAdvisorAndEngine

The Problem

The recent blueprints of policy-based-scheduler and solver-scheduler highlight a possible weakness in the current design, as despite their completely independent contributions (i.e. which filters to apply per request vs. how to compute a valid placement) their implementation as drivers makes combining them non-trivial.

Separating the Scheduler's Concerns

Our approach to this issue is to think of the driver's work as split between two entities:

• A Placement Advisor, that constructs Placement Requests for scheduling requests (filter-scheduler and policy-based-scheduler)
• A Placement Engine, that answers Placement Requests (HostManager in get_filtered_hosts() and solver-scheduler with its LP engine).

Such modularity should allow developing independent mechanisms that can be combined seamlessly through a unified & well-defined protocol based on constructing Placement Request objects by the Placement Advisor and then passing them to the Placement Engine, which returns the solution. The protocol can be orchestrated by the scheduler manager.

As can be see at this point already, the policy-based-scheduler blueprint can now be positioned as an improvement of the placement advisor. Similarly, the solver-scheduler blueprint can be positioned as an improvement of the placement engine.

Placement Requests

Definition

A Placement Request object manifests a specific scheduling request. It would include:

• The Context of Scheduling:
  • What kind of request: currently DEPLOY, RELOCATE , RESIZE, but later possibly also EVACUATE, OPTIMIZE and so on.
  • Optional parameters: for example, the subject instance reference (i.e., the instance being deployed or relocated
• The Current State
  • Hosts with their capacities (total, reserved, current usage)
  • [Optional] Instances with their resource demands and [if placed] current placement on the hosts. This information can be optional for DEPLOY, but may be needed for RELOCATE (target verification), RESIZE, etc.
• Constraints based on a placement-oriented dictionary
  • Hard constraints (placement must not violate), e.g.:
    • MAY_NOT_LOCATE <instances> <hosts>
    • MAY_NOT_COLOCATE <instances>
  • Soft constraints (placement goals, additively quantifying placement quality), e.g.:
    • SHOULD_NOT_LOCATE <instances> <hosts> <weight>
    • SHOULD_LOCATE <instances> <hosts> <weight>
    • SHOULD_NOT_COLOCATE <instances> <weight>
    • SHOULD_COLOCATE <instances> <weight>
  • Optionally, desired properties of the (global) target placement and their respective weights, e.g.,:
    • LOAD_BALANCE <resource> <weight>
    • SAVE_ENERGY <weight>

Constraints vs Filters

The existing filtering mechanism mixes placement considerations (namely anti-location) with the semantic knowledge that induces them (e.g. flavor extra-specs). Constraints aim to model pure placement considerations. This separation is what enables Advisors to utilize highly complex semantics while saving Engines the need to model and support them. An example to this is MAY_NOT_LOCATE which models anti-location using the most primitive entities required to model it - hosts and instances - instead of the more complex entities (e.g. host-aggregates, tenants) that induce it indirectly using some highly-expressive logic (e.g. extra-specs filter, policies) which is of no use to the Engine. This approach means that:

• Constraint are as general as possible: Engines need only support one kind of anti-location constraint (whether induced by host incompatibility or tenant policy), one kind of anti-colocation constraint (whether induced by security, load-balancing or high-availability) and so on.
• New concepts would be added to the Placement Request dictionary only when the existing concepts are insufficient to model (efficiently) a new constraint.

This slow-growing dictionary should let Engine developers concentrate on efficient implementations of constraints rather than on supporting new scheduling semantics. At the same time, translating some new scheduling feature into the available constraints dictionary is enough to assure Advisor developers that their feature is supported by the Engine.

In this framework the existing filters would become constraint generators. Note, however, that the Advisor is free to apply or modify the resulting constraints when constructing the Placement Request. For example, instead of including the full state and set of constraints, an Advisor constructing a DEPLOY Placement Request of a single flavor may omit from Current State:

• All existing instances (while replacing any MAY_NOT_COLOCATE with a MAY_NOT_LOCATE based on the current placement).
• Any host included in a MAY_NOT_LOCATE constraint.

The Advisor's control over the problem size given to the Engine (by pre-processing some of the constraints) can maintain today's filtering behavior (i.e. a small Placement Request for a simple single-flavored DEPLOY) while preserving the ability to construct larger problems for more complex contexts (e.g. a future OPTIMIZE or even a heterogeneous group DEPLOY).

Notes

• Placement Requests are self-contained, which means Engines are stateless. It is up to the Advisor to maintain state (i.e. project the result of previous computations (placement changes) onto the Engine's Current State input).

• All current filters essentially focus on [anti-]location information, i.e., where to place or not a specific instance. Therefore, covering the location semantics should be enough to keep the filters inside the advisor and separate from the engine. The additional co-location constraints can be effective for multiple-instance operations of optimize and evacuate.

• The current weight functions can also be supported as soft placement constraints, not as free-form python code but as simple algebraic expressions over pre-defined host quantities (e.g. CORES_NUMBER, FREE_MEM, PLACED_INSTANCES). A constraint such as "LOCATION_COST <hosts> <expression>" could rank its hosts according to the evaluation of <expression> on each of them. Multiple such soft constraints create a combined rank similar to what is done today.

• The Engine's output would always be a map that includes the computed placement of all (and only) the instances affected by the Placement Request (e.g. the result for DEPLOY would include an entry for each of the deployed instances).