Difference between revisions of "Host-aggregates"
| Line 5: | Line 5: | ||
== Summary == | == Summary == | ||
| − | + | This blueprint will introduce the concept of aggregate into Nova. Host aggregates are different from zones and availability zones: while the former allows the partition of Nova deployments into logical groups for load balancing and instance distribution, the latter are for providing some form of physical isolation and redundancy from other availability zones (e.g. by using separate power supply and network gears). Availability zones do not necessarily mean geographic distribution whereas zones usually do. Host aggregates can be regarded as a mechanism to further partitioning an availability zone, i.e. into multiple groups of hosts that share common resources like storage and network. | |
| − | |||
== Release Note == | == Release Note == | ||
| − | + | Support for host aggregates (i.e. clusters, groups or pools of hypervisor hosts) in Nova. The Aggregates concept is another level of scaling for Nova deployments, after zones and availability zones. | |
| − | |||
| − | |||
| − | |||
== Rationale == | == Rationale == | ||
| + | Host aggregates enable a finer level of granularity in which to structure an entire [[OpenStack]] deployment. Aggregates also allow higher availability of a single guest instance within an availability zone, it enables advanced VM placement strategies, and more importantly it enables hosts' zero-downtime upgrades. Aggregates are exposed via Admin OSAPI API. Check OSAPI documentation for more details. | ||
== User stories == | == User stories == | ||
| + | TBC | ||
== Assumptions == | == Assumptions == | ||
| + | |||
| + | For further details, see implementation section below. | ||
| + | |||
| + | * In the first release of this feature, things like shared storage and networking are manually setup and configured. | ||
| + | * There might be changes required on the specific virt layer support. | ||
| + | ** For instance in the xenapi case, it may be needed to ensure that instances can be explicitly streamed to the local SR or to another storage. At the moment instances get streamed to the default SR and this typically is the local SR. This is fine for non-HA instances, however for HA-enabled instances some form of shared storage is needed. In a nutshell, it is expected that some coordination with whoever is in charge of https://blueprints.launchpad.net/nova/+spec/guest-ha will take place. | ||
| + | ** With the introduction of hypervisor pools (e.g. [[XenServer]] pools or ESX resource pools), live migration becomes bonus; this means that there will be work to be coordinated with whoever is in charge of https://blueprints.launchpad.net/nova/+spec/xenapi-live-migration. | ||
| + | ** Effort is to be coordinated with the wider community team to get this stuff working primarely for both [[XenServer]] and KVM (ESXi may be in doubt). Primary focus for Essex is to close the gap between the two hypervisors, rather than widening it. | ||
== Design == | == Design == | ||
| + | * Nova has zones and availability zones, the relationship is N-M. | ||
| + | * Aggregates introduce a sub-level of grouping. The relationship between availability zones and aggregates is 1-M (i.e. 1 availability zone contains 0 or more aggregates, and an aggregate can belong to one and only one availability zone). | ||
| + | * Aggregates must be visible to admin only. | ||
| + | * Aggregates is an abstract concept, i.e. it does not necessarily translated to a resource pool in the hypervisor-land. The implication of this statement is that if it is decided to have, for instance, [[XenServer]]/VMWare resource pools to map to an aggregate in the host-aggregates-v1, this might no longer the case in host-aggregates-v2. | ||
| − | + | == Implementation == | |
| + | Implementation – Stage 1a: | ||
| + | |||
| + | Operations at 1 and 2 are model operations, so there is no pollution due to hypervisors details. This means that most of the work to be done on Nova is related to adding admin extensions to OSAPI, extending the data model, and providing sqlalchemy migration snippets: | ||
| + | |||
| + | * Basic operations on aggregates. For instance: | ||
| + | ** Create: create empty aggregate with some attributes (e.g. name-label, kwargs, availability_zone). | ||
| + | ** Read: get list of aggregates per specified availability zone. | ||
| + | ** ... | ||
| + | * Basic operations on a single aggregate. For instance: | ||
| + | ** Read/Update properties. | ||
| + | ** Add/Remove host. | ||
| + | * Unit test the lot. | ||
| − | + | Implementation – Stage 1b: | |
| + | |||
| + | * Basic operations on a host within aggregate. For instance: | ||
| + | ** Enable host: I see the hypervisor pool being created here on the first enable call – the host in question becomes master; subsequent calls put host as slave). | ||
| + | ** Disable host: this removes host from pool – note that host cannot be removed if it has not been disabled first; if pool becomes empty it gets destroyed). | ||
| + | * Put/Resume host in/from maintenance: this triggers a lot of intervention on the platform. This means that guest instances on the host being put in maintenance need to move to other hosts in the aggregate (and back once the maintenance window has been closed). This move has ramifications on instance networking, the controllability of the instances, etc. There is extensive work to be done in this area to ensure that instances can still be reached and controlled by the orchestration layer. | ||
| + | * Write CI tests for the lot. | ||
| + | * Write user documentation to get this up and running. | ||
| + | |||
| + | In Stage 2 of the implementation the following aspects need to be looked at: | ||
| − | + | * Admin API to create, associate and manage properties for the aggregate (e.g. shared storage etc). | |
| + | * Scheduling layer needs to be aggregate-aware (e.g. to provide headroom in face of maintenance windows). | ||
=== UI Changes === | === UI Changes === | ||
| − | + | Extensions may be provided to Horizon to support Admin tasks, like aggregates creation and management. | |
| − | |||
=== Code Changes === | === Code Changes === | ||
| − | + | To be detailed when the implementation starts. | |
| − | |||
=== Migration === | === Migration === | ||
| − | + | Because of the introduction of new abstractions to the Nova conceptual model, DB Schema may be required. These changes will be documented here when the implementation commences. | |
| − | |||
| − | |||
| − | |||
| − | |||
== Test/Demo Plan == | == Test/Demo Plan == | ||
| − | |||
This need not be added or completed until the specification is nearing beta. | This need not be added or completed until the specification is nearing beta. | ||
== Unresolved issues == | == Unresolved issues == | ||
| − | |||
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved. | This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved. | ||
== BoF agenda and discussion == | == BoF agenda and discussion == | ||
| − | |||
Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected. | Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected. | ||
---- | ---- | ||
[[Category:Spec]] | [[Category:Spec]] | ||
Revision as of 12:27, 4 November 2011
- Launchpad Entry: NovaSpec:host-aggregates
- Created: 10 October 2011
- Contributors: Armando Migliaccio
Summary
This blueprint will introduce the concept of aggregate into Nova. Host aggregates are different from zones and availability zones: while the former allows the partition of Nova deployments into logical groups for load balancing and instance distribution, the latter are for providing some form of physical isolation and redundancy from other availability zones (e.g. by using separate power supply and network gears). Availability zones do not necessarily mean geographic distribution whereas zones usually do. Host aggregates can be regarded as a mechanism to further partitioning an availability zone, i.e. into multiple groups of hosts that share common resources like storage and network.
Release Note
Support for host aggregates (i.e. clusters, groups or pools of hypervisor hosts) in Nova. The Aggregates concept is another level of scaling for Nova deployments, after zones and availability zones.
Rationale
Host aggregates enable a finer level of granularity in which to structure an entire OpenStack deployment. Aggregates also allow higher availability of a single guest instance within an availability zone, it enables advanced VM placement strategies, and more importantly it enables hosts' zero-downtime upgrades. Aggregates are exposed via Admin OSAPI API. Check OSAPI documentation for more details.
User stories
TBC
Assumptions
For further details, see implementation section below.
- In the first release of this feature, things like shared storage and networking are manually setup and configured.
- There might be changes required on the specific virt layer support.
- For instance in the xenapi case, it may be needed to ensure that instances can be explicitly streamed to the local SR or to another storage. At the moment instances get streamed to the default SR and this typically is the local SR. This is fine for non-HA instances, however for HA-enabled instances some form of shared storage is needed. In a nutshell, it is expected that some coordination with whoever is in charge of https://blueprints.launchpad.net/nova/+spec/guest-ha will take place.
- With the introduction of hypervisor pools (e.g. XenServer pools or ESX resource pools), live migration becomes bonus; this means that there will be work to be coordinated with whoever is in charge of https://blueprints.launchpad.net/nova/+spec/xenapi-live-migration.
- Effort is to be coordinated with the wider community team to get this stuff working primarely for both XenServer and KVM (ESXi may be in doubt). Primary focus for Essex is to close the gap between the two hypervisors, rather than widening it.
Design
- Nova has zones and availability zones, the relationship is N-M.
- Aggregates introduce a sub-level of grouping. The relationship between availability zones and aggregates is 1-M (i.e. 1 availability zone contains 0 or more aggregates, and an aggregate can belong to one and only one availability zone).
- Aggregates must be visible to admin only.
- Aggregates is an abstract concept, i.e. it does not necessarily translated to a resource pool in the hypervisor-land. The implication of this statement is that if it is decided to have, for instance, XenServer/VMWare resource pools to map to an aggregate in the host-aggregates-v1, this might no longer the case in host-aggregates-v2.
Implementation
Implementation – Stage 1a:
Operations at 1 and 2 are model operations, so there is no pollution due to hypervisors details. This means that most of the work to be done on Nova is related to adding admin extensions to OSAPI, extending the data model, and providing sqlalchemy migration snippets:
- Basic operations on aggregates. For instance:
- Create: create empty aggregate with some attributes (e.g. name-label, kwargs, availability_zone).
- Read: get list of aggregates per specified availability zone.
- ...
- Basic operations on a single aggregate. For instance:
- Read/Update properties.
- Add/Remove host.
- Unit test the lot.
Implementation – Stage 1b:
- Basic operations on a host within aggregate. For instance:
- Enable host: I see the hypervisor pool being created here on the first enable call – the host in question becomes master; subsequent calls put host as slave).
- Disable host: this removes host from pool – note that host cannot be removed if it has not been disabled first; if pool becomes empty it gets destroyed).
- Put/Resume host in/from maintenance: this triggers a lot of intervention on the platform. This means that guest instances on the host being put in maintenance need to move to other hosts in the aggregate (and back once the maintenance window has been closed). This move has ramifications on instance networking, the controllability of the instances, etc. There is extensive work to be done in this area to ensure that instances can still be reached and controlled by the orchestration layer.
- Write CI tests for the lot.
- Write user documentation to get this up and running.
In Stage 2 of the implementation the following aspects need to be looked at:
- Admin API to create, associate and manage properties for the aggregate (e.g. shared storage etc).
- Scheduling layer needs to be aggregate-aware (e.g. to provide headroom in face of maintenance windows).
UI Changes
Extensions may be provided to Horizon to support Admin tasks, like aggregates creation and management.
Code Changes
To be detailed when the implementation starts.
Migration
Because of the introduction of new abstractions to the Nova conceptual model, DB Schema may be required. These changes will be documented here when the implementation commences.
Test/Demo Plan
This need not be added or completed until the specification is nearing beta.
Unresolved issues
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.
BoF agenda and discussion
Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.
