Jump to: navigation, search

Difference between revisions of "VMware-vSphere-support"

(Updated the specification to include a risk assessment information.)
Line 41: Line 41:
  
 
== Assumptions ==
 
== Assumptions ==
A1. The [[OpenStack]] project will not ship any VMware products or SDKs. In the current design, there is no need for a SDK from VMware (we are using Python-ZSI to address VMware's SOAP API directly).
+
A1. The [[OpenStack]] project will not ship any VMware products or SDKs. In the current design, there is no need for a SDK from VMware (we are using python-suds (version 0.4)to address VMware's SOAP API directly).
  
 
A2. Users will require licenses for VMware vSphere Essentials (or greater) for all "clouds" with up to three compute hosts, and VMware vSphere Standard (or greater) for all sensible clouds. See Licensing below.
 
A2. Users will require licenses for VMware vSphere Essentials (or greater) for all "clouds" with up to three compute hosts, and VMware vSphere Standard (or greater) for all sensible clouds. See Licensing below.
Line 57: Line 57:
  
 
== Design ==
 
== Design ==
The requirement to support ESXi implies that nothing can be installed inside the console domain. Instead, we will install nova-compute inside a virtual machine, one per compute node. nova-compute will be given the credentials to the local ESXi (or the controlling vCenter), and with these credentials it will perform all necessary actions through the VMware vSphere API over SOAP (see [http://www.vmware.com/support/developer/vc-sdk/visdk41pubs/ApiReference/index.html VMware vSphere API Reference Documentation] and [http://www.vmware.com/support/pubs/sdk_pubs.html VMware APIs and SDK Documentation]). [http://pywebsvcs.sourceforge.net/ Python-ZSI] will be used to make the SOAP calls.
+
The requirement to support ESXi implies that nothing can be installed inside the console domain. Instead, we will install nova-compute inside a virtual machine, one per compute node. nova-compute will be given the credentials to the local ESXi (or the controlling vCenter), and with these credentials it will perform all necessary actions through the VMware vSphere API over SOAP (see [http://www.vmware.com/support/developer/vc-sdk/visdk41pubs/ApiReference/index.html VMware vSphere API Reference Documentation] and [http://www.vmware.com/support/pubs/sdk_pubs.html VMware APIs and SDK Documentation]). [http://http://sourceforge.net/projects/python-suds/ python-suds] will be used to make the SOAP calls.
  
 
Disk streaming from Glance will also be handled within this VM. The VM will ask for a new virtual disk to be created, and attached to itself for writing. Once the streaming is finished, that disk will be detached and made available for the new customer VM.
 
Disk streaming from Glance will also be handled within this VM. The VM will ask for a new virtual disk to be created, and attached to itself for writing. Once the streaming is finished, that disk will be detached and made available for the new customer VM.

Revision as of 11:59, 16 March 2011

Summary

This blueprint proposes adding support to Nova for the use of VMware vSphere as a compute provider.

Requirements

R1. Support VMware vSphere 4.1 as a compute provider within Nova. As the deployment architecture, support both ESX and ESXi. VM storage is restricted to VMFS volumes on local drives. Note that vCenter is not required by the current design, and is not currently supported. Instead, Nova Compute talks directly to ESX/ESXi.

R2. Integrate with Glance, so that VM images can be streamed from there for boot on ESXi.

R3. Suppport Nova's VLAN-based network model (VlanManager).

R4. Support Nova's flat networking model (FlatManager).

R5. When vSphere support is not in use, the presence of this code will add no additional dependencies to nova-compute as a whole. This implies that any new dependencies must be loaded on-demand.

R6. - Removed -

R7. CPU load from Nova on the compute host should be negligible when idle.

Non-requirements

NR1. Shared storage support is excluded.

NR2. Live migration support is excluded.

NR3. Clustering support is excluded.

NR4. Nova is not required to handle any automatic installation or configuration of the underlying hypervisor. The user will be responsible for configuring vSphere appropriately.

NR5. Nova is not required to interrogate vCenter to find compute hosts. The user will be responsible for configuring Nova with the details for each compute host.

NR6. Nova is not required to cope gracefully when compute nodes are concurrently managed by automatic vCenter operations or other third parties. Nova may assume that it "owns" the compute node entirely.

NR7. Nova is not required to cope gracefully when compute nodes are subject to VMware Update Manager dynamic updates. A future blueprint will discuss the interaction between Nova's scheduler and update management on vSphere, to ensure that machines are drained of customer workloads before being upgraded.

NR8. Support for iSCSI volume attach/detach is a future feature, not on the plan for Cactus.

Assumptions

A1. The OpenStack project will not ship any VMware products or SDKs. In the current design, there is no need for a SDK from VMware (we are using python-suds (version 0.4)to address VMware's SOAP API directly).

A2. Users will require licenses for VMware vSphere Essentials (or greater) for all "clouds" with up to three compute hosts, and VMware vSphere Standard (or greater) for all sensible clouds. See Licensing below.

A3. The QA emphasis will be on ESXi, rather than ESX, with the assumption that anything supported by ESXi is also supported by ESX.

Rationale

Regarding A3's emphasis on ESXi: On http://www.vmware.com/products/vsphere/esxi-and-esx/overview.html, VMware state "vSphere 4.1 and its subsequent update and patch releases are the last releases to include both ESX and ESXi hypervisor architectures. Future major releases of VMware vSphere will include only the VMware ESXi architecture. For this reason, VMware recommends that deployments of vSphere 4.x utilize the ESXi hypervisor architecture." Following this, we shift the QA burden towards ESXi, as we expect this to be the deployment architecture selected in most cases.

Regarding R1's restriction to vSphere 4.1: this is to reduce the QA burden, as there is no known need to support older versions.

NRs 1, 2, 3, 6, and 7 have been added to avoid increasing the QA burden.

NRs 4 and 5 have been added to avoid increasing the development burden.

Design

The requirement to support ESXi implies that nothing can be installed inside the console domain. Instead, we will install nova-compute inside a virtual machine, one per compute node. nova-compute will be given the credentials to the local ESXi (or the controlling vCenter), and with these credentials it will perform all necessary actions through the VMware vSphere API over SOAP (see VMware vSphere API Reference Documentation and VMware APIs and SDK Documentation). python-suds will be used to make the SOAP calls.

Disk streaming from Glance will also be handled within this VM. The VM will ask for a new virtual disk to be created, and attached to itself for writing. Once the streaming is finished, that disk will be detached and made available for the new customer VM.

The connection from Nova to vCenter/ESXi will be through a nova.virt module, parallel with nova.virt.libvirt_conn and nova.virt.xenapi_conn (i.e. not through libvirt).

QA

Unit testing will be of similar style to that used by XenAPI, with a mock ESX stubbed in at the RPC layer (i.e. below nova.virt.vmware_conn).

The system will be certified using a single host, 7 day stress test: * random deploy/undeploy, power-on/off cycle * 32 x 1GB VMs, each with a 10GB HDD * up to 2 VMs at a time being deployed

The system test must succeed with zero failures in those 7 days: i.e. no failures to deploy, undeploy, power-on, or power-off VMs, and no nova-compute crashes.

Future

We need to figure out how machines can be updated automatically. Presumably, we want to drain a machine of customer workloads before installing updates and rebooting. It is an open question how best to integrate this with VMware Update Manager.

Support for iSCSI volume attach/detach.

Development Resources

Citrix has committed a development team to implement the functionality specified here. Prototyping is underway, and will be pushed to Launchpad soon.

No commitment has been made with regard to any "Future" features.

Licensing

Why we can't use vSphere Hypervisor

From http://www.vmware.com/download/eula/esx_esxi_eula.html:

"3. GRANT AND USE RIGHTS FOR SOFTWARE

"3.3 Restrictions. You may not (i) sell, lease, license, sublicense, distribute or otherwise transfer in whole or in part the Software or the Software License Key to another party; (ii) provide, disclose, divulge or make available to, or permit use of the Software in whole or in part by, any third party (except Designated Administrative Access) without VMware’s prior written consent; (iii) modify or create derivative works based upon the Software; or (iv) create, develop, license, install, use, or deploy any third party software or services to circumvent, enable, modify or provide access, permissions or rights which violate the technical restrictions of the Software, any additional licensing terms provided by VMware via product documentation, notification, and/or policy change posted at www.vmware.com, and the terms of this EULA."

Also, from http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1023990:

"vCLI, PowerCLI, and vSphere SDK for Perl are limited to read-only access for the free vSphere Hypervisor edition. To enable full functionality of vCLI on a VMware ESXi host, the host must be licensed with vSphere Essentials, vSphere Essential Plus, vSphere Standard, vSphere Advanced, vSphere Enterprise, or vSphere Enterprise Plus."

We understand this to mean that vSphere Hypervisor (previously known as "Free ESXi") may not be managed programmatically in a read-write manner, since VMware's own tools make that restriction. See also http://www.veeam.com/blog/veeam-and-free-esxi.html and http://www.virtualizationpractice.com/blog/?p=274.

Why we can't use vSphere Essentials or Essentials Plus for more than three compute nodes

From http://www.vmware.com/download/eula/esx_esxi_eula.html:

"9. SOFTWARE PRODUCT SPECIFIC TERMS AND CONDITIONS

"9.1 ESX/ESXi

"c) VMware vSphere Essentials and VMware vSphere Essentials Plus

"If you have licensed VMware vSphere Essentials or VMware vSphere Essentials Plus, the following additional license terms apply:

"VMware licenses VMware vSphere Essentials or VMware vSphere Essentials Plus editions (collectively "Editions") to you solely for managing up to three (3) server hosts and your use of these Editions is limited to servers with up to two processors. For these Editions, the server hosts must be managed by the VMware vCenter Server that is provided with these Editions, and that same VMware vCenter Server cannot be used to manage other server hosts not included with these Editions."

We understand this to mean that it is forbidden to use Essentials or Essentials Plus for any installation larger than three hosts.

Release Note

Nova may now use VMware vSphere as a compute provider.

Risk Assessment

The code changes are encapsulated in a module named vsphereapi that fits inside nova.virt (hypervisor abstraction layer). The risk of destabilization is "low" because the changes are well contained in vsphereapi module, and the bulk of the code will be executed only if the connection type is set to 'vsphereapi_conn', which is the case of VMware ESXi server as backend. The changes to the existing codepaths is minimal.

Discussion