Difference between revisions of "CowInstances"
(talk) |
|||
| Line 1: | Line 1: | ||
__NOTOC__ | __NOTOC__ | ||
| − | * '''Launchpad Entry''': [[NovaSpec]]: | + | <!-- ## page was renamed from [[LvmInstances]] --> |
| + | * '''Launchpad Entry''': [[NovaSpec]]:cow-instances | ||
* '''Created''': 11/01/10 | * '''Created''': 11/01/10 | ||
* '''Contributors''': Vish Ishaya | * '''Contributors''': Vish Ishaya | ||
| Line 6: | Line 7: | ||
== Summary == | == Summary == | ||
| − | Right now, instance disks are stored as files in the system and manually partitioned. This leads to very slow load times and a lot of redundant data. Instances should | + | Right now, instance disks are stored as files in the system and manually partitioned. This leads to very slow load times and a lot of redundant data. Instances should images with Copy on Write to conserve space and speed up launch times. |
== Release Note == | == Release Note == | ||
| − | Nova now | + | Nova now uses copy on write images by default. This allows for fast allocation of new images, with no need to manually partition the image. Since disks are no longer manually partitioned, the root drive is available as /dev/vda and local storage for new instances will be available as /dev/vdb. This replaces the older usage of /dev/vda1 and /dev/vda2. To use the older method of allocation, you can turn off cow support by setting --nouse_cow in nova-compute. |
| − | + | == Rationale == | |
| − | |||
| − | |||
| − | + | The code to create image files for instances is inefficient. Images, which represent a single partition are copied with dd into a new file, an mbr is created, and a second partition for local storage is created. Converting to Logical Volumes with copy-on-write will make this process much faster and save disk space on the host. | |
== User stories == | == User stories == | ||
| + | |||
| + | Jack would like to rapidly provision instances. Currently, and m1.large can take as much as 5 minutes to launch properly. With cow volumes, instance launch time can be decreased to under one minute. | ||
| + | |||
| + | Bill is managing a small private cloud on older hardware. His company uses the same base image for every vm. He would like to support a large number of virtual machines, but does not have a huge amount of storage space. CoW images allow him to maximize use of disk resources. | ||
== Assumptions == | == Assumptions == | ||
| Line 24: | Line 27: | ||
== Design == | == Design == | ||
| − | + | Compute host should create a base logical volume the first time an image is used by the host. Kernels and ramdisks could potentially be links to read-only images. Future launches of the image, will just create a copy-on-write snapshot and use that as the disk. The same strategy can be used for local storage. Each host can have a blank image for the local storage for each possible instance type (or flavor in os api). Rather than partitioning these two disks into a single file with a master boot record, these two disks can just be attached separately to the instance through the hypervisor. | |
== Implementation == | == Implementation == | ||
| − | + | Disk.py needs to be rewritten. Some concept of caching needs to be added to the image download. This could be integrated with glance, or be part of nova-compute. Attaching of disks in the hypervisor needs to be reworked a bit in order to allow for multiple attached disks. | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | + | The specific image format is hypervisor specific. The initial version will attempt to abstract out as much as possible using libvirt, but the initial version is only guaranteed to support KVM. Hopefully some of the code can be shared with other hypervisor implementations. | |
=== Migration === | === Migration === | ||
| − | + | This should not interfere with existing instances, but it may be useful to include automated migration of old instances to use CoW. | |
| − | |||
| − | |||
| − | |||
== Test/Demo Plan == | == Test/Demo Plan == | ||
| Line 51: | Line 45: | ||
== Unresolved issues == | == Unresolved issues == | ||
| − | This | + | This has the potential of breaking tools/images designed to be used with ec2, as they may expect the local storage to be a second partition instead of a second disk. |
== BoF agenda and discussion == | == BoF agenda and discussion == | ||
Revision as of 18:30, 17 November 2010
- Launchpad Entry: NovaSpec:cow-instances
- Created: 11/01/10
- Contributors: Vish Ishaya
Summary
Right now, instance disks are stored as files in the system and manually partitioned. This leads to very slow load times and a lot of redundant data. Instances should images with Copy on Write to conserve space and speed up launch times.
Release Note
Nova now uses copy on write images by default. This allows for fast allocation of new images, with no need to manually partition the image. Since disks are no longer manually partitioned, the root drive is available as /dev/vda and local storage for new instances will be available as /dev/vdb. This replaces the older usage of /dev/vda1 and /dev/vda2. To use the older method of allocation, you can turn off cow support by setting --nouse_cow in nova-compute.
Rationale
The code to create image files for instances is inefficient. Images, which represent a single partition are copied with dd into a new file, an mbr is created, and a second partition for local storage is created. Converting to Logical Volumes with copy-on-write will make this process much faster and save disk space on the host.
User stories
Jack would like to rapidly provision instances. Currently, and m1.large can take as much as 5 minutes to launch properly. With cow volumes, instance launch time can be decreased to under one minute.
Bill is managing a small private cloud on older hardware. His company uses the same base image for every vm. He would like to support a large number of virtual machines, but does not have a huge amount of storage space. CoW images allow him to maximize use of disk resources.
Assumptions
Design
Compute host should create a base logical volume the first time an image is used by the host. Kernels and ramdisks could potentially be links to read-only images. Future launches of the image, will just create a copy-on-write snapshot and use that as the disk. The same strategy can be used for local storage. Each host can have a blank image for the local storage for each possible instance type (or flavor in os api). Rather than partitioning these two disks into a single file with a master boot record, these two disks can just be attached separately to the instance through the hypervisor.
Implementation
Disk.py needs to be rewritten. Some concept of caching needs to be added to the image download. This could be integrated with glance, or be part of nova-compute. Attaching of disks in the hypervisor needs to be reworked a bit in order to allow for multiple attached disks.
The specific image format is hypervisor specific. The initial version will attempt to abstract out as much as possible using libvirt, but the initial version is only guaranteed to support KVM. Hopefully some of the code can be shared with other hypervisor implementations.
Migration
This should not interfere with existing instances, but it may be useful to include automated migration of old instances to use CoW.
Test/Demo Plan
This need not be added or completed until the specification is nearing beta.
Unresolved issues
This has the potential of breaking tools/images designed to be used with ec2, as they may expect the local storage to be a second partition instead of a second disk.
BoF agenda and discussion
Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.
