ImageNotes

= Nova Image Notes =

Image Implementation In a Nutshell
The system currently provides two backends for retrieving images:

reside. Images in node-local storage should already in raw format (not     AMI)
 * Node-local Store (my term) - A directory on the node where images

interface to the filesystem
 * S3 Object Store - Currently this is an fake which provides an S3 ReST

The Node-local Store appears to be a convenience for testing and perhaps for use by an Ops team when it goes live, whereas, S3 provides a registration mechanism (with hints at Image sharing) that would be exposed to customers.

An Unregistered Image is mapped to a bucket in the S3 object store. Within the bucket is:


 * The encrypted AMI
 * A JSON manifest (containing the decryption keys)

The Image is Registered by calling endpoints/node.py/register which in turn issues a PUT request to the object store. Contained within the PUT request is:


 * Image Location: account and bucket for the image
 * Image ID: unique identifier

The S3-ObjectStore handler (objectstore/handler.py) receives the PUT request and forks of a process to continue the registration process (Image.create found in objectstore/image.py) which includes:

API
 * Fetching the image from Image Location
 * Decrypting the Image using the keys provided by the manifest
 * Untarring the AMI contents
 * Moving the Image to the destination directory for serving using the ReST

The state of the Image is contained with the JSON manifest which is updated as the image progresses through the unpacking process. Eventually the image reaches the 'available' status and is said to be 'registered'.

Now that the image is registered, the spawn method of compute/node.py can use curl to fetch the image from the objectstore (util.fetchfile) to instantiate a new VM.

It should be noted both the consumer (node.py) and the provider (Image API/objectstore) think of images as having a type:


 * Kernel
 * Ramdisk
 * Image

So, in order spawn a VM Nova you really need to specify *three* images, one of each type.

What the Implementation Gets Right
useful for OPs teams when the system goes live
 * The Node-local Store - useful for testing and debugging, plus, could be

(create/detroy VMS)
 * Separates the Image API (register/deregister) from Instance API


 * Using fakes for the backends of the object stores.

Where the Implemenation Needs Work
large images this could be painful.
 * Registering AMI hits disk at least 3 times (fetch, decrypt, untar). For

metadata, etc)
 * The Image registraion API is missing key features (image sharing,

What the Implmentation Gets Wrong
store and AMI's as the image format
 * Amazon-stack specific: The Image API is coupled to S3 for the object

sense in the manifest.ovf file (others parts would not)
 * No Caching - each Spawn would require a fetch from Image API
 * Does not use OVF container - some of the JSON manifest data would make

describes the VM (architecture, etc) versus Image metadata which describes the Image itself (isPublic, isShared, image location). I would argue that these are fundamentally different and make sense stored in     two different places. The important question being: does this metadata make sense *outside* of our system.
 * Conflates two different types of metadata: Image metadata which

Ideas for Improvement
within OVF container
 * Use Pyfilesystem to store Images. This would provide support for:
 * S3
 * CloudFiles (once the plugin is written, I'm working on that now)
 * Local disk storage
 * Memory storage (this could be great for unit-testing)
 * Add a caching layer. This could be:
 * A separate caching utility
 * Part of Pyfilesystems caching layer
 * Some combination of both
 * Create robust Image registry. This would:
 * Backed by Cassandra or Redis
 * Support an API for sharing images
 * Support metadata on Image Registry entry in addition to metadata stored


 * There is no need to fetch Images from external storage, if network distributed filesystem (GlusterFS) or over the network block device (NBD or SheepDog) is used. It would be nice to support also those solutions (at least by friendly architecture, so they can be developed by 3rd party).

Style-notes
(most of the time they don't)
 * Staticmethods everywhere - just make sure staticmethods make sense here
 * Remove pyc from source tree
 * multiprocessing + uuid4 may cause issues (dup uuids if not careful)