Jump to: navigation, search

Difference between revisions of "ImageNotes"

(talk)
(Adding ImageNotes)
 
(talk)
Line 3: Line 3:
 
<pre><nowiki>
 
<pre><nowiki>
  
Nova Notes Real
+
Nova Image Notes
===============
+
================
 
 
 
Image Implementation In a Nutshell
 
Image Implementation In a Nutshell
  

Revision as of 19:41, 22 June 2010



Nova Image Notes
================
Image Implementation In a Nutshell

The system currently provides two backends for retrieving images:
    
    * Node-local Store (my term) - A directory on the node where images
      reside. Images in node-local storage should already in raw format (not
      AMI)

    * S3 Object Store - Currently this is an fake which provides an S3 ReST
      interface to the filesystem

The Node-local Store appears to be a convenience for testing and perhaps for
use by an Ops team when it goes live, whereas, S3 provides a registration
mechanism (with hints at Image sharing) that would be exposed to customers.

An Unregistered Image is mapped to a bucket in the S3 object store. Within the
bucket is:

    * The encrypted AMI
    * A JSON manifest (containing the decryption keys)

The Image is Registered by calling endpoints/node.py/register() which in turn
issues a PUT request to the object store. Contained within the PUT request is:

    * Image Location: account and bucket for the image
    * Image ID: unique identifier

The S3-ObjectStore handler (objectstore/handler.py) receives the PUT request
and forks of a process to continue the registration process (Image.create
found in objectstore/image.py) which includes:

    * Fetching the image from Image Location
    * Decrypting the Image using the keys provided by the manifest
    * Untarring the AMI contents
    * Moving the Image to the destination directory for serving using the ReST
      API

The state of the Image is contained with the JSON manifest which is updated as
the image progresses through the unpacking process. Eventually the image
reaches the 'available' status and is said to be 'registered'.

Now that the image is registered, the spawn method of compute/node.py can use
curl to fetch the image from the objectstore (util.fetchfile) to instantiate a
new VM.

It should be noted both the consumer (node.py) and the provider (Image
API/objectstore) think of images as having a type:

    * Kernel
    * Ramdisk
    * Image

So, in order spawn a VM Nova you really need to specify *three* images, one of
each type.


What the Implementation Gets Right

    * The Node-local Store - useful for testing and debugging, plus, could be
      useful for OPs teams when the system goes live

    * Separates the Image API (register/deregister) from Instance API
      (create/detroy VMS)

    * Using fakes for the backends of the object stores.


Where the Implemenation Needs Work

    * Registering AMI hits disk at least 3 times (fetch, decrypt, untar). For
      large images this could be painful. 

    * The Image registraion API is missing key features (image sharing,
      metadata, etc)

What the Implmentation Gets Wrong

    * Amazon-stack specific: The Image API is coupled to S3 for the object
      store and AMI's as the image format

    * No Caching - each Spawn would require a fetch from Image API

    * Does not use OVF container - some of the JSON manifest data would make
      sense in the manifest.ovf file (others parts would not)

    * Conflates two different types of metadata: Image metadata which
      describes the VM (architecture, etc) versus Image metadata which
      describes the Image itself (isPublic, isShared, image location). I would
      argue that these are fundamentally different and make sense stored in
      two different places. The important question being: does this metadata
      make sense *outside* of our system.

Ideas for Improvement
=====================

* Use Pyfilesystem to store Images. This would provide support for:
    - S3
    - CloudFiles (once the plugin is written, I'm working on that now)
    - Local disk storage
    - Memory storage (this could be great for unit-testing)

* Add a caching layer. This could be:
    - A separate caching utility
    - Part of Pyfilesystems caching layer
    - Some combination of both

* Create robust Image registry. This would:
    - Backed by Cassandra or Redis
    - Support an API for sharing images
    - Support metadata on Image Registry entry in addition to metadata stored
      within OVF container

Style-notes
===========


* Staticmethods everywhere - just make sure staticmethods make sense here
  (most of the time they don't)
* Remove pyc from source tree
* multiprocessing + uuid4 may cause issues (dup uuids if not careful)