Difference between revisions of "Glance-tasks-import"

Latest revision as of 21:38, 20 August 2013

This is the full specification for blueprint:https://blueprints.launchpad.net/glance/+spec/new-upload-workflow

Context

We want to make a Glance API endpoint public to enable end-user (not admin) upload of images.

Problems Facing a Public Glance Endpoint

Making a Glance API endpoint public causes these problems:

• need some kind of quality control on what's being stuffed into glance as "images"
  • this enhances end-user experience (so you don't keep trying to boot an uploaded movie or something -- want to reject bad images before this point if possible)
• don't want the extra bandwidth, processing requirements of these uploads to interfere with normal glance functionality as an image source for nova
  • want to be able to push the upload, validation process out to the cloud periphery

"Import"

We propose to introduce as a Glance task an end-user upload-type operation called import.

An objection to this proposal is that Glance v1 has "copy-from" and "location" ... why aren't these sufficient?

Difference between "import" and "copy-from":

• copy-from: you've got an Image (capital "I") to upload
• import: you've got some bits that you want turned into an Image

(In other words, we can trust nova but don't know about ordinary glance "users")

Difference between "import" and "location":

• location: you've got an Image stored elsewhere; glance grabs the bits from there when the Image is requested
• import: you've got some bits stored elsewhere, but there's no way to know whether they are an "Image"

So, for these reasons, we will introduce an import operation.

Example Workflow (API View)

Initial Import Request

The initial request would be similar to an image-create, but would be directed to the tasks path and would return the location of a task resource.

There are several advantages to using a task resource instead of simply creating an image that could be polled until it turned active:

• if the import is a failure, we don't have to worry about what to do with the image record
• the user's image-list won't be clogged with images that are being imported and can't be used to boot yet, anyway
• the task resource can be structured to allow for helpful error messages; otherwise, we'd have to try to find a way to shoehorn them into the current image resource
• we can create useful status values for this import task resource without having to modify the current image statuses
• decoupling the task resource from the image resource allows for independent development of each

An import request will adhere to the task_request schema described in ^[1].

Request 1: POST /v2/tasks
{ "type": "import",
   "input": {
      "import_from": "swift://cloud.foo/myaccount/mycontainer/path",
      "import_from_format": "qcow2",
      "image_properties" : {
          "name": "GreatStack 1.22",
          "tags": ["lamp", "custom"]
       }
    }
}

The content of the image_properties field is the same content defined for the image create request in the Images v2 "Metadata" API ^[2].

Response 1: 201 Created
Location: "http://glance-server/v2/tasks/12345678-1234-1234-1234-123456789abc"

(Note: probably return the task resource in the body) The input passed in the request are included in the task body so that (a) the user can tell which task this is a record for (in case the user has multiple tasks underway), (b) we'll have them available when the image record finally gets created after a successful import, and (c) they will be available to the user to use in generating a new task request.

Import Task Request input Content

(no ordering, just using the numerals to keep count)

1. import_from (a URL where glance can get the image data)
2. import_from_format (e.g., 'qcow2')
3. image_properties (this would be the json object of properties you can specify on the current image create call)

Polling for Import Status

The newly-created task resource can now be polled for import status information. Ultimately, it would contain either the location of the successfully created image or an informative message indicating why the import failed. The ultimate response (either success or failure) would contain an expires_at field indicating when this particular task resource may be deleted so we wouldn't have these things accumulating indefinitely.

An import task response will adhere to the task schema described in ^[3].

Request 2 - N: GET /v2/tasks/12345678-1234-1234-1234-123456789abc

Response 2 - N-1: 200 OK
{ "status": "pending", "input" : {... }, "id": "12345678-1234-1234-1234-123456789abc", ... }
 or
{ "status": "processing", "input" : {... }, "id": "12345678-1234-1234-1234-123456789abc", ...  }

Response N: 200 OK
{ "status": "success",
  "result" : {
      "image_uuid": "deadbeef-dead-dead-dead-beefbeefbeef"
   }
  "expires_at": "2013-05-21T15:19:56+0000",
  "id": "12345678-1234-1234-1234-123456789abc",
  "input" : {... },
   /* etc */
}
or
{ "status": "failure",
  "message": "Could not retrieve image.  Received 404 response for url 'swift://cloud.foo/myaccount/mycontainer/path'.",
  "expires_at": "2013-05-21T15:19:56+0000",
  "input" : {... },
  "id": "12345678-1234-1234-1234-123456789abc",
   /* etc */
}

Import Task Response result Content

1. image_uuid : the uuid of the image in glance

Image Retrieval

If the import is successful, the imported image is available through Glance in the normal way:

Request N+1: GET /v2/images/deadbeef-dead-dead-dead-beefbeefbeef

Response N+1: 200 OK
{ "status": "active",
   "name": "GreatStack 1.22",
   "tags": ["lamp", "custom"],
    ... }

Example Workflow (Internal View)

1. create task import object (status: pending)
2. get the data from the location specified (status: processing)
3. verify the data (status: processing)
   • not sure what exactly this "validation" will consist of ... probably want to make it pluggable for the cloud provider ?
   • this is also the place where image conversion could be performed if, for example, the user is uploading an image in qcow2 (or whatever is decided to be the openstack standard interchange format) and the cloud being loaded to uses images in a different format
4. if success, create the image in Glance
5. record result in task import object, set appropriate status, and set expiration date

Formerly Open Questions (with Answers!)

1. The current v2 upload workflow allows 3 options: copy_from, location, and direct data upload. The proposed import workflow offers only a "copy_from" functionality. We definitely don't want to allow the current "location" functionality, but what about direct upload of data?
   • No direct upload of data for this first implementation
2. What would be appropriate import statuses?
   • Tasks will have very generic states. A provider's import plugin may do various things, and rather than try to capture that specifically, we'll just cover it under 'processing'.
3. Do we want to allow cancellation of an import task by a DELETE on /v2/tasks/UUID ?
   • For the first implementation, no.

Revison History

This is a revised version of ^[4]. The revision is mostly to make this part of the tasks API ^[5].

References

1. ↑ https://wiki.openstack.org/wiki/Glance-tasks-api#Task_Request_Schema
2. ↑ http://docs.openstack.org/api/openstack-image-service/2.0/content/create-an-image.html
3. ↑ https://wiki.openstack.org/wiki/Glance-tasks-api#Task_Schema
4. ↑ https://wiki.openstack.org/wiki/Glance-new-upload-workflow
5. ↑ https://wiki.openstack.org/wiki/Glance-tasks-api