Jump to: navigation, search

Occi

Revision as of 22:07, 12 June 2012 by AndyEdmonds (talk)

<<TableOfContents()>>

Summary

This will implement the Open Cloud Computing Interface (OCCI) within nova/api. OCCI is one of the first standards in Cloud Computing. The specification of OCCI can be found here: http://www.occi-wg.org

A demonstration of the implementation is also available.

Implementation details

The OCCI interface is integrated using an WSGI application - it can coexist to the current APIs but offers a rich, flexible interoperable way to interact with OpenStack through a standardized interface.

This implements an OpenStack service that runs out of nova-api. It is implemented using both the OpenStack service and WSGI frameworks. On start it will serve its functionality over HTTP on port 8787 as described in the OCCI specifications. It is compliant as per the set of OCCI specifications (GFD.183, GFD.184 and GFD.185) and implements all mandatory features. It also leverages the OCCI core model to expose OpenStack-specific features in an OCCI fashion. There is further detail on its core and OpenStack specific usage at http://wiki.openstack.org/occi. The implementation is co-funded by Intel Labs Europe Cloud Services Lab and FI-ware.

How to use the OCCI interface

This guide will explain what you can do with the current OCCI implementation for OpenStack.

If you are evaluating or playing with the implementation, it is best followed sequentially.

First up, prerequisites:

Get a running instance of OpenStack

Lots of ways to do this:

  • Install with apt-get
  • Install with puppet
  • Install with chef
  • Install with crowbar
  • Install with devstack
  • Use the prebuilt devstack VM image (see: "Pre-prepared OCCI Devstack VM")

More details can be found in this wiki

The easiest for experimentation and evaluation is devstack.

Running OpenStack with OCCI

Below will show you how to create an OpenStack environment that is OCCI compliant.

Creating a devstack Environment

Note: Creating such an environment is only for development, evaluation purposes.

Create a VM

Pre-requisite: an installed hypervisor. The VM can be created on any desktop hypervisor (e.g. VMware Workstation, VMware Fusion, Parallels Desktop, VirtualBox)

- Create a VM with Ubuntu Server 11.10 with 1 core and 1024MB of RAM. Disk space is at your discretion but 6GB should be enough. See the following video if you need further instructions.
- The hypervisor driver automatically selected will be `qemu` so don't expect your test OpenStack-managed VMs to be insanely fast!
- `sudo apt-get install git`
Install devstack & OCCI

Note: updated install instructions can be found here

With your freshly created VM, install OCCI dependencies and [devstack](http://www.devstack.org) on it.

A pre-requisite to this is the python dev tools: `sudo apt-get install python-pip python-dev build-essential`

1. Install pyssf

   `pip install pyssf`

2. Install devstack

   `git clone git://github.com/openstack-dev/devstack.git`

3. Configure devstack. Here we have to change the `NOVA_REPO` location (done with the `sed` command).

   `cd devstack`
   `sed -i 's/NOVA_REPO=https\:\/\/github.com\/openstack\/nova.git/NOVA_REPO=https\:\/\/github.com\/dizz\/nova.git/' stackrc`
   `sed -i 's/NOVA_BRANCH=master/NOVA_BRANCH=bp\/bexar-open-cloud-compute-interface/' stackrc`

4. Set the contents of `localrc` (you may have to create the file) to:

   `EXTRA_OPTS=( --allow_resize_to_same_host=True --libvirt_inject_password=True --enabled_apis=ec2,occiapi,osapi_compute,osapi_volume,metadata )`
   `ENABLED_SERVICES=g-api,g-reg,key,n-api,n-crt,n-obj,n-cpu,n-net,n-sch,n-novnc,n-xvnc,n-cauth,horizon,mysql,rabbit,n-vol,openstackx`
   `OFFLINE=False`

4. Run devstack

  `./stack.sh`

The first run will be longer than successive runs if this is the first time executing the `stack.sh` command.

For more configuration options of devstack please see the devstack.

The OCCI API will be available at http://$HOST_IP:8787/

Pre-prepared OCCI Devstack VM

A pre-prepared devstack instance with the OCCI API can be downloaded here (OVA). It was generated with VirtualBox and the disk image format is VMDK.

  • Username: `occi`
  • Password: `occi`

All devstack administrative passwords are set to `admin`.

Create some Custom Flavors

Do this if you want to experiment with scaling up a VM on devstack.

1. In devstack ensure that this is in `localrc`:

{
       EXTRA_FLAGS=(--allow_resize_to_same_host=True)

} 2. Create custom flavors:

{
       nova-manage flavor create --name=itsy --cpu=1 --memory=128 --flavor=98 --root_gb=1 --ephemeral_gb=1
       nova-manage flavor create --name=bitsy --cpu=1 --memory=256 --flavor=99 --root_gb=1 --ephemeral_gb=1

}

  • **Note**: your VM must have **at least** 1GB of RAM*

Get Authentication Credentials from Keystone

{
    curl -d '{"auth": {"tenantName": "$YOUR_TENANT_NAME", "passwordCredentials":{"username": "$YOUR_USER_NAME", "password": "$YOUR_PASSWORD"

' -H "Content-type: application/json" http://$KEYSTONE_SERVICE_IP:35357/v2.0/tokens

   export KID=<<Token from Keystone>>

</nowiki></pre> }

Get the Tenant ID from OpenStack

Get it from the dashboard Get it from the command line

{
    export TEN_ID=<tenant ID>

}

 Get a valid Tenant User

{
    export OS_USER=<open stack user name>

} This requirement will go by having an OCCI-specific authentication middleware

OCCI-ness

The examples below use the OCCI header format for terseness, however the recommended format is the OCCI text body format (Content-Type: text/plain).

See What Can be Provisioned

{
    curl -v -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -X GET localhost:8787/-/

}

Create a VM

{
    curl -v -X POST localhost:8787/compute/ -H 'Category: compute; scheme="http://schemas.ogf.org/occi/infrastructure#"; class="kind"' -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: itsy; scheme="http://schemas.openstack.org/template/resource#"; class="mixin"' -H 'Category: cirros-0.3.0-x86_64-uec; scheme="http://schemas.openstack.org/template/os#"; class="mixin"'

}

  • **Note**: you can supply the admin password and/or public ssh key pairs in this request*

For ease of this OCCI exercise, place the VM id into a shell variable e.g.

{
    export VM=d54b4344-16be-486a-9871-2c566ef2263d

}

Get a Listing of VMs

{
    curl -v -X GET localhost:8787/compute/ -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER

}

Get an Individual VM's Details

{    
    curl -v -X GET localhost:8787/compute/$VM -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER

}

Execute a Stop Action Upon a VM

{
    curl -v -X POST "localhost:8787/compute/$VM?action=stop" -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: stop; scheme="http://schemas.ogf.org/occi/infrastructure/compute/action#"; class="action"'

}

Execute a Start Action Upon a VM

{
    curl -v -X POST localhost:8787/compute/$VM?action=start -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: start; scheme="http://schemas.ogf.org/occi/infrastructure/compute/action#"; class="action"'

} _Note: this will probably result in an error state. Currently looking into the issue._

Create Some a Block Storage Volume

{
    curl -v -X POST localhost:8787/storage/ -H 'Category: storage; scheme="http://schemas.ogf.org/occi/infrastructure#"; class="kind"' -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'X-OCCI-Attribute: occi.storage.size = 1.0'

} For ease of this OCCI exercise, place the volume id into a shell variable e.g.

{
    export VOL=1

}

Show the Volume Details:

{
    curl -v -X GET localhost:8787/storage/$VOL -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER

}

Link and Associate that Volume to the New Instance

{
    curl -v -X POST localhost:8787/storage/link/ -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: storagelink; scheme="http://schemas.ogf.org/occi/infrastructure#"; class="kind"' -H 'X-OCCI-Attribute: occi.core.source="http://localhost:8787/compute/'$VM'"' -H 'X-OCCI-Attribute: occi.core.target="http://localhost:8787/storage/'$VOL'"' -H 'X-OCCI-Attribute: occi.storagelink.deviceid="/dev/vdc"' -H 'Content-Type: text/occi'

} For ease of this OCCI exercise, place the volume id into a shell variable e.g.

{
    export VOL_LINK=aa49b313-9714-4cb3-92e3-13ab484235b

}

Inspect the Storage Link

{
    curl -v -X GET localhost:8787/storage/link/$VOL_LINK -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER

}

Unlink and disassociate that volume with the new instance

{
    curl -v -X DELETE localhost:8787/storage/link/$VOL_LINK -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Content-Type: text/occi'

}

Delete Storage Volume

{
    curl -v -X DELETE localhost:8787/storage/$VOL -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Content-Type: text/occi'

}

Scale Up a VM

Let's bump the current instance from itsy (128 RAM, 1 Core) to a bitsy flavour (256 RAM, 1 Core).

{
    curl -v -X POST localhost:8787/compute/$VM -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: bitsy; scheme="http://schemas.openstack.org/template/resource#"; class="mixin"'

} _Notes:_

  • This is a partial update with respect to OCCI.

Confirm the Scaled Up VM

This is OpenStack specific. Users must confirm that the scaled up VM is what they want and that their previous configuration is as expected.

{
    curl -v -X POST "localhost:8787/compute/$VM?action=confirm_resize" -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: confirm_resize; scheme="http://schemas.openstack.org/instance/action#"; class="action"'

}

Scale Down a VM

Let's reduce the current instance from bitsy (256 RAM, 1 Core) to a itsy flavour (128 RAM, 1 Core).

{
    curl -v -X POST localhost:8787/compute/$VM -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: itsy; scheme="http://schemas.openstack.org/template/resource#"; class="mixin"'

} _Notes:_

  • This is a partial update with respect to OCCI.

Confirm the Scaled Down VM

This is OpenStack specific. Users must confirm that the scaled up VM is what they want and that their previous configuration is as expected.

{
    curl -v -X POST "localhost:8787/compute/$VM?action=confirm_resize" -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: confirm_resize; scheme="http://schemas.openstack.org/instance/action#"; class="action"'

}

Delete a VM

{
    curl -v -X DELETE localhost:8787/compute/$VM -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER

}

Update a VM: Change the OS

As an example, let's use SmartOS as the new OS

{
    curl -v -X POST localhost:8787/compute/$VM -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: SmartOS; scheme="http://schemas.openstack.org/template/os#"; class="mixin"'

} _Notes:_

  • this is in effect a partial update.
  • this destroys any data directly related to the VM. It does not destroy connected volumes

Create a Security Group

{
    curl -v -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Content-Type: text/occi' -H 'Category: my_grp; scheme="http://www.mystuff.org/sec#"; class="mixin"; rel="http://schemas.ogf.org/occi/infrastructure/security#group"; location="/mygroups/"' -X POST localhost:8787/-/

}

- Follows the OpenStack model. Groups of rules are associated with a compute resource.
- A security group is associated with a compute. This is done by the addition or removal of a mixin to/from a compute instance
- Security rules can be added and removed to a security group (mixin). This is done by (dis)associating the rule with the group (mixin). Rules can be created and deleted.

List Security Groups

{
    curl -v -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Content-type: text/occi' -H 'Accept: text/plain' -H 'Category: group; scheme="http://schemas.ogf.org/occi/infrastructure/security#"; class="mixin"' -X GET localhost:8787/-/ 

}

'Note': this will only return the specified category in the request and not the related categories. This will be fixed in pyssf in the next release.

Create a Security Rule

To do so specify the rule parameters (as a Kind) and the group the rule (as a mixin) is to belong to. This associates the rule with the respective group. Let's add a rule for inbound SSH traffic to a security rule group. This group can then be supplied when provisioning VMs. The group can also be applied an already provisioned VM.

{    
    curl -v -X POST localhost:8787/network/security/rule/ -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: my_grp; scheme="http://www.mystuff.org/sec#"; class="mixin"' -H 'Category: rule; scheme="http://schemas.openstack.org/occi/infrastructure/network/security#"; class="kind"' -H 'X-OCCI-Attribute: occi.network.security.protocol = "TCP"' -H 'X-OCCI-Attribute: occi.network.security.to = 22' -H 'X-OCCI-Attribute: occi.network.security.from = 22' -H 'X-OCCI-Attribute: occi.network.security.range = "0.0.0.0/24"'

} For ease of this OCCI exercise, place the volume id into a shell variable e.g.

{
    export RULE=1233323

}

List the Associated Rules/Compute Resources to a Group

{
    curl -v -X GET localhost:8787/mygroups/ -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER

}

Get Security Rule's Details

{
    curl -v -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -X GET http://localhost:8787/network/security/rule/$RULE

}

Delete a Security Rule

{
    curl -v -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -X DELETE http://localhost:8787/network/security/rule/$RULE

}

Delete a Security Group

{
    curl -v -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Content-Type: text/occi' -H 'Category: my_grp; scheme="http://www.mystuff.org/sec#"; class="mixin"' -X DELETE localhost:8787/-/

}

Create a Secured VM with a Security Group

{
    curl -v -X POST localhost:8787/compute/ -H 'Category: compute; scheme="http://schemas.ogf.org/occi/infrastructure#"; class="kind"' -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: itsy; scheme="http://schemas.openstack.org/template/resource#"; class="mixin"' -H 'Category: cirros-0.3.0-x86_64-uec; scheme="http://schemas.openstack.org/template/os#"; class="mixin"' -H 'Category: my_grp; scheme="http://www.mystuff.org/sec#"; class="mixin"'

}

List Floating IP Pools

Request the Query Interface. Optionally filter on pool mixin.

{
    curl -v -X GET localhost:8787/-/ -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER

}

Allocate Floating IP to VM

{
    curl -v -X POST "localhost:8787/compute/$VM?action=alloc_float_ip" -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: alloc_float_ip; scheme="http://schemas.openstack.org/instance/action#"; class="action"' -H 'X-OCCI-Attribute: org.openstack.network.floating.pool="nova"'

}

View Allocated Floating IP

GET the VM and inspect the `org.openstack.network.floating.ip` attribute

Deallocate Floating IP to VM

{
    curl -v -X POST "localhost:8787/compute/$VM?action=dealloc_float_ip" -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: dealloc_float_ip; scheme="http://schemas.openstack.org/instance/action#"; class="action"'

}

Current Issue

OS allows multiple floating IPs per server. Current OCCI implementation only allows one IP per compute resource. Multiple IP support will be implemented and can be done by links, once pyssf supports arbitrary values in the Link category.

Change VM Administrative (root) Password

Note to use this functionality the `libvirt_inject_password` parameter must be set to `True` in `/etc/nova/nova.conf`

Issue the following action:

{
curl -v -X POST "localhost:8787/compute/$VM?action=chg_pwd" -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: chg_pwd; scheme="http://schemas.openstack.org/instance/action#"; class="action"' -H 'X-OCCI-Attribute: org.openstack.credentials.admin_pwd="new_pass"'

}

Create a Image from an Active VM

{
curl -v -X POST "localhost:8787/compute/$VM?action=create_image" -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: create_image; scheme=" http://schemas.openstack.org/instance/action#; class="action"' -H 'X-OCCI-Attribute: org.openstack.snapshot.image_name="awesome_ware"'

}

Work in Progress

Integration of Quantum

Create a Network

{
    curl -v -X POST localhost:8787/network/ -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: network; scheme="http://schemas.ogf.org/occi/infrastructure#"; class="kind"' -H 'occi.network.label="www"'

} For ease of this OCCI exercise, place the volume id into a shell variable e.g.

{
    export NET=aa49b313-9714-4cb3-92e3-13ab484235b

}

Get Network Details

{
    curl -v -X GET localhost:8787/network/$NET_ID -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER

}

Attach a VM to the Network

{
    curl -v -X POST localhost:8787/networklink/ -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER -H 'Category: networkinterface; scheme="http://schemas.ogf.org/occi/infrastructure#"; class="kind"' -H 'X-OCCI-Attribute: occi.core.source="http://localhost:8787/compute/'$VM'"' -H 'X-OCCI-Attribute: occi.core.target="http://localhost:8787/network/'$NET'"'

}

Delete Network

{
    curl -v -X DELETE localhost:8787/network/$NET_ID -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-Auth-Tenant-Id: '$TEN_ID -H 'X-Auth-User: '$OS_USER

}

TODOs

  • Write an OCCI registry backend that reads from the OS Nova DB
  • Integrate CDMI volume management like in rOCCI
  • Import/Export OVF functionality
  • Support meta-data and file injection