Difference between revisions of "Ironic-python-agent"

Revision as of 21:28, 24 March 2014

Motivation

At the moment, Ironic uses exactly the same deployment ramdisk as the Nova baremetal driver does. The current deploy model exposes a hard drive via iSCSI. The conductor driver then mounts this volume and writes an OS image to it.

We believe that a deployment ramdisk should be able to do more than just expose an iSCSI volume. This ramdisk should have direct access to the hardware, and be able to:

Erase disks
Partition disks
Install bootloaders
Install an OS image
Update firmware
Configure RAID
And more...

Ironic-python-agent allows this functionality to be exposed to Ironic.

What is it?

The agent is a small python application that is meant to be embedded in a ramdisk.
The target machine boots this ramdisk, which starts the agent process.
The agent then exposes a REST API that Ironic uses to interact with the agent.
The API calls pluggable backends to interact with the machine.

Architecture

The REST API will expose a set of fine-grained commands, such as "wipe disk" and "write image". The REST API will also expose an endpoint to send a list of commands to be run asynchronously. The agent will register as being on a given node, and get that node's UUID, on startup, by POSTing to the Ironic API The agent will heartbeat to Ironic on a given interval. All functionality, with the exception of the heartbeater and API, should be fully pluggable.

Full Ironic system architecture when using the agent.

Pluggable Backends

Preliminary list of backends available:

Hardware (can handle both inventory and manipulation)
Provisioning (bootloader/imaging/configdrive)

This could maybe be split into separate drivers, but is not yet necessary.

Workflows - there are two pieces to this:

"Canned" workflows: A single endpoint that runs multiple tasks, defined by the driver. Example: a "deploy" endpoint.
Arbitrary workflows: A single endpoint that runs an arbitrary list of multiple tasks.

Agent driver

The Ironic conductor will run a driver corresponding to this agent. This driver will live in the Ironic tree as an official driver. This driver may eventually be merged into Ironic's PXE driver - it will depend how much is common between the two.

@@ Line 1: / Line 1: @@
+==Motivation==
+At the moment, Ironic uses exactly the same deployment ramdisk as the Nova baremetal driver does. The current deploy model exposes a hard drive via iSCSI. The conductor driver then mounts this volume and writes an OS image to it.
+We believe that a deployment ramdisk should be able to do more than just expose an iSCSI volume. This ramdisk should have direct access to the hardware, and be able to:
+* Erase disks
+* Partition disks
+* Install bootloaders
+* Install an OS image
+* Update firmware
+* Configure RAID
+* And more...
-==Motivation==
+Ironic-python-agent allows this functionality to be exposed to Ironic.
-At the moment Ironic uses exactly the same deployment ramdisk as nova baremetal driver does. Current implementation of deployment agent exposes a hard drive via ISCSI and then conductor driver mounts it and copies OS image to it. However, there are a number of other possible much more scalable mechanisms such as downloading OS images via HTTP or BitTorrent or any kind of multicast protocols. Besides, default Ironic approach at the moment is to avoid installing bootloader on a node hard drive, but it is better to have such a possibility as well. Another point is that user often wants to have a comprehensive disk partitioning scheme with support of LVM, MD (software RAID), plain partitions, msdos and gpt partition tables, etc. User often needs to have a convenient mechanism of updating firmware, configuring hardware RAIDs, etc. As a conclusion, we need to have a comprehensive pluggable agent to deal with all that stuff.
+==What is it?==
+* The agent is a small python application that is meant to be embedded in a ramdisk.
+* The target machine boots this ramdisk, which starts the agent process.
+* The agent then exposes a REST API that Ironic uses to interact with the agent.
+* The API calls pluggable backends to interact with the machine.
-==What is ironic python agent?==
+==Architecture==
-* It supposed to be a built-in component of so called utility ramdisk.
+The REST API will expose a set of fine-grained commands, such as "wipe disk" and "write image".
-* Ironic conductor drivers are supposed to be able to interact with this component via REST API.
+The REST API will also expose an endpoint to send a list of commands to be run asynchronously.
-* It is supposed to use JSON as a transport layer.
+The agent will register as being on a given node, and get that node's UUID, on startup, by POSTing to the Ironic API
-* It is supposed to have a pluggable architecture in order to make user able to easily add their own functionality (whether gathering data about the hardware or applying changes to it). Plugins are supposed to be JSON driven components (python modules, bash scripts, what ever).
+The agent will heartbeat to Ironic on a given interval.
-* Agent itself is supposed to route the incoming JSON to the appropriate plugin, and to route the response back via the transport layer.
+All functionality, with the exception of the heartbeater and API, should be fully pluggable.
-==Preliminary architecture==
+[https://www.lucidchart.com/publicSegments/view/532335f8-edc4-40c2-804e-11060a009433/image.pdf Full Ironic system architecture when using the agent.]
-* Drow.io editable diagram https://drive.google.com/file/d/0BybDDjx4oqkYOVZ5a2hURUViblk/edit?usp=sharing
-[[File:Ironic python agent.png|650px]]
-Full system architecture when using the ironic python agent: https://www.lucidchart.com/publicSegments/view/532335f8-edc4-40c2-804e-11060a009433/image.pdf
+==Pluggable Backends==
+Preliminary list of backends available:
+* Hardware (can handle both inventory and manipulation)
+* Provisioning (bootloader/imaging/configdrive)
+:* This could maybe be split into separate drivers, but is not yet necessary.
+* Workflows - there are two pieces to this:
+:* "Canned" workflows: A single endpoint that runs multiple tasks, defined by the driver. Example: a "deploy" endpoint.
+:* Arbitrary workflows: A single endpoint that runs an arbitrary list of multiple tasks.
-Here are etherpad [https://etherpad.openstack.org/p/IronicPythonAgent notes]
+==Agent driver==
-=On Ironic conductor side=
+The Ironic conductor will run a driver corresponding to this agent. This driver will live in the Ironic tree as an official driver. This driver may eventually be merged into Ironic's PXE driver - it will depend how much is common between the two.
-Currently Ironic pxe driver is able to interact with only nova baremetal agent. It is supposed that all that stuff about provisioning and using comprehensive agent will also be implemented as a part of Ironic pxe driver (NOT separate driver).