NovaImageBuilding

Nova Image Building

This page considers what work is required in order to enable OpenStack to be used as a service by user tools wishing to build cloud images for arbitrary operating system. It is not about defining or providing an official OpenStack blessed image building service, although it is expected that work done for the former will facilitate the design & creation of the latter.

Background: Oz

There are many tools for building cloud images, but most of them are tailored towards specific operating systems. This is typically due to an approach focused on installing packages in a chroot environment, or similar. The Oz project is somewhat different in that it aims to be a tool that is portable to building images for any common operating system. The key architectural design of Oz is to make use of KVM to run the operating systems native installer, using its built in automation capabilities. For Fedora/RHEL this implies using kickstart scripts, for Debian/Ubuntu this would use the installer answer files, for Windows this involves SIF files, and so it. It has been ported to all commonly used operating systems.

Of course Oz requires a host which has KVM available in order to do its, unless the user is happy to fallback to the slower QEMU. The latter is not really a practical option speedwise. Requirement for KVM means Oz requires a Linux host of some kind. Consumers of OpenStack services may be using Windows or OS-X, or any number of operating systems. It is desirable that they have an easy way to build images to be used with OpenStack. A key observation is that while we can't assume the consumer's machine can run KVM, OpenStack of course has virtualization capabilities. There is nothing especially KVM specific in what Oz does, it was merely the best choice for its original developer.

Oz' approach to building images is as follows

• Obtain a installer automation file for building a JEOS image. The Oz codebase has such automation files available for most common operating systems
• Modify the original installer ISO image to include the automation file
• Determine the optimal hardware to run the guest OS installer in KVM. The Oz codebase has a set of classes which determine hardware for each OS
• Launch a KVM guest with an empty disk and the installer ISO
• Wait for the guest to shutdown (indicating completion of install), monitoring disk/net I/O to determine if it hangs
• Boot the resulting JEOS image, SSH into it and install extra packages required by the final appliance
• Convert the final raw image into qcow2 format

It is possible to see from this, that it might be possible to re-write Oz to make use of Nova/Glance APIs to accomplish the work it currently does using local KVM instances.

Background: libosinfo

In the years since virtualization has become commonplace, a great many applications have been developed to manage virtual machines. Many of these applications have duplicated each others functionality in a many areas. For example, every application needs to figure out the optimal hardware devices for running a particular OS as a guest on a particular hypervisor. Any application may want to know the recommended / minimum disk/ram resources to allocate for a particular guest. Any application which wants todo automated installs of new guest OS images, has to write out the same installer automation scripts. Any application may want to know how to identify which OS a particular ISO image corresponds to, and whether it is an installer ISO or a LiveCD image or something else. Any application may want to know where to find installer trees or appliance images online. The applications GNOME Boxes, oVirt, virt-manager and virt-install already duplicate this information in their respective codebases.

The libosinfo project is an attempt to remove this extensive duplication of work / knowledge from virtualization management applications. At its core it provides a database containing information about operating systems and hypervisors. The database includes details of supported hardware devices, minimum/recommended hardware devices, metadata required to identify ISO images, templates for creating installer automation scripts, and much more besides. Alongside the database is a library API for querying information in the database. The database and library have no dependancy on any specific virtualization technology, intending to be completely agnostic and focus exclusively on being an information source.

The libosinfo library is already used by the GNOME Boxes tool for doing 100% automated installations of Fedora / RHEL and Windows operating systems on KVM. Work is also underway or planned, for libosinfo to be used by virt-manager, virt-install and oVirt applications.

Much of the work/information that the Oz tool does/has related to configuring a guest OS for automated installation can be replaced by usage the libosinfo database.

Image building requirements

Considering what the Oz tool does and what libosinfo provides, the requirements that need to be satisfied to build OS appliance images are fairly minimal

• Ability to boot a virtual machine from a kernel + initrd + provide custom kernel command line args
• Ability to boot a virtual machine from an ISO image
• Ability to boot a virtual machine with 1 harddisk, 1 cdrom and either an additional cdrom or floppy or usb disk
• Ability to download the harddisk image after shutting down the VM
• Ability to monitor network & disk I/O in the virtual machine
• Ability to upload installer ISO images
• Ability to upload floppy/iso/usb disk images
• Ability to specify custom hardware models for disk, cdrom, floppy and network devices

As can be seen from this list, Nova and Glance can already satisfy many of these requirements and those which are missing are fairly easy to address. Specifically what is missing in Nova

• Ability to specify custom kernel command line args when booting a kerne/initrd (blueprint custom-kernel-args)
• Ability to use > 1 CDROM with VM (blueprint flexible-block-device-config)
• Ability to specify disk/network hardware models (blueprint libvirt-custom-hardware)
• Ability to download an image from Nova
• Ability to copy an image back into glance
• Ability to use floppy or usb disks (blueprint flexible-block-device-config)

What is nice to have in Glance

• Ability to obtain ISO image header metadata fields (blueprint iso-image-metadata)

Image building design

With the above gaps addressed it will be practical to create an appliance building tool that uses OpenStack for its work.

• Obtain an installer JEOS automation script for the desired OS from libosinfo
• Build a cdrom, floppy or usb disk image containing the automation script
• Upload the custom built image to glance
• Upload the OS installer ISO to glance
• Boot a VM in nova using a new empty disk, the installer ISO and automation image
• Monitor progress in Nova using diagnostics API to look at disk/net I/O, until VM shuts down
• Boot the VM image again and let cloud-init customize the JEOS image to the desired role then shutdown again
• Copy VM disk image back into glance or download to client

While such a tool is easily run standalone by a client user, its codebase could easily be leveraged by an OpenStack server side service for image building.