GuestFilesystemParity

• Created: Tue, 20 Dec 2011 13:32:16 -0800
• Contributors: Joshua Harlow

Guest filesytem parity

Summary

We propose a blueprint to help openstack perform actions with/on images in a uniform manner. This would mean unifying (or abstracting) the different methods via ephemeral volumes are created, mounted, and modified (ie network injection). This is useful since not all methods are available on all operating systems (most importantly rhel6 lacks qemu-nbd but has guestfs and vice versa). We would also like to propose a possible solution to the long term problem of unifying all image modification and introspection methods into one utility, if this makes sense for the community.

Rationale

Openstack users would like to be able to develop and run on various linux distributions, with RHEL (and derivatives) being one of primary distributions that many companies and people use, as well as ubuntu being the other major distribution. Unfortunately those two major distributions right now have different priorities and different utilities available for mounting and modifying guest images. We would like to make the different operating systems work in a single abstracted manner so all actions with images eventually go through one well defined api. This will make it easier to define what the valid image operations are and what a distribution needs to do to have those features thus enabling a wider variety of distribution support and making everyone's lives better and a cleaner design.

Design

From a high level this would involve the following:

1. Modularize injection code to have a path for ubuntu and a path for libguestfs, with the end goal that once libguestfs is in "native" ubuntu (v12?) that the standard path will be the libguestfs path. This will involve modularizing all code that does image injection into a plugin layer with a plugin for ubuntu (which will use qemu-nbd or loopback devices) and a plugin for libguestfs (that will use the python guestfs library for injection/modification)
2. Modularize any other code that calls into qemu-img or qemu-nbd on nova compute and hide it behind the plugin layers via an abstraction instead of executing qemu system calls directly. This may involve the following other actions.
   1. Image resizing
   2. Image creation (ephemeral and swap volumes)
3. Create network injection code that will work with RHEL and ubuntu since both of these operating systems perform network configuration in a different manner.
   1. Hide this behind an abstraction layer and use guestfs introspection (or config options) to determine which network type to use.

Expected Code Changes

Diablo branch (may change in essex):

nova/virt/disk.py
nova/virt/images.py
nova/virt/injector.py
nova/virt/interfaces.template
nova/virt/libvirt.xml.template
nova/virt/libvirt/connection.py
... (might encounter others)
... (new files for abstraction layer and implementations) 

Dependencies

• https://blueprints.launchpad.net/nova/+spec/package-stabilization (needed so that we know what packages to use for every distribution - in detail)

The following packages would need to be built and made available via a RHEL repo (may change for essex release):

Note: this list is what grid-dynamics has come up with, since there does not seem to be a listing of exactly what versions/packages are needed for each release.

python-libguestfs >= 1.7.17
(others as needed…)

One large issue is that rhel 6 and libguestfs by default require qemu-kvm, this dependency does not seem valid when using a xen dom0 compute node, since unless I am mistaken qemu-kvm can not be used in a dom0 of a xen hypervisor. If it can then this is not a problem.

Test/Demo Plan

In order to test this we will need to be able to run the current unit tests on ubuntu and on RHEL (and any other operating systems) and ensure that injection, image creation,.. is working according to those unit tests. If those unit tests do not exist or are themselves operating system specific we will need to create new unit tests that do not have this restriction and we will have to replace the previous unit tests.

As for integration tests we will need to ensure that the openstack hudson instance creates RHEL integration tests (with guests from RHEL 4.5 -> RHEL 6) and tests those to ensure functionality is working as expected as well as testing those same functionality points on other operating systems (ubuntu…). If those integration tests/E2E do not exist, we will have to make them. We will have to depend on unit tests, integration tests and any existing <nop>E2E</nop> tests to ensure that this is correct, and will be for future code changes that others make.

Migration Plan

Previously existing code should work with this new set of abstraction code. Since there is no existing RHEL approved implementation there is no need for a migration plan for that distro. It is the goal that all unit tests should work as expected since the ubuntu mechanism should still continue to function (qemu-nbd, loopback...) as much as possible until ubuntu 12 where we can remove that code. Or we can remove it earlier if we do not depend on ubuntu packages but provide our own...

Unresolved Issues

1. Who will build/maintain these packages that are not in the base RHEL 6 repository (see above)?
2. Should we switch to libguestfs right now and provide our own ubuntu packages?
   • This is easier in the long run and would remove some unneeded ubuntu specific code (but again might not work with xen)
3. How do we approach rhel to disconnect libguestfs from kvm so that xen dom0 can work (if this is needed)?

Contacts

harlowja@yahoo-inc.com