Jump to: navigation, search

LibvirtAPI

Revision as of 14:57, 14 April 2011 by Brian (talk)

Libvirt API

The current method for connecting to a libvirt-based hypervisor, through nova/virt/libvirt_conn.py, is functional but is lacking support for a number of great libvirt features. Some of the items we are missing are optimizations and others are simply new features that we could provide to the world if we take a step back and look at the design of the LibvirtConnection class.

Current Areas of Weakness

Long/Untestable Methods 
The number of unit-tests for LibvirtConnection is close to 0. There are a few functional tests, but individual unit tests are non-existent. Having long, complex function such as to_xml, destroy, spawn, lead to poor testability and inflexibility as the libvirt project grows and changes. The better we architect the link between OpenStack and libvirt, the more features we can provide in a stable fashion.
Database/Context Access 
I'm very much open to debate on this, but I feel like virt drivers have no business connecting to the database...nor should they know or care what a nova 'context' is. Contexts should be handled at the API level and database interactions need to occur in the compute manager. Right now there is an odd false security feeling because the only time context is used in the driver is to call "context.get_admin_context()".
KVM/QEMU-based Design 
The most popular backend for libvirt (I lack any sort of evidence for this statement.) is KVM/QEMU. As such it feels like a lot of the code is very much geared towards that connection type. Libvirt supports the following backends:
  • LXC - Linux Containers
  • OpenVZ
  • QEMU
  • Test - Used for testing
  • UML - User Mode Linux
  • VirtualBox
  • VMware ESX
  • VMware Workstation/Player
  • Xen
As such we should design our libvirt interface to allow for support of at least this list of hypervisor backends. One thing to note is that although libvirt supports the above 'drivers', using native APIs (such as the Xen XMLRPC API) is almost always going to provide more functionality vs. libvirt. This is why we have the XenAPI (nova/virt/xenapi/*) implementation as well as the libvirt implementation.
LoopingCall Inefficiencies 
The concept and initial implementation of "LoopingCall" is sound, but largely unneccesary when dealing with libvirt. Currently LoopingCall is used in 4 places (reboot, rescue, spawn, live_migration) and should be used in a fifth place (destroy). All of these calls can be replaced by linking in with libvirt's event notification service, which provide an asynchrounous way to detect when a VM changes state. No more polling! It's currently unclear to me if this works for all libvirt drivers, or if it only works with a subset (such as just QEMU).
Seperate Out Firewall Code 
While not directly related to LibvirtConnection, the netfilter/iptables firewall implementations as well as the baseclass is located in nova/virt/libvirt_conn.py and really should be moved out to someplace like nova/virt/firewall.py.
Same-Host-Assumption 
LibvirtConnection should seemingly work for qemu+ssh:// connections, but it will not due to functions which check for resources on the local host. Support for controlling non-local hypervisors would be a "wishlist" item for me, but something that should be in the back of the mind of whomever re-tools the code.

Proposal for Fixing

While this doesn't have to happen all at once and the classes don't need to live in the same file, cleaning up the code in some places is as easy as abstracting out libvirt-driver-based logic into relevant subclasses. An LXC connection might require "spawn" to create/mount 'rootfs', where QEMU doesn't.

Splitting up logic on a driver-connection-type-basis:

#!highlight python

class LibvirtConnection(driver.ComputeDriver):
    pass

class LinuxContainers(LibvirtConnection):
    pass

class UserModeLinux(LibvirtConnection):
    pass

class Xen(LibvirtConnection):
    pass

class Qemu(LibvirtConnection):
    pass


Support callbacks rather than polling:

#!highlight python

class LibvirtConnection(driver.ComputeDriver):
    def _process_event(self, domain, event, detail, extra): pass
    def _on_instance_start(self, instance_name): pass
    def _on_instance_stop(self, instance_name): pass
    def _on_instance_suspend(self, instance_name): pass
    def _on_instance_resume(self, instance_name): pass
    ...
    def connect(...):
        """Connect to a libvirt hypervisor and process callbacks."""
        ...
        connection.domainEventRegister(self._process_event,None)