Nova/USB device&USB controller

USB device and USB controller features

Background

Currently, nova has supported function of pci-passthrough, but hasn't supported usb-passthrough. I think usb-passthrough is also a important and necessary function especially in private cloud.

User scenarios

1. When a user transfers money online, him may need a usb-key provided by bank for authentication.

2. When a user runs a ERP software, him may need a usb-key provided by software provider for authentication.

3. When a user wants to copy data to a removable media for business trip him may want a usb data disk.

Technical verification

1. Test case

    Creating vm with different type and number usb controller and usb device.

2. Xml definition

  (1)Sample of usb controller:
   <controller type='usb' index='2' model='piix3-uhci'/>
   <controller type='usb' index='1' model='ehci'/>

  (2)Sample of pass-through usb devices
   <hostdev mode='subsystem' type='usb'>
    < source>
     <vendor id='0x136b'/>
     <product id='0x0003'/>
     <address bus='2' device='2'/>
    </source>
    <address type='usb' bus='1' port='1'/>
   </hostdev>

  (3)Sample of emulated usb disks
     <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      < source file='/home/vms/usb/sdb.qcow2'/>
      <target dev='sdb' bus='usb'/>
      <address type='usb' bus='1' port='1'/>
    </disk>

3. The test results

  (1)Usb controllers have a maximum port restriction. Uhci controller support maxmum of 2 while ehci controller support maxmum of 6.
  (2)If create vm without usb controller, qemu will default create a piix3-usb-uhci controller but not creating other type's controllers by default.
  (3)If create vm with usb device but no usb controller, the usb device will default be attached to the default uhci controller, but speed may mismatch.
  (4)There are two ways for supporting more usb devices. 1. Add cascaded hubs and connect usb devices to them. 2.Create more controllers and connect usb devices to different controllers.
  (5)When create vm with a pair of usb hub and ehci controller, the vm crashed. Qemu thrown a error(why?)

Proposed solution

1. Use cases

   Case 1:
   (1)Admin/User request to create a flavor with usb controllers(key arguments: type, count).
   (2)System creates a flavor.
   (3)Admin/User request to create a vm with flavor created above. 
   (4)System first validates usb arguments specified in flavor, if valid then create a vm with usb controllers.

   Case 2:
   (1)Admin/User request to create a flavor with usb devices(key arguments: unique identifiers of devices).
   (2)System creates a flavor.
   (3)Admin/User request to create a vm with flavor created above. 
   (4)System first validates if usb devices are exist, available, and in the same host, if valid then create a vm with usb devices in the host. The usb devices will be attached to the default uhci controller created by qemu .

   Case 3:    
   (1)Admin/User request to create a flavor with usb controllers(key argument: type, count) and usb devices(key arguments: identifiers and usb controller type).
   (2)System creates a flavor.
   (3)Admin/User request to create a vm with flavor created above. 
   (4)Admin/User first validate if usb devices are exist, available,in the same host, and matching with controllers(type, free port and so on), if valid then create a vm with usb devices and usb controllers in the host. The usb devices will be attached to the controllers by type.

2. System requirements and things should be considered for usb controller(An optional Implementation):

   (1)System supports creating a flavor with usb controller.
       Add a property about usb controller to extra_specs field of flavor. The scheme may like {"usb_controllers":[{"type":"ehci", "count": 1}]} 
       Note: Because libvirt/qemu will create a default pii3-uhi controller, so system will create all type's controller but pii3-uhci.

   (2)System supports constructing usb controller xml in libvirt driver and config.

   (3)System supports managing usb controller index/port<--->usb device's relationship so that we can attach usb device to unique usb controller and port. 
        The usb controller schema may like
        {
          "usb_controller_requests":
          [
            {
               "type": "ehci",
               "count": 2,
               "usb_controllers":
               [
                 {
                   "index": 1,
                   "ports":
                   [
                     {
                        "port": 1,
                        "usb_device": "dev_address"
                     }
                   ]
                 } 
               ]
            }
          ]
        }

   (4)System supports saving usb controller information in instance_system_metadata table so that the next starting of vm will create usb controllers.

3. System requirements and things should be considered for usb device(An optional Implementation):

   (1)System supports discovering of usb devices and saving them to DB.
        The main process can refer to pci-passthrough:
        <1>Using white-list and lsusb function to retrieve available usb devices of a host.
        <2>Adding a table usb_devices in DB to save usb devices information.
        <3>Adding auto-update process to synchronize information between compute node and DB.

   (2)System supports creating vm with usb devices.
        <1>Adding a property about usb device to extra_specs field of flavor. The scheme may like {"usb_devices":[{"node_id":"1", "dev_id": "usb_2_1"}]}
              Note: Because most user scenarios should be admin/user specify which usb device to which vm, so it's necessary to exposure physical information to admin/user. The mechanism of automatically finding compute node to provide pci devices is not comfortable here.
        <2>System schedulers request to appropriate compute node.
              Add a UsbDeviceFilter.py filter for filtering compute node. 
        <3>System allocates usb controllers for usb devices.
              If not specifying usb controllers in request, libvirt/qemu will create a default uhci controller for using.
              If specifying usb controllers(type and count) in request, system will choose a matched controller and port for every usb device. If controllers are not matched or ports are not enough then system throws an error. If some controllers and ports are used then the structure usb_controller_requests described above will be updated.

   (3)System supports saving usb devices information in instance_system_metadata table so that the next starting of vm will automatically attach usb devices.
       The usb device schema may like
       {
          "usb_devices_requests":
          [
            {
               "dev_address": "usb_2_1"
            }
          ]
        }

   (4)System supports attaching usb devices when starting a vm with usb devices.
        System will use usb_controller_requests to find relationship between usb devices and usb controller/ports and construct xml configuration.

   (5)System supports detaching usb devices when suspending a vm with usb devices, and re-attaching usb devices when resuming the vm.

   (6)System supports detaching usb devices before making snapshot of a vm and re-attaching usb devices after making snapshot. System also support the protection of usb device when using snapshot to resume a vm.

   (7)System supports reporting error when migrating vm with usb device.
        If a vm has usb devices attached, it is not allowed to migrate.