PCI Hotplug Support - API and Features

Background

In current PCI passthrough implementation, a pci device is only allowed to be assigned to a instance while the instance is being created, it is not allowed to be assigned or removed from the instance while the instance is running or stop.

Concept

1. pci_alias is configured in nova.conf on the controller node, for example,

   pci_alias={"vendor_id":"8086", "product_id":"10c9", "name":"a1"}

2. pci_passthrough_whitelist is configured in nova.conf on the compute node,for example,

   pci_passthrough_whitelist=[{ "vendor_id":"8086","product_id":"10c9"}]

Current implementation

1. Config pci_passthrough_whitelist on the compute node and pci_alias on the controller node, which means the devices you permit to be assigned to vms.
2. Create a floavor and set the "pci_passthrough:alias",for example,

   nova flavor-key gzm set "pci_passthrough:alias"="a1:2".
   
   one more request is supported too,like this,
   nova flavor-key gzm set "pci_passthrough:alias"="a1:2,b1:1".

3. Show details of the flavor, you'll see ,

   nova flavor-show gzm
   +----------------------------+--------------------------------------+
   | Property                   | Value                                |
   +----------------------------+--------------------------------------+
   | name                       | gzm                                  |
   | ram                        | 1024                                 |
   | OS-FLV-DISABLED:disabled   | False                                |
   | vcpus                      | 2                                    |
   | extra_specs                | {u'pci_passthrough:alias': u'a1:2'}  |
   | swap                       |                                      |
   | os-flavor-access:is_public | True                                 |
   | rxtx_factor                | 1.0                                  |
   | OS-FLV-EXT-DATA:ephemeral  | 0                                    |
   | disk                       | 10                                   |
   | id                         | 62686729-aa43-4fc6-99e0-b50bb84e2f60 |
   +----------------------------+--------------------------------------+

4. Create instance using the flavor above,then you'll see two pci devices have been assigned to the instance and you won't see them on the old compute node.
5. Query pci device in nova DB, you'll see,

   mysql> select * from pci_devices;
   +---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+--------------------------------------+
   | created_at          | updated_at          | deleted_at | deleted | id | compute_node_id | address      | product_id | vendor_id | dev_type | dev_id           | label           | status    | extra_info | instance_uuid                        |
   +---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+--------------------------------------+
   | 2014-02-22 06:59:16 | 2014-02-22 07:01:59 | NULL       |       0 |  1 |               1 | 0000:02:00.0 | 10c9       | 8086      | type-PCI | pci_0000_02_00_0 | label_8086_10c9 | allocated | {}         | 294c0426-693a-4514-907e-3eb17dc1de1c |
   | 2014-02-22 06:59:16 | 2014-02-22 07:01:59 | NULL       |       0 |  2 |               1 | 0000:02:00.1 | 10c9       | 8086      | type-PCI | pci_0000_02_00_1 | label_8086_10c9 | allocated | {}         | 294c0426-693a-4514-907e-3eb17dc1de1c |
   | 2014-02-22 06:59:16 | NULL                | NULL       |       0 |  3 |               1 | 0000:03:00.0 | 10c9       | 8086      | type-PCI | pci_0000_03_00_0 | label_8086_10c9 | available | {}         | NULL                                 |
   | 2014-02-22 06:59:16 | NULL                | NULL       |       0 |  4 |               1 | 0000:03:00.1 | 10c9       | 8086      | type-PCI | pci_0000_03_00_1 | label_8086_10c9 | available | {}         | NULL                                 |
   +---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+--------------------------------------+

The Hotplug APIs Based on Current Implementation

API for hotplug a pci device

V2 API specification: POST: v2/{tenant_id}/servers/{server_id}/os-pci_attachments V3 API specification: POST: v3/servers/{server_id}/os-pci_attachments

JSON Request: {

      "pciAttachment":
      {
         "pci_passthrough:alias":"a1:2"
      }

}

JSON Response: {

   "pci_devices": [
       {
            "attachment_id":"a26887c6-c47b-4654-abb5-dfadf7d3f803",
            "serverId": "4d8c3732-a248-40ed-bebc-539a6ffd25c0",
            "dev_id":"pci_0000_02_00_0",
            "address":"0000:02:00.0",
            "inner_address":"0000:03:00.0",
            "vendor_id":"8086",
            "product_id":"10c9"
       },
       {
            "attachment_id":"a26887c6-c47b-4654-abb5-dfadf7d3f704",
            "serverId": "4d8c3732-a248-40ed-bebc-539a6ffd25c0",
            "dev_id":"pci_0000_02_00_1",
            "address":"0000:02:00.1",
            "inner_address":"0000:03:01.0",
            "vendor_id":"8086",
            "product_id":"10c9"
       }
   ]

}

API for hotunplug a pci device

V2 API specification: DELETE: v2/{tenant_id}/servers/{server_id}/os-pci_attachments/{attachment_id} V3 API specification: DELETE: v3/servers/{server_id}/os-pci_attachments/{attachment_id}

HTTP response codes: v2: Normal HTTP Response Code: 200 on success v3: Normal HTTP Response Code: 202 on success

API for query plugged pci devices of an instance

V2 API specification: GET: v2/{tenant_id}/servers/{server_id}/os-pci_attachments V3 API specification: GET: v3/servers/{server_id}/os-pci_attachments