Difference between revisions of "Nova/pci hotplug"
(→Concept) |
(→The Hotplug APIs Based on Current Implementation) |
||
(15 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
− | =PCI Hotplug Support - API and | + | =PCI Hotplug Support - API and Features= |
==Background== | ==Background== | ||
Line 6: | Line 6: | ||
==Concept== | ==Concept== | ||
− | 1. pci_alias is | + | 1. pci_alias is configured in nova.conf on the controller node, for example,<br/> |
− | pci_alias={"vendor_id":"8086", "product_id":"10c9", "name":"a1"} | + | pci_alias={"vendor_id":"8086", "product_id":"10c9", "name":"a1"} |
− | 2. pci_passthrough_whitelist is | + | 2. pci_passthrough_whitelist is configured in nova.conf on the compute node,for example,<br/> |
− | pci_passthrough_whitelist=[{ "vendor_id":"8086","product_id":"10c9"}] | + | pci_passthrough_whitelist=[{ "vendor_id":"8086","product_id":"10c9"}] |
==Current implementation== | ==Current implementation== | ||
− | 1. Config pci_passthrough_whitelist on the compute node and pci_alias on the controller node, which means the devices you permit to be assigned to vms. | + | 1. Config pci_passthrough_whitelist on the compute node and pci_alias on the controller node, which means the devices you permit to be assigned to vms.<br/> |
− | 2. Create a floavor and set the "pci_passthrough:alias",for example, nova flavor-key gzm set "pci_passthrough:alias"="a1:2" | + | 2. Create a floavor and set the "pci_passthrough:alias",for example, <br/> |
− | 3. Show details of the flavor, you'll see , | + | nova flavor-key gzm set "pci_passthrough:alias"="a1:2". |
− | nova flavor-show gzm | + | |
− | +----------------------------+--------------------------------------+ | + | one more request is supported too,like this, |
− | | Property | Value | | + | nova flavor-key gzm set "pci_passthrough:alias"="a1:2,b1:1". |
− | +----------------------------+--------------------------------------+ | + | 3. Show details of the flavor, you'll see ,<br/> |
− | | name | gzm | | + | nova flavor-show gzm |
− | | ram | 1024 | | + | +----------------------------+--------------------------------------+ |
− | | OS-FLV-DISABLED:disabled | False | | + | | Property | Value | |
− | | vcpus | 2 | | + | +----------------------------+--------------------------------------+ |
− | | extra_specs | {u'pci_passthrough:alias': u'a1:2'} | | + | | name | gzm | |
− | | swap | | | + | | ram | 1024 | |
− | | os-flavor-access:is_public | True | | + | | OS-FLV-DISABLED:disabled | False | |
− | | rxtx_factor | 1.0 | | + | | vcpus | 2 | |
− | | OS-FLV-EXT-DATA:ephemeral | 0 | | + | | extra_specs | {u'pci_passthrough:alias': u'a1:2'} | |
− | | disk | 10 | | + | | swap | | |
− | | id | 62686729-aa43-4fc6-99e0-b50bb84e2f60 | | + | | os-flavor-access:is_public | True | |
− | +----------------------------+--------------------------------------+ | + | | rxtx_factor | 1.0 | |
− | 4. Create instance using the flavor above,then you'll see two pci devices have been assigned to the instance and you won't see them on the old compute node. | + | | OS-FLV-EXT-DATA:ephemeral | 0 | |
− | 5. Query pci device in nova DB, you'll see, | + | | disk | 10 | |
− | mysql> select * from pci_devices; | + | | id | 62686729-aa43-4fc6-99e0-b50bb84e2f60 | |
− | +---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+--------------------------------------+ | + | +----------------------------+--------------------------------------+ |
− | | created_at | updated_at | deleted_at | deleted | id | compute_node_id | address | product_id | vendor_id | dev_type | dev_id | label | status | extra_info | instance_uuid | | + | 4. Create instance using the flavor above,then you'll see two pci devices have been assigned to the instance and you won't see them on the old compute node.<br/> |
− | +---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+--------------------------------------+ | + | 5. Query pci device in nova DB, you'll see,<br/> |
− | | 2014-02-22 06:59:16 | 2014-02-22 07:01:59 | NULL | 0 | 1 | 1 | 0000:02:00.0 | 10c9 | 8086 | type-PCI | pci_0000_02_00_0 | label_8086_10c9 | allocated | {} | 294c0426-693a-4514-907e-3eb17dc1de1c | | + | mysql> select * from pci_devices; |
− | | 2014-02-22 06:59:16 | 2014-02-22 07:01:59 | NULL | 0 | 2 | 1 | 0000:02:00.1 | 10c9 | 8086 | type-PCI | pci_0000_02_00_1 | label_8086_10c9 | allocated | {} | 294c0426-693a-4514-907e-3eb17dc1de1c | | + | +---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+--------------------------------------+ |
− | | 2014-02-22 06:59:16 | NULL | NULL | 0 | 3 | 1 | 0000:03:00.0 | 10c9 | 8086 | type-PCI | pci_0000_03_00_0 | label_8086_10c9 | available | {} | NULL | | + | | created_at | updated_at | deleted_at | deleted | id | compute_node_id | address | product_id | vendor_id | dev_type | dev_id | label | status | extra_info | instance_uuid | |
− | | 2014-02-22 06:59:16 | NULL | NULL | 0 | 4 | 1 | 0000:03:00.1 | 10c9 | 8086 | type-PCI | pci_0000_03_00_1 | label_8086_10c9 | available | {} | NULL | | + | +---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+--------------------------------------+ |
− | +---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+--------------------------------------+ | + | | 2014-02-22 06:59:16 | 2014-02-22 07:01:59 | NULL | 0 | 1 | 1 | 0000:02:00.0 | 10c9 | 8086 | type-PCI | pci_0000_02_00_0 | label_8086_10c9 | allocated | {} | 294c0426-693a-4514-907e-3eb17dc1de1c | |
+ | | 2014-02-22 06:59:16 | 2014-02-22 07:01:59 | NULL | 0 | 2 | 1 | 0000:02:00.1 | 10c9 | 8086 | type-PCI | pci_0000_02_00_1 | label_8086_10c9 | allocated | {} | 294c0426-693a-4514-907e-3eb17dc1de1c | | ||
+ | | 2014-02-22 06:59:16 | NULL | NULL | 0 | 3 | 1 | 0000:03:00.0 | 10c9 | 8086 | type-PCI | pci_0000_03_00_0 | label_8086_10c9 | available | {} | NULL | | ||
+ | | 2014-02-22 06:59:16 | NULL | NULL | 0 | 4 | 1 | 0000:03:00.1 | 10c9 | 8086 | type-PCI | pci_0000_03_00_1 | label_8086_10c9 | available | {} | NULL | | ||
+ | +---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+--------------------------------------+ | ||
− | == | + | ==The Hotplug APIs Based on Current Implementation== |
− | + | * API for hotplug a pci device | |
+ | V2 API specification: <br/> | ||
+ | POST: v2/{tenant_id}/servers/{server_id}/os-pci_attachments <br/> | ||
+ | V3 API specification: <br/> | ||
+ | POST: v3/servers/{server_id}/os-pci_attachments <br/> | ||
− | + | JSON Request: | |
− | |||
{ | { | ||
− | + | "pciAttachment": | |
− | + | { | |
− | + | "pci_passthrough:alias":"a1:2" | |
− | + | } | |
− | " | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
} | } | ||
− | + | JSON Response: | |
− | |||
{ | { | ||
− | " | + | "pci_devices": [ |
− | + | { | |
− | + | "attachment_id":"a26887c6-c47b-4654-abb5-dfadf7d3f803", | |
− | + | "serverId": "4d8c3732-a248-40ed-bebc-539a6ffd25c0", | |
− | + | "dev_id":"pci_0000_02_00_0", | |
− | + | "address":"0000:02:00.0", | |
− | + | "inner_address":"0000:03:00.0", | |
+ | "vendor_id":"8086", | ||
+ | "product_id":"10c9" | ||
+ | }, | ||
+ | { | ||
+ | "attachment_id":"a26887c6-c47b-4654-abb5-dfadf7d3f704", | ||
+ | "serverId": "4d8c3732-a248-40ed-bebc-539a6ffd25c0", | ||
+ | "dev_id":"pci_0000_02_00_1", | ||
+ | "address":"0000:02:00.1", | ||
+ | "inner_address":"0000:03:01.0", | ||
+ | "vendor_id":"8086", | ||
+ | "product_id":"10c9" | ||
+ | } | ||
] | ] | ||
} | } | ||
− | + | * API for hotunplug a pci device | |
+ | V2 API specification:<br/> | ||
+ | DELETE: v2/{tenant_id}/servers/{server_id}/os-pci_attachments/{attachment_id}<br/> | ||
+ | V3 API specification:<br/> | ||
+ | DELETE: v3/servers/{server_id}/os-pci_attachments/{attachment_id}<br/> | ||
+ | |||
+ | HTTP response codes:<br/> | ||
+ | v2:<br/> | ||
+ | Normal HTTP Response Code: 200 on success<br/> | ||
+ | v3:<br/> | ||
+ | Normal HTTP Response Code: 202 on success<br/> | ||
+ | |||
+ | * API for query plugged pci devices of an instance | ||
+ | V2 API specification: | ||
+ | GET: v2/{tenant_id}/servers/{server_id}/os-pci_attachments | ||
+ | V3 API specification: | ||
+ | GET: v3/servers/{server_id}/os-pci_attachments | ||
+ | |||
+ | JSON Response: | ||
{ | { | ||
− | " | + | "pciAttachment": [ |
− | + | { | |
− | + | "attachment_id":"a26887c6-c47b-4654-abb5-dfadf7d3f803", | |
− | + | "serverId": "4d8c3732-a248-40ed-bebc-539a6ffd25c0", | |
− | + | "dev_id":"pci_0000_02_00_0", | |
− | + | "address":"0000:02:00.0", | |
− | + | "inner_address":"0000:03:00.0", | |
+ | "vendor_id":"8086", | ||
+ | "product_id":"10c9" | ||
+ | }, | ||
+ | { | ||
+ | "attachment_id":"a26887c6-c47b-4654-abb5-dfadf7d3f803", | ||
+ | "serverId": "4d8c3732-a248-40ed-bebc-539a6ffd25c0", | ||
+ | "dev_id":"pci_0000_02_00_1", | ||
+ | "address":"0000:02:00.1", | ||
+ | "inner_address":"0000:03:01.0", | ||
+ | "vendor_id":"8086", | ||
+ | "product_id":"10c9" | ||
+ | } | ||
] | ] | ||
} | } |
Latest revision as of 11:53, 22 April 2014
Contents
PCI Hotplug Support - API and Features
Background
In current PCI passthrough implementation, a pci device is only allowed to be assigned to a instance while the instance is being created, it is not allowed to be assigned or removed from the instance while the instance is running or stop.
Concept
1. pci_alias is configured in nova.conf on the controller node, for example,
pci_alias={"vendor_id":"8086", "product_id":"10c9", "name":"a1"}
2. pci_passthrough_whitelist is configured in nova.conf on the compute node,for example,
pci_passthrough_whitelist=[{ "vendor_id":"8086","product_id":"10c9"}]
Current implementation
1. Config pci_passthrough_whitelist on the compute node and pci_alias on the controller node, which means the devices you permit to be assigned to vms.
2. Create a floavor and set the "pci_passthrough:alias",for example,
nova flavor-key gzm set "pci_passthrough:alias"="a1:2". one more request is supported too,like this, nova flavor-key gzm set "pci_passthrough:alias"="a1:2,b1:1".
3. Show details of the flavor, you'll see ,
nova flavor-show gzm +----------------------------+--------------------------------------+ | Property | Value | +----------------------------+--------------------------------------+ | name | gzm | | ram | 1024 | | OS-FLV-DISABLED:disabled | False | | vcpus | 2 | | extra_specs | {u'pci_passthrough:alias': u'a1:2'} | | swap | | | os-flavor-access:is_public | True | | rxtx_factor | 1.0 | | OS-FLV-EXT-DATA:ephemeral | 0 | | disk | 10 | | id | 62686729-aa43-4fc6-99e0-b50bb84e2f60 | +----------------------------+--------------------------------------+
4. Create instance using the flavor above,then you'll see two pci devices have been assigned to the instance and you won't see them on the old compute node.
5. Query pci device in nova DB, you'll see,
mysql> select * from pci_devices; +---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+--------------------------------------+ | created_at | updated_at | deleted_at | deleted | id | compute_node_id | address | product_id | vendor_id | dev_type | dev_id | label | status | extra_info | instance_uuid | +---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+--------------------------------------+ | 2014-02-22 06:59:16 | 2014-02-22 07:01:59 | NULL | 0 | 1 | 1 | 0000:02:00.0 | 10c9 | 8086 | type-PCI | pci_0000_02_00_0 | label_8086_10c9 | allocated | {} | 294c0426-693a-4514-907e-3eb17dc1de1c | | 2014-02-22 06:59:16 | 2014-02-22 07:01:59 | NULL | 0 | 2 | 1 | 0000:02:00.1 | 10c9 | 8086 | type-PCI | pci_0000_02_00_1 | label_8086_10c9 | allocated | {} | 294c0426-693a-4514-907e-3eb17dc1de1c | | 2014-02-22 06:59:16 | NULL | NULL | 0 | 3 | 1 | 0000:03:00.0 | 10c9 | 8086 | type-PCI | pci_0000_03_00_0 | label_8086_10c9 | available | {} | NULL | | 2014-02-22 06:59:16 | NULL | NULL | 0 | 4 | 1 | 0000:03:00.1 | 10c9 | 8086 | type-PCI | pci_0000_03_00_1 | label_8086_10c9 | available | {} | NULL | +---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+--------------------------------------+
The Hotplug APIs Based on Current Implementation
- API for hotplug a pci device
V2 API specification:
POST: v2/{tenant_id}/servers/{server_id}/os-pci_attachments
V3 API specification:
POST: v3/servers/{server_id}/os-pci_attachments
JSON Request: {
"pciAttachment": { "pci_passthrough:alias":"a1:2" }
}
JSON Response: {
"pci_devices": [ { "attachment_id":"a26887c6-c47b-4654-abb5-dfadf7d3f803", "serverId": "4d8c3732-a248-40ed-bebc-539a6ffd25c0", "dev_id":"pci_0000_02_00_0", "address":"0000:02:00.0", "inner_address":"0000:03:00.0", "vendor_id":"8086", "product_id":"10c9" }, { "attachment_id":"a26887c6-c47b-4654-abb5-dfadf7d3f704", "serverId": "4d8c3732-a248-40ed-bebc-539a6ffd25c0", "dev_id":"pci_0000_02_00_1", "address":"0000:02:00.1", "inner_address":"0000:03:01.0", "vendor_id":"8086", "product_id":"10c9" } ]
}
- API for hotunplug a pci device
V2 API specification:
DELETE: v2/{tenant_id}/servers/{server_id}/os-pci_attachments/{attachment_id}
V3 API specification:
DELETE: v3/servers/{server_id}/os-pci_attachments/{attachment_id}
HTTP response codes:
v2:
Normal HTTP Response Code: 200 on success
v3:
Normal HTTP Response Code: 202 on success
- API for query plugged pci devices of an instance
V2 API specification: GET: v2/{tenant_id}/servers/{server_id}/os-pci_attachments V3 API specification: GET: v3/servers/{server_id}/os-pci_attachments
JSON Response: {
"pciAttachment": [ { "attachment_id":"a26887c6-c47b-4654-abb5-dfadf7d3f803", "serverId": "4d8c3732-a248-40ed-bebc-539a6ffd25c0", "dev_id":"pci_0000_02_00_0", "address":"0000:02:00.0", "inner_address":"0000:03:00.0", "vendor_id":"8086", "product_id":"10c9" }, { "attachment_id":"a26887c6-c47b-4654-abb5-dfadf7d3f803", "serverId": "4d8c3732-a248-40ed-bebc-539a6ffd25c0", "dev_id":"pci_0000_02_00_1", "address":"0000:02:00.1", "inner_address":"0000:03:01.0", "vendor_id":"8086", "product_id":"10c9" } ]
}