Jump to: navigation, search

Difference between revisions of "PCI configration Database and API"

(API for PCI configration)
(admin create a pci flavor (white-list))
Line 121: Line 121:
 
* I think you might want to add key-value metadata to these groups, like host-aggregates, but maybe lets ignore that for now, it can be added later
 
* I think you might want to add key-value metadata to these groups, like host-aggregates, but maybe lets ignore that for now, it can be added later
  
 +
====Take advantage of host aggregate (T.B.D) ====
 +
 +
* create aggregate
 +
nova aggregate-create  pci-aware-group
 +
nova aggregate-add-host  host1
 +
nova aggregate-add-host  host2
 +
 +
* map flavor to aggregate
 +
nova aggregate-set-metadata pci-aware-group set 'pci-flavor:ntel.INC'=true
 +
 +
* set aggregate extra spec to enhancement PCI scheduler
 +
nova flavor-create --is-public true m1.iwantPCI 100 2048 20 2
 +
nova flavor-key 100 set  pci-flavor:ntel.NIC=true
  
 
====admin delete a  pci flavor (white-list)  ====
 
====admin delete a  pci flavor (white-list)  ====

Revision as of 08:56, 9 December 2013

API for PCI configration

To get a better between user and amdin, and remove redudent code from pci, alias will fade out, white-list will be used to map devices to an pci-flavor: the group to use for scheduler and configration. this approach keep the possibility to take advange of aggregate.


User will see flavors like:

  • flavor that gives you a cheap GPU
  • flavor that gives you a big GPU
  • flavor that gives you two SSD disks (of varying types depending on where it lands) and a big GPU
  • flavor that gives you your public network via SRIOV (which could be one of several makes of network card, depending on the host picked)

And for the admin...

Admin sees:

  • per host devices (adding things to os-host/<host>/pci-device)
    • lists pci devices present
    • lists pci devices that are exposed to users, and which are in use or free
  • per pci-flavor
    • creates a pci-device description
    • specifies vendor-id, address, uuid, name, etc
    • this used to be a combination of alias and whitelist
    • this could be overlapping descriptions
  • flavor extra specs
    • this has entries that describe:
      • list of possible pci-flavors that could be picked
      • use the key: pci-passthrough:<label_just_to_make_uniqe value:<pci-flavor-uuid-1>,<pci-flavor-uuid-2>
    • for multiple devices, you just add multiple entries

Take advantage of host aggregate:

  • host aggregates used to map hosts to pci-flavor
    • use host aggregates to expose specific pci-flavors as available on a particular host


Review comments (WIP)

Overall review comments:

  • need a better split between admin (setup) and user use cases
  • need to clearly define what the user will see

I think users will see... (move to text)

Use cases

admin check PCI devices avaliable per host

admin might want to know if there are some pci device avaliable to use, eventhough admin should know such infomation but if the inspect works, it's convenience. there is 2 methods to implemt inspect.

   nova  host-list pci-device 
   GET  v2/​{tenant_id}​/os-hosts/​{host_name}​os-host/<host>/os-pci-devices

return a summary infomation about pci devices on this host:

        [  {'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:00.7',  'pci-type'VF', 'status': 'avaliable' , 'pci-flavor':'xxx'  }}

to find which pci-flavor this device belong to, status, we had to query the database.

currently pci device in the databse is after filter, if want inpect the device on node from db, we should let all device going in to database. db will too large then eventually slow down query, became a scale problem. so we'd use RPC call for this goal, use RPC call to get reulst from a node, show it to admin.

Review comments: (addressed, by yongli)

  • I think we need a REST API to list the PCI devices present on a host
  • That API could also show the groups
  • Also maybe it could say which are in use (as currently reported to the scheduler)
  • I think we should list them here: v2/​{tenant_id}​/os-hosts/​{host_name}​/os-pci-devices (or similar)
  • See http://api.openstack.org/api-ref-compute.html and look for os-hosts


admin check avaliable pci flavor (white-list)

  • list all avaliable pci flavor (whitelist)
   nova pci-flavor-show 

   GET v2/​{tenant_id}​/os-pci-flavors
   data:
    [  { 'description':'xxxx' 'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx',  'host':'compute-id' ,'UUID':'xxxx-xx-xx' }  }
  • list avaliable pci flavor on host (white list)
   nova host-list pci-flavor
   GET  v2/​{tenant_id}​/os-hosts/​{host_name}​os-host/<host>/os-pci-flavors
   data:
    [  { 'description':'xxxx' 'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx','host':'compute-id' , 'UUID':'xxxx-xx-xx'}  }
  • get detailed infomation about one pci-flavor:
    nova pci-flavor-show  <white-list UUID>
  
   GET v2/​{tenant_id}​/os-pci-flavor/<UUID>
   data:
   { 'description':'xxxx'  'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx', 'UUID':'xxxx-xx-xx' ,'host':'compute-id' }

admin create a pci flavor (white-list)

  1. create flavor
  nova pci-flavor-create  name 'GetMePowerfulldevice'  description "xxxxx"
  API:
  POST  v2/​{tenant_id}​/os-pci-flavors
  
  data: 
       { 'pci-flavor':'GetMePowerfulldevice', description: "xxxxx" }
  action:  create database entry for this flavor.
  1. update flavor defination
   nova pci-flavor-update UUID  set    'description'='xxxx'   'address'= '0000:01:*.7', 'host'='compute-id'
      
    PUT v2/​{tenant_id}​/os-pci-flavors/<UUID>
    with data  :
         { 'description':'xxxx',  'address': '0000:01:*.7'}
    action: set this as the new defination of the pci flavor.


Review comments:

  • I would go for: create/v2/​{tenant_id}​/os-pci-device-flavor
  • please add a uuid for the group/flavor
  • why is there no vendor id here?
  • please consider the APIs for listing the current white-lists
  • we need all the CRUD operations really. Please see other APIs to copy the patterns we use there (http://api.openstack.org/api-ref-compute.html see host-aggregates)
  • I think you might want to add key-value metadata to these groups, like host-aggregates, but maybe lets ignore that for now, it can be added later

Take advantage of host aggregate (T.B.D)

  • create aggregate

nova aggregate-create pci-aware-group nova aggregate-add-host host1 nova aggregate-add-host host2

  • map flavor to aggregate
nova aggregate-set-metadata pci-aware-group set 'pci-flavor:ntel.INC'=true
  • set aggregate extra spec to enhancement PCI scheduler

nova flavor-create --is-public true m1.iwantPCI 100 2048 20 2 nova flavor-key 100 set pci-flavor:ntel.NIC=true

admin delete a pci flavor (white-list)

   nova pci-flavor-delete <UUID>
   API will be:
   DELETE v2/​{tenant_id}​/os-pci-flavor/<UUID>
   flow: delete it from database
  

Review comments:

  • this is good information, but I don't like the flow or the API
  • please rework after seeing my comments above
  • I think this could be simpler and more consistent


admin configures extra spec in flavor request pci device

to allocate the device from a pci flavor, just fill pci flavor into the flavor's extra spec:

   nova flavor-key m1.small set pci_passthrough:pci_flavor=GetMePowerfulldevice:2  

Review comments:

  • again, this just becomes the pci-device-flavor

admin boot VM with this flavours

   nova boot  mytest  --flavor m1.small  --image=cirros-0.3.1-x86_64-uec

Review:

  • perfect, this is what I hoped for

admin configures SRIOV flavor

  • create a pci flavor for the SRIOV
  nova pci-flavor-create  name 'vlan-SRIOV'  description "xxxxx"
  nova pci-flavor-update UUID  set    'description'='xxxx'   'address'= '0000:01:*.7',  'host'='*'


Reviewer comments:

  • this info is missing
  • how does the admin associate PCI devices to neutron networks?
  • I think the answer is that the pci-flavor has some metadata that tells you what neutron network it maps to, but its messy

user requests SRIOV

  • config the instance flavor
    nova flavor-key m1.small set "pci_passthrough:sriov:<nic1-tag>:pci-flavor"="vlan-SRIOV:2"

for sriov there should be a marker <nic1-tag> for nic, the <nic1-tag> will be store in the device allocated to this nic. after allocation finised on compute node, related network module can find the device via the <nic1-tag> to perferm interface configration.

  • 5) admin boot VM with this flavours
   nova boot mytest --flavor m1.small --image=cirros-0.3.1-x86_64-uec

NOTE: neotron guy want use this style for request pci for NIC:

                                                --nic
                                                    --pci-flavor  GetmePowerfullNIC

this is under discuss.

Reviewer comments:

  • I really don't understand this SRIOV stuff:
 A(yongli):  
        SRIOV need 2 things,  
          1) group device like pci-flavor,  we given it as nature 
          2) SRIOV need to find the device allocation from the pci-flavor, <nic1-tag> here give the device allocated as a marker, later, the SRIOV code can find it from DB.
 
  • I think the VIF attach logic should do the deciding, and we probably need more config in the flavor extra specs to make the mapping
 A( yongli ):
      nova flavor-key m1.small set  pci_passthrough:sriov:<nic1-tag>:pci-flavor"="vlan-SRIOV:2"   map the  SRIOV tage <nic1-tag> to the device, capibale to find this device, SRIOV code can do it's work. 

transite config file to API

  1. the config file for alias and whitelist defination is going to deprecated.
  2. if database is not NULL , configration is ommit and given deprecated warning.
  3. if database is NULL, config if read from the file,
    *white list/alias schema still work
    * And also  given a deprecated notice, alias will fade out  which will be remove start from next release.

with this solution, we move pci config from file to API.

DB for pci configration

each pci flavor will be a set of (k,v), and the pci flavor don't need to contain same k, v pair. another problem this define try to slove is, i,.e SRIOV also want feature autodiscovery(under discuss), with this, the flavor might need a 'feature' key to be added if not store it as (k,v) pair. the (k,v) paire define let more extra infomation can be store in the pci device.

  talbe: pci_flavor{
               id   :  data base of this k,v pair
               UUID:  which pci-flavor the  k,v belong to
               compute_node_id:  easy to query per host configration, NULL for global flavor defination( if aggreate avaliable, this fade out)
               key 
               value
               value_type:  ( value might be a simple value or Regular express, don't sure for now.)
           }

PCI releated Objects

  Objects: white list/ alias/ group keys(on demand grop)
  objects property:  sets of k,v
   

API interface

  • get pci devices infomation on host
  nova  host-list pci-device 
  GET  v2/​{tenant_id}​/os-hosts/​{host_name}​os-host/<host>/os-pci-devices
  return a summary infomation about pci devices on this host:
   [  {'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:00.7',  'pci-type'VF', 'status': 'avaliable' , 'pci-flavor':'xxx'  }}
* list avaliable pci flavor on host (white list) [might not needed]
  nova host-list pci-flavor
  GET  v2/​{tenant_id}​/os-hosts/​{host_name}​os-host/<host>/os-pci-flavors
  data:
   [  { 'description':'xxxx' 'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx','host':'compute-id' , 'UUID':'xxxx-xx-xx'}  }
  • list all avaliable pci flavor (whitelist)
  nova pci-flavor-show 
  GET v2/​{tenant_id}​/os-pci-flavors
   data:
   [  { 'description':'xxxx' 'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx',  'host':'compute-id' ,'UUID':'xxxx-xx-xx' }  }


  • get detailed infomation about one pci-flavor:
   nova pci-flavor-show  <white-list UUID>
  GET v2/​{tenant_id}​/os-pci-flavor/<UUID>
  data:
  { 'description':'xxxx'  'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx', 'UUID':'xxxx-xx-xx' ,'host':'compute-id' }
  • create pci flavor
 nova pci-flavor-create  name 'GetMePowerfulldevice'  description "xxxxx"
 POST  v2/​{tenant_id}​/os-pci-flavors
 data: 
      { 'pci-flavor':'GetMePowerfulldevice', description: "xxxxx" }
  • update the pci flavor
   nova pci-flavor-update UUID  set    'description'='xxxx'   'address'= '0000:01:*.7', 'host'='compute-id'
   PUT v2/​{tenant_id}​/os-pci-flavors/<UUID>
   with data  :
        { 'description':'xxxx',  'address': '0000:01:*.7'}
  • delete a pci flavor
  nova pci-flavor-delete <UUID>
  DELETE v2/​{tenant_id}​/os-pci-flavor/<UUID>