Difference between revisions of "PCI configration Database and API"

Revision as of 07:23, 9 December 2013

API for PCI configration

To get a better between user and amdin, and remove redudent code from pci, alias will fade out, white-list will be used to map devices to an pci-flavor: the group to use for scheduler and configration. cuase KVM not support aggregator , and PCI support only KVM for now, we had to povide a global white list filter to define flavor, then partly relove the aggregating problem, this approach keep the possibility to take advange of aggregtor later.

User will see flavors like:

• flavor that gives you a cheap GPU
• flavor that gives you a big GPU
• flavor that gives you two SSD disks (of varying types depending on where it lands) and a big GPU
• flavor that gives you your public network via SRIOV (which could be one of several makes of network card, depending on the host picked)

And for the admin...

Admin sees:

• per host devices (adding things to os-host/<host>/pci-device)
  • lists pci devices present
  • lists pci devices that are exposed to users, and which are in use or free

• per pci-flavor
  • creates a pci-device description
  • specifies vendor-id, address, uuid, name, etc
  • this used to be a combination of alias and whitelist
  • this could be overlapping descriptions

• flavor extra specs
  • this has entries that describe:
    • list of possible pci-flavors that could be picked
    • use the key: pci-passthrough:<label_just_to_make_uniqe value:<pci-flavor-uuid-1>,<pci-flavor-uuid-2>
  • for multiple devices, you just add multiple entries

take advantage of host aggregate:

• host aggregates used to map hosts to pci-flavor ()
  • use host aggregates to expose specific pci-flavors as available on a particular host

Review comments (WIP)

Overall review comments:

• need a better split between admin (setup) and user use cases
• need to clearly define what the user will see

I think users will see... (move to text)

Use cases

admin check PCI devices avaliable per host

admin might want to know if there are some pci device avaliable to use, eventhough admin should know such infomation but if the inspect works, it's convenience. there is 2 methods to implemt inspect.

   nova  host-list pci-device 
   GET  v2/​{tenant_id}​/os-hosts/​{host_name}​os-host/<host>/os-pci-devices

return a summary infomation about pci devices on this host:

        [  {'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:00.7',  'pci-type'VF', 'status': 'avaliable' , 'pci-flavor':'xxx'  }}

to find which pci-flavor this device belong to, status, we had to query the database.

currently pci device in the databse is after filter, if want inpect the device on node from db, we should let all device going in to database. db will too large then eventually slow down query, became a scale problem. so we'd use RPC call for this goal, use RPC call to get reulst from a node, show it to admin.

Review comments: (addressed, by yongli)

• I think we need a REST API to list the PCI devices present on a host
• That API could also show the groups
• Also maybe it could say which are in use (as currently reported to the scheduler)
• I think we should list them here: v2/​{tenant_id}​/os-hosts/​{host_name}​/os-pci-devices (or similar)
• See http://api.openstack.org/api-ref-compute.html and look for os-hosts

admin check avaliable pci flavor (white-list)

• list all avaliable pci flavor (whitelist)

   nova pci-flavor-show 

   GET v2/​{tenant_id}​/os-pci-flavors
   data:
    [  { 'description':'xxxx' 'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx',  'host':'compute-id' ,'UUID':'xxxx-xx-xx' }  }

• list avaliable pci flavor on host (white list)

   nova host-list pci-flavor

   GET  v2/​{tenant_id}​/os-hosts/​{host_name}​os-host/<host>/os-pci-flavors
   data:
    [  { 'description':'xxxx' 'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx','host':'compute-id' ， 'UUID':'xxxx-xx-xx'}  }

• get detailed infomation about one pci-flavor:

    nova pci-flavor-show  <white-list UUID>
  
   GET v2/​{tenant_id}​/os-pci-flavor/<UUID>
   data:
   { 'description':'xxxx'  'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx', 'UUID':'xxxx-xx-xx' ,'host':'compute-id' }

admin create a pci flavor (white-list)

1. create flavor

  nova pci-flavor-create  name 'GetMePowerfulldevice'  description "xxxxx"

  API:
  POST  v2/​{tenant_id}​/os-pci-flavors
  
  data: 
       { 'pci-flavor':'GetMePowerfulldevice', description: "xxxxx" }
  action:  create database entry for this flavor.

1. update flavor defination

   nova pci-flavor-update UUID  set    'description'='xxxx'   'address'= '0000:01:*.7', 'host'='compute-id'
      
    PUT v2/​{tenant_id}​/os-pci-flavors/<UUID>
    with data  :
         { 'description':'xxxx',  'address': '0000:01:*.7'}
    action: set this as the new defination of the pci flavor.

Review comments:

• I would go for: create/v2/​{tenant_id}​/os-pci-device-flavor
• please add a uuid for the group/flavor
• why is there no vendor id here?
• please consider the APIs for listing the current white-lists
• we need all the CRUD operations really. Please see other APIs to copy the patterns we use there (http://api.openstack.org/api-ref-compute.html see host-aggregates)
• I think you might want to add key-value metadata to these groups, like host-aggregates, but maybe lets ignore that for now, it can be added later

admin delete a pci flavor (white-list)

   nova pci-flavor-delete <UUID>

   API will be:
   DELETE v2/​{tenant_id}​/os-pci-flavor/<UUID>
   flow: delete it from database
  

Review comments:

• this is good information, but I don't like the flow or the API
• please rework after seeing my comments above
• I think this could be simpler and more consistent

admin configures extra spec in flavor request pci device

to allocate the device from a pci flavor, just fill pci flavor into the flavor's extra spec:

nova flavor-key m1.small set pci_passthrough:pci_flavor=GetMePowerfulldevice:2

Review comments:

• again, this just becomes the pci-device-flavor

admin boot VM with this flavours

nova boot mytest --flavor m1.small --image=cirros-0.3.1-x86_64-uec

Review:

• perfect, this is what I hoped for

admin configures SRIOV flavor

• create a pci flavor for the SRIOV

  nova pci-flavor-create  name 'vlan-SRIOV'  description "xxxxx"
  nova pci-flavor-update UUID  set    'description'='xxxx'   'address'= '0000:01:*.7',  'host'='*'

Reviewer comments:

• this info is missing
• how does the admin associate PCI devices to neutron networks?
• I think the answer is that the pci-flavor has some metadata that tells you what neutron network it maps to, but its messy

user requests SRIOV

• config the instance flavor

    nova flavor-key m1.small set "pci_passthrough:sriov:<nic1-tag>:pci-flavor"="vlan-SRIOV:2"

for sriov there should be a marker <nic1-tag> for nic, the <nic1-tag> will be store in the device allocated to this nic. after allocation finised on compute node, related network module can find the device via the <nic1-tag> to perferm interface configration.

• 5) admin boot VM with this flavours

   nova boot mytest --flavor m1.small --image=cirros-0.3.1-x86_64-uec

NOTE: neotron guy want use this style for request pci for NIC:

                                                --nic
                                                    --pci-flavor  GetmePowerfullNIC

this is under discuss.

Reviewer comments:

• I really don't understand this SRIOV stuff:

 A(yongli):  
        SRIOV need 2 things,  
          1) group device like pci-flavor,  we given it as nature 
          2) SRIOV need to find the device allocation from the pci-flavor, <nic1-tag> here give the device allocated as a marker, later, the SRIOV code can find it from DB.
 

• I think the VIF attach logic should do the deciding, and we probably need more config in the flavor extra specs to make the mapping

 A( yongli ):
      nova flavor-key m1.small set  pci_passthrough:sriov:<nic1-tag>:pci-flavor"="vlan-SRIOV:2"   map the  SRIOV tage <nic1-tag> to the device, capibale to find this device, SRIOV code can do it's work. 

transite config file to API

1. the config file for alias and whitelist defination is going to deprecated.
2. if database is not NULL , configration is ommit and given deprecated warning.
3. if database is NULL, config if read from the file,

    *white list/alias schema still work
    * And also  given a deprecated notice, alias will fade out  which will be remove start from next release.

with this solution, we move pci config from file to API.

DB for pci configration

  talbe: pci_config{
               id   : data base of this k,v pair
               UUID: which alias/whitelist k,v belong to
               compute_node_id:  easy to query per host configration, NULL for global configration
               object : easy to query by object whitelist/alias
               key 
               value
               value_type:  string, RE, list  # for further enhancement 
           }

alias is always global.

PCI releated Objects

  Objects: white list/ alias/ group keys(on demand grop)
  objects property:  sets of k,v
   

API interface

• get summary info mation

   GET/v2/​{tenant_id}​/os-pci-config/{config-object-name}
   show the global configraion summary.
    i.e {config-object-name} is pci_alias:
    [{"pci_alias":'nic-group', 'UUID":"alias-UUID',  'description':"blabla"}  ]

• get summary info mation on compute node

   GET/v2/​{tenant_id}​/os-pci-config/{config-object-name}/node/{compute-node-id'}
   show the per compute node  pci configraion summary:
   i.e {config-object-name} is pci_whitelist:
    [{"pci_whitelist":'NIC-2', 'UUID":"whitelist-UUID', 'description':"blabla" } ]
  

• get details infomation about the config

    GET/v2/​{tenant_id}​/os-pci-config/{config-object-name}/uuid/<UUID>
    i.e {config-object-name} is pci_whitelist:
     {'product_id':'8086', 'group':'nic-g1', 'compute-node-id':"3" ....}

• update a exist config

   UPDATE//v2/​{tenant_id}​/os-pci-config/{config-object-name}/<UUID>
   with data :
       {  'sriov-type':"macvtap" ....}
   or with array data :
       {  'address': ['a1', 'a2'...] ....}

• unset a exist config's key value (the value support list)

   UPDATE//v2/​{tenant_id}​/os-pci-config/{config-object-name}/<UUID>/unset
   with data :
       {  'address':"0000:01:00:7"}
    this wil remove the "0000:01:00:7" from address key's value set.
 

• delete a whole config item

   DELETE/v2/​{tenant_id}​/os-pci-config/{config-object-name}/<UUID>