Jump to: navigation, search

Difference between revisions of "PCI configration Database and API"

(admin configures SRIOV flavor)
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
+
merge back to:
===API for PCI configration===
+
https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support
To get a better between user and amdin, and remove redudent code from pci, alias will fade out, white-list will be used to map devices to an  pci-flavor:  the group to use for scheduler and configration.  this approach keep the  possibility to take advange of aggregate.
 
 
 
 
 
User will see flavors like:
 
* flavor that gives you a cheap GPU
 
* flavor that gives you a big GPU
 
* flavor that gives you two SSD disks (of varying types depending on where it lands) and a big GPU
 
* flavor that gives you your public network via SRIOV (which could be one of several makes of network card, depending on the host picked)
 
 
 
And for the admin...
 
 
 
Admin sees:
 
 
 
* per host devices (adding things to os-host/<host>/pci-device)
 
** lists pci devices present
 
** lists pci devices that are exposed to users, and which are in use or free
 
 
 
* per pci-flavor
 
** creates a pci-device description
 
** specifies vendor-id, address, uuid, name, etc
 
** this used to be a combination of alias and whitelist
 
** this could be overlapping descriptions
 
 
 
* flavor extra specs
 
** this has entries that describe:
 
*** list of possible pci-flavors that could be picked
 
*** use the key: pci-passthrough:<label_just_to_make_uniqe  value:<pci-flavor-uuid-1>,<pci-flavor-uuid-2>
 
** for multiple devices, you just add multiple entries
 
 
 
Take advantage of host aggregate:
 
* host aggregates used to map hosts to pci-flavor
 
** use host aggregates to expose specific pci-flavors as available on a particular host
 
 
 
 
 
==== Review comments ====
 
 
 
Overall review comments:
 
* need a better split between admin (setup) and user use cases
 
* need to clearly define what the user will see
 
 
 
I think users will see... (move to text)
 
 
 
===Use cases ===
 
 
 
====admin check PCI devices avaliable per host ====
 
admin might want to know if there are some pci device avaliable to use, eventhough admin should know such infomation but if the inspect works, it's convenience. there is 2 methods to implemt inspect.
 
 
 
    nova  host-list pci-device
 
    GET  v2/​{tenant_id}​/os-hosts/​{host_name}​os-host/<host>/os-pci-devices
 
 
 
return a summary infomation about pci devices on this host:
 
 
 
        [  {'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:00.7',  'pci-type'VF', 'status': 'avaliable' , 'pci-flavor':'xxx'  }}
 
 
 
to find which pci-flavor this device belong to, status, we had to query the database.
 
 
 
currently pci device in the  databse is after filter, if want inpect the device on node from db, we should let all device going in to database. db will too large then eventually slow down query, became a scale problem. so we'd use RPC call for this goal, use RPC call to get reulst from a node, show it to admin. 
 
 
 
Review comments: (addressed, by yongli)
 
* I think we need a REST API to list the PCI devices present on a host
 
* That API could also show the groups
 
* Also maybe it could say which are in use (as currently reported to the scheduler)
 
* I think we should list them here: v2/​{tenant_id}​/os-hosts/​{host_name}​/os-pci-devices (or similar)
 
* See http://api.openstack.org/api-ref-compute.html and look for os-hosts
 
 
 
 
 
====admin check avaliable pci flavor (white-list)  ====
 
 
 
* list all avaliable pci flavor (whitelist)
 
    nova pci-flavor-show
 
 
    GET v2/​{tenant_id}​/os-pci-flavors
 
    data:
 
    [  { 'description':'xxxx' 'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx',  'host':'compute-id' ,'UUID':'xxxx-xx-xx' }  }
 
 
 
* list avaliable pci flavor on host (white list)
 
    nova host-list pci-flavor
 
 
 
    GET  v2/​{tenant_id}​/os-hosts/​{host_name}​os-host/<host>/os-pci-flavors
 
    data:
 
    [  { 'description':'xxxx' 'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx','host':'compute-id' , 'UUID':'xxxx-xx-xx'}  }
 
 
 
* get detailed infomation about one pci-flavor:  
 
 
    nova pci-flavor-show  <white-list UUID>
 
 
 
    GET v2/​{tenant_id}​/os-pci-flavor/<UUID>
 
    data:
 
    { 'description':'xxxx'  'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx', 'UUID':'xxxx-xx-xx' ,'host':'compute-id' }
 
 
 
====admin create a  pci flavor (white-list)  ====
 
 
 
#  create flavor
 
  nova pci-flavor-create  name 'GetMePowerfulldevice'  description "xxxxx"
 
 
 
  API:
 
  POST  v2/​{tenant_id}​/os-pci-flavors
 
 
 
  data:
 
        { 'pci-flavor':'GetMePowerfulldevice', description: "xxxxx" }
 
  action:  create database entry for this flavor.
 
 
 
#  update flavor defination
 
    nova pci-flavor-update UUID  set    'description'='xxxx'  'address'= '0000:01:*.7', 'host'='compute-id'
 
     
 
    PUT v2/​{tenant_id}​/os-pci-flavors/<UUID>
 
    with data  :
 
          { 'description':'xxxx',  'address': '0000:01:*.7'}
 
    action: set this as the new defination of the pci flavor.
 
 
 
 
 
 
 
Review comments:
 
* I would go for: create/v2/​{tenant_id}​/os-pci-device-flavor
 
* please add a uuid for the group/flavor
 
* why is there no vendor id here?
 
* please consider the APIs for listing the current white-lists
 
* we need all the CRUD operations really. Please see other APIs to copy the patterns we use there (http://api.openstack.org/api-ref-compute.html see host-aggregates)
 
* I think you might want to add key-value metadata to these groups, like host-aggregates, but maybe lets ignore that for now, it can be added later
 
 
 
====Take advantage of host aggregate  ====
 
host aggregate can be used to enhancement the schduler for PCI.
 
 
 
* create aggregate
 
    nova aggregate-create  pci-aware-group
 
    nova aggregate-add-host  host1
 
    nova aggregate-add-host  host2
 
 
 
* map flavor to aggregate
 
    nova aggregate-set-metadata pci-aware-group set 'pci-flavor:ntel.INC'=true
 
 
 
* set aggregate extra spec to enhancement PCI scheduler
 
    nova flavor-create --is-public true m1.iwantPCI 100 2048 20 2
 
    nova flavor-key 100 set  pci-flavor:ntel.NIC=true
 
 
 
====admin delete a  pci flavor (white-list)  ====
 
 
 
    nova pci-flavor-delete <UUID>
 
 
 
    API will be:
 
    DELETE v2/​{tenant_id}​/os-pci-flavor/<UUID>
 
    flow: delete it from database
 
 
 
Review comments:
 
* this is good information, but I don't like the flow or the API
 
* please rework after seeing my comments above
 
* I think this could be simpler and more consistent
 
 
 
 
 
====admin configures extra spec in flavor request pci device ====
 
 
 
to allocate the device from a pci flavor, just fill pci flavor into the flavor's extra spec:
 
 
 
    nova flavor-key m1.small set pci_passthrough:pci_flavor=GetMePowerfulldevice:2 
 
 
 
Review comments:
 
* again, this just becomes the pci-device-flavor
 
 
 
====admin boot VM with this  flavours ====
 
    nova boot  mytest  --flavor m1.small  --image=cirros-0.3.1-x86_64-uec
 
 
 
Review:
 
* perfect, this is what I hoped for
 
 
 
====admin configures SRIOV flavor ====
 
 
 
* create a pci flavor for the SRIOV
 
  nova pci-flavor-create  name 'vlan-SRIOV'  description "xxxxx"
 
  nova pci-flavor-update UUID  set    'description'='xxxx'  'address'= '0000:01:*.7'
 
 
 
 
 
Reviewer comments:
 
* this info is missing
 
* how does the admin associate PCI devices to neutron networks?
 
* I think the answer is that the pci-flavor has some metadata that tells you what neutron network it maps to, but its messy
 
 
 
====user requests SRIOV====
 
 
 
*  config the instance flavor
 
    nova flavor-key m1.small set "pci_passthrough:sriov:<nic1-tag>:pci-flavor"="vlan-SRIOV:2"
 
 
 
for sriov there should be a marker <nic1-tag>  for nic, the <nic1-tag> will be store in the device allocated to this nic.
 
after allocation finised on compute node, related network module can find the device via the <nic1-tag> to perferm interface configration.
 
 
 
* 5) admin boot VM with this flavours
 
    nova boot mytest --flavor m1.small --image=cirros-0.3.1-x86_64-uec
 
 
 
NOTE: neotron guy want use this style for request pci for NIC:
 
                                                --nic
 
                                                    --pci-flavor  GetmePowerfullNIC
 
this is under discuss.
 
 
 
Reviewer comments:
 
* I really don't understand this SRIOV stuff: 
 
  A(yongli): 
 
        SRIOV need 2 things, 
 
          1) group device like pci-flavor,  we given it as nature
 
          2) SRIOV need to find the device allocation from the pci-flavor, <nic1-tag> here give the device allocated as a marker, later, the SRIOV code can find it from DB.
 
 
 
* I think the VIF attach logic should do the deciding, and we probably need more config in the flavor extra specs to make the mapping
 
  A( yongli ):
 
      nova flavor-key m1.small set  pci_passthrough:sriov:<nic1-tag>:pci-flavor"="vlan-SRIOV:2"  map the  SRIOV tage <nic1-tag> to the device, capibale to find this device, SRIOV code can do it's work.
 
 
 
===transite config file to API ===
 
#  the config file for alias and whitelist defination is going to deprecated.
 
#  if database is not NULL , configration is ommit and given deprecated  warning.
 
#  if database is NULL, config if read from the file,
 
    *white list/alias schema still work
 
    * And also  given a deprecated notice, alias will fade out  which will be remove start from next release.
 
 
 
with this solution, we move pci config from file to API.
 
 
 
===DB for pci configration===
 
each pci flavor will be a set of (k,v), and the pci flavor don't need to contain same k, v pair.  another problem this define try to slove is, i,.e  SRIOV also want feature autodiscovery(under discuss), with this, the flavor might need a 'feature' key to be added if not store it as (k,v) pair.  the (k,v) paire define let more extra infomation can be store in the pci device.
 
 
 
  talbe: pci_flavor{
 
                id  :  data base of this k,v pair
 
                UUID:  which pci-flavor the  k,v belong to
 
                compute_node_id:  easy to query per host configration, NULL for global flavor defination( if aggreate avaliable, this fade out)
 
                key
 
                value
 
                value_type:  ( value might be a simple value or Regular express, don't sure for now.)
 
            }
 
 
 
===PCI releated Objects===
 
  Objects: white list/ alias/ group keys(on demand grop)
 
  objects property:  sets of k,v
 
   
 
===API interface===
 
 
 
*  get pci devices infomation on host
 
  nova  host-list pci-device
 
  GET  v2/​{tenant_id}​/os-hosts/​{host_name}​os-host/<host>/os-pci-devices
 
  return a summary infomation about pci devices on this host:
 
    [  {'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:00.7',  'pci-type'VF', 'status': 'avaliable' , 'pci-flavor':'xxx'  }}
 
 
 
* list avaliable pci flavor on host (white list) [might not needed]
 
  nova host-list pci-flavor
 
  GET  v2/​{tenant_id}​/os-hosts/​{host_name}​os-host/<host>/os-pci-flavors
 
  data:
 
    [  { 'description':'xxxx' 'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx','host':'compute-id' , 'UUID':'xxxx-xx-xx'}  }
 
 
 
* list all avaliable pci flavor (whitelist)
 
  nova pci-flavor-show
 
  GET v2/​{tenant_id}​/os-pci-flavors
 
    data:
 
    [  { 'description':'xxxx' 'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx',  'host':'compute-id' ,'UUID':'xxxx-xx-xx' }  }
 
 
 
 
 
* get detailed infomation about one pci-flavor:
 
    nova pci-flavor-show  <white-list UUID>
 
  GET v2/​{tenant_id}​/os-pci-flavor/<UUID>
 
  data:
 
  { 'description':'xxxx'  'vendor_id':'8086', 'product_id':'xxx',  'address': '0000:01:*.7',  'pci-flavor':'xxx', 'UUID':'xxxx-xx-xx' ,'host':'compute-id' }
 
 
 
* create pci flavor
 
  nova pci-flavor-create  name 'GetMePowerfulldevice'  description "xxxxx"
 
  POST  v2/​{tenant_id}​/os-pci-flavors
 
  data:
 
      { 'pci-flavor':'GetMePowerfulldevice', description: "xxxxx" }
 
 
 
*update the pci flavor
 
    nova pci-flavor-update UUID  set    'description'='xxxx'  'address'= '0000:01:*.7', 'host'='compute-id'
 
    PUT v2/​{tenant_id}​/os-pci-flavors/<UUID>
 
    with data  :
 
        { 'description':'xxxx',  'address': '0000:01:*.7'}
 
 
 
* delete a pci flavor
 
  nova pci-flavor-delete <UUID>
 
  DELETE v2/​{tenant_id}​/os-pci-flavor/<UUID>
 

Latest revision as of 06:31, 11 December 2013

merge back to: https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support