PCI passthrough SRIOV support Icehouse

Background

This document describes the generic PCI pass-through enhancement that we want to achieve in IceHouse release, to support the SRIO-IOV NIC pass-through.

The corresponding long-term design documents is at https://docs.google.com/document/d/1vadqmurlnlvZ5bv3BlUbFeXRS_wh-dsgi5plSjimWjU/edit?pli=1# . Because of tight schedule and still some disagreement left, we will only implement part of the design document in I release.

Please refer to https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support for final changes plan.

Use cases

General PCI pass through

given compute nodes contain 1 GPU with vendor:device 8086:0001

• on the compute nodes, config the pci_information

   pci_information =  { { 'device_id': "8086", 'vendor_id': "0001" }, {} }

• on controller

  pci_flavor_attrs = ['device_id', 'vendor_id']
  pci_alias = {'vendor_id':'8086', 'product_id':'0001', 'name':bigGPU', description: 'passthrough Intel's on-die GPU'}

the compute node would report PCI stats group by ('device_id', 'vendor_id'). pci stats will report one pool:

 {'device_id':'0001', 'vendor_id':'8086', 'count': 1 }

• create flavor and boot with it

 nova flavor-key m1.small set pci_passthrough:pci_flavor= 1:bigGPU
 nova boot  mytest  --flavor m1.tiny  --image=cirros-0.3.1-x86_64-uec

General PCI pass through with multi PCI flavor candidate

given compute nodes contain 2 type GPU with , vendor:device 8086:0001, or vendor:device 8086:0002

• on the compute nodes, config the pci_information

   pci_information =  { { 'device_id': "8086", 'vendor_id': "000[1-2]" }, {} }

• on controller

  pci_flavor_attrs = ['device_id', 'vendor_id']
  pci_alias = [{'vendor_id':'8086', 'product_id':'0001', 'name':bigGPU', description: 'Intel's on-die GPU'},  {'vendor_id':'8086', 'product_id':'0002', 'name':bigGPU2', description: ' New Intel's on-die GPU'}]

the compute node would report PCI stats group by ('device_id', 'vendor_id'). pci stats will report 2 pool:

 {'device_id':'0001', 'vendor_id':'8086', 'count': 1 }
 {'device_id':'0002', 'vendor_id':'8086', 'count': 1 }

• create flavor and boot with it

 nova flavor-key m1.small set pci_passthrough:pci_flavor= '1:bigGPU,1:bigGPU2;'
 nova boot  mytest  --flavor m1.tiny  --image=cirros-0.3.1-x86_64-uec

General PCI pass through wild-cast PCI flavor

given compute nodes contain 2 type GPUs with vendor:device 8086:0001, or vendor:device 8086:0002

• on the compute nodes, config the pci_information

   pci_information =  { { 'device_id': "8086", 'vendor_id': "000[1-2]" }, {} }

• on controller

  pci_flavor_attrs = ['device_id', 'vendor_id']
  pci_alias = [{'vendor_id':'8086', 'product_id':'000[1-2]', 'name':bigGPU', description: 'Intel's on-die GPU'}]

the compute node would report PCI stats group by ('device_id', 'vendor_id'). pci stats will report 2 pool:

 {'device_id':'0001', 'vendor_id':'8086', 'count': 1 }
 {'device_id':'0002', 'vendor_id':'8086', 'count': 1 }

• create flavor and boot with it

 nova flavor-key m1.small set pci_passthrough:pci_flavor= '1:bigGPU;'
 nova boot  mytest  --flavor m1.tiny  --image=cirros-0.3.1-x86_64-uec

PCI pass through support grouping tag

given compute nodes contain 2 type GPU with , vendor:device 8086:0001, or vendor:device 8086:0002

• on the compute nodes, config the pci_information

   pci_information =  { { 'device_id': "8086", 'vendor_id': "000[1-2]" }, { 'e.group':'gpu' } } 

• on controller

  pci_flavor_attrs = ['e.group']
  pci_alias = [{'e.group':'gpu', 'name':bigGPU', description: 'Intel's on-die GPU'}]

the compute node would report PCI stats group by ('e.group'). pci stats will report 1 pool:

{'e.group':'gpu', 'count': 2 }

• create flavor and boot with it

 nova flavor-key m1.small set pci_passthrough:pci_flavor= '1:bigGPU;'
 nova boot  mytest  --flavor m1.tiny  --image=cirros-0.3.1-x86_64-uec

PCI SRIOV with tagged flavor

given compute nodes contain 5 PCI NIC , vendor:device 8086:0022, and it connect to physical network "X".

• on the compute nodes, config the pci_information

   pci_information =  { { 'device_id': "8086", 'vendor_id': "000[1-2]" }, { 'e.physical_netowrk': 'X' } }

• on controller

  pci_flavor_attrs = ['e.physical_netowrk']
  pci_alias = [{'e.physical_netowrk':'X', 'name':phyX_NIC', description: 'NIC connect to physical network X'}]

the compute node would report PCI stats group by ('e.group'). pci stats will report 1 pool:

 {'e.physical_netowrk':'X', 'count': 1 }

• create flavor and boot with it

 nova boot  mytest  --flavor m1.tiny  --image=cirros-0.3.1-x86_64-uec  --nic  net-id=network_X  pci_flavor= 'phyX_NIC:1'

Changes Plan

PCI Infomation Config Item

A new configuration item added to extend current PCI white list configuration items:

Select assignable PCI devices based on reduced regular expression

Configure additional arbitrary info for these devices, as any (k,v) pair, k and v is string.

pci_flavor_attrs Config Items

Specify the PCI information and extra information that can be used for PCI alias and PCI stats.

Extending PCI Stats

Current PCI stats group devices only on [vendor_id, product_id, extra_info]. This will be extended to group by keys specified in pci_flavor_attrs.

Extending PCI Alias

PCI alias will be extended in IceHouse release and is planed to be replaced by PCI flavor if agreement achieved.

• Currently PCI alias can only accept combination of keys 'vendor_id, product_id,device_type' . This will be extended to support any keys specified in pci_flavor_attrs.
• Currently PCI alias can only compare if the PCI device information is equal to the one specified in PCI alias. This comparison will be extended to support reduced regular expression.