Difference between revisions of "PCI passthrough SRIOV support Icehouse"

Revision as of 07:23, 2 February 2014

Background

This document describes the generic PCI pass-through enhancement that we want to achieve in IceHouse release, to support the SRIO-IOV NIC pass-through.

The corresponding long-term design documents is at https://docs.google.com/document/d/1vadqmurlnlvZ5bv3BlUbFeXRS_wh-dsgi5plSjimWjU/edit?pli=1# . Because of tight schedule and still some disagreement left, we will only implement part of the design document in I release.

Please refer to https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support for final changes plan.

Use cases

General PCI pass through

given compute nodes contain 1 GPU with vendor:device 8086:0001

• on the compute nodes, config the pci_information

   pci_information =  { { 'device_id': "8086", 'vendor_id': "0001" }, {} }

• on controller

  pci_flavor_attrs = ['device_id', 'vendor_id']
  pci_alias = {'vendor_id':'8086', 'product_id':'0001', 'name':bigGPU', description: 'passthrough Intel's on-die GPU'}

the compute node would report PCI stats group by ('device_id', 'vendor_id'). pci stats will report one pool:

 {'device_id':'0001', 'vendor_id':'8086', 'count': 1 }

• create flavor and boot with it

 nova flavor-key m1.small set pci_passthrough:pci_flavor= 1:bigGPU
 nova boot  mytest  --flavor m1.tiny  --image=cirros-0.3.1-x86_64-uec

General PCI pass through with multi PCI flavor candidate

given compute nodes contain 2 type GPU with , vendor:device 8086:0001, or vendor:device 8086:0002

• on the compute nodes, config the pci_information

   pci_information =  { { 'device_id': "8086", 'vendor_id': "000[1-2]" }, {} }

• on controller

  pci_flavor_attrs = ['device_id', 'vendor_id']
  pci_alias = [{'vendor_id':'8086', 'product_id':'0001', 'name':bigGPU', description: 'Intel's on-die GPU'},  {'vendor_id':'8086', 'product_id':'0002', 'name':bigGPU2', description: ' New Intel's on-die GPU'}]

the compute node would report PCI stats group by ('device_id', 'vendor_id'). pci stats will report 2 pool:

 {'device_id':'0001', 'vendor_id':'8086', 'count': 1 }
 {'device_id':'0002', 'vendor_id':'8086', 'count': 1 }

• create flavor and boot with it

 nova flavor-key m1.small set pci_passthrough:pci_flavor= '1:bigGPU,1:bigGPU2;'
 nova boot  mytest  --flavor m1.tiny  --image=cirros-0.3.1-x86_64-uec

General PCI pass through wild-cast PCI flavor

given compute nodes contain 2 type GPUs with vendor:device 8086:0001, or vendor:device 8086:0002

• on the compute nodes, config the pci_information

   pci_information =  { { 'device_id': "8086", 'vendor_id': "000[1-2]" }, {} }

• on controller

  pci_flavor_attrs = ['device_id', 'vendor_id']
  pci_alias = [{'vendor_id':'8086', 'product_id':'000[1-2]', 'name':bigGPU', description: 'Intel's on-die GPU'}]

the compute node would report PCI stats group by ('device_id', 'vendor_id'). pci stats will report 2 pool:

 {'device_id':'0001', 'vendor_id':'8086', 'count': 1 }
 {'device_id':'0002', 'vendor_id':'8086', 'count': 1 }

• create flavor and boot with it

 nova flavor-key m1.small set pci_passthrough:pci_flavor= '1:bigGPU;'
 nova boot  mytest  --flavor m1.tiny  --image=cirros-0.3.1-x86_64-uec

PCI pass through support grouping tag

given compute nodes contain 2 type GPU with , vendor:device 8086:0001, or vendor:device 8086:0002

• on the compute nodes, config the pci_information

   pci_information =  { { 'device_id': "8086", 'vendor_id': "000[1-2]" }, { 'e.group':'gpu' } } 

• on controller

  pci_flavor_attrs = ['e.group']
  pci_alias = [{'e.group':'gpu', 'name':bigGPU', description: 'Intel's on-die GPU'}]

the compute node would report PCI stats group by ('e.group'). pci stats will report 1 pool:

{'e.group':'gpu', 'count': 2 }

• create flavor and boot with it

 nova flavor-key m1.small set pci_passthrough:pci_flavor= '1:bigGPU;'
 nova boot  mytest  --flavor m1.tiny  --image=cirros-0.3.1-x86_64-uec

PCI SRIOV with tagged flavor

given compute nodes contain 5 PCI NIC , vendor:device 8086:0022, and it connect to physical network "X".

• on the compute nodes, config the pci_information

   pci_information =  { { 'device_id': "8086", 'vendor_id': "000[1-2]" }, { 'e.physical_netowrk': 'X' } }

• on controller

  pci_flavor_attrs = ['e.physical_netowrk']
  pci_alias = [{'e.physical_netowrk':'X', 'name':phyX_NIC', description: 'NIC connect to physical network X'}]

the compute node would report PCI stats group by ('e.group'). pci stats will report 1 pool:

 {'e.physical_netowrk':'X', 'count': 1 }

• create flavor and boot with it

 nova boot  mytest  --flavor m1.tiny  --image=cirros-0.3.1-x86_64-uec  --nic  net-id=network_X  pci_flavor= 'phyX_NIC:1'

Changes Plan

PCI Infomation Config Item

A new configuration item added to extend current PCI white list configuration items:

Select assignable PCI devices based on reduced regular expression

Configure additional arbitrary info for these devices, as any (k,v) pair, k and v is string.

Pci_flavor_attrs Config Items

Specify the PCI information and extra information that can be used for PCI alias and PCI stats.

Extending PCI Stats

Current PCI stats group devices only on [vendor_id, product_id, extra_info]. This will be extended to group by keys specified in pci_flavor_attrs.

Extending PCI Alias

PCI alias will be extended in IceHouse release and is planed to be replaced by PCI flavor if agreement achieved.

• Currently PCI alias can only accept combination of keys 'vendor_id, product_id,device_type' . This will be extended to support any keys specified in pci_flavor_attrs.
• Currently PCI alias can only compare if the PCI device information is equal to the one specified in PCI alias. This comparison will be extended to support reduced regular expression.

More PCI Request Information

Currently PCI request is passed to Nova through instance flavor. This will be extended to passing through network parameter.