Jump to: navigation, search

Cyborg/TestReport/IntelNic

Intel x710 Nic Cyborg Driver Test Report

This test report shows the current intel nic driver is well tested in local env where Intel x710 Niccard located. Instead of a 3rd party CI, this report can be the proof.

report part

Enable intel nic driver in cyborg agent section

  • add intel_nic_driver in enabled_drivers
  • set "enabled_nic_types" to "x710_static", this means the pre-porgrammed NICs are supported.
  • set "physical_device_mappings" and "function_device_mappings" in "x710_static" section, indicating tha mapping relation between physnet/device, function/device.
   $ vim /etc/cyborg/cyborg.conf
   [agent]
   enabled_drivers = fake_driver,intel_nic_driver
   [nic_devices]
   enabled_nic_types = x710_static
   [x710_static]
   physical_device_mappings = physnet1:eth2|eth3
   function_device_mappings = GTPv1:eth3|eth2
  • restart cyborg-agent
   $ systemctl restart devstack@cyborg-agent
Report logs
   cyborg-conductor: we can see cyborg conductor call placement API to create releated resource providerFeb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 
   03:10:04.476 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca-2595b464051c - - - - -] Creating resource provider newbuy1217.maas_0000:86:02.3
   Feb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 03:10:04.485 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca- 
   2595b464051c - - - - -] [req-b6cbe2bb-201a-4c86-9789-02c0a4648be1] Created resource provider record via placement API for resource provider with UUID 5fe904cf-5d52-3dff-9a25- 
   635841deed1a and name newbuy1217.maas_0000:86:02.3.
   Feb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 03:10:04.520 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca- 
   2595b464051c - - - - -] Trait CUSTOM_VF already existed
   Feb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 03:10:04.530 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca- 
   2595b464051c - - - - -] Trait CUSTOM_GTPV1 already existed
   Feb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 03:10:04.540 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca- 
   2595b464051c - - - - -] Trait CUSTOM_PHYSNET1 already existed
   Feb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 03:10:04.580 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca- 
   2595b464051c - - - - -] Resource class  CUSTOM_NIC already exists
   Feb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 03:10:04.587 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca- 
   2595b464051c - - - - -] Creating resource provider newbuy1217.maas_0000:86:02.1


check placement record
   $ openstack resource provider list
   +--------------------------------------+------------------------------+------------+
   | uuid                                 | name                         | generation |
   +--------------------------------------+------------------------------+------------+
   | 5977653e-12df-319f-a420-17716b7e5123 | newbuy1217.maas_0000:86:00.1 |          2 |
   | e71b41e9-da06-3b85-8131-b753c01c224b | newbuy1217.maas_0000:86:02.2 |          2 |
   | 84cd0653-049d-36c7-8ad3-d02f9bad4a8c | newbuy1217.maas_0000:86:02.0 |          2 |
   | 5fe904cf-5d52-3dff-9a25-635841deed1a | newbuy1217.maas_0000:86:02.3 |          2 |
   | e95f79d4-8ac1-3b0d-b1ce-002cb073fc9c | newbuy1217.maas_0000:86:02.1 |          2 |
   +--------------------------------------+------------------------------+------------+


There are 5 RPs created because in this env, there are 2 NIC PFs, one of them is virtualized into 6 VFs, the other is not virtualized.

   $ openstack resource provider inventory list 5fe904cf-5d52-3dff-9a25-635841deed1a
   +----------------+------------------+----------+----------+----------+-----------+-------+
   | resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total |
   +----------------+------------------+----------+----------+----------+-----------+-------+
   | CUSTOM_NIC     |              1.0 |        1 |        1 |        0 |         1 |     1 |
   +----------------+------------------+----------+----------+----------+-----------+-------+

Boot a VM with NIC VF attached

create device profile and flavor

   $ export GRP=[{\"resources\:CUSTOM_NIC\"\:\"1\"\,\"trait:CUSTOM_GTPV1\":\"required\"\,\"trait:CUSTOM_VF\":\"required\"}]
   $ openstack accelerator device profile create sriovdp $GRP
   $ openstack accelerator device profile show d30efbf9-bdf5-45d1-9d34-74d720121971
   +------------+--------------------------------------------------------------------------------------------------+
   | Field      | Value                                                                                            |
   +------------+--------------------------------------------------------------------------------------------------+
   | created_at | 2021-01-14T08:11:32+00:00                                                                        |
   | updated_at | None                                                                                             |
   | uuid       | d30efbf9-bdf5-45d1-9d34-74d720121971                                                             |
   | name       | sriovdp                                                                                       |
   | groups     | [{'resources:CUSTOM_NIC': '1', 'trait:CUSTOM_GTPV1': 'required', 'trait:CUSTOM_VF': 'required'}] |
   +------------+--------------------------------------------------------------------------------------------------+
   $ openstack port create test --network public  --device-profile new-sriovdp --vnic-type=accelerator-direct
   $ neutron port-show 299e665f-502c-42f8-b744-058d8ead5a55
   neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
   +-----------------------+--------------------------------------------------------------------------------------+
   | Field                 | Value                                                                                | 
   +-----------------------+--------------------------------------------------------------------------------------+
   | admin_state_up        | True                                                                                 |
   | allowed_address_pairs |                                                                                      |
   | binding:host_id       |                                                                                      |
   | binding:profile       | {"physical_network": "physnet1"}                                                     |
   | binding:vif_details   | {}                                                                                   |
   | binding:vif_type      | unbound                                                                              |
   | binding:vnic_type     | accelerator-direct                                                                   |
   | created_at            | 2021-01-14T08:13:05Z                                                                 |
   | description           |                                                                                      |
   | device_id             |                                                                                      |
   | device_owner          |                                                                                      |
   | device_profile        | new-sriovdp                                                                          |
   | extra_dhcp_opts       |                                                                                      |
   | fixed_ips             | {"subnet_id": "26921a61-a89b-44d9-ba3a-6135cfe2ffec", "ip_address": "172.24.4.173"}  |
   |                       | {"subnet_id": "82914187-5c21-4a78-a787-a7befc10c8d1", "ip_address": "2001:db8::3aa"} |
   | id                    | 299e665f-502c-42f8-b744-058d8ead5a55                                                 |
   | mac_address           | fa:16:3e:3e:28:7b                                                                    |
   | name                  | test                                                                                 |
   | network_id            | db1a7132-6469-4b75-965e-dc103b1b04cb                                                 |
   | project_id            | 6aa3db65287949cfab355baff912a5f3                                                     |
   | revision_number       | 143                                                                                  |
   | security_groups       | 2fcaedb7-b148-4597-9bc4-6a5173034125                                                 |
   | status                | DOWN                                                                                 |
   | tags                  |                                                                                      |
   | tenant_id             | 6aa3db65287949cfab355baff912a5f3                                                     |
   | updated_at            | 2021-01-29T07:41:41Z                                                                 |
   +-----------------------+--------------------------------------------------------------------------------------+


boot a VM with x710 NIC VF attached

   $ export ID=299e665f-502c-42f8-b744-058d8ead5a55
   $ nova boot testbackend  --flavor m1.tiny --image cirros-0.5.1-x86_64-disk  --nic port-id=$ID
Nova get DP details
   Feb 03 03:24:30 newbuy1217.maas devstack@cyborg-api.service[1836730]: [pid: 1836744|app: 0|req: 8/181] 192.168.0.186 () {60 va
   rs in 1207 bytes} [Wed Feb  3 03:24:30 2021] GET /accelerator/v2/device_profiles => generated 738 bytes in 126 msecs (HTTP/1.1
   200) 8 headers in 286 bytes (2 switches on core 0)
   Feb 03 03:29:34 newbuy1217.maas devstack@cyborg-api.service[1836730]: 2021-02-03 03:29:34.389 1836748 INFO cyborg.api.controll
   ers.v2.device_profiles [req-31b7d118-2df2-403e-8d5d-205be1de2145 50fe054b2deb4669abccf2f3ef980eb9 6aa3db65287949cfab355baff912
   a5f3 - default default] [device_profiles] get_all returned: <cyborg.api.controllers.v2.device_profiles.DeviceProfileCollection
   object at 0x7f915b0e6e10>
   Feb 03 03:29:34 newbuy1217.maas devstack@cyborg-api.service[1836730]: [pid: 1836748|app: 0|req: 8/182] 192.168.0.186 () {60 va
   rs in 1375 bytes} [Wed Feb  3 03:29:34 2021] GET /accelerator/v2/device_profiles?name=new-sriovdp => generated 414 bytes in 22
   msecs (HTTP/1.1 200) 8 headers in 286 bytes (1 switches on core 0)


ARQ creation
    Feb 03 03:29:34 newbuy1217.maas devstack@cyborg-api.service[1836730]: 2021-02-03 03:29:34.409 1836750 INFO cyborg.api.controll
   ers.v2.arqs [req-a7f6c123-f415-49d9-8aba-ba21582a41e5 50fe054b2deb4669abccf2f3ef980eb9 6aa3db65287949cfab355baff912a5f3 - defa
   ult default] [arq] post req = ({'device_profile_name': 'new-sriovdp'})


ARQ binding
   Feb 03 03:29:34 newbuy1217.maas devstack@cyborg-api.service[1836730]: 2021-02-03 03:29:34.926 1836747 INFO cyborg.api.controll
   ers.v2.arqs [req-dc455c20-9b91-46b9-a3a5-076a8e5f3688 50fe054b2deb4669abccf2f3ef980eb9 6aa3db65287949cfab355baff912a5f3 - defa
   ult default] [arqs] patch. list=({'b9bf6023-f1c2-40ec-ab0d-04c3c4958e94': [{'path': '/hostname', 'op': 'add', 'value': 'newbuy
   1217.maas'}, {'path': '/device_rp_uuid', 'op': 'add', 'value': 'e95f79d4-8ac1-3b0d-b1ce-002cb073fc9c'}, {'path': '/instance_uu
   id', 'op': 'add', 'value': '1fd31114-40cb-4f0f-af49-794545ba8e38'}]})
Get bound ARQ
   Feb 03 03:29:35 newbuy1217.maas devstack@cyborg-api.service[1836730]: [pid: 1836755|app: 0|req: 8/186] 192.168.0.186 () {58 vars in 1248 bytes} [Wed Feb  3 03:29:35 2021] 
   GET /accelerator/v2/accelerator_requests?instance=1fd31114-40cb-4f0f-af49-794545ba8e38 => generated 680 bytes in 18 msecs (HTTP/1.1 200) 8 headers in 286 bytes (2 switches 
   on core 0)

Check the VM

Log in to the VM , run`lspci`

It will show a pci device which means the x710 NIC vf has been passthrough to the VM successfully.

   $ lspci
   00:01.2 Class 0c03: 8086:7020
   00:01.0 Class 0601: 8086:7000
   00:04.0 Class 0200: 8086:154c
   00:00.0 Class 0600: 8086:1237
   00:01.3 Class 0680: 8086:7113
   ...

We can see 8086:154c which is a x710 Nic' VF.


Nova would reject the server move or suspend

$ openstack server shelve testbackend Forbidden with instances that have accelerators. (HTTP 403) (Request-ID: req-9b988fb9-159f-43df-8f70-ef358f6b1672)

$ openstack server resize testbackend --flavor 3 Forbidden with instances that have accelerators. (HTTP 403) (Request-ID: req-9779d326-647d-4ad4-80b9-a5e1bdaaa436)

]$ openstack server suspend testbackend Forbidden with instances that have accelerators. (HTTP 403) (Request-ID: req-c827af77-c37b-4d66-833b-741781a7e03c)

$ openstack server evacuate testbackend Forbidden with instances that have accelerators. (HTTP 403) (Request-ID: req-21fdf02b-32a0-45b1-b424-8b54614943dc)

$ openstack server migrate --live-migration testbackend Forbidden with instances that have accelerators. (HTTP 403) (Request-ID: req-75591909-29f2-465a-9d16-a31e955e498a)