Jump to: navigation, search

Difference between revisions of "Cyborg/TestReport/IntelQAT"

(Created page with "== Intel QAT Cyborg Driver Test Report == This test report shows the current qat driver is well tested in local env where QAT card located. Instead of a 3rd party CI, this re...")
 
(No difference)

Latest revision as of 09:37, 27 August 2020

Intel QAT Cyborg Driver Test Report

This test report shows the current qat driver is well tested in local env where QAT card located. Instead of a 3rd party CI, this report can be the proof.

report part

Enable qat driver in cyborg agent section

   $ cat /etc/cyborg/cyborg.conf  | grep intel
   enabled_drivers = intel_qat_driver
Report logs
   cyborg-conductor: we can see cyborg conductor call placement API to create releated resource provider
   Aug 18 01:34:28 vca-server cyborg-conductor[2243750]: 2020-08-18 01:34:28.929 2243750 INFO cyborg.conductor.manager [req-08b86085-4bc4-4cce-9fcd-9c60e99fc
   c33 - - - - -] Start differing devices.
   Aug 18 01:34:29 vca-server cyborg-conductor[2243750]: 2020-08-18 01:34:29.085 2243750 INFO cyborg.common.placement_client [req-08b86085-4bc4-4cce-9fcd-9c6
   0e99fcc33 - - - - -] Resource class  CUSTOM_QAT already exists
   Aug 18 01:34:29 vca-server cyborg-conductor[2243750]: 2020-08-18 01:34:29.093 2243750 INFO cyborg.common.placement_client [req-08b86085-4bc4-4cce-9fcd-9c6
   0e99fcc33 - - - - -] Creating resource provider intel-qat-dev_0000:3d:02.7
   Aug 18 01:34:29 vca-server cyborg-conductor[2243750]: 2020-08-18 01:34:29.102 2243750 INFO cyborg.common.placement_client [req-08b86085-4bc4-4cce-9fcd-9c6
   0e99fcc33 - - - - -] [req-418258f7-a564-4d25-a81a-9e51062a27f3] Created resource provider record via placement API for resource provider with UUID 17cedae
   e-f4ad-3521-b815-b2f14c4fd621 and name intel-qat-dev_0000:3d:02.7.
   Aug 18 01:34:29 vca-server cyborg-conductor[2243750]: 2020-08-18 01:34:29.160 2243750 INFO cyborg.common.placement_client [req-08b86085-4bc4-4cce-9fcd-9c6
   0e99fcc33 - - - - -] Resource class  CUSTOM_QAT already exists
   Aug 18 01:34:29 vca-server cyborg-conductor[2243750]: 2020-08-18 01:34:29.168 2243750 INFO cyborg.common.placement_client [req-08b86085-4bc4-4cce-9fcd-9c6
   0e99fcc33 - - - - -] Creating resource provider intel-qat-dev_0000:3d:02.0
   Aug 18 01:34:29 vca-server cyborg-conductor[2243750]: 2020-08-18 01:34:29.180 2243750 INFO cyborg.common.placement_client [req-08b86085-4bc4-4cce-9fcd-9c6
   0e99fcc33 - - - - -] [req-2d09804c-89a8-4299-bf78-0afa1760a85f] Created resource provider record via placement API for resource provider with UUID b937b6d
   5-2ac0-3746-b3ba-f7792887a81f and name intel-qat-dev_0000:3d:02.0.
   Aug 18 01:34:29 vca-server cyborg-conductor[2243750]: 2020-08-18 01:34:29.237 2243750 INFO cyborg.common.placement_client [req-08b86085-4bc4-4cce-9fcd-9c6
   0e99fcc33 - - - - -] Resource class  CUSTOM_QAT already exists
   ...
check placement record
   $ openstack resource provider list
   +--------------------------------------+----------------------------+---------------+
   | uuid                                  | name                         | generation |
   +--------------------------------------+----------------------------+---------------+
   | 8fc1b918-04e2-4ba1-be7d-69387d6803d8 | vca-server                 |         68 |
   | 17cedaee-f4ad-3521-b815-b2f14c4fd621 | intel-qat-dev_0000:3d:02.7 |          1 |
   | b937b6d5-2ac0-3746-b3ba-f7792887a81f | intel-qat-dev_0000:3d:02.0 |          1 |
   | 9e189749-2304-3aaf-b590-cab4209a5cbc | intel-qat-dev_0000:3d:02.5 |          1 |
   | 57841a47-b674-3fe4-a54d-12fbd6addeeb | intel-qat-dev_0000:3d:01.6 |          1 |
   | d127b5b3-39a0-3c89-9c5d-88bc4afa14bc | intel-qat-dev_0000:3d:02.3 |          1 |
   | 456bf96e-5a8d-36bb-b5ff-11c2a7a8ba3b | intel-qat-dev_0000:3d:01.4 |          1 |
   | 5bfd1cff-5c51-3cdc-a992-0cc0c92431b4 | intel-qat-dev_0000:3d:01.2 |          1 |
   | f19ed211-f71e-3d14-9149-4c06e47a965c | intel-qat-dev_0000:3d:01.0 |          1 |
   ....

There are 48 RPs created, because in this env, there are 3 QAT PFs, each of them is virtualized into 16 VFs.

   $ openstack resource provider list | grep qat | wc -l
   48
   $ openstack resource provider inventory list d6cdd568-2d87-3668-a0aa-5d19787d4ce2
   +----------------+------------------+----------+----------+----------+-----------+-------+
   | resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total |
   +----------------+------------------+----------+----------+----------+-----------+-------+
   | CUSTOM_QAT     |              1.0 |        1 |        1 |        0 |         1 |     1 |
   +----------------+------------------+----------+----------+----------+-----------+-------+

Boot a VM with QAT attached

create device profile and flavor

   $ echo $BODY | python3 -m json.tool
   [
       {
           "name": "QAT",
           "groups": [
               {
                   "resources:CUSTOM_QAT": "1"
               }
           ]
       }
   ]
   $ curl -s -H "$CTYPE" -H "$AUTH" POST -d "$BODY" $CYURL/device_profiles
   {"name": "QAT", "description": null, "uuid": "dd9d2759-eec3-4bfe-af95-24773d28e8f4", "groups": [{"resources:CUSTOM_QAT": "1"}], "created_at": "2020-08-18
   05:50:48+00:00", "updated_at": "None", "links": [{"href": "http://localhost/accelerator/v2/device_profiles/dd9d2759-eec3-4bfe-af95-24773d28e8f4", "rel": $self"}]}
   $ openstack flavor create fpga-flavor --property accel:device_profile='QAT' --disk 5
   +----------------------------+--------------------------------------+
   | Field                      | Value                                |
   +----------------------------+--------------------------------------+
   | OS-FLV-DISABLED:disabled   | False                                |
   | OS-FLV-EXT-DATA:ephemeral  | 0                                    |
   | disk                       | 5                                    |
   | id                         | f518569f-812a-4382-bfd3-9dc61ed848c9 |
   | name                       | fpga-flavor                          |
   | os-flavor-access:is_public | True                                 |
   | properties                 | accel:device_profile='QAT'           |
   | ram                        | 256                                  |
   | rxtx_factor                | 1.0                                  |
   | swap                       |                                      |
   | vcpus                      | 1                                    |
   +----------------------------+--------------------------------------+

boot a VM with QAT attached

In this step, I gave some output of cyborg-api log to show how Nova interact with Cyborg.

Nova get DP details
   Aug 18 02:03:09 vca-server devstack@cyborg-api.service[2243671]: 2020-08-18 02:03:09.631 2243678 INFO cyborg.api.controllers.v2.device_profiles [req-75741c76-593a-40bf- 
   8955-80eb0ec29c63 1f86481df3024d8bb74e213e23dcc2e4 7cd6c62783934a148605e271b4f6f922 - default default] [device_profiles] get_all returned: {
   'device_profiles': [{'name': 'QAT', 'description': None, 'uuid': 'dd9d2759-eec3-4bfe-af95-24773d28e8f4', 'groups': [{'resources:CUSTOM_QAT': '1'}], 'created_at': '2020-08-18 
   05:50:48+00:00', 'updated_at': 'None', 'links': [{'href': 'http://10.238.153.121/accelerator/v2/device_profiles/dd9d2759-eec3-4bfe-af9
   5-24773d28e8f4', 'rel': 'self'}]}]}  
ARQ creation
   Aug 18 02:03:10 vca-server devstack@cyborg-api.service[2243671]: 2020-08-18 02:03:10.052 2243674 INFO cyborg.api.controllers.v2.arqs [req-73c06ce1-b9df-4d94-8760- 
  e4ab5f595328 1f86481df3024d8bb74e213e23dcc2e4 7cd6c62783934a148605e271b4f6f922 - default default] [arq] post req = ({'device_profile_name': 'QAT'})
ARQ binding
   Aug 18 02:03:10 vca-server devstack@cyborg-api.service[2243671]: 2020-08-18 02:03:10.118 2243675 INFO cyborg.api.controllers.v2.arqs [req-fba1cad3-bccc-4c
   4a-9a78-2cc145c4798b 1f86481df3024d8bb74e213e23dcc2e4 7cd6c62783934a148605e271b4f6f922 - default default] [arqs] patch. list=({'0fff41ad-487e-4a3d-8395-2c
   4d35142c1f': [{'path': '/hostname', 'op': 'add', 'value': 'vca-server'}, {'path': '/device_rp_uuid', 'op': 'add', 'value': 'fb328624-7775-3d1e-bb36-478200
   ea1a5a'}, {'path': '/instance_uuid', 'op': 'add', 'value': '92080153-a69a-4f1e-9ee7-34298673ad1e'}]})
Get bound ARQ
   Aug 18 02:03:10 vca-server devstack@cyborg-api.service[2243671]: 2020-08-18 02:03:10.711 2243672 INFO cyborg.api.controllers.v2.arqs [req-bb8b6928-137e-4b
   87-aac0-5fe2cecc22d7 1f86481df3024d8bb74e213e23dcc2e4 7cd6c62783934a148605e271b4f6f922 - default default] [arqs:get_all] Returned: <cyborg.api.controllers
   .v2.arqs.ARQCollection object at 0x7f8c14002128>

Check the VM

Log in to the VM , run`lspci | grep 37c9`

It will show a pci device which means the QAT vf has been passthrough to the VM successfully.

   $ lspci |  grep 37c9
   08:00.1 Intel Corporation Device 37c9 (rev 04)