Cyborg/TestReport/InspurFPGA
< Cyborg | TestReport
Revision as of 06:34, 1 September 2020 by Zhangbailin (talk | contribs) (→Cyborg api start delete arq)
Contents
Inspur FPGA Cyborg Driver Test Report
This test report shows the current Inspur FPGA driver is well tested in local env where Inspur FPGA card located. Instead of a 3rd party CI, this report can be the proof.
report part
Enable Inspur FPGA driver in cyborg agent section
[root@node11 cyborg]$ cat /etc/kolla/cyborg/cyborg.conf | grep inspur enabled_drivers = inspur_fpga_driver
Report logs
Aug 20 17:06:48 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:48.909 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
4480-8178-134037b60524 - - - - -] Start differing devices.
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.501 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
eff4-4480-8178-134037b60524 - - - - -] Creating resource provider
Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.553 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
eff4-4480-8178-134037b60524 - - - - -] [req-c5566486-40b5-4a20-b4f4-b0b96b7812c9] Created resource provider record via placement
API for resource provider with UUID bcedfee6-c1f0-349c-acfa-7d39033b7b8c and name
Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0.
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.715 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
eff4-4480-8178-134037b60524 - - - - -] Created trait CUSTOM_FPGA_INSPUR
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.788 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
eff4-4480-8178- 134037b60524 - - - - -] Created trait CUSTOM_FPGA_PRODUCT_ID_A115
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.950 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
4480-8178-134037b60524 - - - - -] Start differing deploybles.
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.955 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
4480-8178-134037b60524 - - - - -] Start differing attributes.
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.972 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
4480-8178-134037b60524 - - - - -] Start differing attach_handles.
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
4480-8178-134037b60524 - - - - -] new info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}']
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
4480-8178-134037b60524 - - - - -] old info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}']
...
check cyborg record
root@ubuntu:~# openstack accelerator device list
+--------------------------------------+------+--------+----------+-----------------------------------------------------------------+
| uuid | type | vendor | hostname | std_board_info |
+--------------------------------------+------+--------+----------+-----------------------------------------------------------------+
| 462a37e7-3654-44ab-b5bf-c462cd10f8a4 | FPGA | 1bd4 | ubuntu | {"product_id": "a115", "controller": "Processing accelerators"} |
+--------------------------------------+------+--------+----------+-----------------------------------------------------------------
root@ubuntu:~# openstack accelerator deployable list +--------------------------------------+----------------------------------------------------------------------+-----------+ | uuid | name | device_id | +--------------------------------------+----------------------------------------------------------------------+-----------+ | 3355050d-6a21-4129-bab8-63280598684e | Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0 | 2 | +--------------------------------------+----------------------------------------------------------------------+-----------+
check placement record
[root@ubuntu cyborg]$ openstack resource provider list +--------------------------------------+----------------------------+------------+ | uuid | name | generation | +--------------------------------------+----------------------------+------------+ | bcedfee6-c1f0-349c-acfa-7d39033b7b8c | Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0 | 2 | | 1b3419ee-f45c-4b4e-ba91-7715b9070da8 | ubuntu | 60 |
[root@ubuntu cyborg]$ openstack resource provider inventory list bcedfee6-c1f0-349c-acfa-7d39033b7b8c +----------------+------------------+----------+----------+----------+-----------+-------+ | resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total | +----------------+------------------+----------+----------+----------+-----------+-------+ | FPGA | 1.0 | 1 | 1 | 0 | 1 | 1 | +----------------+------------------+----------+----------+----------+-----------+-------+
Boot a VM with Inspur FPGA attached
create device profile and flavor
root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile create inspur-fpga '[{"resources:FPGA":1, "trait:CUSTOM_FPGA_PRODUCT_ID_A115": "required", "trait:CUSTOM_FPGA_INSPUR": "required"}]'
+------------+--------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+---------------+-----------------------------------------------------------------------------------------------------------------+
| created_at | 2020-08-21 10:47:06+00:00 |
| updated_at | None |
| uuid | 63cac6f6-1122-46d1-825e-b288886ebdf5 |
| name | inspur-fpga |
| groups | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required',
'trait:CUSTOM_FPGA_INSPUR': 'required'}] |
+---------------+------------------------------------------------------------------------------------------------------------- ---+
root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile list
+--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+
| uuid | name | groups |
+--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+
| 63cac6f6-1122-46d1-825e-b288886ebdf5 | inspur-fpga | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required',
'trait:CUSTOM_FPGA_INSPUR': 'required'}] |
root@ubuntu:/opt/stack/cyborg# nova flavor-key 3 set accel:device_profile='inspur-fpga'
root@ubuntu:/opt/stack/cyborg# nova flavor-show 3
+----------------------------+-------------------------------------------------------------------+
| Property | Value |
+----------------------------+-------------------------------------------------------------------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| description | - |
| disk | 40 |
| extra_specs | {"accel:device_profile": "inspur-fpga", "hw_rng:allowed": "True"} |
| id | 3 |
| name | m1.medium |
| os-flavor-access:is_public | True |
| ram | 4096 |
| rxtx_factor | 1.0 |
| swap | 0 |
| vcpus | 2 |
+----------------------------+-------------------------------------------------------------------+
boot a VM with Inspur FPGA attached
In this step, I gave some output of cyborg-api log to show how Nova interact with Cyborg.
Nova get DP details
Aug 21 17:33:48 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:48.594 9365 INFO
cyborg.api.controllers.v2.device_profiles
[req-e89db88c-80f1-420f-b4b3-a38998f41f98 613b4ba7e86a4058b458e4e9496a0507
fef0949e2ce6415ab5cd001ba254e563 - default default] [device_profiles] get_all returned:
{'device_profiles': [{'name': 'inspur-fpga', 'description': None, 'uuid': '63cac6f6-1122-46d1-825e-
b288886ebdf5', 'groups': [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required',
'trait:CUSTOM_FPGA_INSPUR': 'required'}], 'created_at': '2020-08-21 09:13:29+00:00',
'updated_at': 'None', 'links': [{'href': 'http://10.180.249.199/accelerator/v2/device
_profiles/63cac6f6-1122-46d1-825e-b288886ebdf5', 'rel': 'self'}]}]}
ARQ creation
Aug 18 02:03:10 vca-server devstack@cyborg-api.service[2243671]: 2020-08-18 02:03:10.052 2243674 INFO
cyborg.api.controllers.v2.arqs [req-73c06ce1-b9df-4d94-8760-e4ab5f595328 1f86481df3024d8bb74e213e23dcc2e4
7cd6c62783934a148605e271b4f6f922 - default default] [arq] post req = ({'device_profile_name': 'inspur-fpga'})
ARQ binding
Aug 21 17:33:49 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:49.980 9360 INFO
cyborg.api.controllers.v2.arqs [req-f92f6792-4730-4cb7-9932-9ca4f393500a 613b4ba7e86a4058b458e4e9496a0507
fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs] patch. list=({'549f40f9-ff45-4b55-8d7d-89f0f6822203': [
{'path': '/hostname', 'op': 'add', 'value': 'ubuntu'}, {'path': '/device_rp_uuid', 'op': 'add',
'value': 'bcedfee6-c1f0-349c-acfa-7d39033b7b8c'}, {'path': '/instance_uuid', 'op': 'add',
'value': '45716b59-7eaf-4663-bd7e- f83416f7ab48'}]})
Get bound ARQ
Aug 21 17:33:51 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:51.025 9366 INFO cyborg.api.controllers.v2.arqs [req-d094f906-3969-49d5-88e2- 8459a1b72158 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs:get_all] Returned: <cyborg.api.controllers.v2.arqs.ARQCollection object at 0x7fc2deb187b8>
Check the VM
log in to the VM , run`lspci | grep Inspur`
It will show a pci device which means the Inspur FPGA has been passthrough to the VM sucessfully.
[root@host-10-180-249-159 ~]# lspci|grep Inspur 00:05.0 Processing accelerators: Inspur Electronic Information Industry Co., Ltd. Device a115 (rev 04)
delete a VM with Inspur FPGA detached
In this step, I gave some output of cyborg and nova log to show how Nova interact with Cyborg.
Nova api delete vm request
Aug 22 09:54:48 ubuntu devstack@n-api.service[1808]: [pid: 1916|app: 0|req: 25/296] 10.180.249.199 ()
{70 vars in 1494 bytes} [Sat Aug 22 09:54:47 2020]
DELETE /compute/v2.1/servers/45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes
in 404 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0)
Nova compute start delete vm
Aug 22 09:54:48 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.manager [#033[01;36m None req-a307b6cf-101a-4e9c-923f-a6fd4d8f6e6f #033[00;36madmin admin#033[00;32m] #033[01;35m [instance: 45716b59-7eaf-4663-bd7e-f83416f7ab48] #033[00;32mStart destroying the instance on the hypervisor. #033[00m #033[00;33mTemplate:(pid=20353) shutdown instance /opt/stack/nova/nova/compute/manager.py:2837#033[00m#033[00m
Nova compute start delete arq
Aug 22 09:54:50 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.utils [#033[01;36mNone
req-a307b6cf-101a-4e9c-923f-a6fd4d8f6e6f
#033[00;36madmin admin#033[00;32m] #033[01;35m#033[00;32mCalling Cyborg to delete ARQs for instance
45716b59-7eaf-4663-bd7e-f83416f7ab48#033[00m #033[00;33m{{(pid=20353) delete_arqs_if_needed
/opt/stack/nova/nova/compute/utils.py:1554}}
Cyborg api start delete arq
Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.466 9364 INFO cyborg.api.controllers.v2.arqs [req-3b7aca17-b96c-4d08-9c99-7df8093fc927 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs] delete. instance=(45716b59-7eaf-4663-bd7e-f83416f7ab48)
Cyborg api delete arq
Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.487 9364 INFO cyborg.objects.ext_arq [req-3b7aca17-b96c-4d08-9c99- 7df8093fc927 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] Deleting obj_extarq uuid 549f40f9-ff45-4b55- 8d7d-89f0f6822203 for instance 45716b59-7eaf-4663-bd7e-f83416f7ab48#033[00m
Arq delete successfully
Aug 22 09:54:50 2020] DELETE /accelerator/v2/accelerator_requests?instance=45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 7 headers in 260 bytes (1 switches on core 0)
