Cyborg/TestReport/InspurFPGA
< Cyborg | TestReport
Revision as of 02:42, 1 September 2020 by Songwenping (talk | contribs) (→Inspur FPGA Cyborg Driver Test Report)
Contents
Inspur FPGA Cyborg Driver Test Report
This test report shows the current Inspur FPGA driver is well tested in local env where Inspur FPGA card located. Instead of a 3rd party CI, this report can be the proof.
report part
Enable Inspur FPGA driver in cyborg agent section
[root@node11 cyborg]$ cat /etc/kolla/cyborg/cyborg.conf | grep inspur enabled_drivers = inspur_fpga_driver
Report logs
Aug 20 17:06:48 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:48.909 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-4480-8178-134037b60524 - - - - -] Start differing devices. Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.501 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-eff4-4480-8178- 134037b60524 - - - - -] Creating resource provider Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0 Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.553 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-eff4-4480-8178- 134037b60524 - - - - -] [req-c5566486-40b5-4a20-b4f4-b0b96b7812c9] Created resource provider record via placement API for resource provider with UUID bcedfee6-c1f0-349c-acfa-7d39033b7b8c and name Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0. Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.715 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-eff4-4480-8178- 134037b60524 - - - - -] Created trait CUSTOM_FPGA_INSPUR Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.788 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-eff4-4480-8178- 134037b60524 - - - - -] Created trait CUSTOM_FPGA_PRODUCT_ID_A115 Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.950 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-4480-8178-134037b60524 - - - - -] Start differing deploybles. Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.955 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-4480-8178-134037b60524 - - - - -] Start differing attributes. Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.972 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-4480-8178-134037b60524 - - - - -] Start differing attach_handles. Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-4480-8178-134037b60524 - - - - -] new info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}'] Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-4480-8178-134037b60524 - - - - -] old info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}'] ...
check cyborg record
root@ubuntu:~# openstack accelerator device list +--------------------------------------+------+--------+----------+-----------------------------------------------------------------+ | uuid | type | vendor | hostname | std_board_info | +--------------------------------------+------+--------+----------+-----------------------------------------------------------------+ | 462a37e7-3654-44ab-b5bf-c462cd10f8a4 | FPGA | 1bd4 | ubuntu | {"product_id": "a115", "controller": "Processing accelerators"} | +--------------------------------------+------+--------+----------+-----------------------------------------------------------------
root@ubuntu:~# openstack accelerator deployable list +--------------------------------------+----------------------------------------------------------------------+-----------+ | uuid | name | device_id | +--------------------------------------+----------------------------------------------------------------------+-----------+ | 3355050d-6a21-4129-bab8-63280598684e | Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0 | 2 | +--------------------------------------+----------------------------------------------------------------------+-----------+
check placement record
[root@ubuntu cyborg]$ openstack resource provider list +--------------------------------------+----------------------------+------------+ | uuid | name | generation | +--------------------------------------+----------------------------+------------+ | bcedfee6-c1f0-349c-acfa-7d39033b7b8c | Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0 | 2 | | 1b3419ee-f45c-4b4e-ba91-7715b9070da8 | ubuntu | 60 |
[root@ubuntu cyborg]$ openstack resource provider inventory list bcedfee6-c1f0-349c-acfa-7d39033b7b8c +----------------+------------------+----------+----------+----------+-----------+-------+ | resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total | +----------------+------------------+----------+----------+----------+-----------+-------+ | FPGA | 1.0 | 1 | 1 | 0 | 1 | 1 | +----------------+------------------+----------+----------+----------+-----------+-------+
Boot a VM with Inspur FPGA attached
create device profile and flavor
root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile create inspur-fpga '[{"resources:FPGA":1, "trait:CUSTOM_FPGA_PRODUCT_ID_A115": "required", "trait:CUSTOM_FPGA_INSPUR": "required"}]' +------------+--------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------+--------------------------------------------------------------------------------------------------------------------+ | created_at | 2020-08-21 10:47:06+00:00 | | updated_at | None | | uuid | 63cac6f6-1122-46d1-825e-b288886ebdf5 | | name | inspur-fpga | | groups | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required', 'trait:CUSTOM_FPGA_INSPUR': 'required'}] | +------------+--------------------------------------------------------------------------------------------------------------------+
root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile list +--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+ | uuid | name | groups | +--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+ | 63cac6f6-1122-46d1-825e-b288886ebdf5 | inspur-fpga | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required', 'trait:CUSTOM_FPGA_INSPUR': 'required'}] |
root@ubuntu:/opt/stack/cyborg# nova flavor-key 3 set accel:device_profile='inspur-fpga' root@ubuntu:/opt/stack/cyborg# nova flavor-show 3 +----------------------------+-------------------------------------------------------------------+ | Property | Value | +----------------------------+-------------------------------------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | description | - | | disk | 40 | | extra_specs | {"accel:device_profile": "inspur-fpga", "hw_rng:allowed": "True"} | | id | 3 | | name | m1.medium | | os-flavor-access:is_public | True | | ram | 4096 | | rxtx_factor | 1.0 | | swap | 0 | | vcpus | 2 | +----------------------------+-------------------------------------------------------------------+
boot a VM with Inspur FPGA attached
In this step, I gave some output of cyborg-api log to show how Nova interact with Cyborg.
Nova get DP details
Aug 21 17:33:48 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:48.594 9365 INFO cyborg.api.controllers.v2.device_profiles [req-e89db88c-80f1-420f-b4b3-a38998f41f98 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [devi ce_profiles] get_all returned: {'device_profiles': [{'name': 'inspur-fpga', 'description': None, 'uuid': '63cac6f6-1122-46d1-825e-b28 8886ebdf5', 'groups': [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required', 'trait:CUSTOM_FPGA_INSPUR': 'required '}], 'created_at': '2020-08-21 09:13:29+00:00', 'updated_at': 'None', 'links': [{'href': 'http://10.180.249.199/accelerator/v2/device _profiles/63cac6f6-1122-46d1-825e-b288886ebdf5', 'rel': 'self'}]}]}
ARQ creation
Aug 18 02:03:10 vca-server devstack@cyborg-api.service[2243671]: 2020-08-18 02:03:10.052 2243674 INFO cyborg.api.controllers.v2.arqs [req-73c06ce1-b9df- 4d94-8760-e4ab5f595328 1f86481df3024d8bb74e213e23dcc2e4 7cd6c62783934a148605e271b4f6f922 - default default] [arq] post req = ({'device_profile_name': 'inspur-fpga'})
ARQ binding
Aug 21 17:33:49 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:49.980 9360 INFO cyborg.api.controllers.v2.arqs [req-f92f6 792-4730-4cb7-9932-9ca4f393500a 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs] patch. li st=({'549f40f9-ff45-4b55-8d7d-89f0f6822203': [{'path': '/hostname', 'op': 'add', 'value': 'ubuntu'}, {'path': '/device_rp_uuid', 'op' : 'add', 'value': 'bcedfee6-c1f0-349c-acfa-7d39033b7b8c'}, {'path': '/instance_uuid', 'op': 'add', 'value': '45716b59-7eaf-4663-bd7e- f83416f7ab48'}]})
Get bound ARQ
Aug 21 17:33:51 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:51.025 9366 INFO cyborg.api.controllers.v2.arqs [req-d094f906-3969-49d5-88e2- 8459a1b72158 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs:get_all] Returned: <cyborg.api.controllers.v2.arqs.ARQCollection object at 0x7fc2deb187b8>
Check the VM
log in to the VM , run`lspci | grep Inspur`
It will show a pci device which means the Inspur FPGA has been passthrough to the VM sucessfully.
[root@host-10-180-249-159 ~]# lspci|grep Inspur 00:05.0 Processing accelerators: Inspur Electronic Information Industry Co., Ltd. Device a115 (rev 04)
delete a VM with Inspur FPGA detached
In this step, I gave some output of cyborg and nova log to show how Nova interact with Cyborg.
Nova api delete vm request
Aug 22 09:54:48 ubuntu devstack@n-api.service[1808]: [pid: 1916|app: 0|req: 25/296] 10.180.249.199 () {70 vars in 1494 bytes} [Sat Au g 22 09:54:47 2020] DELETE /compute/v2.1/servers/45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes in 404 msecs (HTTP/1.1 204 ) 9 headers in 363 bytes (2 switches on core 0)
Nova compute start delete vm
Aug 22 09:54:48 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.manager [#033[01;36mNone req-a307b6cf-101a-4e9c-923f-a6fd4d 8f6e6f #033[00;36madmin admin#033[00;32m] #033[01;35m[instance: 45716b59-7eaf-4663-bd7e-f83416f7ab48] #033[00;32mStart destroying the instance on the hypervisor.#033[00m #033[00;33mTemplate:(pid=20353) shutdown instance /opt/stack/nova/nova/compute/manager.py:2837#033[00m#033[00m
Nova compute start delete arq
Aug 22 09:54:50 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.utils [#033[01;36mNone req-a307b6cf-101a-4e9c-923f-a6fd4d8f6e6f #033[00;36madmin admin#033[00;32m] #033[01;35m#033[00;32mCalling Cyborg to delete ARQs for instance 45716b59-7eaf-4663-bd7e- f83416f7ab48#033[00m #033[00;33mTemplate:(pid=20353) delete arqs if needed /opt/stack/nova/nova/compute/utils.py:1554
Cyborg api start delete arq
Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.466 9364 INFO cyborg.api.controllers.v2.arqs [req-3b7aca17-b96c-4d08-9c99- 7df8093fc927 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs] delete. instance=(45716b59-7eaf-4663- bd7e-f83416f7ab48)
Cyborg api delete arq
Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.487 9364 INFO cyborg.objects.ext_arq [req-3b7aca17-b96c-4d08-9c99- 7df8093fc927 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] Deleting obj_extarq uuid 549f40f9-ff45-4b55- 8d7d-89f0f6822203 for instance 45716b59-7eaf-4663-bd7e-f83416f7ab48#033[00m
Arq delete successfully
Aug 22 09:54:50 2020] DELETE /accelerator/v2/accelerator_requests?instance=45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 7 headers in 260 bytes (1 switches on core 0)