Cyborg/TestReport/InspurFPGA
< Cyborg | TestReport
Contents
Inspur FPGA Cyborg Driver Test Report
This test report shows the current Inspur FPGA driver is well tested in local env where Inspur FPGA card located. Instead of a 3rd party CI, this report can be the proof.
report part
Enable Inspur FPGA driver in cyborg agent section
[root@node11 cyborg]$ cat /etc/kolla/cyborg/cyborg.conf | grep inspur enabled_drivers = inspur_fpga_driver
Report logs
Aug 20 17:06:48 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:48.909 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4- 4480-8178-134037b60524 - - - - -] Start differing devices. Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.501 9441 INFO cyborg.common.placement_client [req-e9c7b9ec- eff4-4480-8178-134037b60524 - - - - -] Creating resource provider Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0 Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.553 9441 INFO cyborg.common.placement_client [req-e9c7b9ec- eff4-4480-8178-134037b60524 - - - - -] [req-c5566486-40b5-4a20-b4f4-b0b96b7812c9] Created resource provider record via placement API for resource provider with UUID bcedfee6-c1f0-349c-acfa-7d39033b7b8c and name Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0. Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.715 9441 INFO cyborg.common.placement_client [req-e9c7b9ec- eff4-4480-8178-134037b60524 - - - - -] Created trait CUSTOM_FPGA_INSPUR Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.788 9441 INFO cyborg.common.placement_client [req-e9c7b9ec- eff4-4480-8178- 134037b60524 - - - - -] Created trait CUSTOM_FPGA_PRODUCT_ID_A115 Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.950 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4- 4480-8178-134037b60524 - - - - -] Start differing deploybles. Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.955 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4- 4480-8178-134037b60524 - - - - -] Start differing attributes. Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.972 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4- 4480-8178-134037b60524 - - - - -] Start differing attach_handles. Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4- 4480-8178-134037b60524 - - - - -] new info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}'] Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4- 4480-8178-134037b60524 - - - - -] old info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}'] ...
check cyborg record
root@ubuntu:~# openstack accelerator device list +--------------------------------------+------+--------+----------+-----------------------------------------------------------------+ | uuid | type | vendor | hostname | std_board_info | +--------------------------------------+------+--------+----------+-----------------------------------------------------------------+ | 462a37e7-3654-44ab-b5bf-c462cd10f8a4 | FPGA | 1bd4 | ubuntu | {"product_id": "a115", "controller": "Processing accelerators"} | +--------------------------------------+------+--------+----------+-----------------------------------------------------------------
root@ubuntu:~# openstack accelerator deployable list +--------------------------------------+----------------------------------------------------------------------+-----------+ | uuid | name | device_id | +--------------------------------------+----------------------------------------------------------------------+-----------+ | 3355050d-6a21-4129-bab8-63280598684e | Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0 | 2 | +--------------------------------------+----------------------------------------------------------------------+-----------+
check placement record
[root@ubuntu cyborg]$ openstack resource provider list +--------------------------------------+----------------------------+------------+ | uuid | name | generation | +--------------------------------------+----------------------------+------------+ | bcedfee6-c1f0-349c-acfa-7d39033b7b8c | Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0 | 2 | | 1b3419ee-f45c-4b4e-ba91-7715b9070da8 | ubuntu | 60 |
[root@ubuntu cyborg]$ openstack resource provider inventory list bcedfee6-c1f0-349c-acfa-7d39033b7b8c +----------------+------------------+----------+----------+----------+-----------+-------+ | resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total | +----------------+------------------+----------+----------+----------+-----------+-------+ | FPGA | 1.0 | 1 | 1 | 0 | 1 | 1 | +----------------+------------------+----------+----------+----------+-----------+-------+
Boot a VM with Inspur FPGA attached
create device profile and flavor
root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile create inspur-fpga '[{"resources:FPGA":1, "trait:CUSTOM_FPGA_PRODUCT_ID_A115": "required", "trait:CUSTOM_FPGA_INSPUR": "required"}]' +------------+--------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------+--------------------------------------------------------------------------------------------------------------------+ | created_at | 2020-08-21 10:47:06+00:00 | | updated_at | None | | uuid | 63cac6f6-1122-46d1-825e-b288886ebdf5 | | name | inspur-fpga | | groups | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required', 'trait:CUSTOM_FPGA_INSPUR': 'required'}] | +------------+--------------------------------------------------------------------------------------------------------------------+
root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile list +--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+ | uuid | name | groups | +--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+ | 63cac6f6-1122-46d1-825e-b288886ebdf5 | inspur-fpga | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required', 'trait:CUSTOM_FPGA_INSPUR': 'required'}] |
root@ubuntu:/opt/stack/cyborg# nova flavor-key 3 set accel:device_profile='inspur-fpga' root@ubuntu:/opt/stack/cyborg# nova flavor-show 3 +----------------------------+-------------------------------------------------------------------+ | Property | Value | +----------------------------+-------------------------------------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | description | - | | disk | 40 | | extra_specs | {"accel:device_profile": "inspur-fpga", "hw_rng:allowed": "True"} | | id | 3 | | name | m1.medium | | os-flavor-access:is_public | True | | ram | 4096 | | rxtx_factor | 1.0 | | swap | 0 | | vcpus | 2 | +----------------------------+-------------------------------------------------------------------+
boot a VM with Inspur FPGA attached
In this step, I gave some output of cyborg-api log to show how Nova interact with Cyborg.
Nova get DP details
Aug 21 17:33:48 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:48.594 9365 INFO cyborg.api.controllers.v2.device_profiles [req-e89db88c-80f1-420f-b4b3-a38998f41f98 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [devi ce_profiles] get_all returned: {'device_profiles': [{'name': 'inspur-fpga', 'description': None, 'uuid': '63cac6f6-1122-46d1-825e-b28 8886ebdf5', 'groups': [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required', 'trait:CUSTOM_FPGA_INSPUR': 'required '}], 'created_at': '2020-08-21 09:13:29+00:00', 'updated_at': 'None', 'links': [{'href': 'http://10.180.249.199/accelerator/v2/device _profiles/63cac6f6-1122-46d1-825e-b288886ebdf5', 'rel': 'self'}]}]}
ARQ creation
Aug 18 02:03:10 vca-server devstack@cyborg-api.service[2243671]: 2020-08-18 02:03:10.052 2243674 INFO cyborg.api.controllers.v2.arqs [req-73c06ce1-b9df- 4d94-8760-e4ab5f595328 1f86481df3024d8bb74e213e23dcc2e4 7cd6c62783934a148605e271b4f6f922 - default default] [arq] post req = ({'device_profile_name': 'inspur-fpga'})
ARQ binding
Aug 21 17:33:49 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:49.980 9360 INFO cyborg.api.controllers.v2.arqs [req-f92f6 792-4730-4cb7-9932-9ca4f393500a 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs] patch. li st=({'549f40f9-ff45-4b55-8d7d-89f0f6822203': [{'path': '/hostname', 'op': 'add', 'value': 'ubuntu'}, {'path': '/device_rp_uuid', 'op' : 'add', 'value': 'bcedfee6-c1f0-349c-acfa-7d39033b7b8c'}, {'path': '/instance_uuid', 'op': 'add', 'value': '45716b59-7eaf-4663-bd7e- f83416f7ab48'}]})
Get bound ARQ
Aug 21 17:33:51 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:51.025 9366 INFO cyborg.api.controllers.v2.arqs [req-d094f906-3969-49d5-88e2- 8459a1b72158 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs:get_all] Returned: <cyborg.api.controllers.v2.arqs.ARQCollection object at 0x7fc2deb187b8>
Check the VM
log in to the VM , run`lspci | grep Inspur`
It will show a pci device which means the Inspur FPGA has been passthrough to the VM sucessfully.
[root@host-10-180-249-159 ~]# lspci|grep Inspur 00:05.0 Processing accelerators: Inspur Electronic Information Industry Co., Ltd. Device a115 (rev 04)
delete a VM with Inspur FPGA detached
In this step, I gave some output of cyborg and nova log to show how Nova interact with Cyborg.
Nova api delete vm request
Aug 22 09:54:48 ubuntu devstack@n-api.service[1808]: [pid: 1916|app: 0|req: 25/296] 10.180.249.199 () {70 vars in 1494 bytes} [Sat Au g 22 09:54:47 2020] DELETE /compute/v2.1/servers/45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes in 404 msecs (HTTP/1.1 204 ) 9 headers in 363 bytes (2 switches on core 0)
Nova compute start delete vm
Aug 22 09:54:48 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.manager [#033[01;36mNone req-a307b6cf-101a-4e9c-923f-a6fd4d 8f6e6f #033[00;36madmin admin#033[00;32m] #033[01;35m[instance: 45716b59-7eaf-4663-bd7e-f83416f7ab48] #033[00;32mStart destroying the instance on the hypervisor.#033[00m #033[00;33mTemplate:(pid=20353) shutdown instance /opt/stack/nova/nova/compute/manager.py:2837#033[00m#033[00m
Nova compute start delete arq
Aug 22 09:54:50 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.utils [#033[01;36mNone req-a307b6cf-101a-4e9c-923f-a6fd4d8f6e6f #033[00;36madmin admin#033[00;32m] #033[01;35m#033[00;32mCalling Cyborg to delete ARQs for instance 45716b59-7eaf-4663-bd7e- f83416f7ab48#033[00m #033[00;33mTemplate:(pid=20353) delete arqs if needed /opt/stack/nova/nova/compute/utils.py:1554
Cyborg api start delete arq
Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.466 9364 INFO cyborg.api.controllers.v2.arqs [req-3b7aca17-b96c-4d08-9c99- 7df8093fc927 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs] delete. instance=(45716b59-7eaf-4663- bd7e-f83416f7ab48)
Cyborg api delete arq
Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.487 9364 INFO cyborg.objects.ext_arq [req-3b7aca17-b96c-4d08-9c99- 7df8093fc927 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] Deleting obj_extarq uuid 549f40f9-ff45-4b55- 8d7d-89f0f6822203 for instance 45716b59-7eaf-4663-bd7e-f83416f7ab48#033[00m
Arq delete successfully
Aug 22 09:54:50 2020] DELETE /accelerator/v2/accelerator_requests?instance=45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 7 headers in 260 bytes (1 switches on core 0)