Jump to: navigation, search

Difference between revisions of "Cyborg/TestReport/InspurFPGA"

(Cyborg api delete arq)
(Get bound ARQ)
 
(25 intermediate revisions by 2 users not shown)
Line 8: Line 8:
 
     enabled_drivers = inspur_fpga_driver
 
     enabled_drivers = inspur_fpga_driver
  
===== Report logs =====
+
==== Report logs ====
     Aug 20 17:06:48 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:48.909 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-4480-8178-134037b60524 -  
+
     Aug 20 17:06:48 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:48.909 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
    - - - -] Start differing devices.
+
    4480-8178-134037b60524 - - - - -] Start differing devices.
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.501 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-eff4-4480-8178-
+
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.501 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
    134037b60524 - - - - -] Creating resource provider Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0
+
    eff4-4480-8178-134037b60524 - - - - -] Creating resource provider
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.553 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-eff4-4480-8178-
+
    Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0
    134037b60524 - - - - -] [req-c5566486-40b5-4a20-b4f4-b0b96b7812c9] Created resource provider record via placement API for resource provider with UUID  
+
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.553 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
    bcedfee6-c1f0-349c-acfa-7d39033b7b8c and name Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0.
+
    eff4-4480-8178-134037b60524 - - - - -] [req-c5566486-40b5-4a20-b4f4-b0b96b7812c9] Created resource provider record via placement
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.715 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-eff4-4480-8178-
+
    API for resource provider with UUID bcedfee6-c1f0-349c-acfa-7d39033b7b8c and name
    134037b60524 - - - - -] Created trait CUSTOM_FPGA_INSPUR
+
    Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0.
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.788 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-eff4-4480-8178-
+
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.715 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
    134037b60524 - - - - -] Created trait CUSTOM_FPGA_PRODUCT_ID_A115
+
    eff4-4480-8178-134037b60524 - - - - -] Created trait CUSTOM_FPGA_INSPUR
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.950 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-4480-8178-134037b60524 -  
+
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.788 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
    - - - -] Start differing deploybles.
+
    eff4-4480-8178- 134037b60524 - - - - -] Created trait CUSTOM_FPGA_PRODUCT_ID_A115
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.955 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-4480-8178-134037b60524 -  
+
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.950 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
    - - - -] Start differing attributes.
+
    4480-8178-134037b60524 - - - - -] Start differing deploybles.
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.972 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-4480-8178-134037b60524 -  
+
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.955 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
    - - - -] Start differing attach_handles.
+
    4480-8178-134037b60524 - - - - -] Start differing attributes.
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-4480-8178-134037b60524 -  
+
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.972 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
    - - - -] new info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}']
+
    4480-8178-134037b60524 - - - - -] Start differing attach_handles.
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-4480-8178-134037b60524 -  
+
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
    - - - -] old info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}']
+
    4480-8178-134037b60524 - - - - -] new info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}']
 +
     Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
 +
    4480-8178-134037b60524 - - - - -] old info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}']  
 
     ...
 
     ...
  
===== check cyborgrecord =====
+
==== check cyborg record ====
 
     root@ubuntu:~# openstack accelerator device list
 
     root@ubuntu:~# openstack accelerator device list
 
     +--------------------------------------+------+--------+----------+-----------------------------------------------------------------+
 
     +--------------------------------------+------+--------+----------+-----------------------------------------------------------------+
Line 47: Line 49:
 
     +--------------------------------------+----------------------------------------------------------------------+-----------+
 
     +--------------------------------------+----------------------------------------------------------------------+-----------+
  
===== check placement record =====
+
==== check placement record ====
 
      
 
      
 
     [root@ubuntu cyborg]$ openstack resource provider list
 
     [root@ubuntu cyborg]$ openstack resource provider list
Line 67: Line 69:
  
 
==== create device profile and flavor ====
 
==== create device profile and flavor ====
     root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile create inspur-fpga '[{"resources:FPGA":1, "trait:CUSTOM_FPGA_PRODUCT_ID_A115":  
+
     root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile create inspur-fpga '[{"resources:FPGA":1, "trait:CUSTOM_FPGA_PRODUCT_ID_A115": "required",  "trait:CUSTOM_FPGA_INSPUR": "required"}]'
    "required",  "trait:CUSTOM_FPGA_INSPUR": "required"}]'
+
 
    +------------+--------------------------------------------------------------------------------------------------------------------+
 
    | Field      | Value                                                                                                              |
 
    +------------+--------------------------------------------------------------------------------------------------------------------+
 
    | created_at | 2020-08-21 10:47:06+00:00                                                                                          |
 
    | updated_at | None                                                                                                              |
 
    | uuid      | 63cac6f6-1122-46d1-825e-b288886ebdf5                                                                              |
 
    | name      | inspur-fpga                                                                                                      |
 
    | groups    | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required', 'trait:CUSTOM_FPGA_INSPUR': 'required'}] |
 
 
     +------------+--------------------------------------------------------------------------------------------------------------------+
 
     +------------+--------------------------------------------------------------------------------------------------------------------+
 +
    | Field      | Value                                                                                                                                  |
 +
    +---------------+-----------------------------------------------------------------------------------------------------------------+
 +
    | created_at  | 2020-08-21 10:47:06+00:00                                                                                          |
 +
    | updated_at | None                                                                                                                              |
 +
    | uuid            | 63cac6f6-1122-46d1-825e-b288886ebdf5                                                                    |
 +
    | name          | inspur-fpga                                                                                                                    |
 +
    | groups        | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required',
 +
                            'trait:CUSTOM_FPGA_INSPUR': 'required'}]                                                              |
 +
    +---------------+------------------------------------------------------------------------------------------------------------- ---+
  
 
     root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile list
 
     root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile list
 
     +--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+
 
     +--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+
     | uuid                                 | name        | groups                                                                                                             |
+
     | uuid                                       | name        | groups                                                                                                                               |
 
     +--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+
 
     +--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+
     | 63cac6f6-1122-46d1-825e-b288886ebdf5 | inspur-fpga | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required', 'trait:CUSTOM_FPGA_INSPUR': 'required'}] |
+
     | 63cac6f6-1122-46d1-825e-b288886ebdf5 | inspur-fpga | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required',
 +
                                                                                                  'trait:CUSTOM_FPGA_INSPUR': 'required'}] |
  
 
     root@ubuntu:/opt/stack/cyborg# nova flavor-key 3 set accel:device_profile='inspur-fpga'  
 
     root@ubuntu:/opt/stack/cyborg# nova flavor-key 3 set accel:device_profile='inspur-fpga'  
Line 108: Line 112:
  
 
===== Nova get DP details =====
 
===== Nova get DP details =====
     Aug 21 17:33:48 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:48.594 9365 INFO cyborg.api.controllers.v2.device_profiles
+
     Aug 21 17:33:48 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:48.594 9365 INFO
     [req-e89db88c-80f1-420f-b4b3-a38998f41f98 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [devi
+
    cyborg.api.controllers.v2.device_profiles
    ce_profiles] get_all returned: {'device_profiles': [{'name': 'inspur-fpga', 'description': None, 'uuid': '63cac6f6-1122-46d1-825e-b28
+
     [req-e89db88c-80f1-420f-b4b3-a38998f41f98 613b4ba7e86a4058b458e4e9496a0507
    8886ebdf5', 'groups': [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required', 'trait:CUSTOM_FPGA_INSPUR': 'required
+
    fef0949e2ce6415ab5cd001ba254e563 - default default] [device_profiles] get_all returned:
    '}], 'created_at': '2020-08-21 09:13:29+00:00', 'updated_at': 'None', 'links': [{'href': 'http://10.180.249.199/accelerator/v2/device
+
    {'device_profiles': [{'name': 'inspur-fpga', 'description': None, 'uuid': '63cac6f6-1122-46d1-825e-
 +
      b288886ebdf5', 'groups': [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required',
 +
    'trait:CUSTOM_FPGA_INSPUR': 'required'}], 'created_at': '2020-08-21 09:13:29+00:00',
 +
    'updated_at': 'None', 'links': [{'href': 'http://10.180.249.199/accelerator/v2/device
 
     _profiles/63cac6f6-1122-46d1-825e-b288886ebdf5', 'rel': 'self'}]}]}
 
     _profiles/63cac6f6-1122-46d1-825e-b288886ebdf5', 'rel': 'self'}]}]}
  
 
===== ARQ creation =====
 
===== ARQ creation =====
     Aug 18 02:03:10 vca-server devstack@cyborg-api.service[2243671]: 2020-08-18 02:03:10.052 2243674 INFO cyborg.api.controllers.v2.arqs [req-73c06ce1-b9df-
+
     Aug 18 02:03:10 vca-server devstack@cyborg-api.service[2243671]: 2020-08-18 02:03:10.052 2243674 INFO
    4d94-8760-e4ab5f595328 1f86481df3024d8bb74e213e23dcc2e4 7cd6c62783934a148605e271b4f6f922 - default default] [arq] post req = ({'device_profile_name':
+
    cyborg.api.controllers.v2.arqs [req-73c06ce1-b9df-4d94-8760-e4ab5f595328 1f86481df3024d8bb74e213e23dcc2e4
    'inspur-fpga'})
+
    7cd6c62783934a148605e271b4f6f922 - default default] [arq] post req = ({'device_profile_name': 'inspur-fpga'})
  
 
===== ARQ binding =====
 
===== ARQ binding =====
     Aug 21 17:33:49 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:49.980 9360 INFO cyborg.api.controllers.v2.arqs [req-f92f6
+
     Aug 21 17:33:49 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:49.980 9360 INFO
    792-4730-4cb7-9932-9ca4f393500a 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs] patch. li
+
    cyborg.api.controllers.v2.arqs [req-f92f6792-4730-4cb7-9932-9ca4f393500a 613b4ba7e86a4058b458e4e9496a0507
    st=({'549f40f9-ff45-4b55-8d7d-89f0f6822203': [{'path': '/hostname', 'op': 'add', 'value': 'ubuntu'}, {'path': '/device_rp_uuid', 'op'
+
    fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs] patch. list=({'549f40f9-ff45-4b55-8d7d-89f0f6822203': [
    : 'add', 'value': 'bcedfee6-c1f0-349c-acfa-7d39033b7b8c'}, {'path': '/instance_uuid', 'op': 'add', 'value': '45716b59-7eaf-4663-bd7e-
+
    {'path': '/hostname', 'op': 'add', 'value': 'ubuntu'}, {'path': '/device_rp_uuid', 'op': 'add',
    f83416f7ab48'}]})
+
    'value': 'bcedfee6-c1f0-349c-acfa-7d39033b7b8c'}, {'path': '/instance_uuid', 'op': 'add',
 +
    'value': '45716b59-7eaf-4663-bd7e- f83416f7ab48'}]})
  
 
===== Get bound ARQ =====
 
===== Get bound ARQ =====
     Aug 21 17:33:51 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:51.025 9366 INFO cyborg.api.controllers.v2.arqs [req-d094f906-3969-49d5-88e2-
+
     Aug 21 17:33:51 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:51.025 9366 INFO cyborg.api.controllers.v2.arqs
    8459a1b72158 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs:get_all] Returned:
+
    [req-d094f906-3969-49d5-88e2-8459a1b72158 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563
    <cyborg.api.controllers.v2.arqs.ARQCollection object at 0x7fc2deb187b8>
+
    - default default] [arqs:get_all] Returned:<cyborg.api.controllers.v2.arqs.ARQCollection object at 0x7fc2deb187b8>
  
 
==== Check the VM ====
 
==== Check the VM ====
Line 145: Line 153:
  
 
==== Nova api delete vm request ====
 
==== Nova api delete vm request ====
     Aug 22 09:54:48 ubuntu devstack@n-api.service[1808]: [pid: 1916|app: 0|req: 25/296] 10.180.249.199 () {70 vars in 1494 bytes} [Sat Au
+
     Aug 22 09:54:48 ubuntu devstack@n-api.service[1808]: [pid: 1916|app: 0|req: 25/296] 10.180.249.199 ()
    g 22 09:54:47 2020] DELETE /compute/v2.1/servers/45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes in 404 msecs (HTTP/1.1 204
+
    {70 vars in 1494 bytes} [Sat Aug 22 09:54:47 2020]
    ) 9 headers in 363 bytes (2 switches on core 0)
+
    DELETE /compute/v2.1/servers/45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes
 +
    in 404 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0)
  
 
==== Nova compute start delete vm ====
 
==== Nova compute start delete vm ====
     Aug 22 09:54:48 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.manager [#033[01;36mNone req-a307b6cf-101a-4e9c-923f-a6fd4d
+
     Aug 22 09:54:48 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.manager [#033[01;36m None
    8f6e6f #033[00;36madmin admin#033[00;32m] #033[01;35m[instance: 45716b59-7eaf-4663-bd7e-f83416f7ab48] #033[00;32mStart destroying the instance on  
+
    req-a307b6cf-101a-4e9c-923f-a6fd4d8f6e6f #033[00;36madmin admin#033[00;32m] #033[01;35m
    the hypervisor.#033[00m #033[00;33m{{(pid=20353) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:2837}}#033[00m#033[00m
+
    [instance: 45716b59-7eaf-4663-bd7e-f83416f7ab48] #033[00;32mStart destroying the instance on the hypervisor.
 +
    #033[00m #033[00;33m{{(pid=20353) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:2837}}#033[00m#033[00m
  
 
==== Nova compute start delete arq ====
 
==== Nova compute start delete arq ====
     Aug 22 09:54:50 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.utils [#033[01;36mNone req-a307b6cf-101a-4e9c-923f-a6fd4d8f6e6f  
+
     Aug 22 09:54:50 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.utils [#033[01;36mNone
     #033[00;36madmin admin#033[00;32m] #033[01;35m#033[00;32mCalling Cyborg to delete ARQs for instance 45716b59-7eaf-4663-bd7e-
+
    req-a307b6cf-101a-4e9c-923f-a6fd4d8f6e6f  
    f83416f7ab48#033[00m #033[00;33m{{(pid=20353) delete_arqs_if_needed /opt/stack/nova/nova/compute/utils.py:1554}}
+
     #033[00;36madmin admin#033[00;32m] #033[01;35m#033[00;32mCalling Cyborg to delete ARQs for instance
 +
    45716b59-7eaf-4663-bd7e-f83416f7ab48#033[00m #033[00;33m{{(pid=20353) delete_arqs_if_needed
 +
    /opt/stack/nova/nova/compute/utils.py:1554}}
  
 
==== Cyborg api start delete arq ====
 
==== Cyborg api start delete arq ====
     Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.466 9364 INFO cyborg.api.controllers.v2.arqs [req-3b7aca17-b96c-4d08-9c99-
+
     Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.466 9364 INFO
     7df8093fc927 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs] delete. instance=(45716b59-7eaf-4663-
+
    cyborg.api.controllers.v2.arqs [req-3b7aca17-b96c-4d08-9c99-7df8093fc927
    bd7e-f83416f7ab48)
+
     613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 -
 +
    default default] [arqs] delete. instance=(45716b59-7eaf-4663-bd7e-f83416f7ab48)
  
 
==== Cyborg api delete arq ====
 
==== Cyborg api delete arq ====
     Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.487 9364 INFO cyborg.objects.ext_arq [req-3b7aca17-b96c-4d08-9c99-
+
     Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.487 9364 INFO
     7df8093fc927 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] Deleting obj_extarq uuid 549f40f9-ff45-4b55-
+
    cyborg.objects.ext_arq [req-3b7aca17-b96c-4d08-9c99-7df8093fc927
    8d7d-89f0f6822203 for instance 45716b59-7eaf-4663-bd7e-f83416f7ab48#033[00m
+
     613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default]
 +
    Deleting obj_extarq uuid 549f40f9-ff45-4b55-8d7d-89f0f6822203
 +
    for instance 45716b59-7eaf-4663-bd7e-f83416f7ab48#033[00m
  
=== Arq delete successfully ===
+
==== Arq delete successfully ====
     Aug 22 09:54:50 2020] DELETE /accelerator/v2/accelerator_requests?instance=45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes
+
     Aug 22 09:54:50 2020] DELETE /accelerator/v2/accelerator_requests?instance=
 +
    45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes
 
     in 80 msecs (HTTP/1.1 204) 7 headers in 260 bytes (1 switches on core 0)
 
     in 80 msecs (HTTP/1.1 204) 7 headers in 260 bytes (1 switches on core 0)

Latest revision as of 06:37, 1 September 2020

Inspur FPGA Cyborg Driver Test Report

This test report shows the current Inspur FPGA driver is well tested in local env where Inspur FPGA card located. Instead of a 3rd party CI, this report can be the proof.

report part

Enable Inspur FPGA driver in cyborg agent section

   [root@node11 cyborg]$ cat /etc/kolla/cyborg/cyborg.conf  | grep inspur
   enabled_drivers = inspur_fpga_driver

Report logs

   Aug 20 17:06:48 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:48.909 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
   4480-8178-134037b60524 - - - - -] Start differing devices.
   Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.501 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
   eff4-4480-8178-134037b60524 - - - - -] Creating resource provider
   Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0
   Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.553 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
   eff4-4480-8178-134037b60524 - - - - -] [req-c5566486-40b5-4a20-b4f4-b0b96b7812c9] Created resource provider record via placement
   API for resource provider with UUID bcedfee6-c1f0-349c-acfa-7d39033b7b8c and name
   Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0.
   Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.715 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
   eff4-4480-8178-134037b60524 - - - - -] Created trait CUSTOM_FPGA_INSPUR
   Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.788 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
   eff4-4480-8178- 134037b60524 - - - - -] Created trait CUSTOM_FPGA_PRODUCT_ID_A115
   Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.950 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
   4480-8178-134037b60524 -  - - - -] Start differing deploybles.
   Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.955 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
   4480-8178-134037b60524 - - - - -] Start differing attributes.
   Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.972 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
   4480-8178-134037b60524 - - - - -] Start differing attach_handles.
   Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
   4480-8178-134037b60524 - - - - -] new info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}']
   Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
   4480-8178-134037b60524 - - - - -] old info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}']    
   ...

check cyborg record

   root@ubuntu:~# openstack accelerator device list
   +--------------------------------------+------+--------+----------+-----------------------------------------------------------------+
   | uuid                                 | type | vendor | hostname | std_board_info                                                  |
   +--------------------------------------+------+--------+----------+-----------------------------------------------------------------+
   | 462a37e7-3654-44ab-b5bf-c462cd10f8a4 | FPGA | 1bd4   | ubuntu   | {"product_id": "a115", "controller": "Processing accelerators"} |
   +--------------------------------------+------+--------+----------+-----------------------------------------------------------------
   root@ubuntu:~# openstack accelerator deployable list
   +--------------------------------------+----------------------------------------------------------------------+-----------+
   | uuid                                 | name                                                                 | device_id |
   +--------------------------------------+----------------------------------------------------------------------+-----------+
   | 3355050d-6a21-4129-bab8-63280598684e | Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0 |         2 |
   +--------------------------------------+----------------------------------------------------------------------+-----------+

check placement record

   [root@ubuntu cyborg]$ openstack resource provider list
   +--------------------------------------+----------------------------+------------+
   | uuid                                 | name                       | generation |
   +--------------------------------------+----------------------------+------------+
   | bcedfee6-c1f0-349c-acfa-7d39033b7b8c  | Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0                 |        2 |
   | 1b3419ee-f45c-4b4e-ba91-7715b9070da8 |  ubuntu |          60 |
   [root@ubuntu cyborg]$ openstack resource provider inventory list bcedfee6-c1f0-349c-acfa-7d39033b7b8c
   +----------------+------------------+----------+----------+----------+-----------+-------+
   | resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total |
   +----------------+------------------+----------+----------+----------+-----------+-------+
   | FPGA         |              1.0 |        1  |        1  |        0  |         1   |     1  |
   +----------------+------------------+----------+----------+----------+-----------+-------+


Boot a VM with Inspur FPGA attached

create device profile and flavor

   root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile create inspur-fpga '[{"resources:FPGA":1, "trait:CUSTOM_FPGA_PRODUCT_ID_A115": "required",  "trait:CUSTOM_FPGA_INSPUR": "required"}]'
   +------------+--------------------------------------------------------------------------------------------------------------------+
   | Field       | Value                                                                                                                                  |
   +---------------+-----------------------------------------------------------------------------------------------------------------+
   | created_at  | 2020-08-21 10:47:06+00:00                                                                                           |
   | updated_at | None                                                                                                                               |
   | uuid            | 63cac6f6-1122-46d1-825e-b288886ebdf5                                                                     |
   | name          | inspur-fpga                                                                                                                     |
   | groups        | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required',
                            'trait:CUSTOM_FPGA_INSPUR': 'required'}]                                                               |
   +---------------+------------------------------------------------------------------------------------------------------------- ---+
   root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile list
   +--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+
   | uuid                                       | name        | groups                                                                                                                               |
   +--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+
   | 63cac6f6-1122-46d1-825e-b288886ebdf5 | inspur-fpga | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required',
                                                                                                 'trait:CUSTOM_FPGA_INSPUR': 'required'}] |
   root@ubuntu:/opt/stack/cyborg# nova flavor-key 3 set accel:device_profile='inspur-fpga' 
   root@ubuntu:/opt/stack/cyborg# nova flavor-show 3
   +----------------------------+-------------------------------------------------------------------+
   | Property                   | Value                                                             |
   +----------------------------+-------------------------------------------------------------------+
   | OS-FLV-DISABLED:disabled   | False                                                             |
   | OS-FLV-EXT-DATA:ephemeral  | 0                                                                 |
   | description                | -                                                                 |
   | disk                       | 40                                                                |
   | extra_specs                | {"accel:device_profile": "inspur-fpga", "hw_rng:allowed": "True"} |
   | id                         | 3                                                                 |
   | name                       | m1.medium                                                         |
   | os-flavor-access:is_public | True                                                              |
   | ram                        | 4096                                                              |
   | rxtx_factor                | 1.0                                                               |
   | swap                       | 0                                                                 |
   | vcpus                      | 2                                                                 |
   +----------------------------+-------------------------------------------------------------------+

boot a VM with Inspur FPGA attached

In this step, I gave some output of cyborg-api log to show how Nova interact with Cyborg.

Nova get DP details
   Aug 21 17:33:48 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:48.594 9365 INFO
   cyborg.api.controllers.v2.device_profiles
   [req-e89db88c-80f1-420f-b4b3-a38998f41f98 613b4ba7e86a4058b458e4e9496a0507
   fef0949e2ce6415ab5cd001ba254e563 - default default] [device_profiles] get_all returned:
   {'device_profiles': [{'name': 'inspur-fpga', 'description': None, 'uuid': '63cac6f6-1122-46d1-825e-
     b288886ebdf5', 'groups': [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required',
   'trait:CUSTOM_FPGA_INSPUR': 'required'}], 'created_at': '2020-08-21 09:13:29+00:00',
   'updated_at': 'None', 'links': [{'href': 'http://10.180.249.199/accelerator/v2/device
   _profiles/63cac6f6-1122-46d1-825e-b288886ebdf5', 'rel': 'self'}]}]}
ARQ creation
   Aug 18 02:03:10 vca-server devstack@cyborg-api.service[2243671]: 2020-08-18 02:03:10.052 2243674 INFO
   cyborg.api.controllers.v2.arqs [req-73c06ce1-b9df-4d94-8760-e4ab5f595328 1f86481df3024d8bb74e213e23dcc2e4
   7cd6c62783934a148605e271b4f6f922 - default default] [arq] post req = ({'device_profile_name': 'inspur-fpga'})
ARQ binding
   Aug 21 17:33:49 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:49.980 9360 INFO
   cyborg.api.controllers.v2.arqs [req-f92f6792-4730-4cb7-9932-9ca4f393500a 613b4ba7e86a4058b458e4e9496a0507
   fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs] patch. list=({'549f40f9-ff45-4b55-8d7d-89f0f6822203': [
   {'path': '/hostname', 'op': 'add', 'value': 'ubuntu'}, {'path': '/device_rp_uuid', 'op': 'add',
   'value': 'bcedfee6-c1f0-349c-acfa-7d39033b7b8c'}, {'path': '/instance_uuid', 'op': 'add',
   'value': '45716b59-7eaf-4663-bd7e- f83416f7ab48'}]})
Get bound ARQ
   Aug 21 17:33:51 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:51.025 9366 INFO cyborg.api.controllers.v2.arqs
   [req-d094f906-3969-49d5-88e2-8459a1b72158 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563
   - default default] [arqs:get_all] Returned:<cyborg.api.controllers.v2.arqs.ARQCollection object at 0x7fc2deb187b8>

Check the VM

log in to the VM , run`lspci | grep Inspur`

It will show a pci device which means the Inspur FPGA has been passthrough to the VM sucessfully.

   [root@host-10-180-249-159 ~]# lspci|grep Inspur
   00:05.0 Processing accelerators: Inspur Electronic Information Industry Co., Ltd. Device a115 (rev 04)


delete a VM with Inspur FPGA detached

In this step, I gave some output of cyborg and nova log to show how Nova interact with Cyborg.

Nova api delete vm request

   Aug 22 09:54:48 ubuntu devstack@n-api.service[1808]: [pid: 1916|app: 0|req: 25/296] 10.180.249.199 ()
   {70 vars in 1494 bytes} [Sat Aug 22 09:54:47 2020]
   DELETE /compute/v2.1/servers/45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes
   in 404 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0)

Nova compute start delete vm

   Aug 22 09:54:48 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.manager [#033[01;36m None
   req-a307b6cf-101a-4e9c-923f-a6fd4d8f6e6f #033[00;36madmin admin#033[00;32m] #033[01;35m
   [instance: 45716b59-7eaf-4663-bd7e-f83416f7ab48] #033[00;32mStart destroying the instance on the hypervisor.
   #033[00m #033[00;33mTemplate:(pid=20353) shutdown instance /opt/stack/nova/nova/compute/manager.py:2837#033[00m#033[00m

Nova compute start delete arq

   Aug 22 09:54:50 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.utils [#033[01;36mNone
   req-a307b6cf-101a-4e9c-923f-a6fd4d8f6e6f 
   #033[00;36madmin admin#033[00;32m] #033[01;35m#033[00;32mCalling Cyborg to delete ARQs for instance
   45716b59-7eaf-4663-bd7e-f83416f7ab48#033[00m #033[00;33m{{(pid=20353) delete_arqs_if_needed
   /opt/stack/nova/nova/compute/utils.py:1554}}

Cyborg api start delete arq

   Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.466 9364 INFO
   cyborg.api.controllers.v2.arqs [req-3b7aca17-b96c-4d08-9c99-7df8093fc927
   613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 -
   default default] [arqs] delete. instance=(45716b59-7eaf-4663-bd7e-f83416f7ab48)

Cyborg api delete arq

   Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.487 9364 INFO
   cyborg.objects.ext_arq [req-3b7aca17-b96c-4d08-9c99-7df8093fc927
   613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default]
   Deleting obj_extarq uuid 549f40f9-ff45-4b55-8d7d-89f0f6822203
   for instance 45716b59-7eaf-4663-bd7e-f83416f7ab48#033[00m

Arq delete successfully

   Aug 22 09:54:50 2020] DELETE /accelerator/v2/accelerator_requests?instance=
   45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes
   in 80 msecs (HTTP/1.1 204) 7 headers in 260 bytes (1 switches on core 0)