Jump to: navigation, search

Difference between revisions of "Cyborg"

(Related Industry Efforts)
(Development Release Schedule)
 
(84 intermediate revisions by 5 users not shown)
Line 1: Line 1:
 
==Overview==
 
==Overview==
  
Cyborg (previously known as [[Nomad]]) is an OpenStack project that aims to provide a general purpose management framework for acceleration resources (i.e. various types of accelerators such as Crypto cards,GPU, FPGA, NVMe/NOF SSDs, ODP, DPDK/SPDK and so on).
+
Cyborg (previously known as [[Nomad]]) is an OpenStack project that aims to provide a general purpose management framework for acceleration resources (i.e. various types of accelerators such as GPU, FPGA, ASIC, NP, SoCs, NVMe/NOF SSDs, ODP, DPDK/SPDK and so on).
  
===Development Trello Board===
+
== Projects ==
 +
=== Cyborg ===
  
https://trello.com/b/4nFtHNSg/queens-dev
+
{| border="1" cellpadding="2"
 +
| Official name
 +
| '''OpenStack Accelerator'''
 +
|-
 +
| Source code
 +
| https://github.com/openstack/cyborg
 +
|-
 +
| Bug and Feature tracker
 +
| https://storyboard.openstack.org/#!/project/openstack/cyborg
 +
|-
 +
| Developer doc
 +
|
 +
|}
  
===Overall Design===
 
  
* API-DB: https://review.openstack.org/445814
+
We align with the official release schedule:https://releases.openstack.org/, with three milestones.
* Conductor-Agent: https://review.openstack.org/446091
+
We use storyboard to track all bugs and features. The cyborg basic board page[], with three worklists, track specific tasks in the three milestones.
* Generic Driver: https://review.openstack.org/447257
 
* Cyborg-Nova: https://review.openstack.org/448228, https://review.openstack.org/508572 (Queens Take 2)
 
* Sandbox: https://review.openstack.org/461220, https://review.openstack.org/444999
 
  
===Initial Architecture===
+
=== Python Cyborg Client ===
 +
{| border="1" cellpadding="2"
 +
| Source code
 +
| https://github.com/openstack/python-cyborgclient
 +
|-
 +
| Bug and Feature tracker
 +
| https://storyboard.openstack.org/#!/project/openstack/python-cyborgclient
 +
|
 +
|}
  
[[File: Cyborg_arch_correct_1.0.png|300px|center|thumb|Cyborg Architecture 1.0]]
+
=== Cyborg Tempest Plugin ===
 +
{| border="1" cellpadding="2"
 +
| Source code
 +
| https://github.com/openstack/cyborg-tempest-plugin
 +
|-
 +
| Bug and Feature tracker
 +
| https://storyboard.openstack.org/#!/project/openstack/cyborg-tempest-plugin
 +
|
 +
|}
  
==Related Industry Efforts==
+
== People ==
FD.IO: https://fd.io/
 
  
SPDK: http://www.spdk.io/
+
* PTL
 +
** Xinran Wang(xinranwang)
 +
* Cyborg Code Reviewers
 +
**[https://review.opendev.org/#/admin/groups/1243,members cyborg-core-team]
 +
* Blueprint Czar (responsible for maintenance of Cyborg's blueprint lists)
 +
** Same as PTL
 +
* [[https://wiki.openstack.org/wiki/CrossProjectLiaisons Release Management Liaisons]]
 +
** Same as PTL
 +
* [[https://wiki.openstack.org/wiki/CrossProjectLiaisons Oslo  Liaisons]]
 +
** Shaohe Feng(shaohe_feng)
 +
* [[https://wiki.openstack.org/wiki/CrossProjectLiaisons QA  Liaisons]]
 +
**Chenke(chenker)
 +
* [[https://wiki.openstack.org/wiki/CrossProjectLiaisons Doc Liaisons]]
 +
**Xinran Wang(xinranwang)
 +
* Nova Group Liaisons(works with nova team around Cyborg things)
 +
**Brin Zhang(brinzhang)
 +
**Xinran Wang(xinranwang)
 +
* Neutron Group Liaisons(works with neutron team around Cyborg things)
 +
**Xinran Wang(Xinran)
  
Oak Ridge National Lab Quantum Computing Acceleration: https://ornl-qci.github.io/xacc/  
+
==Meeting/Materials==
 +
 
 +
* Please refer to [[Meetings/CyborgTeamMeeting]] for Cyborg weekly meeting agenda and logistic info
 +
* Please refer to [[Cyborg/Materials]] for additional materials regarding Cyborg.
 +
* Please refer to [[Cyborg/MeetingLogs]] for meeting minutes archives.
 +
* Please refer to [[Cyborg/FirstContact]] for local on-boarding meeting agenda and logistic info
 +
 
 +
== General Resources ==
 +
NOTE: Please be aware that this page is incomplete, we will continually update this page! We will not remove this note until we think that all the imperative documentation is done here. If you need any help in using Cyborg, please just ping in the IRC channel #openstack-cyborg.
 +
 
 +
=== Supported Device Driver List and Test Report===
 +
[https://wiki.openstack.org/wiki/Cyborg/TestReport Driver Test Report]
 +
 
 +
=== Accelerator API reference ===
 +
* [https://docs.openstack.org/api-ref/accelerator/ Accelerator API reference]
 +
 
 +
=== End User Guide ===
 +
* [https://docs.openstack.org/cyborg/latest/#for-end-users End User Guide]
 +
 
 +
=== Releasenotes ===
 +
* cyborg (and other projects under the [https://governance.openstack.org/tc/reference/projects/nova.html compute program in governance]) are released through the standard openstack release process: https://github.com/openstack/releases/blob/master/README.rst
 +
* [https://releases.openstack.org/teams/cyborg.html All cyborg releasenotes]
 +
*[https://releases.openstack.org/ussuri/highlights.html#cyborg Latest cyborg releasenote]
  
Neuromorphic Computing: https://github.com/nengo/nengo
+
== Resources for Contributors ==
  
OpenCL: https://www.khronos.org/opencl/
+
=== Contributor Documentation ===
  
OpenML: http://www.openmp.org/
+
* [http://docs.openstack.org/infra/manual/developers.html General Developer's Guide]
 +
* [ TODO(yumeng): add doc here once get merged Cyborg Architecture and long term plan Docs]
 +
* [ TODO(yumeng): add doc here once get merged Cyborg-driver process]
  
CUDA: https://developer.nvidia.com/cuda-toolkit
+
=== Bug Report and Feature Registration Guide===
 +
After Rocky, Cyborg has migrated from [https://launchpad.net/openstack-cyborg cyborg-launchpad] to [https://storyboard.openstack.org/#!/project_group/77 cyborg-storyboard] to track bugs and features. But we were not using it very well. From Victoria, we will use it, use it well, and use it often.
 +
If time is very limited for you to walk through all the documentation, go directly to the Cyborg Specific Storyboard Usage Guide, and please pay close attention to the tags!
  
OpenACC: https://developer.nvidia.com/openacc
+
*[https://docs.openstack.org/infra/storyboard/#using-storyboard General Storyboard Usage]
 +
*[https://wiki.openstack.org/wiki/Cyborg/CyborgStoryboard Cyborg Specific Storyboard Usage Guide]
  
RISC-V: https://riscv.org/  
+
=== Development Release Schedule===
 +
* [https://wiki.openstack.org/wiki/Cyborg/Yoga_Release_Schedule Cyborg Yoga Release Schedule]
 +
* [https://wiki.openstack.org/wiki/Cyborg/Xena_Release_Schedule Cyborg Xena Release Schedule]
 +
* [https://wiki.openstack.org/wiki/Cyborg/Wallaby_Release_Schedule Cyborg Wallaby Release Schedule]
 +
* [https://wiki.openstack.org/wiki/Cyborg/Victoria_Release_Schedule Cyborg Victoria Release Schedule]
  
eBPF: https://www.iovisor.org/technology/ebpf
+
== History ==
  
==Meeting/Materials==
+
Howard (Zhipeng) Huang started OpenStack acceleration mission discussion from Telco requirements and build the Great Founding Team of Cyborg.
 +
Under his leadership, we established the basic infrastructure for sustained growth, resulting in our official release in Rocky. Stein release, with Li Liu as the PTL, we have made substantial progress, particularly in defining a clean database schema and device model that will form the basis for future work. In both previous cycles, we had several illuminating discussions with Nova developers to define how Cyborg would interact with Nova. In the following Train and Ussuri releases, under Sundar Nadathur's leadership, Cyborg has reached an exciting and significant milestone, where Cyborg integration with Nova (compute service) has been completed. Users can now use Nova to launch server instances with accelerators provisioned with the Cyborg service.
  
* Please refer to [[Meetings/CyborgTeamMeeting]] for meeting agenda and logistic info
 
* Please refer to [[Cyborg/Materials]] for additional materials regarding Cyborg.
 
* Please refer to [[Cyborg/MeetingLogs]] for meeting minutes archives.
 
* Please refer to [[Cyborg/FirstContact]] for local on-boarding meeting agenda and logistic info
 
  
==Team Photo==
+
* Previous PTLs
 +
**Zhipeng Huang, Rocky Release
 +
**Li Liu, Stein Release
 +
**Sundar Nadathur, Train and Ussuri Releases
 +
** Yumeng Bao, Victoria and Wallaby Release
  
[[File:Cyborg_team_photo_denver.jpeg|thumb|center]]
+
* Previous or current core reviewers:
 +
**Wenping Song
 +
**Brin Zhang
 +
**Shogo Saito
 +
**Chen Ke
 +
**Sundar Nadathur
 +
**Xinran WANG
 +
**Jinghua Gao
 +
**Yumeng Bao
 +
**Zhenghao Wang
 +
**ShaoHe Feng
 +
**Li Liu
 +
**Li Zhu
 +
**Zhipeng Huang
  
==How to contribute==
 
  
# git clone https://git.openstack.org/openstack/cyborg
+
*Team Photo
# Make the changes to your entry, be sure to include what’s changed and why
 
# Commit the change for review
 
# The changes will be reviewed, merged within a day or so.
 
  
Cyborg is designed to use the same tools for submission and review as other OpenStack projects.  As such we follow the [http://docs.openstack.org/infra/manual/developers.html#development-workflow OpenStack development workflow]. New contributors should follow the [http://docs.openstack.org/infra/manual/developers.html#getting-started getting started] steps before proceeding, as a Launchpad ID and signed contributor license are required to add new entries.
 
  
The Cyborg Launchpad page can be found at https://launchpad.net/openstack-cyborg.
+
[[File:Cyborg_team_photo_denver.jpeg|thumb|center]]
 +
[[File:Cyborg team photo dublin.jpg|thumb|center]]
 +
[[File:Cyborg team photo denver 2018.jpg|thumb|center]]

Latest revision as of 06:02, 16 December 2021

Overview

Cyborg (previously known as Nomad) is an OpenStack project that aims to provide a general purpose management framework for acceleration resources (i.e. various types of accelerators such as GPU, FPGA, ASIC, NP, SoCs, NVMe/NOF SSDs, ODP, DPDK/SPDK and so on).

Projects

Cyborg

Official name OpenStack Accelerator
Source code https://github.com/openstack/cyborg
Bug and Feature tracker https://storyboard.openstack.org/#!/project/openstack/cyborg
Developer doc


We align with the official release schedule:https://releases.openstack.org/, with three milestones. We use storyboard to track all bugs and features. The cyborg basic board page[], with three worklists, track specific tasks in the three milestones.

Python Cyborg Client

Source code https://github.com/openstack/python-cyborgclient
Bug and Feature tracker https://storyboard.openstack.org/#!/project/openstack/python-cyborgclient

Cyborg Tempest Plugin

Source code https://github.com/openstack/cyborg-tempest-plugin
Bug and Feature tracker https://storyboard.openstack.org/#!/project/openstack/cyborg-tempest-plugin

People

  • PTL
    • Xinran Wang(xinranwang)
  • Cyborg Code Reviewers
  • Blueprint Czar (responsible for maintenance of Cyborg's blueprint lists)
    • Same as PTL
  • [Release Management Liaisons]
    • Same as PTL
  • [Oslo Liaisons]
    • Shaohe Feng(shaohe_feng)
  • [QA Liaisons]
    • Chenke(chenker)
  • [Doc Liaisons]
    • Xinran Wang(xinranwang)
  • Nova Group Liaisons(works with nova team around Cyborg things)
    • Brin Zhang(brinzhang)
    • Xinran Wang(xinranwang)
  • Neutron Group Liaisons(works with neutron team around Cyborg things)
    • Xinran Wang(Xinran)

Meeting/Materials

General Resources

NOTE: Please be aware that this page is incomplete, we will continually update this page! We will not remove this note until we think that all the imperative documentation is done here. If you need any help in using Cyborg, please just ping in the IRC channel #openstack-cyborg.

Supported Device Driver List and Test Report

Driver Test Report

Accelerator API reference

End User Guide

Releasenotes

Resources for Contributors

Contributor Documentation

  • General Developer's Guide
  • [ TODO(yumeng): add doc here once get merged Cyborg Architecture and long term plan Docs]
  • [ TODO(yumeng): add doc here once get merged Cyborg-driver process]

Bug Report and Feature Registration Guide

After Rocky, Cyborg has migrated from cyborg-launchpad to cyborg-storyboard to track bugs and features. But we were not using it very well. From Victoria, we will use it, use it well, and use it often. If time is very limited for you to walk through all the documentation, go directly to the Cyborg Specific Storyboard Usage Guide, and please pay close attention to the tags!

Development Release Schedule

History

Howard (Zhipeng) Huang started OpenStack acceleration mission discussion from Telco requirements and build the Great Founding Team of Cyborg. Under his leadership, we established the basic infrastructure for sustained growth, resulting in our official release in Rocky. Stein release, with Li Liu as the PTL, we have made substantial progress, particularly in defining a clean database schema and device model that will form the basis for future work. In both previous cycles, we had several illuminating discussions with Nova developers to define how Cyborg would interact with Nova. In the following Train and Ussuri releases, under Sundar Nadathur's leadership, Cyborg has reached an exciting and significant milestone, where Cyborg integration with Nova (compute service) has been completed. Users can now use Nova to launch server instances with accelerators provisioned with the Cyborg service.


  • Previous PTLs
    • Zhipeng Huang, Rocky Release
    • Li Liu, Stein Release
    • Sundar Nadathur, Train and Ussuri Releases
    • Yumeng Bao, Victoria and Wallaby Release
  • Previous or current core reviewers:
    • Wenping Song
    • Brin Zhang
    • Shogo Saito
    • Chen Ke
    • Sundar Nadathur
    • Xinran WANG
    • Jinghua Gao
    • Yumeng Bao
    • Zhenghao Wang
    • ShaoHe Feng
    • Li Liu
    • Li Zhu
    • Zhipeng Huang


  • Team Photo


Cyborg team photo denver.jpeg
Cyborg team photo dublin.jpg
Cyborg team photo denver 2018.jpg