Jump to: navigation, search

Difference between revisions of "Cinder"

(OpenStack Block Storage ("Cinder"))
(PTG and Summit Meeting Summaries)
(44 intermediate revisions by 13 users not shown)
Line 1: Line 1:
 +
'''Note:''' The wiki.openstack.org pages are for development team collaboration and documentation. If you are looking for official project documentation, please go to https://docs.openstack.org/cinder/latest/.
  
 +
'''Official Title:''' OpenStack Block Storage Cinder<br />
  
= OpenStack Block Storage ("Cinder") =
+
'''PTL:''' Jay Bryant <jsbryant at electronicjungle d0t net><br />
  
{| border="1" cellpadding="2" cellspacing="0"
+
'''Mission Statement:''' <blockquote>To implement services and libraries to provide on demand, self-service access to Block Storage resources.  Provide Software Defined Block Storage via abstraction and automation on top of various traditional backend block storage devices.</blockquote>
|  [[https://launchpad.net/cinder/ Cinder on launchpad (including bug tracker and blueprints)]]
 
|-
 
|  [[https://github.com/openstack/cinder Source code]]
 
|-
 
|  [[http://docs.openstack.org/developer/cinder/ Developer docs]]
 
|}
 
 
 
== Mission Statement ==
 
To implement services and libraries to provide on demand, self-service access to Block Storage resources.  Provide Software Defined Block Storage via abstraction and automation on top of various traditional backend block storage devices.
 
  
 
== Description ==
 
== Description ==
Cinder is a Block Storage service for OpenStack.  It's designed to allow the use of either a reference implementation (LVM) to present storage resources to end users that can be consumed by the OpenStack Compute Project (Nova).  The short description of Cinder is that it virtualizes pools of block storage devices and provides end users with a self service API to request and consume those resources without requiring any knowledge of where their storage is actually deployed or on what type of device.
+
Cinder is a Block Storage service for OpenStack.  It's designed to present storage resources to end users that can be consumed by the OpenStack Compute Project (Nova).  This is done through the use of either a reference implementation (LVM) or plugin drivers for other storage. The short description of Cinder is that it virtualizes the management of block storage devices and provides end users with a self service API to request and consume those resources without requiring any knowledge of where their storage is actually deployed or on what type of device.
  
== Related projects ==
+
== Documentation ==
* Python Cinder client
+
See https://docs.openstack.org/cinder
* Block Storage API documentation
 
  
== What is Cinder ? ==
+
== Core Team ==
 +
See [https://review.openstack.org/#/admin/groups/83,members current members].
  
Cinder provides an infrastructure for managing volumes in OpenStack.  It was originally a Nova component called nova-volume, but has become an independent project since the Folsom release.
+
== Project Meetings ==
 +
See [[CinderMeetings|Meetings/Cinder]].
  
== Reasoning: ==
+
== Getting in Touch ==
# Nova is currently a very large project; managing all of the dependencies in linkages of services within Nova can make the  ability to advance new features and functionality very difficult.
+
We use the [http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev openstack-dev@lists.openstack.org] mailing list for discussions using subjects with the prefix "[cinder]". We also hang out on IRC in #openstack-cinder on freenode. IRC logs are available in: [http://eavesdrop.openstack.org/irclogs/%23openstack-cinder/ http://eavesdrop.openstack.org/irclogs/#openstack-cinder/]
# As a result of the many components and dependencies in Nova, it's difficult for anybody to really have a complete view of Nova and to be a true expert.  This makes the job of core team member on Nova very difficult, and inhibits good thorough reviews of bug and blueprint submissions.
 
# Block storage is a critical component of [[OpenStack]], as such it warrants focused and dedicated attention.
 
# Having Block Storage as a dedicated core project in [[OpenStack]] enables the ability to greatly improve functionality and reliability of the block storage component of [[OpenStack]]
 
  
== Documents: ==
+
== Related projects ==
* Cinder deep dive (updated for Grizzly): [[File:cinder-grizzly-deep-dive-pub.pdf]]
+
* [https://github.com/openstack/python-cinderclient Python Cinder client]
 +
* [https://wiki.openstack.org/wiki/CinderBrick Brick]
  
== Minimum Driver Features ==
+
== Core Volume Drivers ==
See [https://github.com/openstack/cinder/blob/master/doc/source/devref/drivers.rst driver dev docs]
+
For a list of the core drivers in each OpenStack release and the volume operations they support, see https://wiki.openstack.org/wiki/CinderSupportMatrix
 
 
=== Keeping consistant with multi backend ===
 
In order to maintain consistency with multi backend, do not directly use FLAGS.my_flag, instead use the self.configuration that is provided to the volume drivers. If this does not exist, look @ lvm.py and add it to your driver. using FLAGS.my_flag instead of self.configuration.my_flag will cause multi backend to not work properly. Multi backend relies on the configurations to be within a specific config group in the config file, and the self.configuration abstracts that away from the drivers.
 
 
 
== Keeping informed and providing '''CONSTRUCTIVE INPUT''' ==
 
The Cinder team currently meets on a weekly basis in #openstack-meeting at 16:00 UTC on Wednesdays.  I try to keep the meetings wiki agenda page http://wiki.openstack.org/CinderMeetings up to date and follow it.  Also keep in mind that '''anybody''' is able to add/suggest agenda items via the meeting wiki page.
 
 
 
Of course, there's also IRC... a number of us monitor #openstack-cinder or you can always send a PM to jgriffith (that's me)
 
  
== Concerns from the community: ==
+
== Contributing Code ==
=== Compatibility and Migration: ===
+
For any new features, significant code changes, new drivers, or major bug fixes, please add a release note along with your patch. See the [http://docs.openstack.org/developer/reno/usage.html#creating-new-release-notes Reno Documentation] for details on how to generate new release notes.
There has been a significant amount of concern raised regarding "compatibility"; unfortunately this seems to mean different things to different people.  For those that haven't looked at the Cinder code or tried a demo in devstack, here are some question/answers:
 
  
* Do the same nova client commands I use for volumes today still work the same? '''YES'''
+
=== How To Contribute A Driver ===
* Do the same euca2ools that I use for volumes today still work the same? '''YES'''
+
See [https://wiki.openstack.org/wiki/Cinder/how-to-contribute-a-driver How to contribute a driver]
* Does block storage still work the same as it does today in terms of LVM, iSCSI and the drivers that are curently in place? '''YES'''
 
* Are the associated database tables the same as they are in the current nova volume code?  '''For the most part YES, all volume related tables and columns are migrated, non-volume related tables however are not present'''
 
* Does it use the same nova database as we use today? '''No, it does require a new independent database'''
 
* Are you going to implement cinder with complete disregard for my current install and completely change everything out from under me?  '''ABSOLUTELY NOT'''
 
* Are you going to test migrating from nova-vol to Cinder? '''YES'''
 
* Are those migration tests going to be done just using fakes/unit tests?  '''NO, we would require running setups, most likely devstack'''
 
* Are you planning to provide migration scripts/tools to move from nova to cinder?  '''YES'''
 
  
=== Additional thoughts to keep in mind: ===
+
=== How To Contribute A New Feature ===
* The Cinder core team is fortunate enough to have a number of members who currently work for companies that are using [[OpenStack]] in production environments. There is a strong representation and the concerns of Providers is in fact a major consideration
+
See [https://wiki.openstack.org/wiki/Cinder/how-to-contribute-new-feature How to contribute a new feature]
* The goal is '''NOT''' to throw away nova-volume as it is today, but to separate it, focus on it and improve it.
 
* Migration is one of the top priorities for introduction of Cinder into Folsom (regardless of whether nova-volume is still in place or not).  This is something that is just considered a part of the requirements for the project.
 
  
== Cinder Core Drivers ==
+
== Sample cinder.conf ==
For a list of the core drivers in each OpenStack release and the volume operations they support, see https://wiki.openstack.org/wiki/CinderSupportMatrix
+
The cinder.conf.sample is no longer maintained and tested in the source tree.  Currently you can obtain a copy by running the command 'tox -e genconfig' in a cloned version of the Cinder project and then look in etc/cinder/ for the cinder.conf.sample file.  
  
== Notes About Submitting Patches ==
+
The newly generated file will have all options in the Cinder project, driver options included.
Everyone is welcome to sign the CLA and submit code.  Please be sure you familiarize yourself with the "how to contribute guide" (https://wiki.openstack.org/wiki/How_To_Contribute#If_you.27re_a_developer).
 
  
Keep in mind, there is a disproportionate number of submitters to reviewers.  YOU can help with this!!  Anybody is welcome to review patches, jump in, give a review.  It's a great way to learn more about the code and to help you make better submissions in the future. It also helps your karma, when you submit a patch if you're an active reviewer core team members are more likely to notice your patch and give it some attention before some others.
+
For more information about the generation of the file, please see: [https://github.com/openstack/cinder/blob/master/doc/source/devref/genconfig.rst Cinder Sample Configuration Devref]
  
== Cinder Plugins ==
+
== Resources ==
How to submit a plugin/driver: https://wiki.openstack.org/wiki/Cinder/how-to-contribute-a-driver
+
===Etherpads===
 +
====Active====
 +
*[https://etherpad.openstack.org/p/newton-cinder-midcycle Newton Midcycle]
 +
*[https://etherpad.openstack.org/p/cinder-spec-review-tracking Newton Spec Review Tracking]
 +
*[https://etherpad.openstack.org/p/cinder-outreachy-project-ideas Outreachy Project Ideas]
 +
*[https://etherpad.openstack.org/p/cinder-default-iscsihelper-lio Default iscsihelper LIO]
 +
*[https://etherpad.openstack.org/p/cinder-nova-api-changes Cinder/Nova API Changes]
  
Cinder Plugin/Driver certification page: https://wiki.openstack.org/wiki/Cinder/certified-drivers
+
====Historic====
 +
*[https://etherpad.openstack.org/p/newton-cinder-summit-ideas Newton Summit Ideas]
 +
*[https://etherpad.openstack.org/p/cinder-mataka-release-final-push Mitaka Final Push]
 +
*[https://etherpad.openstack.org/p/mitaka-cinder-spec-review-tracking Mitaka Spec Review Tracking]
 +
*[https://etherpad.openstack.org/p/mitaka-cinder-midcycle Mitaka Midcycle Meetup- Planning]
 +
*[https://etherpad.openstack.org/p/cinder-mitaka-summit-topics Mitaka Summit- Planning]
 +
*[https://etherpad.openstack.org/p/cinder-meetup-summer-2015 Liberty Midcycle Meetup- Notes]
 +
*[https://etherpad.openstack.org/p/cinder-liberty-midcycle-meetup Liberty Midcycle Meetup- Planning]
  
The following plugins (from other sources) are avaialble for this project
+
=== Review Links ===
* [https://wiki.openstack.org/wiki/Mellanox-Cinder Mellanox Cinder Plugin] Mellanox Cinder Plugin
+
* [https://review.openstack.org/#/dashboard/?foreach=%28project%3Aopenstack%2Fcinder+OR%0Aproject%3Aopenstack%2Fpython%2Dcinderclient+OR%0Aproject%3Aopenstack%2Fos%2Dbrick+OR%0Aproject%3Aopenstack%2Fcinder%2Dspecs%29+status%3Aopen&title=Cinder+Review+Dashboard&Cinder+Specs=project%3Aopenstack%2Fcinder%2Dspecs&Needs+Final+%2B2=label%3ACode%2DReview%3E%3D2+NOT+label%3ACode%2DReview%3C%3D%2D2+NOT+reviewedby%3Aself&Small+Patches=NOT+label%3ACode%2DReview%3C%3D%2D1%2Ccinder%2Dcore+delta%3A%3C%3D10&Bug+Fixes+without+Negative+Feedback=NOT+label%3AWorkflow%3E%3D1+NOT+label%3AWorkflow%3C%3D%2D1+NOT+owner%3Aself+limit%3A50+branch%3Amaster+topic%3A%5Ebug.%2A+NOT+reviewedby%3Aself&Blueprints+without+Negative+Feedback=NOT+label%3AWorkflow%3E%3D1+NOT+label%3AWorkflow%3C%3D%2D1+label%3AVerified%3E%3D1+NOT+owner%3Aself+NOT+label%3ACode%2DReview%3C%3D%2D1+NOT+label%3ACode%2DReview%3E%3D2+limit%3A50+branch%3Amaster+topic%3A%5Ebp.%2A+NOT+reviewedby%3Aself&Without+Negative+Feedback=NOT+label%3AWorkflow%3E%3D1+NOT+label%3AWorkflow%3C%3D%2D1+label%3AVerified%3E%3D1+NOT+owner%3Aself+NOT+label%3ACode%2DReview%3C%3D%2D1+NOT+label%3ACode%2DReview%3E%3D2+limit%3A50+branch%3Amaster+NOT+topic%3A%5Ebug.%2A+NOT+topic%3A%5Ebp.%2A+NOT+reviewedby%3Aself&5+Days+Without+Feedback=NOT+label%3ACode%2DReview%3C%3D%2D1+NOT+label%3ACode%2DReview%3E%3D1+NOT+is%3Areviewed+age%3A5d&Own+Patches=owner%3Aself&Patches+I+%2D2%27d=label%3ACode%2DReview%3C%3D%2D2%2Cself&Stable+Branches=branch%3A%5Estable%2F.%2A+NOT+reviewedby%3Aself Cinder Projects Review Inbox]
 +
* [https://bugs.launchpad.net/cinder/+bugs?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=INPROGRESS&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_commenter=&field.subscriber=&field.structural_subscriber=&field.tag=-drivers&field.tags_combinator=ANY&field.has_cve.used=&field.omit_dupes.used=&field.omit_dupes=on&field.affects_me.used=&field.has_patch.used=&field.has_branches.used=&field.has_no_branches.used=&field.has_blueprints.used=&field.has_no_blueprints.used= In progress bugs]
 +
* [https://bugs.launchpad.net/cinder/+bugs?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=NEW&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_commenter=&field.subscriber=&field.structural_subscriber=&field.tag=-drivers&field.tags_combinator=ANY&field.has_cve.used=&field.omit_dupes.used=&field.omit_dupes=on&field.affects_me.used=&field.has_patch.used=&field.has_branches.used=&field.has_no_branches.used=&field.has_blueprints.used=&field.has_no_blueprints.used= New bugs]
 +
* [https://review.openstack.org/#/dashboard/?foreach=project:%5Eopenstack/.*python-brick-cinderclient-ext.*%20status:open%20NOT%20owner:self%20NOT%20label:Workflow%3C=%252D1%20label:Verified%3E=1%20NOT%20label:Code%252DReview%3C=%252D1%252cself%20NOT%20label:Code%252DReview%3E=1%252cself&title=python-brick-cinderclient-ext%20Review%20Inbox&Cinder%20Specs=project:openstack/cinder%252Dspecs&Bug%20Fixes=topic:%5Ebug/.*&Needs%20Feedback%20(Changes%20older%20than%205%20days%20that%20have%20not%20been%20reviewed%20by%20anyone)=NOT%20label:Code%252DReview%3C=2%20age:5d&You%20are%20a%20reviewer%252c%20but%20haven't%20voted%20in%20the%20current%20revision=reviewer:self&Needs%20final%20+2=label:Code%252DReview%3E=2%20limit:50&New%20Contributors=reviewer:10068&Passed%20Jenkins%252c%20No%20Negative%20Feedback=NOT%20label:Code%252DReview%3E=2%20NOT%20label:Code%252DReview%3C=%252D1%20limit:50&Wayward%20Changes%20(Changes%20with%20no%20code%20review%20in%20the%20last%202days)=NOT%20label:Code%252DReview%3C=2%20age:2d python-brick-cinderclient-ext Review Inbox]
  
== Configuring devstack to use your driver and backend ==
+
=== PTG and Summit Meeting Summaries ===
One of the things you'll be required to do when submitting a new driver is running your backend and driver in a devstack environment and executing the tempest volume tests against it.  Currently we provide a driver_cert wrapper (mentioned in the how-to-contribute-a-driver section).  One thing that causes some confusion is how do I configure devstack to use my backend device.  It used to be that your driver info would have to be added to lib/cinder in devstack to set your options.  We then created a cinder/plugin module in devstack.  Fortunately though it's MUCH easier than that.  For *most* drivers, the only changes that are made consist of cinder.conf file changes.  That can easily be accomplished by using devstacks local.conf file (more info here: http://devstack.org/configuration.html).  For more complex actions (like the need to install packages etc, the plugin directory in devstack can be used).  An example of what this file would look like to add driver FOO is shown below, the default localrc section is included for completeness, but the section of interest is the post-config cinder.conf section:
 
  
<nowiki>[[local|localrc]]</nowiki><br />
+
*[[VancouverSummit2018Summary|Vancouver Summit 2018 Summary]]
<sub>:# Passwords<br />
+
*[[CinderRockyPTGSummary|Rocky PTG Summary]]
ADMIN_PASSWORD=password<br />
+
*[[CinderQueensPTGSummary|Queens PTG Summary]]
MYSQL_PASSWORD=password<br />
+
*[[CinderPikePTGSummary|Pike PTG Summary]]
RABBIT_PASSWORD=password<br />
 
SERVICE_PASSWORD=password<br />
 
SERVICE_TOKEN=password<br />
 
SCREEN_LOGDIR=/opt/stack/logs<br />
 
HOST_IP=172.16.140.246<br />
 
disable_service n-net<br />
 
enable_service q-svc<br />
 
enable_service q-agt<br />
 
enable_service q-dhcp<br />
 
enable_service q-l3<br />
 
enable_service q-meta<br />
 
enable_service neutron<br />
 
<br />
 
  
<nowiki># These options define  expected driver capabilities </nowiki><br />
+
=== Cinder YouTube Channel ===
TEMPEST_VOLUME_DRIVER=foo<br />
+
* [https://www.youtube.com/channel/UCJ8Koy4gsISMy0qW3CWZmaQ/videos Midcycle/PTG Videos and Related Content]
TEMPEST_VOLUME_VENDOR="Foo Inc"<br />
 
TEMPEST_STORAGE_PROTOCOL=iSCSI<br />
 
<br />
 
<nowiki># These options allow you to specify a branch other than "master" be used </nowiki><br />
 
CINDER_REPO=https://review.openstack.org/openstack/cinder<br />
 
CINDER_BRANCH=refs/changes/83/72183/4<br />
 
<br />
 
<nowiki># Disable security groups entirely</nowiki><br />
 
Q_USE_SECGROUP=False<br />
 
LIBVIRT_FIREWALL_DRIVER=nova.virt.firewall.NoopFirewallDriver<br />
 
CINDER_SECURE_DELETE=False<br />
 
<br />
 
<nowiki>[[post-config|$CINDER_CONF]]</nowiki><br />
 
volume_driver = cinder.volume.drivers.foo.FooDriver<br />
 
foos_var = something<br />
 
another_foo_var = something-else</sub><br />
 
<br />
 
  
== Cinder Brick Proposal ==
+
[[Category: Cinder]]
https://wiki.openstack.org/wiki/CinderBrick
 

Revision as of 12:59, 30 May 2018

Note: The wiki.openstack.org pages are for development team collaboration and documentation. If you are looking for official project documentation, please go to https://docs.openstack.org/cinder/latest/.

Official Title: OpenStack Block Storage Cinder

PTL: Jay Bryant <jsbryant at electronicjungle d0t net>

Mission Statement:
To implement services and libraries to provide on demand, self-service access to Block Storage resources. Provide Software Defined Block Storage via abstraction and automation on top of various traditional backend block storage devices.

Description

Cinder is a Block Storage service for OpenStack. It's designed to present storage resources to end users that can be consumed by the OpenStack Compute Project (Nova). This is done through the use of either a reference implementation (LVM) or plugin drivers for other storage. The short description of Cinder is that it virtualizes the management of block storage devices and provides end users with a self service API to request and consume those resources without requiring any knowledge of where their storage is actually deployed or on what type of device.

Documentation

See https://docs.openstack.org/cinder

Core Team

See current members.

Project Meetings

See Meetings/Cinder.

Getting in Touch

We use the openstack-dev@lists.openstack.org mailing list for discussions using subjects with the prefix "[cinder]". We also hang out on IRC in #openstack-cinder on freenode. IRC logs are available in: http://eavesdrop.openstack.org/irclogs/#openstack-cinder/

Related projects

Core Volume Drivers

For a list of the core drivers in each OpenStack release and the volume operations they support, see https://wiki.openstack.org/wiki/CinderSupportMatrix

Contributing Code

For any new features, significant code changes, new drivers, or major bug fixes, please add a release note along with your patch. See the Reno Documentation for details on how to generate new release notes.

How To Contribute A Driver

See How to contribute a driver

How To Contribute A New Feature

See How to contribute a new feature

Sample cinder.conf

The cinder.conf.sample is no longer maintained and tested in the source tree. Currently you can obtain a copy by running the command 'tox -e genconfig' in a cloned version of the Cinder project and then look in etc/cinder/ for the cinder.conf.sample file.

The newly generated file will have all options in the Cinder project, driver options included.

For more information about the generation of the file, please see: Cinder Sample Configuration Devref

Resources

Etherpads

Active

Historic

Review Links

PTG and Summit Meeting Summaries

Cinder YouTube Channel