Jump to: navigation, search

Difference between revisions of "CinderPikePTGSummary"

(DB code Cleanup and Fixes)
 
(15 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
<br />
 +
<br />
 
=== Introduction ===
 
=== Introduction ===
 
This page contains a summary of the subjects covered during the Pike PTG in Atlanta Georgia, February 22nd, through February 24th, 2017.
 
This page contains a summary of the subjects covered during the Pike PTG in Atlanta Georgia, February 22nd, through February 24th, 2017.
  
 
The full etherpad and all associated notes may be found  [https://etherpad.openstack.org/p/ATL-cinder-ptg-notes here.]  The etherpad also includes links to YouTube recordings of our sessions.
 
The full etherpad and all associated notes may be found  [https://etherpad.openstack.org/p/ATL-cinder-ptg-notes here.]  The etherpad also includes links to YouTube recordings of our sessions.
 
+
<br />
 +
<br />
 +
<br />
 
== Wednesday, 2/22/2017 ==
 
== Wednesday, 2/22/2017 ==
 +
[https://www.youtube.com/watch?v=rxDNJ2G-odo&feature=youtu.be Video:  Cinder PTG -Pike- Day 1, Part 1 ]
  
 
=== How to Capture Decisions and Find Them Later ===
 
=== How to Capture Decisions and Find Them Later ===
Line 18: Line 23:
 
*'''Decision:'''  We will get some functional tests in place that are more than just extended unit tests.  We will move to actually using devstack and see how things go from there.  Moving to using our own cinder-tempest rep will hopefully be faster.  In the past getting tempest patches in took a very long time.
 
*'''Decision:'''  We will get some functional tests in place that are more than just extended unit tests.  We will move to actually using devstack and see how things go from there.  Moving to using our own cinder-tempest rep will hopefully be faster.  In the past getting tempest patches in took a very long time.
 
*'''Action:'''  (e0ne) Have a devstack plugin for fake drivers.  A new repo will be needed for this.  eharney has experience with this and will help out e0ne.
 
*'''Action:'''  (e0ne) Have a devstack plugin for fake drivers.  A new repo will be needed for this.  eharney has experience with this and will help out e0ne.
 
+
<br />
 +
<br />
 +
[https://www.youtube.com/watch?v=NlHYgqfEfjo&feature=youtu.be Video:  Cinder PTG -Pike- Day 1, Part2 ]
  
 
=== Testing Drivers for Active/Active High Availability ===
 
=== Testing Drivers for Active/Active High Availability ===
Line 32: Line 39:
 
=== Having Each API Have its Own Policy Action ===
 
=== Having Each API Have its Own Policy Action ===
 
*'''Action:'''  (rb560u) File a bug for the policies that are not granular enough and then submit patches to resolve the issue.
 
*'''Action:'''  (rb560u) File a bug for the policies that are not granular enough and then submit patches to resolve the issue.
 +
<br />
 +
<br />
 +
[https://www.youtube.com/watch?v=mhChWBUHabI&feature=youtu.be Video:  Cinder PTG -Pike- Day 1, Part 3 ]
  
 
=== Status of Migration CGs to Generic Volume Groups ===
 
=== Status of Migration CGs to Generic Volume Groups ===
Line 41: Line 51:
 
*'''Action:'''  Follow up with driver teams that have not moved to generic groups.
 
*'''Action:'''  Follow up with driver teams that have not moved to generic groups.
 
*'''Action:'''  (xyang) to update CG API state in Queens.
 
*'''Action:'''  (xyang) to update CG API state in Queens.
 +
<br />
 +
<br />
 +
[https://www.youtube.com/watch?v=OcpY9iV--CU&feature=youtu.be Video: Cinder PTG -Pike- Day 1, Part 4 ]
  
 
=== Cinder Volume Multi-attach ===
 
=== Cinder Volume Multi-attach ===
Line 51: Line 64:
 
*'''Action:'''  We should update the dothill driver so that the classes in there cannot be loaded as a standalone driver.
 
*'''Action:'''  We should update the dothill driver so that the classes in there cannot be loaded as a standalone driver.
 
*'''Action:'''  Add comments or a readme explaining why this isn't a standalone driver so we don't accidentally remove it in the future.  
 
*'''Action:'''  Add comments or a readme explaining why this isn't a standalone driver so we don't accidentally remove it in the future.  
 +
<br />
 +
<br />
 +
<br />
  
 
== Thursday, 2/23/2017 ==
 
== Thursday, 2/23/2017 ==
 
+
<br />
 +
<br />
 +
[https://www.youtube.com/watch?v=eGhIFfk1QXk&feature=youtu.be Video:  Cinder PTG -Pike- Day 2, Part 1 ]
 
===  reset-state, states of volumes/snaps/other things ===
 
===  reset-state, states of volumes/snaps/other things ===
 
*'''Decision:'''  Collapse all the commands down to one 'cinder reset-state' command.  Deprecate the old <name>-reset-state comands.
 
*'''Decision:'''  Collapse all the commands down to one 'cinder reset-state' command.  Deprecate the old <name>-reset-state comands.
Line 66: Line 84:
 
*'''Decision:'''  Team is ok with splitting up the cinder/db/sqlalchemy/api.py
 
*'''Decision:'''  Team is ok with splitting up the cinder/db/sqlalchemy/api.py
 
*'''Action:'''  (bluex) will work on proposing changes to split up the api.py file.
 
*'''Action:'''  (bluex) will work on proposing changes to split up the api.py file.
 +
<br />
 +
<br />
 +
[https://www.youtube.com/watch?v=MMTAJSYHxgA&feature=youtu.be  Video:  Cinder/Nova PTG -Pike- Day 2, Part 2 ]
  
 
=== Interaction Between Nova and the New Cinder attach/detach API ===
 
=== Interaction Between Nova and the New Cinder attach/detach API ===
Line 75: Line 96:
 
*'''Action:'''  (scottda) will get support for Cinder v3 into Nova.
 
*'''Action:'''  (scottda) will get support for Cinder v3 into Nova.
 
*'''Action:'''  New attachment flow will be implemented once the above items are completed.
 
*'''Action:'''  New attachment flow will be implemented once the above items are completed.
 +
<br />
 +
<br />
 +
[https://www.youtube.com/watch?v=OI55mKlm8pY&feature=youtu.be Video:  Cinder PTG -Pike- Day 2, Part 3 ]
  
 
=== Capacity-based QoS ===
 
=== Capacity-based QoS ===
Line 87: Line 111:
 
*'''Decision:'''  After much discussion we clearly have a number of concerns with the proposal that need to be addressed.  Since we can't come to a decision now it seems that we need to handle the details through the spec review.
 
*'''Decision:'''  After much discussion we clearly have a number of concerns with the proposal that need to be addressed.  Since we can't come to a decision now it seems that we need to handle the details through the spec review.
 
*'''Action:'''  All need to review the spec and comment.
 
*'''Action:'''  All need to review the spec and comment.
 +
<br />
 +
<br />
 +
[https://www.youtube.com/watch?v=w2eHP_CDFQc&feature=youtu.be Video:  Cinder PTG -Pike- Day 2, Part 4 ]
  
 
===  Inspection Mechanism For Capacity Limited Host ===
 
===  Inspection Mechanism For Capacity Limited Host ===
Line 100: Line 127:
 
*'''Action:'''  (gman-tx) to write a spec for this feature.
 
*'''Action:'''  (gman-tx) to write a spec for this feature.
  
=== Cinder Client Command Review:
+
=== Cinder Client Command Review: ===
 
*'''Decision:'''  There are some commands that could be condensed.
 
*'''Decision:'''  There are some commands that could be condensed.
 
*'''Action:'''  Anyone interested can propose patches.
 
*'''Action:'''  Anyone interested can propose patches.
 +
<br />
 +
<br />
 +
<br />
 +
 +
== Friday 2/24/2017 ==
  
 
=== Cinder Pike Priorities ===
 
=== Cinder Pike Priorities ===
Line 118: Line 150:
 
*Cinder Ephemeral Driver:
 
*Cinder Ephemeral Driver:
  
 +
=== Nova Placement API ===
 +
*'''Action:'''  We need to add the type of traits (capabilities) we want to report up to the placement API by updating os-traits.
 +
 +
=== Removing V1 API Endpoint ===
 +
*'''Action:'''  Sean is going to test V2 with V1 tempest.  We will then redirect V1 to V2 internally if that works.
 +
 +
=== 3rd Party CI Requirements ===
 +
*'''Decision:'''  Everyone will continue to run as we currently have it.  We enforce a passing vendor CI for patches against that vendor's driver.  We will then have a no-op patch that we rebase once a week that we use to verify that your CI is running and able to pass.  We will also run a test that should cause all CI's to fail.  Run the two tests back-to-back.
 +
*'''Action:''' (jgriffith) is going to bring this proposal up to the community in next Wednesday's meeting.
 +
*'''Action:'''  Propose public shaming of drivers that are not working sufficiently will be done at the first community meeting of the month.
 +
*'''Action:'''  Propose adding a patch that should fail for every CI.
 +
 +
=== Change to Make the Default Section Act Like the Default Section and Shared Backend Config ===
 +
*'''Action:'''  Get the spec merged for Pike and review Patrick's code change.
  
 
[[Category: Cinder]]
 
[[Category: Cinder]]

Latest revision as of 19:11, 1 March 2017



Introduction

This page contains a summary of the subjects covered during the Pike PTG in Atlanta Georgia, February 22nd, through February 24th, 2017.

The full etherpad and all associated notes may be found here. The etherpad also includes links to YouTube recordings of our sessions.


Wednesday, 2/22/2017

Video: Cinder PTG -Pike- Day 1, Part 1

How to Capture Decisions and Find Them Later

  • Decision: Jay will do post processing of the etherpad notes taken and summarize the the decisions on the Wiki page [1] Sean will also review.
  • Action: Jay to create this page and add links to the Cinder Wiki page accordingly.

Testing Priority for Pike

  • Decision: Focus on getting tempest Cinder API V1 testing removed.
  • Action: (jgriffith) will look at what needs to change with the tempest tests to get away from v1.
  • Action: (jgriffith) will also add a README in the testing directory that will be a symbolic link to the devref that has more details about our testing infrastructure.

One More Discussion about Functional Tests

  • Decision: We will get some functional tests in place that are more than just extended unit tests. We will move to actually using devstack and see how things go from there. Moving to using our own cinder-tempest rep will hopefully be faster. In the past getting tempest patches in took a very long time.
  • Action: (e0ne) Have a devstack plugin for fake drivers. A new repo will be needed for this. eharney has experience with this and will help out e0ne.



Video: Cinder PTG -Pike- Day 1, Part2

Testing Drivers for Active/Active High Availability

  • Decision: At this point, when someone submits a patch for A/A support we will refer the developer to the devref noted above and ask if they have read it, executed the recommended testing and feel comfortable that their driver meets the needs. At this point, the A/A HA support is still a bit of a tech preview so it is up to the driver maintainers to help do the right thing for this support.
  • Action: (gorka) is going to continue to work on getting automated tests configured but it is going to take time.
  • Action: (gorka) document the tests that we feel driver maintainers should be doing to certify their driver. The documentation will start in the etherpad linked above and then, once we have some confidence, we will move the documentation into the devref.

API Philosophy and Reality

  • Decision: We need to stick with the program that everyone else is following. We don't want diverge from what everyone else is doing with microversions. If we try to bump on release boundaries we are going to break the model.
  • Action: (jungleboyj) We need somone to backfill Scott and Walt on the API Working group meeting. Meeting is at 10 am Central time on Thursdays.
  • Action: (jungleboyj) Work on fixing up the docstrings so that we include the API version in the documentation.

Having Each API Have its Own Policy Action

  • Action: (rb560u) File a bug for the policies that are not granular enough and then submit patches to resolve the issue.



Video: Cinder PTG -Pike- Day 1, Part 3

Status of Migration CGs to Generic Volume Groups

  • Decision: We are going to make it so you cannot modify volume types with existing volumes. Same as group types.
  • Decision: We should file bugs against the current drivers that have CG implemented but don't move to generic groups.
  • Decision: Mark the CG API as deprecated in Queens. Remove it and clean it up at some point in the future, perhaps.
  • Decision: We need quotas for groups.
  • Action: (jgriffith) will fix volume_types so they can't be deleted when volumes are associated - [2]
  • Action: Follow up with driver teams that have not moved to generic groups.
  • Action: (xyang) to update CG API state in Queens.



Video: Cinder PTG -Pike- Day 1, Part 4

Cinder Volume Multi-attach

DotHill Driver Deprecation Discussion

  • Decision: Ok to leave the driver in the volume/drivers directory. Would like to change the name of the directory to disassociate it from DotHill. Should have the driver so it can't be inserted by itself.
  • Action: We should update the dothill driver so that the classes in there cannot be loaded as a standalone driver.
  • Action: Add comments or a readme explaining why this isn't a standalone driver so we don't accidentally remove it in the future.




Thursday, 2/23/2017



Video: Cinder PTG -Pike- Day 2, Part 1

reset-state, states of volumes/snaps/other things

  • Decision: Collapse all the commands down to one 'cinder reset-state' command. Deprecate the old <name>-reset-state comands.
  • Decision: Change default functionality to require a state be specified.
  • Action: (eharney) will work on implementing these features.

Quota Tests and Cleanup

  • Decision: Team is ok with adding testing/cleaning up this code.
  • Action: (bluex) will start coding up the tests.

DB code Cleanup and Fixes

  • Decision: Team is ok with splitting up the cinder/db/sqlalchemy/api.py
  • Action: (bluex) will work on proposing changes to split up the api.py file.



Video: Cinder/Nova PTG -Pike- Day 2, Part 2

Interaction Between Nova and the New Cinder attach/detach API

  • Decision: If cinder v3 is not available, or v3 is available but 3.27 is not, nova falls back to the old (existing) behavior.
  • Decision: Nova will default to Cinder API v3 in Pike.
  • Decision: In nova-api, nova calls create_attachment to effectively reserve the volume and put it into 'attaching' status.
  • Action: (johnthetubaguy) update the Nova spec for multi-attach.
  • Action: (ildikov) will get the check_attach patch landed.
  • Action: (scottda) will get support for Cinder v3 into Nova.
  • Action: New attachment flow will be implemented once the above items are completed.



Video: Cinder PTG -Pike- Day 2, Part 3

Capacity-based QoS

  • Decision: John has concerns and would like to better understand it. Thinks a spec will be helpful in creating understanding.
  • Action: (eharney) will write up a spec for review.

New Driver Submission Policies

  • Decision: Recommend that submitters start with a base functionality patch. Not a hard requirement. Then encourage them create additional patches per added functionality. The separation could also be a protocol boundary for splitting the page ... I.E. iSCSI vs FibreChannel.
  • Action: e0ne to add the recommendation to the 'How to Submit a Driver' wiki page.

Design detail about Cinder volume revert to snapshot

  • Decision: After much discussion we clearly have a number of concerns with the proposal that need to be addressed. Since we can't come to a decision now it seems that we need to handle the details through the spec review.
  • Action: All need to review the spec and comment.



Video: Cinder PTG -Pike- Day 2, Part 4

Inspection Mechanism For Capacity Limited Host

  • Decision: We should implement the checks in the manager now for actions that skip the scheduler. This is a good first step. In the future we can work on getting things routed through the scheduler.
  • Action: Winston-D will update his current WIP patch to cover more things that could fail due to insufficient capacity and we will review it. (https://review.openstack.org/#/c/437677/ )

Items from OpenStack Client Discussion

  • Decision: manage/unmanage commands are going to be changed to adopt/abandon to be consistent with other projects.
  • Decision: osc will get the latest supported microversion from Cinder and use it.

Backup Service that Operates on a Group of Volumes

  • Decision: It could be good to have a volume group backup feature.
  • Action: (gman-tx) to write a spec for this feature.

Cinder Client Command Review:

  • Decision: There are some commands that could be condensed.
  • Action: Anyone interested can propose patches.




Friday 2/24/2017

Cinder Pike Priorities

Etherpad: https://etherpad.openstack.org/p/cinder-spec-review-tracking See Etherpad for links to associated reviews.

  • Attach/Detach API Changes:
  • CG to Generic Group migration:
  • Generic Volume Group Driver Updates.x
  • Cinderclient microversion support
  • Finish getting replication v2.1 patches in place.
  • Replication Group (Tiramisu)
  • New Driver Reviews.
  • Driver Refactors
  • Dynamic Reconfiguration:
  • Cinder Ephemeral Driver:

Nova Placement API

  • Action: We need to add the type of traits (capabilities) we want to report up to the placement API by updating os-traits.

Removing V1 API Endpoint

  • Action: Sean is going to test V2 with V1 tempest. We will then redirect V1 to V2 internally if that works.

3rd Party CI Requirements

  • Decision: Everyone will continue to run as we currently have it. We enforce a passing vendor CI for patches against that vendor's driver. We will then have a no-op patch that we rebase once a week that we use to verify that your CI is running and able to pass. We will also run a test that should cause all CI's to fail. Run the two tests back-to-back.
  • Action: (jgriffith) is going to bring this proposal up to the community in next Wednesday's meeting.
  • Action: Propose public shaming of drivers that are not working sufficiently will be done at the first community meeting of the month.
  • Action: Propose adding a patch that should fail for every CI.

Change to Make the Default Section Act Like the Default Section and Shared Backend Config

  • Action: Get the spec merged for Pike and review Patrick's code change.