CinderTrainMidCycleSummary

Introduction
This page contains a summary of the subjects covered during the Train Mid-Cycle held in Morrisville, North Carolina, USA, August 21 and 22, 2019.

The full etherpad and all associated notes may be found here.

=Train Mid-Cycle Summary=

Wednesday 8/21/2019
Video Recording Part 1

Python-Cinderclient major version bump work

 * Summary: Reviewed the open patches that were appropriate to release with a major version bump. Agreed that we should work on getting the patches merged and released in time to ship with Train.
 * Action (whoami-rajat): To fix up his patch for options being sent to --sort.
 * Action (team): To review outstanding patches and get them merged before 9/9/19.

Cinder PDF Documentation Creation

 * Summary: Walt has been working towards the Train goal of getting PDFs generated for our documentation. Has patches up for each of the Cinder projects and had some issues to work through for Cinder due to the size of the documentation.
 * Action (hemna): To remove WIP from the patches that are ready to be reviewed/merged.
 * Action (rosmaita): To look at problems building cinder-lib documentation and try to fix it up.

Multi-Attach

 * Summary: Multi-attach has been in place for a while now but we are not requiring any 3rd Party CI for it and are not sure how well this is actually working in the wild. Red Hat is planning to test it for their next release as it is one of the highlighted features.
 * Action (rosmaita): Going to draft documentation as to what tests should be run before submitting a multi-attach enablement patch.
 * Action (eharney): Concerned about the read only functionality for multi-attach. He is going to check into what, if any, support there is for this in libvirt.
 * Action (jungleboyj): To take a look if the driver that have support enabled are running the muti-attach test case and is it passing?
 * Action (jungleboyj): To open bugs in the case that they aren't testing multi-attach. The flag will be removed if they don't respond to the bug.

Discussion of Stable Backport Policies

 * Summary: Concerns have been raised by the stable branch team about some of the things we have allowed for backport. The problem is that with longer lived stable branches and the removal of the driver fixes branches we are missing a method for backporting some changes that distributors need. We could not come up with another way to resolve this issue.
 * Action (jungleboyj): To update our documentation to explain why our backport policies are looser than other projects.
 * Action (jungleboyj): To communicate this to the stable release team and work to address/resolve any concerns.

Video Recording Part 2

Review of the Support Matrix

 * Summary: In the last couple of summits it has been useful to review the support matrix for accuracy and needed additions. Once again this proved to be a useful exercise.
 * Action (jungleboyj): Remove the old matrix from the Wiki and move it to a place that is clearly old.
 * Action (jungleboyj): Add manage/unmanage support.
 * Action (jungleboyj): Add manage/unmanage snapshot
 * Action (jungleboyj): Investigate whether list manageable is needed as a separate item.
 * Action (jungleboyj): For replication check to see if we need to adding failover/failback support as a separate item.

iSCSI Ceph Driver Update

 * Summary: The driver has been created and appears to work properly. The problem is that it has proven very hard to get a CI to work for the Ceph iSCSI driver.  There aren't good pypi packages to support it and Ubuntu doesn't come with the right level of support.  SuSE's LEAP supports it but is not supported by devstack.  So, due to many complications, the driver missed Train but may be able to make it into the U release.
 * Action (hemna): To continue working on the driver and CI with the hope of solving issues for the U release.
 * Action (hemna): To work to see if there is a better way to implement deploying Ceph for Devstack. Ceph-Ansible may be a better fit.

How to Deal with 3rd Party CI Testing Irregularities

 * Summary: We continue to have issues getting many vendors to consistently run 3rd Party CI. The difficulty getting systems moved over to using Py3.7 recently has just once again highlighted the issues here.  We were happy that some vendors have responded to the notes about Py3.7 but many still haven't.  We will have to unsupport them as they will not work by the end of the U release.
 * Action (jungleboyj): Start unsupporting all the drivers that are not testing Py3.
 * Action (jungleboyj): To unsupport the IBM drivers as they have been out of compliance for quite a while now.
 * Action (jungleboyj): Review/update the Third Party CI requirements page.
 * Action (HELP NEEDED): Someone needs to check into what tests are being run by each vendor to make sure that they appear correct.

Video Recording Part 3

Appropriate Upgrade Checks

 * Summary: There was some confusion/disagreement as to what have upgrade checks created for it. Also some confusion as to why the checks are backported to the previous release so that they can be run before upgrading the environment in question.
 * Action (jungleboyj): Ensure that we are checking for all the right drivers that have been removed in Train. Make any updates to things that aren't accurate.
 * Action (jungleboyj): Check to see what happens for drivers that are added to the check and then removed. Don't think we have an issue here, but need to ensure that is the case.
 * Action (jungleboyj): Look into adding an option to check for unsupported drivers and then ensure that the right flag is set for using unsupported drivers.
 * Action (jungleboyj): To create a forum topic proposal for this to find out how people are using this in their environments and what, if anything, should be done to improve the functionality.

Improve Automated Test Coverage

 * Summary: There is no disagreement that we have gaps in our automated test coverage. There are definitely Tempest API tests that need to be added.  This would be something good for an Outreachy intern to help with perhaps.
 * Action (eharney): To put together a list of tests that need to be written. Perhaps open bugs to document all of them.
 * Action (eharney): Determine a tag to use in LaunchPad to use for all the bugs. 'test-coverage' maybe?

Review of open reviews for client and non-client libraries

 * Summary: Wanted to make sure that we were on track for our client and non-client library freeze dates. A scan of the open reviews showed that we were more or less in good shape.
 * Action (team): To do reviews and stay on top of changes as they come in.

Deprecate the Back-up Service

 * Summary: Not really the goal of the discussion, the topic was designed to get our attention that backup may not be tested anymore. Backup testing has been removed from many check/gate jobs due to frequent failures.  Wasn't clear as to whether it was still be tested anywhere.  It appears that some of them are being run in check jobs but not all of them.
 * Action (eharney): To dig into the failures that are being seen to figure out if there is any pattern to the failures.
 * Action (team): Need to figure out where to run these tests long term. We don't really have a good job right now that is focused on testing Cinder.  Do we need to create one?

Thursday 8/22/2019
Video Recording Part 1

v2 API Removal

 * Summary: Been working for some time on getting the V2 API removed as it is a subset of the V3 API. Sean has a patch out there to propose the removal but it is failing the checks.  We have decided that we don't want to try to get this into Train but it is something that we want to get fixed soon.
 * Action (jungleboyj): To follow up with Sean to find out if he is still working on this.
 * Action (HELP NEEDED): Devstack needs to be updated to not use V2 anymore.

Active/Active HA Support

 * Summary: Discussion about who, with anyone, is using this. Red Hat is interested as they are wanting to ship this with their next release.  Currently Macrosan and Ceph are the only drivers that list it as supported.  Unfortunately testing it is hard.  Jon Bernard has done some verification with Ceph and will continue to do so.
 * Action (HELP NEEDED): Really should get some automated testing for this in place.

Default Volume Type Change

 * Summary: Agreed that we want to try to get this into Train and still have time. There was some discussion as to whether we needed to add a check to ensure that the new default type isn't deleted.  The type, however, can't be deleted if it is in use so we agreed to not change the default behavior as it covers the concern raised.
 * Action (team): To review and work to merge the patch.
 * Action (jungleboyj): Schedule a Forum Session to understand how people are using Volume Types.

Backup Test Leaking Notifications

 * Summary: Had a ToDo from Denver to follow up on this issue. It appears that patches that Eric created to avoid having notifications impact other tests has worked.  So, we can take this one off the list.

Dependency Install Mechanism for Containers

 * Summary: Had a follow up from the Denver PTG to discuss this. Walt merged patch that moved the code from the driver requirements file into setup.py.  This will help to get containers properly configured for drivers.  There are a few requirements that couldn't be included as they would not pass the global requirements, requirements.

SQLAlchemy to Alembic Migration

 * Summary: We have gotten the database migrations collapsed down which was one of the goals after Denver. There isn't documentation we are aware of on how to do this but there are examples in what Glance and Manila have done.  We should get this done in the U release.  Could possibly be a good work item for an Outreachy person.
 * Action (jungleboyj): Find out if there is a Deadline that this has to happen by.

IPv6 Impact on Drivers
Video Recording Part 2
 * Summary: Some unexpected issues were found in the LVM driver when using IPv6. Wanted to start discussion to make sure other drivers don't have an issue.  There isn't a requirement for IPv6 but it is considered good practice to support it.  Walt's experience has been that Cinder works with IPv6 and can even be deployed by Devstack to use IPV6.
 * Action (jungleboyj): Add a note to the driver development documentation that we strongly encourage IPv6 support and testing. Should also encourage people to use the host address option for IP addresses rather than a string or IP option.
 * Action (HELP NEEDED): We should look at driver config options to make sure that they are using the host address option.
 * Action (HELP NEEDED): Test os-brick with IPv6.

Replication

 * Summary: Not really clear what the state of this function is. DId a lot of work to get it included but it isn't clear how well it works now.  Gorka tested it with RBD a while ago and made some fixes.  Not sure how well this works for other drivers.
 * Action (jungleboyj): Follow up with the vendors and find out if they are using this and how well it is working.

Capabilities Reporting
Video Recording Part 3
 * Summary: Continue to discuss this and the fact that it is something that we need to do for the good of Cinder. We have gotten beyond the blocks that, we believe, prevented this from happening in the past.  Need to move forward to make this happen.
 * Action (eharney and hemna): Restore the old specs to restart this effort. The existing specs might be able to be combined into one.
 * Action (jungleboyj): Create a pointer to the links from the Denver Summit/PTG.

Capabilities Reporting

 * Summary: We did a review of the specs that we were shooting to get into Train and made updates in the etherpad. Details can be seen there.
 * Action (team): Review the etherpad and help to review code changes that need to go into Train.
 * Action (jungleboyj): Clean up specs that aren't going to make it as planned.
 * Action (jungleboyj): Create an untargeted folder for specs that are approved but not assigned to a release.

Cinder Mutable Options

 * Summary: Continued discussion around this feature. Something we have been discussing for quite some time.  Doesn't appear that it has ever been fully implemented but there is continued interest in it.
 * Action (jungleboyj): Move the spec as it did not land in the release where it was targeted.
 * Action (jungleboyj): Follow up with NetApp to see if they are still working on this.

Future Mid-Cycles

 * Summary: Brian and the team had mixed feelings on this topic. The face to face meetings have been very productive but it is hard to get everyone to a physical location for the meetings.    If not everyone can make it, it is better to just have an all virtual event.  We weren't really able to reach an agreement so there will be follow-up.
 * Action (rosmaita): Put together a google survey to get a feeling from the team as to how they would like to proceed.