- 1 Introduction
- 2 Session One: 21 January 2020
- 2.1 "Drop Py2 Support" Community Goal
- 2.2 Volume Local Cache Spec
- 2.3 Using six
- 2.4 The current state of 3rd Party CIs
Since the Ussuri mid-cycle meeting will be held virtually, we don't have to worry about physical arrangements, and decided to hold it in two separate two-hour sessions. The first session will be around the Cinder Spec Freeze (last week in January) and the second session will be around the "Cinder New Feature Status Checkpoint" (week of 16 March 2020), which is 3 weeks before the final release for client libraries.
Session One: 21 January 2020
We met in BlueJeans from 1300 to 1500 UTC.
recording: (spam filter preventing direct link) https:// youtu.be/ Dz28U1pQnqA (remove the spaces in the URL)
"Drop Py2 Support" Community Goal
We've been tracking this on an etherpad: https://etherpad.openstack.org/p/cinder-ussuri-community-goal-drop-py27-support
At this point, all Cinder project deliverables no longer check/gate on Python 2. The official Ussuri Python runtimes are 3.6 and 3.7, and tox has been configured so that there are unit and functional test environments for both 3.6 and 3.7, and the check/gate config in .zuul.yaml for each project has been configured to run both 3.6 and 3.7 as well.
There's going to be a major version bump on all components to indicate that Python 2.7 is no longer supported.
The upper-constraints files on the stable branches prevent py3-only libraries from being used with a stable release. Deployers who manually patch systems need to pay attention to upper-constraints so that they don't inadvertently drop in a recent library release that hasn't been tested with Python 2.
- rosmaita - while changing tox.ini and .zuul.yaml, it turned out that for some of our previous py3 testing, what version was actually being tested against depended on the system python3. The later patches for "drop-py2-support" were set up to test against both py3.6 and py3.7 for unit and functional tests in the gate; need to verify that this is true for all Cinder deliverables.
Volume Local Cache Spec
Liang has updated the spec. We're trying to keep things simple on the Cinder and os-brick side because the configuration of the cache, including the cache mode(s) used, is completely independent of Cinder (and could change dynamically). Current proposal is to have a 'cacheable' volume-type and users can control whether it's actually cached or not by selecting an appropriate flavor (or whatever nova decides is OK). For some cache modes, you cannot live migrate or retype the volume, so it looks like somehow info about the cache mode is going to have to make it into cinder so inappropriate operations can be blocked. Maybe for the first implementation, os-brick could only allow safe cache modes at attach time, and we can document that operators should only use safe modes and should not change cache modes dynamically.
- Liang will update the spec. It's still not clear how exactly cache-mode info will be propagated to Cinder; we'll discuss at the weekly meeting.
We agreed to some guidelines about using python-3-only language features at the PTG, but they have turned out to be kind of vague, especially with respect to how to handle new code in drivers that uses the six compatability library. For example, there is this patch: https://review.opendev.org/#/c/701542/
These are the PTG guidelines:
- Reminder to reviewers: we need to be checking the code coverage of test cases very carefully so that new code has excellent coverage and will be likely to fail when tested with py2 in stable branches when a backport is proposed
- Reminder to committers: be patient with reviewers when they ask for more tests!
- Reminder to community: new features can use py3 language constructs; bugfixes likely to be backported should be more conservative and write for py2 compatabilty
- Reminder to driver maintainers: ^^
- Ivan and Sean have a green light to start removing py2 compatability
After some discussion, we decided that:
- we will allow drivers to continue to use six at their discretion
- we will not remove six from code that impacts the drivers (e.g., classes they inherit from)
- we can remove six from code that doesn't impact drivers, keeping in mind that backports may be more problematic, and hence making sure that we have really good test coverage
- rosmaita - write this up as a change to the contributor docs and we can all fight it out on the review
The current state of 3rd Party CIs
Jay has been working on removing drivers that were marked as 'unsupported' in Train and subject to removal in Ussuri, as well as marking drivers whose CIs are not being maintained as 'unsupported' in Ussuri. It is turning out to be a distressing number of drivers.
We discussed a few issues around this general topic.
Do we need upgrade checks for unsupported drivers?
No, we decided that the current log notice and release note and the fact that you have to enable an unsupported driver in config gives operators plenty of notice of impending driver removal.
Should we hold off on removing drivers right now?
Yes, at least for a few weeks. Eric suggested that we reconsider the removal policy, and Walt pointed out that some of the drivers in question are pretty widely used (looking at you, HPE), and it's going to be a PITA for operators who want to use them. So it might make sense to extend the 'unsupported' period. On the other hand, Sean pointed out that we could have problems with keeping non-maintained drivers in-tree, especially as libraries in master are updated to versions that have only been tested on python 3.6 & 3.7.
What about drivers that have already been removed?
Maybe the ones that have been removed in the past few weeks could be restored if the vendors are not completely disinterested. For example, Brocade issued a statement that they will not support their OpenStack driver beyond the Train release (they only want to support Python 2). So that's an easy call. In addition to assessing vendor "interestedness", Sean pointed out that this brings up a fairness issue, namely, what about the drivers removed in the last cycle (or the cycle before that ...). We need to discuss this some more so we can have a consistent policy.
Can we make running 3rd Party CI easier?
Luigi reported on the "Software Factory" effort: https://www.softwarefactory-project.io/
Software Factory is basically a packaging of a CI system so that it doesn't have to be assembled and built from disparate parts. It is used by RDO to do their CI. There's a doc patch up for Software Factory that could use some reviews/feedback: https://softwarefactory-project.io/r/#/c/17097/
There's already some documentation up there that explains how to hook the CI up to openstack gerrit: https://www.softwarefactory-project.io/docs/3.4/operator/quickstart.html#third-party-ci-quickstart
Our hope is that this will make it easier to set up and maintain 3rd party CI. Plus, if a lot of people use Software Factory, it will be easier for maintainers to get help.
Should we reconsider the 3rd Party CI requirements?
Current requirement is that 3rd Party CI must run on all changes, whether they impact that driver or not. What we're winding up with on most patches is a big list of failures that you scroll past to get to the list of comments on the review, and you only pay attention to them when you're reviewing a driver change, and then you only look at that vendor's CI results.
Sean pointed out that this isn't a new discussion. Here are a few things to think about from 2017: https://wiki.openstack.org/wiki/CinderPikePTGSummary#3rd_Party_CI_Requirements http://eavesdrop.openstack.org/meetings/cinder/2017/cinder.2017-03-15-16.00.log.html#l-267
Some other things that came up were: if this is a resource issue, maybe only require the CI to run once a day? What it comes down to is that we have a big constraint satisfaction problem here.
- everyone - look over the Software Factory patch https://softwarefactory-project.io/r/#/c/17097/
- jungleboyj - will try to contact vendors with problematic CIs and report back at the 29 January meeting
- rosmaita - get something together to organize this discussion (either a patch to our docs or an etherpad) for the 29 January meeting