Jump to: navigation, search

Difference between revisions of "CinderWallabyMidCycleSummary"

m
m (Cinder Project Business)
Line 138: Line 138:
  
 
===Cinder Project Business===
 
===Cinder Project Business===
 +
* stable releases
 +
** planning stable releases in 2 weeks; tracking on https://etherpad.opendev.org/p/stable-releases-review-tracker-22-02-2021
 +
* new drivers status
 +
** community drivers: s3 backup, ceph iSCSI
 +
*** CI-related patches have not merged yet
 +
*** ceph-iscsi: https://review.opendev.org/q/topic:%22ceph-iscsi%22+(status:open%20OR%20status:merged)
 +
*** s3 backup: https://review.opendev.org/q/topic:%22feature%252Fsupport-s3-backup-driver%22+(status:open%20OR%20status:merged)
 +
** third-party merged: open-e jovian dss, DellEMC PowerVault, TOYOU
 +
** third-party unmerged: kioxia kumoscale (proposed merge deadline of Friday 12 Feb)
 +
*** http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020362.html
 +
*** everyone seems OK with the extension, though we hope not to have this happen in non-pandemic times
 +
** proposal: release cinder 18.0.0.0b1 after all drivers have merged
 +
*** give people an opportunity to consume & test the new drivers
 +
* rbd-iscsi-client
 +
** all governance, project-config patches accepted
 +
** outstanding patches
 +
*** reformat as a Cinder project: https://review.opendev.org/c/openstack/rbd-iscsi-client/+/774748
 +
*** update Cinder docs: https://review.opendev.org/c/openstack/cinder/+/773158
 +
 
===FS backend format info documentation brainstorming===
 
===FS backend format info documentation brainstorming===
 
===Avoid raw to raw conversions when creating volume from raw image===
 
===Avoid raw to raw conversions when creating volume from raw image===

Revision as of 03:06, 11 February 2021

Introduction

At the Wallaby Virtual PTG we decided to hold two mid-cycle meetings, each two hours long. The first meeting would be before the Cinder Spec Freeze, and the second would be around the New Feature Status Checkpoint. So the mid-cycle meetings will be the week of R-18 and the week of R-9.

What follows is a brief summary of what was discussed. Full details are on the etherpad: https://etherpad.opendev.org/p/cinder-wallaby-mid-cycles

Session One: R-18: 9 December 2020

We met in BlueJeans from 1400 to 1600 UTC.
etherpad: https://etherpad.openstack.org/p/cinder-wallaby-mid-cycles
recording: https://www.youtube.com/watch?v=rKmlpBTIDGA

Cinder Project Updates

upcoming deadlines

  • 18 December - spec freeze
  • 21 January - Wallaby Milestone-2 and new driver merge deadline
  • 1 March - os-brick wallaby release
  • 8 March - python-cinderclient wallaby release

stable releases

Our current release cadence on the stable branches is (roughly) every two months. We've recently released:

  • os-brick: 4.1.0 (wallaby), 3.0.4 (ussuri)
  • cinder: 17.0.1 (victoria), 16.2.1 (ussuri), 15.4.1 (train)

We managed to slip a critical fix past the Train gate, but three or four other patches that had been approved and targeted for 15.4.1 are stuck in the gate. Once we get the Train gate sorted out, we'll most likely do an extra release from stable/train out of cadence.

end of year meetings

The last meeting of the year will be 23 December. Being also the last meeting of the month, it will be held in videoconference. We won't meet on 30 December.

requirements/lower-constraints update

The lower-constraints job was broken recently by a pip update that included a new dependencies resolver that is apparently more strict than the old one, and as a result, it was detecting incompatible constraints from cinder specifying a lower constraint for some libraries than its dependencies allowed as a minimum. We started to change the values in lower-constraints.txt one by one, and it became pretty obvious that a larger change was needed. So we merged this patch yesterday: https://review.opendev.org/c/openstack/cinder/+/766085

That patch raised the minimums in requirements.txt to the versions in a pip freeze of a recent py38 testenv and raised the values in lower-constraints.txt accordingly. This included some big jumps in major versions for some libraries, but it does bring our lower-constraints.txt much closer to reality. The lower-constraints job just uses the lower-constraints.txt as constraints and basically checks to make sure the requirements can be satisfied and runs unit tests. Since functional and tempest tests and the third party CIs are using versions of libraries much closer to the openstack upper-constraints, that left a big gap between what we were claiming as lower constraints and what was actually being tested.

actions
  • rosmaita - look into what exactly is the function of lower-constraints; it was supposed to be a guideline for packagers, but we need to keep a balance between what's claimed and what's actually being tested
  • driver maintainers - check the values in setup.cfg for dependencies used by your driver and make sure that the minimum isn't too low; also make sure the values in driver-requirements.txt are consistent with setup.cfg (setup.cfg is what's actually used; the driver-requirements.txt is a legacy convenience file for packagers)

Block Storage API v2 removal

The goal of this session was to make a list of task/owner/completion dates so we can move this along and get it done during this cycle. Ivan (e0ne) is acting as v2 removal coordinator.

other projects using v2

tempest

Whether tempest uses v2 or not is controlled by a variable. Tempest has a tempest-cinder-v2-api job: https://opendev.org/openstack/tempest/src/commit/dd84940a81316129bb6adccf7ae4b5cdcbb37382/zuul.d/project.yaml#L123 At some point during Wallaby, tempest will no longer be able to run this job against cinder master.

action
  • rosmaita - talk to gmann about this and figure out how he wants to handle the change
grenade
action
  • e0ne - is looking into this
devstack
actions
other services

Ivan pointed out that when we removed the v1 API, several other services broke even though all the cinder tests were passing. So we need to test horizon, nova, glance, and heat gates when v2 is disabled.

actions
openstackclient, openstackSDK

Give them a heads-up that if they plan to continue Block Storage API v2 support, they'll have to make arrangements to test against an older cinder release.

actions
  • rosmaita - contact [osc][sdk] on the ML
others?
actions
  • rosmaita - general announcement to the ML

how much v2 code must be removed from Cinder in Wallaby?

  • We need to remove the v2 endpoint.
  • v3 uses a lot of v2 classes, which we'll eventually want to refactor. Before doing this, though, we need to verify that there is good v3 test coverage.
  • documentation
    • remove the v2 api-ref
    • refactor the api-ref functional tests to handle only v3
    • make sure the general documentation does not contain references to any v2-specific stuff

cinder-tempest-plugin

actions

rosmaita - tag if necessary and then patch to make sure it only runs v3

python-cinderclient

We already announced on the ML that we'd be removing v2 support in Wallaby:


This has to happen before the client library freeze, which is the week of 8 March.

The cinderclient is structured similar to the Block Storage API in that a lot of v2 classes are reused for v3 functionality. This may make removing v2 functionality from the CLI tricky.

actions

somebody - needs to dig in and see what needs to happen

consistent and secure policies

Keystone made some changes back in Queens (enhanced scoping) and Rocky (default roles). Combining roles and scoping provides common "personas" that can be supported by openstack services in the default policy configuration.

There's a patch up that describes what personas cinder will support, and which provides a matrix of cinder policies and how each of the supported personas should behave by default: https://review.opendev.org/c/openstack/cinder/+/763306

This will give us a document under version control that describes the expected behavior and which will make it easier to verify that the new policies are doing the correct thing.

In reviewing the permissions matrix, Rajat noticed that the default policy to reset the status of a group snapshot is currently admin-or-owner, which seemed weird. A bit of digging revealed that this may have been a mistake when the old policy.json was converted to policies-in-code:

We had a quick discussion over whether anyone remembered if this was an intentional change (nobody remembered). Eric pointed out that exposing this to end-users can be dangerous if they set the state to something like 'CREATING'. The consensus was that this should really be an administrative action by default, and we should treat the current value as a bug, and backport it as far as possible.

We discussed a few other possible changes, but the key takeaway is that even though the "project-admin" persona has "admin" in its name, such a person is *not* an admin in the way we're used to thinking of an "admin". This is discussed in more detail on the permissions matrix patch: https://review.opendev.org/c/openstack/cinder/+/763306

actions

Windows RBD os-brick connector

Lucian (lpetrut) led a followup discussion of a topic he brought up at last week's Cinder meeting, namely adding a mixin class to os-brick to provide a simplified RBD connector for Windows given that Ceph Pacific will support mounting RBD images on Windows.


There was a concern that the windows RBD connector would have to be version aware, or should maybe pass version info through the connection properties. We discussed this a bit, and it wasn't clear that having this information would actually be helpful, and it might expose information via the connector that we don't want exposed. Also, Nova might use cached connector properties that contain old ceph version info. The consensus was that we don't need to be passing the ceph version in the connection info at the current time.

review of proposed specs

migration support for a volume with replication status enabled

https://review.opendev.org/c/openstack/cinder-specs/+/766130

The problem is that the current workflow disallows a migration of a volume that's got replication enabled too high in the stack. The way cinder currently handles multiattach isn't a good model for what needs to be done here. Gorka suggested that this should probably be done the way snapshots are currently handled to go through the scheduler to find out if the operation will be allowed. He dug up two old patches from when snapshots were changed to work this way:

This looks like a promising model for the new spec.

storing volume format info

https://review.opendev.org/c/openstack/cinder-specs/+/760999

The format info will be stored as admin metadata, but shouldn't be exposed in the volume-show call; instead, it should show up in the Attachments API.

We had a brief discussion over what "admin metadata" is supposed to be, because it's ambiguous over whether it's arbitrary info that administrators can add to volumes, or whether it's required for administrating the volume. The consensus was that it should be more like the latter; some drivers use admin metadata to hold useful info for themselves. So the general idea is:

  • drivers can add stuff in the admin metadata
  • system-admins can see it for troubleshooting
  • system-admins should not touch the fields that they themselves don't create

Session Two: R-9: 10 February 2021

We met in BlueJeans from 1400 to 1600 UTC.
etherpad: https://etherpad.openstack.org/p/cinder-wallaby-mid-cycles
recording: <not yet available>

Cinder Project Business

FS backend format info documentation brainstorming

Avoid raw to raw conversions when creating volume from raw image

"ospurge" via openstackclient via openstacksdk

Refactoring cinder.utils / volume_utils

Block Storage API v2 removal

Backup drivers issue with the container parameter

Consistent & Secure RBAC status

mypy

Test with current versions of Ceph

Reviewing XS patches

CI failures unittest

R-9 Bug Review