OpenStack - User contributions [en]

Manila/Meetings

2017-04-27T11:46:45Z

Vponomaryov: /* Next meeting */

= Weekly Manila team meeting =
'''NOTE MEETING TIME: Thursday at 15:00 UTC'''

If you're interested in management of shared filesystems for OpenStack, we have a weekly meetings in <code><nowiki>#openstack-meeting-alt</nowiki></code>, on Thursdays at 15:00 UTC. Please feel free to add items to the agenda below. NOTE: When adding topics please include your IRC name so we know who's topic it is and how to get more info.

'''Previous meeting logs can be found here:''' http://eavesdrop.openstack.org/meetings/manila/

== Next meeting ==
'''NOTE:''' ''Include your IRC nickname next to agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. You might want to put this on your calendar if you are adding items.''

'''Scheduled for 20 Apr, 2017, 15:00 UTC'''

# Ensure shares (zhongjun)
#* https://review.openstack.org/#/c/446494/
# Update shares (zhongjun)
#* https://review.openstack.org/#/c/453553/
# Infinite shares (zhongjun)
#* https://review.openstack.org/#/c/452097/
#* What "infinite" shares ought to be?
#* How should we collect actual share usage?
# API filtering (zhongjun)
#* https://review.openstack.org/#/c/447775/
# Notification about vponomaryov's stepping out of project
#* Valeriy Ponomaryov, known as "vponomaryov" in IRC, will be involved in Manila project till the middle of June. Then he will have other responsibilities.
# Open Discussion

== Previous meetings ==

'''Scheduled for 13 Apr, 2017, 15:00 UTC'''
# Spec Freeze
# Open Discussion

'''Scheduled for 6 Apr, 2017, 15:00 UTC'''
# Concurrency bugs with LVM driver? [http://lists.openstack.org/pipermail/openstack-dev/2017-April/114945.html email ] (tbarron)
# Pike Specs
#* https://etherpad.openstack.org/p/manila-pike-spec-review-focus

'''Scheduled for 30 Mar, 2017, 15:00 UTC'''
# ensure share(zhongjun_)
#*https://review.openstack.org/#/c/446494

'''Scheduled for 23 Mar, 2017, 15:00 UTC'''
# SSL certificate verification
# Specs
#* https://review.openstack.org/#/q/status:open+project:openstack/manila-specs
# Open Discussion

'''Scheduled for 9 Mar, 2017, 15:00 UTC'''
# Specs Deadline (continued) (bswartz)
# Community Goals (bswartz)
#* https://github.com/openstack/governance/blob/master/goals/pike/deploy-api-in-wsgi.rst
#* https://github.com/openstack/governance/blob/master/goals/pike/python35.rst
# Experimental APIs (bswartz)
# Refactor of Tempest scenario base classes (bswartz)
#* http://lists.openstack.org/pipermail/openstack-dev/2017-March/113071.html

'''Scheduled for 2 Mar, 2017, 15:00 UTC'''
# Experimental Features (bswartz)
# Specs Deadline (bswartz)
# Open Discussion

'''Scheduled for 16 Feb, 2017, 15:00 UTC'''
# i18n liason needed
#* https://wiki.openstack.org/wiki/CrossProjectLiaisons#I18n
# PTG
#* https://etherpad.openstack.org/p/manila-pike-ptg-topics

'''Scheduled for 2 Feb, 2017, 15:00 UTC'''
# Microversions (rhagarty)
#* Do we follow Cinder's lead and provide an easy way for other projects to know our current supported microversion? (see https://review.openstack.org/#/c/420119). This would provide the mechanism for the Manila UI to determine if features should/should not be made available to the user.
# Share groups bugs
#* https://bugs.launchpad.net/manila/+bug/1660321
#* https://bugs.launchpad.net/manila/+bug/1660319
#* https://bugs.launchpad.net/manila/+bug/1659023
#* https://bugs.launchpad.net/manila/+bug/1661266
#* https://bugs.launchpad.net/manila/+bug/1661268
# LVM revert to snapshot

'''Scheduled for 26 Jan, 2017, 15:00 UTC'''
# Feature Freeze
#* IPv6
# Open Discussion

'''Scheduled for 19 Jan, 2017, 15:00 UTC'''
# Specs implementation status (bswartz)
#* https://etherpad.openstack.org/p/manila-ocata-code-review-focus
# Filterscheduler warning in logs (ravichandran)
#* "WARNING manila.scheduler.manager [-] Scheduler driver path manila.scheduler.filter_scheduler.FilterScheduler is deprecated, update your configuration to the new path manila.scheduler.drivers.filter.FilterScheduler "
# How to see the recently committed changes to (master branch ) /config-reference/shared-file-systems/drivers/hpe-3par-share-driver.html in docs.openstack.org.(ravichandran)
# How to update the tables inside manuals.Is it auto updated from the code. (ravichandran)
#* Is there any way to call some scripts or manila cli commands with in setupClass of tempest tests to address the prerequisite steps of backedn
# Open Discussion

'''Scheduled for 22 Dec, 2016, 15:00 UTC'''
# Specs implementation status (bswartz)
# briefly clarify the "24 hour rule" (tbarron)
# Manila CI Tempest Test Cases are Failing (ravichandran)

'''Scheduled for 8 Dec, 2016, 15:00 UTC'''
# PTG (bswartz)
# Default share type (tbarron)
# migration-start new command syntax (ganso)
# remove access-update fallback to access_allow/deny? (tbarron)
# decision of deadline for driver config docs in devref removal (ganso)
# Specs implementation status (bswartz)

'''Scheduled for 1 Dec, 2016, 15:00 UTC'''
# Discussion about driver docs in devref (ganso)
# Specs implementation status (bswartz)

'''Scheduled for 17 Nov, 2016, 15:00 UTC'''
# Spec review & prioritization (bswartz)
#* https://etherpad.openstack.org/p/manila-ocata-spec-review-focus
#* https://review.openstack.org/#/q/status:open+project:openstack/manila-specs

'''Scheduled for 10 Nov, 2016, 15:00 UTC'''
# Decide defaults for migration options (ganso)
#* https://review.openstack.org/#/c/392291

'''Scheduled for 3 Nov, 2016, 15:00 UTC'''
# Spec review & prioritization
#* https://review.openstack.org/#/q/project:%22%255Eopenstack/manila-specs%22++status:open
#* https://etherpad.openstack.org/p/manila-ocata-spec-review-focus
#* List of open specs
#** manila-ipv6
#** add-db-manage-purge
#** create-share-from-snapshot-extra-spec
#** support-quota-usage-in-detail
#** ocata-migration-improvements
#** data-service-jobs-table
#** add-share-type-filter-to-get-pools
#** manila-share-revert-to-snapshot
#** scenario-tests
#** mountable_snapshots
#** share-network-multiple-subnets
#** share-retype
#** multiple_protocol_share
#** access-groups
#** backup_share
#** manila-share-groups
#** data-service-api
#* vponomaryov's prioritiesː https://docs.google.com/document/d/1zdMNuQTIqrtDtzbdcxyZVn8SoPN9pbtv_HSLVR3Eld0/edit

'''Scheduled for 20 Oct, 2016, 15:00 UTC'''
# Specs and Deadlines (bswartz)
#* https://review.openstack.org/#/c/374883/
# Adding invalid user to the access rule changes valid access rule state from 'active' to 'error' (ravichandran)
# Open Discussion

'''Scheduled for 13 Oct, 2016, 15:00 UTC'''
# Ocata Design Summit
#* https://www.openstack.org/summit/barcelona-2016/summit-schedule/global-search?t=Manila%3A
# PTG

'''Scheduled for 6 Oct, 2016, 15:00 UTC'''
# Microversion-ing addition or removal of transitional ('micro') states (gouthamr)
#* https://review.openstack.org/#/c/369668/

'''Scheduled for 29 Sep, 2016, 15:00 UTC'''
# Ocata Design Summit
#* https://etherpad.openstack.org/p/manila-ocata-design-summit-topics
# Open Discussion

'''Scheduled for 22 Sep, 2016, 15:00 UTC'''
# Specs process
#* https://review.openstack.org/#/c/374883

'''Scheduled for 8 Sep, 2016, 15:00 UTC'''
# RC1 status
#* https://launchpad.net/manila/+milestone/newton-rc1
# Project Logo
#* http://civs.cs.cornell.edu/cgi-bin/vote.pl?id=E_6f0d111cec78c5ef&akey=d34a751f2d084d79
# Container Driver Security (aovchinnikov)
#* https://review.openstack.org/#/c/353463/
# Security "vulnerability-managed" tag for manila (tbarron)
# Zanata translations for manila-client and manila-ui (tbarron)
# Tempest direction (tbarron)

'''Scheduled for 1 Sep, 2016, 15:00 UTC'''
# Mascot (bswartz)
# Vulnerability Management

'''Scheduled for 25 Aug, 2016, 15:00 UTC'''
# Huawei share replication and CI
# IPv6 (cknight)
# Open Discussion

'''Scheduled for 18 Aug, 2016, 15:00 UTC'''
# Driver private data admin API
#* Disagreement abounds on how this API is to be designed, tested, and presented.
#* https://review.openstack.org/#/c/315346/
# Scenario tests are broken
#* http://logs.openstack.org/86/309286/34/check/gate-manila-tempest-dsvm-scenario/610f96e/console.html
# Revert to snapshot
#* https://review.openstack.org/#/c/340502/
#* https://review.openstack.org/#/c/356682/

"Scheduled for 11 Aug, 2016, 15:00 UTC"
# None

'''Scheduled for 4 Aug, 2016, 15:00 UTC'''
# Migration APIs
#* http://lists.openstack.org/pipermail/openstack-dev/2016-August/100682.html
# Correct approach for avoiding races

'''Scheduled for 28 Jul, 2016, 15:00 UTC'''
# Stepping Down as Docs Liaison (dustins)
#* Current job responsibilities don't give me the time to devote to this that I'd like to have, so it should go to someone else with a passion for documentation and has the willingness and desire for taking up the mantle.
# N-3 Review focus
#* https://etherpad.openstack.org/p/manila-newton3-priorities

'''Scheduled for 21 Jul, 2016, 15:00 UTC'''
# Migrate API reference into tree (dgonzalez)
#* https://review.openstack.org/#/c/313874/

'''Scheduled for 14 Jul, 2016, 15:00 UTC'''
# N-2 milestone
# Managing initial security (of the root directory) for new shares
# Mascot/Logo discussion
# Tempest alternatives

'''Scheduled for 30 Jun, 2016, 15:00 UTC'''
# non-disruptive nfs-ganesha export modifications (rraja/tbarron)
#* https://sourceforge.net/p/nfs-ganesha/mailman/message/35173839
# share backup spec discussion (zhongjun)
#* https://review.openstack.org/#/c/330306

'''Scheduled for 23 Jun, 2016, 15:00 UTC'''
# Implement extend and shrink features for glusterfs driver (DaiDV)
# Do all drivers have to inherit from driver.ShareDriver? (mkoderer)
# Creating a second share instance for driver-assisted migration (ganso)

'''Scheduled for 16 Jun, 2016, 15:00 UTC'''
# A share support multiple protocols (zhongjun)
#* https://review.openstack.org/#/c/329392/
# Drivers with minimum size for shares (tpsilva)

'''Scheduled for 9 Jun, 2016, 15:00 UTC'''
# HPB feature (mkoderer)
# the fate of nova-network support in manila (gouthamr)
# multi-AZ tests in the gate

'''Scheduled for 2 Jun, 2016, 15:00 UTC'''
# N-1 milestone (bswartz)
# Decision on APIv1 removal for newton (bswartz)
# Model updates from drivers (bswartz)
# Midcycle meetup
# Spec update
#* New spec - Extending share networks to span subnets - https://review.openstack.org/#/c/323646/ (gouthamr)

'''Scheduled for 26 May, 2016, 15:00 UTC'''
# N-1 milestone
# Update access remaining work
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* Nova-network removal
#* APIv1 removal
#* Model updates from the driver

'''Scheduled for 19 May, 2016, 15:00 UTC'''
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* UI Customization
# Share Backup(zhongjun)
# Snapshot Restore (cknight)
# Specs (ganso)

'''Scheduled for 12 May, 2016, 15:00 UTC'''
# Driver private storage: https://review.openstack.org/#/c/315346/ (xyang)
#* Admin wants to check driver private storage before doing some actions on a resource, i.e., force delete a share or unmanage a share.
#* The patch proposes to add admin API to retrieve driver private storage data. Alternative approach proposed by Valeriy to use manila-manage (i.e., manila-manage driver-private-storage <uuid of a resource>). Which one should we choose?
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* Py3 functional tests
#* Replication with share networks
#* Service-check
#* Container driver

'''Scheduled for 5 May, 2016, 15:00 UTC'''
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* Documentation update (dustins/gouthamr)
#* User messages
#* Experimental features in UI
#* Share/snapshot size mismatch
# BP Specs (tbarron)
#* Need to ensure that the repo doesn't become "a wasteland of partial ideas where hope goes to die." (See http://lists.openstack.org/pipermail/openstack-dev/2016-May/094026.html)

'''Scheduled for 14 Apr, 2016, 15:00 UTC'''
# Manage API retrying (gouthamr/nidhimittalhada)
# Tempest test structuring - resource sharing and cleanup issues (akerr)
# Snapshot support common capability in the documentation (gouthamr)

'''Scheduled for 7 Apr, 2016, 15:00 UTC'''
# Access rules (bswartz)
# Design summit planning (bswartz)
#* Link: https://etherpad.openstack.org/p/manila-newton-summit-topics
# Release notes, continued (gouthamr)
#* Link: https://review.openstack.org/#/c/300656/
# Midcycle meetup
# Access Groups
#* https://wiki.openstack.org/wiki/Manila/design/access_groups
# Revert to snapshot

'''Scheduled for 31 Mar, 2016, 15:00 UTC'''
# Mitaka release status
#* RC1 expected to be the Mitaka release build
#* Last day for a re-spin is today
#** http://lists.openstack.org/pipermail/openstack-dev/2016-March/090877.html
# Design summit planning
#* Final schedule confirms 2 fishbowls, 4 working sessions, 1 half-day meetup
#** http://lists.openstack.org/pipermail/openstack-dev/2016-March/091032.html
#* Voting open now for summit topics
#** https://etherpad.openstack.org/p/manila-newton-summit-topics
# Manila Documentation (gouthamr)
#* Where do our docs live? how do we maintain them? (In Tree: devref, adminref | out of tree: install-guide, config-ref, cloud-admin guide)
#* Need a docs liaison
#** https://wiki.openstack.org/wiki/CrossProjectLiaisons#Documentation
#* Release Notes
#** http://docs.openstack.org/developer/reno/usage.html
# Managing/unmanaging replicated shares and snapshots (gouthamr)

'''Scheduled for 24 Mar, 2016, 15:00 UTC'''
# Design summit planning
#* https://etherpad.openstack.org/p/manila-newton-summit-topics
# Question about share servers (Jay.Xu)
#* Is there a way to get gateway IP address when the API setup_server() is invoked. I did not see this attribute in the input parameter network_info. Gateway IP is mandatory to create the interface for EMC storage which will be release coming soon, so it is expected that we can add the gateway IP into NetworkAllocation model if not.

'''Scheduled for 10 Mar, 2016, 15:00 UTC'''
# Common Capabilities Matrix (dustins)
#* https://review.openstack.org/#/c/288504/
#* I'm adding a table in the Developer Reference documentation to show which common capabilities are supported by which backends and when that support started.
#* I've got most drivers covered now.
#* If you want me to add your capabilities, leave a review on the patch and I'll update the table!

'''Scheduled for 25 Feb, 2016, 15:00 UTC'''

# OpenStack bug smash event (toabctl, mkoderer)
## link: https://etherpad.openstack.org/p/OpenStack-Bug-Smash-Mitaka
# Status of new drivers
## M.3 deadline is a week away.
## Only a few things have merged in the last couple days. At this rate, many things won't make it.
## Zuul is busy and will only get worse until the deadline.
## Multiple Manila gate breakages have occurred recently, and more are likely.
## The Manila tests are very unreliable at concurrency > 1.
## The generic driver is too slow to pass the gate at concurrency = 1.
## The new drivers *are* fast enough to run at concurrency = 1.
## The new drivers may not be 100% done, but they are done enough to pass reliably at concurrency = 1.
## We should switch voting to the new drivers NOW, or start throwing stuff overboard for the release.
# Next Manila mid-cycle location
## Proposal is to do it in Germany
## Need to know who could attend at that location.
## Would it make a difference if the Cinder mid-cycle happens in Dublin, and the Manila meeting is immediately before or after the Cinder one?

'''Scheduled for 18 Feb, 2016, 15:00 UTC'''

# Feature Proposal Freeze (bswartz)
# Open Discussion

'''Scheduled for 11 Feb, 2016, 15:00 UTC'''

# Hierarchical port binding support (mkoderer)
#* https://wiki.openstack.org/wiki/Manila/design/manila-newton-hpb-support
#* https://blueprints.launchpad.net/manila/+spec/manila-hpb-support
#* https://review.openstack.org/#/c/277731/
# Next Manila mid-cycle (mkoderer)
#* https://etherpad.openstack.org/p/manila-n-release-midcycle
# Replication + ZFS Driver (bswartz)
#* https://review.openstack.org/#/c/238572
# Driver FPF (bswartz)
#* https://review.openstack.org/#/c/279090/
#* https://review.openstack.org/#/c/278169/
# HDFS CI (bswartz)
'''Scheduled for 4 Feb, 2016, 15:00 UTC'''
# Data service handling protocol mount commands (ganso)
#* https://etherpad.openstack.org/p/manila-data-service-mount-meeting
# Separate process for copy on data service (tpsilva)
# ZFS driver (bswartz)
# First party drivers share migration support cooperation (ganso)

'''Scheduled for 28 Jan, 2016, 15:00 UTC'''
# Hierarchical port binding support (mkoderer)
#* https://etherpad.openstack.org/p/manila-hierarchical-port-binding
# Core reviewer team and other manila-related projects (bswartz)
# Replication status (bswartz)

'''Scheduled for 21 Jan, 2016, 15:00 UTC'''
# Update_access backwards compatibility approach discussion. (ganso)
# Exposing the Data Service to drivers. (ganso)
# Snapshot manage/unmanage (bswartz)

'''Scheduled for 17 Dec, 2015, 15:00 UTC'''
# Proposal for share groups (cknight)
# Liberty.1 stable release (bswartz)

'''Scheduled for 10 Dec, 2015, 15:00 UTC'''
# The most of the driver already report the "'QoS_support': False" to scheduler. Does we need to remove it or save it as Qos support flat? and write 'QoS_support' to capabilities_and_extra_specs.rst?(zhongjun)
##https://github.com/openstack/manila/blob/master/manila/share/drivers/hpe/hpe_3par_driver.py#L391
##https://github.com/openstack/manila/blob/master/manila/share/drivers/hitachi/hds_hnas.py#L354
##https://github.com/openstack/manila/blob/master/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py#L257
##https://github.com/openstack/manila/blob/master/manila/scheduler/host_manager.py#L111
##https://review.openstack.org/#/c/247286/4/doc/source/devref/capabilities_and_extra_specs.rst
# Architectural concerns and interoperability of new features (cknight)

'''Scheduled for 3 Dec, 2015, 15:00 UTC'''
# Midcycle meetup
# Add "data loss" flag in DB migrations which lead to losing of existing data and update "manila-manage db sync" command to forbid applying such migrations without "--force" flag (u_glide)
# Reorganization of Manila scheduler to merge oslo-incubator code and simplify future maintenance (cknight)
## https://review.openstack.org/#/c/252060/
## Mostly for awareness, this is now merged and should be backwards compatible.
## Cinder could benefit from a similar treatment. And with the incubator gone, we could explore a common library with the Cinder team, since the heretofore shared components are certain to diverge over time, but I'm not sure how receptive they would be. Xing?
# CI reliability
## Why should we merge patches for drivers whose CI systems consistently fail? This negates the value of CI. I think that accepting patches that aren't proven by CI does a disservice to everyone, most of all to the driver owners themselves. Refusing to merge driver patches without a good CI result for that driver seems like a gentler and more uniform-over-time approach than threatening to throw out a driver near the end of a release. And why make our illustrious and overworked PTL be the CI policeman when all core reviewers could simply glance at the corresponding CI result before giving +A? (cknight)
## http://ci-watch.tintri.com/project?project=manila
## https://review.openstack.org/#/c/232293/
## https://review.openstack.org/#/c/235522/
## https://review.openstack.org/#/c/235726/
## https://review.openstack.org/#/c/249999/
## https://review.openstack.org/#/c/237454/

'''Scheduled for 19 Nov, 2015, 15:00 UTC'''
# GB --> GiB (markstur)
# OpenStack Manuals and Devref (markstur)
#* Config Ref (and others) are moving to .RST!
#* How should we document prior releases in devref?
# Midcycle Meetup Date (bswartz)
# Deadlines for features (bswartz)
#* Feature proposal freeze vs Big Features
#* New drivers
# Data Copy Service network approach discussion (ganso)
#* https://drive.google.com/file/d/0B30HPZmaePCHMlAzcjJlYm5rRTA/view?usp=sharing

'''Scheduled for 12 Nov, 2015, 15:00 UTC'''
# Which method is better for Manila QoS(independent way or just use extra specs ) (zhongjun)
#* link:http://paste.openstack.org/show/477677/
#*If use extra specs to config QoS, Does we first need add “QoS_support=true/false” in share_type. Does we need to add QoS_support flag in common capabilities
#**https://github.com/openstack/manila/blob/master/doc/source/devref/capabilities_and_extra_specs.rst)?
#** eg: share_type: qos_support='<is> True' or '<is> False'
#** netapp:max_iops=100 or qos:netapp:max_iops=100 or qos:max_iops=100
#** driver update: qos_support=true
#** An alternative could be to have the driver report a range
# Manila DR update (ameade)
#* Still no reviews on https://review.openstack.org/#/c/238572/
#* First-party driver implementation? Which driver, current generic or new solution? When do we need this implemented and who wants to do it?
#* Replication_change share state - This is the state of the share when any of it's replicas are being promoted. Is 'replication_change' good or is there a better phrase?
# Manila Driver minimum requirements document update (ganso)
# Manila Data Copy service name (bswartz)
#*https://review.openstack.org/#/c/244286/
# Upcoming change for Manila CI hooks (vponomaryov)
#*https://review.openstack.org/#/c/243233/
# Manilaclient enhancement to provide request_id when set http_log_debug is True. It will co-ordinate the request and its id, so it will be helpful to filter the log and the failed request. (Jay Xu)
#*http://paste.openstack.org/show/478675/

'''Scheduled for 5 Nov, 2015, 15:00 UTC'''
# Decide on read-only access rules as a required feature
# Manila DR - Design/Code feedback (gouthamr)
#* [https://wiki.openstack.org/wiki/Manila/design/manila-mitaka-data-replication Design Document]
#* [https://review.openstack.org/#/c/238572/ Core Implementation on Gerrit]
#* [https://review.openstack.org/#/c/235448/ Client Implementation on Gerrit]

'''Scheduled for 1 Oct, 2015, 15:00 UTC'''
# Liberty RC2 status
#* https://launchpad.net/manila/+milestone/liberty-rc2
#* https://review.openstack.org/#/q/project:openstack/manila+branch:stable/liberty,n,z
# Mitaka Design Summit
#* https://etherpad.openstack.org/p/manila-mitaka-summit-topics
# Vendor driver docs
#* https://github.com/openstack/manila/blob/master/doc/source/devref/share_back_ends_feature_support_mapping.rst
# Manila Client Nova Style Microversion Support (cfouts)
#* https://review.openstack.org/#/c/229142/
#* http://docs.openstack.org/developer/manila/devref/api_microversion_dev.html

'''Scheduled for 24 Sep, 2015, 15:00 UTC'''
# 3rd-party CI status
#* http://ec2-54-67-102-119.us-west-1.compute.amazonaws.com:5000/?project=openstack/manila&user=&timeframe=72&start=&end=&page_size=500
#* http://ci-watch.tintri.com/project?project=manila
# Liberty RC status
# Support for QoS in Manila (zhongjun2)
#* https://blueprints.launchpad.net/manila/+spec/manila-support-qos
#* https://wiki.openstack.org/wiki/Manila/QoS
# Reminder for backend driver maintainers to update feature support table (toabctl)
#* http://docs.openstack.org/developer/manila/devref/share_back_ends_feature_support_mapping.html
# Regression when using NetApp - license issue with netapp_lib (toabctl)
#* https://bugs.launchpad.net/cinder/+bug/1499334
# Mitaka Design Summit
#* Propose topics - https://etherpad.openstack.org/p/manila-mitaka-summit-topics
# Vendor driver docs
# Open Discussion

Manila/Meetings

2016-11-10T12:23:18Z

Vponomaryov: /* Next meeting */

= Weekly Manila team meeting =
'''NOTE MEETING TIME: Thursday at 15:00 UTC'''

If you're interested in management of shared filesystems for OpenStack, we have a weekly meetings in <code><nowiki>#openstack-meeting-alt</nowiki></code>, on Thursdays at 15:00 UTC. Please feel free to add items to the agenda below. NOTE: When adding topics please include your IRC name so we know who's topic it is and how to get more info.

'''Previous meeting logs can be found here:''' http://eavesdrop.openstack.org/meetings/manila/

== Next meeting ==
'''NOTE:''' ''Include your IRC nickname next to agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. You might want to put this on your calendar if you are adding items.''

'''Scheduled for 10 Nov, 2016, 15:00 UTC'''
# Decide defaults for migration options (ganso)
#* https://review.openstack.org/#/c/392291
# Spec review & prioritization (bswartz)
#* https://etherpad.openstack.org/p/manila-ocata-spec-review-focus
#* https://review.openstack.org/#/q/status:open+project:openstack/manila-specs
# Topics not covered at design summit (bswartz)
#* ensure share
#* manage API requirements (vponomaryov)
#** Add possibility for share drivers to set all existing rules of just managed share?
#** Should it be renamed on storage based on new share_instance_id or not?
#** Should its export locations be changed or not?
#* manila-ui displaying only enabled share protocols
#* app-catalog use case
#* share replica quotas (vponomaryov)
# Open Discussion

== Previous meetings ==

'''Scheduled for 3 Nov, 2016, 15:00 UTC'''
# Spec review & prioritization
#* https://review.openstack.org/#/q/project:%22%255Eopenstack/manila-specs%22++status:open
#* https://etherpad.openstack.org/p/manila-ocata-spec-review-focus
#* List of open specs
#** manila-ipv6
#** add-db-manage-purge
#** create-share-from-snapshot-extra-spec
#** support-quota-usage-in-detail
#** ocata-migration-improvements
#** data-service-jobs-table
#** add-share-type-filter-to-get-pools
#** manila-share-revert-to-snapshot
#** scenario-tests
#** mountable_snapshots
#** share-network-multiple-subnets
#** share-retype
#** multiple_protocol_share
#** access-groups
#** backup_share
#** manila-share-groups
#** data-service-api
#* vponomaryov's prioritiesː https://docs.google.com/document/d/1zdMNuQTIqrtDtzbdcxyZVn8SoPN9pbtv_HSLVR3Eld0/edit

'''Scheduled for 20 Oct, 2016, 15:00 UTC'''
# Specs and Deadlines (bswartz)
#* https://review.openstack.org/#/c/374883/
# Adding invalid user to the access rule changes valid access rule state from 'active' to 'error' (ravichandran)
# Open Discussion

'''Scheduled for 13 Oct, 2016, 15:00 UTC'''
# Ocata Design Summit
#* https://www.openstack.org/summit/barcelona-2016/summit-schedule/global-search?t=Manila%3A
# PTG

'''Scheduled for 6 Oct, 2016, 15:00 UTC'''
# Microversion-ing addition or removal of transitional ('micro') states (gouthamr)
#* https://review.openstack.org/#/c/369668/

'''Scheduled for 29 Sep, 2016, 15:00 UTC'''
# Ocata Design Summit
#* https://etherpad.openstack.org/p/manila-ocata-design-summit-topics
# Open Discussion

'''Scheduled for 22 Sep, 2016, 15:00 UTC'''
# Specs process
#* https://review.openstack.org/#/c/374883

'''Scheduled for 8 Sep, 2016, 15:00 UTC'''
# RC1 status
#* https://launchpad.net/manila/+milestone/newton-rc1
# Project Logo
#* http://civs.cs.cornell.edu/cgi-bin/vote.pl?id=E_6f0d111cec78c5ef&akey=d34a751f2d084d79
# Container Driver Security (aovchinnikov)
#* https://review.openstack.org/#/c/353463/
# Security "vulnerability-managed" tag for manila (tbarron)
# Zanata translations for manila-client and manila-ui (tbarron)
# Tempest direction (tbarron)

'''Scheduled for 1 Sep, 2016, 15:00 UTC'''
# Mascot (bswartz)
# Vulnerability Management

'''Scheduled for 25 Aug, 2016, 15:00 UTC'''
# Huawei share replication and CI
# IPv6 (cknight)
# Open Discussion

'''Scheduled for 18 Aug, 2016, 15:00 UTC'''
# Driver private data admin API
#* Disagreement abounds on how this API is to be designed, tested, and presented.
#* https://review.openstack.org/#/c/315346/
# Scenario tests are broken
#* http://logs.openstack.org/86/309286/34/check/gate-manila-tempest-dsvm-scenario/610f96e/console.html
# Revert to snapshot
#* https://review.openstack.org/#/c/340502/
#* https://review.openstack.org/#/c/356682/

"Scheduled for 11 Aug, 2016, 15:00 UTC"
# None

'''Scheduled for 4 Aug, 2016, 15:00 UTC'''
# Migration APIs
#* http://lists.openstack.org/pipermail/openstack-dev/2016-August/100682.html
# Correct approach for avoiding races

'''Scheduled for 28 Jul, 2016, 15:00 UTC'''
# Stepping Down as Docs Liaison (dustins)
#* Current job responsibilities don't give me the time to devote to this that I'd like to have, so it should go to someone else with a passion for documentation and has the willingness and desire for taking up the mantle.
# N-3 Review focus
#* https://etherpad.openstack.org/p/manila-newton3-priorities

'''Scheduled for 21 Jul, 2016, 15:00 UTC'''
# Migrate API reference into tree (dgonzalez)
#* https://review.openstack.org/#/c/313874/

'''Scheduled for 14 Jul, 2016, 15:00 UTC'''
# N-2 milestone
# Managing initial security (of the root directory) for new shares
# Mascot/Logo discussion
# Tempest alternatives

'''Scheduled for 30 Jun, 2016, 15:00 UTC'''
# non-disruptive nfs-ganesha export modifications (rraja/tbarron)
#* https://sourceforge.net/p/nfs-ganesha/mailman/message/35173839
# share backup spec discussion (zhongjun)
#* https://review.openstack.org/#/c/330306

'''Scheduled for 23 Jun, 2016, 15:00 UTC'''
# Implement extend and shrink features for glusterfs driver (DaiDV)
# Do all drivers have to inherit from driver.ShareDriver? (mkoderer)
# Creating a second share instance for driver-assisted migration (ganso)

'''Scheduled for 16 Jun, 2016, 15:00 UTC'''
# A share support multiple protocols (zhongjun)
#* https://review.openstack.org/#/c/329392/
# Drivers with minimum size for shares (tpsilva)

'''Scheduled for 9 Jun, 2016, 15:00 UTC'''
# HPB feature (mkoderer)
# the fate of nova-network support in manila (gouthamr)
# multi-AZ tests in the gate

'''Scheduled for 2 Jun, 2016, 15:00 UTC'''
# N-1 milestone (bswartz)
# Decision on APIv1 removal for newton (bswartz)
# Model updates from drivers (bswartz)
# Midcycle meetup
# Spec update
#* New spec - Extending share networks to span subnets - https://review.openstack.org/#/c/323646/ (gouthamr)

'''Scheduled for 26 May, 2016, 15:00 UTC'''
# N-1 milestone
# Update access remaining work
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* Nova-network removal
#* APIv1 removal
#* Model updates from the driver

'''Scheduled for 19 May, 2016, 15:00 UTC'''
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* UI Customization
# Share Backup(zhongjun)
# Snapshot Restore (cknight)
# Specs (ganso)

'''Scheduled for 12 May, 2016, 15:00 UTC'''
# Driver private storage: https://review.openstack.org/#/c/315346/ (xyang)
#* Admin wants to check driver private storage before doing some actions on a resource, i.e., force delete a share or unmanage a share.
#* The patch proposes to add admin API to retrieve driver private storage data. Alternative approach proposed by Valeriy to use manila-manage (i.e., manila-manage driver-private-storage <uuid of a resource>). Which one should we choose?
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* Py3 functional tests
#* Replication with share networks
#* Service-check
#* Container driver

'''Scheduled for 5 May, 2016, 15:00 UTC'''
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* Documentation update (dustins/gouthamr)
#* User messages
#* Experimental features in UI
#* Share/snapshot size mismatch
# BP Specs (tbarron)
#* Need to ensure that the repo doesn't become "a wasteland of partial ideas where hope goes to die." (See http://lists.openstack.org/pipermail/openstack-dev/2016-May/094026.html)

'''Scheduled for 14 Apr, 2016, 15:00 UTC'''
# Manage API retrying (gouthamr/nidhimittalhada)
# Tempest test structuring - resource sharing and cleanup issues (akerr)
# Snapshot support common capability in the documentation (gouthamr)

'''Scheduled for 7 Apr, 2016, 15:00 UTC'''
# Access rules (bswartz)
# Design summit planning (bswartz)
#* Link: https://etherpad.openstack.org/p/manila-newton-summit-topics
# Release notes, continued (gouthamr)
#* Link: https://review.openstack.org/#/c/300656/
# Midcycle meetup
# Access Groups
#* https://wiki.openstack.org/wiki/Manila/design/access_groups
# Revert to snapshot

'''Scheduled for 31 Mar, 2016, 15:00 UTC'''
# Mitaka release status
#* RC1 expected to be the Mitaka release build
#* Last day for a re-spin is today
#** http://lists.openstack.org/pipermail/openstack-dev/2016-March/090877.html
# Design summit planning
#* Final schedule confirms 2 fishbowls, 4 working sessions, 1 half-day meetup
#** http://lists.openstack.org/pipermail/openstack-dev/2016-March/091032.html
#* Voting open now for summit topics
#** https://etherpad.openstack.org/p/manila-newton-summit-topics
# Manila Documentation (gouthamr)
#* Where do our docs live? how do we maintain them? (In Tree: devref, adminref | out of tree: install-guide, config-ref, cloud-admin guide)
#* Need a docs liaison
#** https://wiki.openstack.org/wiki/CrossProjectLiaisons#Documentation
#* Release Notes
#** http://docs.openstack.org/developer/reno/usage.html
# Managing/unmanaging replicated shares and snapshots (gouthamr)

'''Scheduled for 24 Mar, 2016, 15:00 UTC'''
# Design summit planning
#* https://etherpad.openstack.org/p/manila-newton-summit-topics
# Question about share servers (Jay.Xu)
#* Is there a way to get gateway IP address when the API setup_server() is invoked. I did not see this attribute in the input parameter network_info. Gateway IP is mandatory to create the interface for EMC storage which will be release coming soon, so it is expected that we can add the gateway IP into NetworkAllocation model if not.

'''Scheduled for 10 Mar, 2016, 15:00 UTC'''
# Common Capabilities Matrix (dustins)
#* https://review.openstack.org/#/c/288504/
#* I'm adding a table in the Developer Reference documentation to show which common capabilities are supported by which backends and when that support started.
#* I've got most drivers covered now.
#* If you want me to add your capabilities, leave a review on the patch and I'll update the table!

'''Scheduled for 25 Feb, 2016, 15:00 UTC'''

# OpenStack bug smash event (toabctl, mkoderer)
## link: https://etherpad.openstack.org/p/OpenStack-Bug-Smash-Mitaka
# Status of new drivers
## M.3 deadline is a week away.
## Only a few things have merged in the last couple days. At this rate, many things won't make it.
## Zuul is busy and will only get worse until the deadline.
## Multiple Manila gate breakages have occurred recently, and more are likely.
## The Manila tests are very unreliable at concurrency > 1.
## The generic driver is too slow to pass the gate at concurrency = 1.
## The new drivers *are* fast enough to run at concurrency = 1.
## The new drivers may not be 100% done, but they are done enough to pass reliably at concurrency = 1.
## We should switch voting to the new drivers NOW, or start throwing stuff overboard for the release.
# Next Manila mid-cycle location
## Proposal is to do it in Germany
## Need to know who could attend at that location.
## Would it make a difference if the Cinder mid-cycle happens in Dublin, and the Manila meeting is immediately before or after the Cinder one?

'''Scheduled for 18 Feb, 2016, 15:00 UTC'''

# Feature Proposal Freeze (bswartz)
# Open Discussion

'''Scheduled for 11 Feb, 2016, 15:00 UTC'''

# Hierarchical port binding support (mkoderer)
#* https://wiki.openstack.org/wiki/Manila/design/manila-newton-hpb-support
#* https://blueprints.launchpad.net/manila/+spec/manila-hpb-support
#* https://review.openstack.org/#/c/277731/
# Next Manila mid-cycle (mkoderer)
#* https://etherpad.openstack.org/p/manila-n-release-midcycle
# Replication + ZFS Driver (bswartz)
#* https://review.openstack.org/#/c/238572
# Driver FPF (bswartz)
#* https://review.openstack.org/#/c/279090/
#* https://review.openstack.org/#/c/278169/
# HDFS CI (bswartz)
'''Scheduled for 4 Feb, 2016, 15:00 UTC'''
# Data service handling protocol mount commands (ganso)
#* https://etherpad.openstack.org/p/manila-data-service-mount-meeting
# Separate process for copy on data service (tpsilva)
# ZFS driver (bswartz)
# First party drivers share migration support cooperation (ganso)

'''Scheduled for 28 Jan, 2016, 15:00 UTC'''
# Hierarchical port binding support (mkoderer)
#* https://etherpad.openstack.org/p/manila-hierarchical-port-binding
# Core reviewer team and other manila-related projects (bswartz)
# Replication status (bswartz)

'''Scheduled for 21 Jan, 2016, 15:00 UTC'''
# Update_access backwards compatibility approach discussion. (ganso)
# Exposing the Data Service to drivers. (ganso)
# Snapshot manage/unmanage (bswartz)

'''Scheduled for 17 Dec, 2015, 15:00 UTC'''
# Proposal for share groups (cknight)
# Liberty.1 stable release (bswartz)

'''Scheduled for 10 Dec, 2015, 15:00 UTC'''
# The most of the driver already report the "'QoS_support': False" to scheduler. Does we need to remove it or save it as Qos support flat? and write 'QoS_support' to capabilities_and_extra_specs.rst?(zhongjun)
##https://github.com/openstack/manila/blob/master/manila/share/drivers/hpe/hpe_3par_driver.py#L391
##https://github.com/openstack/manila/blob/master/manila/share/drivers/hitachi/hds_hnas.py#L354
##https://github.com/openstack/manila/blob/master/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py#L257
##https://github.com/openstack/manila/blob/master/manila/scheduler/host_manager.py#L111
##https://review.openstack.org/#/c/247286/4/doc/source/devref/capabilities_and_extra_specs.rst
# Architectural concerns and interoperability of new features (cknight)

'''Scheduled for 3 Dec, 2015, 15:00 UTC'''
# Midcycle meetup
# Add "data loss" flag in DB migrations which lead to losing of existing data and update "manila-manage db sync" command to forbid applying such migrations without "--force" flag (u_glide)
# Reorganization of Manila scheduler to merge oslo-incubator code and simplify future maintenance (cknight)
## https://review.openstack.org/#/c/252060/
## Mostly for awareness, this is now merged and should be backwards compatible.
## Cinder could benefit from a similar treatment. And with the incubator gone, we could explore a common library with the Cinder team, since the heretofore shared components are certain to diverge over time, but I'm not sure how receptive they would be. Xing?
# CI reliability
## Why should we merge patches for drivers whose CI systems consistently fail? This negates the value of CI. I think that accepting patches that aren't proven by CI does a disservice to everyone, most of all to the driver owners themselves. Refusing to merge driver patches without a good CI result for that driver seems like a gentler and more uniform-over-time approach than threatening to throw out a driver near the end of a release. And why make our illustrious and overworked PTL be the CI policeman when all core reviewers could simply glance at the corresponding CI result before giving +A? (cknight)
## http://ci-watch.tintri.com/project?project=manila
## https://review.openstack.org/#/c/232293/
## https://review.openstack.org/#/c/235522/
## https://review.openstack.org/#/c/235726/
## https://review.openstack.org/#/c/249999/
## https://review.openstack.org/#/c/237454/

'''Scheduled for 19 Nov, 2015, 15:00 UTC'''
# GB --> GiB (markstur)
# OpenStack Manuals and Devref (markstur)
#* Config Ref (and others) are moving to .RST!
#* How should we document prior releases in devref?
# Midcycle Meetup Date (bswartz)
# Deadlines for features (bswartz)
#* Feature proposal freeze vs Big Features
#* New drivers
# Data Copy Service network approach discussion (ganso)
#* https://drive.google.com/file/d/0B30HPZmaePCHMlAzcjJlYm5rRTA/view?usp=sharing

'''Scheduled for 12 Nov, 2015, 15:00 UTC'''
# Which method is better for Manila QoS(independent way or just use extra specs ) (zhongjun)
#* link:http://paste.openstack.org/show/477677/
#*If use extra specs to config QoS, Does we first need add “QoS_support=true/false” in share_type. Does we need to add QoS_support flag in common capabilities
#**https://github.com/openstack/manila/blob/master/doc/source/devref/capabilities_and_extra_specs.rst)?
#** eg: share_type: qos_support='<is> True' or '<is> False'
#** netapp:max_iops=100 or qos:netapp:max_iops=100 or qos:max_iops=100
#** driver update: qos_support=true
#** An alternative could be to have the driver report a range
# Manila DR update (ameade)
#* Still no reviews on https://review.openstack.org/#/c/238572/
#* First-party driver implementation? Which driver, current generic or new solution? When do we need this implemented and who wants to do it?
#* Replication_change share state - This is the state of the share when any of it's replicas are being promoted. Is 'replication_change' good or is there a better phrase?
# Manila Driver minimum requirements document update (ganso)
# Manila Data Copy service name (bswartz)
#*https://review.openstack.org/#/c/244286/
# Upcoming change for Manila CI hooks (vponomaryov)
#*https://review.openstack.org/#/c/243233/
# Manilaclient enhancement to provide request_id when set http_log_debug is True. It will co-ordinate the request and its id, so it will be helpful to filter the log and the failed request. (Jay Xu)
#*http://paste.openstack.org/show/478675/

'''Scheduled for 5 Nov, 2015, 15:00 UTC'''
# Decide on read-only access rules as a required feature
# Manila DR - Design/Code feedback (gouthamr)
#* [https://wiki.openstack.org/wiki/Manila/design/manila-mitaka-data-replication Design Document]
#* [https://review.openstack.org/#/c/238572/ Core Implementation on Gerrit]
#* [https://review.openstack.org/#/c/235448/ Client Implementation on Gerrit]

'''Scheduled for 1 Oct, 2015, 15:00 UTC'''
# Liberty RC2 status
#* https://launchpad.net/manila/+milestone/liberty-rc2
#* https://review.openstack.org/#/q/project:openstack/manila+branch:stable/liberty,n,z
# Mitaka Design Summit
#* https://etherpad.openstack.org/p/manila-mitaka-summit-topics
# Vendor driver docs
#* https://github.com/openstack/manila/blob/master/doc/source/devref/share_back_ends_feature_support_mapping.rst
# Manila Client Nova Style Microversion Support (cfouts)
#* https://review.openstack.org/#/c/229142/
#* http://docs.openstack.org/developer/manila/devref/api_microversion_dev.html

'''Scheduled for 24 Sep, 2015, 15:00 UTC'''
# 3rd-party CI status
#* http://ec2-54-67-102-119.us-west-1.compute.amazonaws.com:5000/?project=openstack/manila&user=&timeframe=72&start=&end=&page_size=500
#* http://ci-watch.tintri.com/project?project=manila
# Liberty RC status
# Support for QoS in Manila (zhongjun2)
#* https://blueprints.launchpad.net/manila/+spec/manila-support-qos
#* https://wiki.openstack.org/wiki/Manila/QoS
# Reminder for backend driver maintainers to update feature support table (toabctl)
#* http://docs.openstack.org/developer/manila/devref/share_back_ends_feature_support_mapping.html
# Regression when using NetApp - license issue with netapp_lib (toabctl)
#* https://bugs.launchpad.net/cinder/+bug/1499334
# Mitaka Design Summit
#* Propose topics - https://etherpad.openstack.org/p/manila-mitaka-summit-topics
# Vendor driver docs
# Open Discussion

Manila/Meetings

2016-11-07T13:18:56Z

Vponomaryov: /* Next meeting */

= Weekly Manila team meeting =
'''NOTE MEETING TIME: Thursday at 15:00 UTC'''

If you're interested in management of shared filesystems for OpenStack, we have a weekly meetings in <code><nowiki>#openstack-meeting-alt</nowiki></code>, on Thursdays at 15:00 UTC. Please feel free to add items to the agenda below. NOTE: When adding topics please include your IRC name so we know who's topic it is and how to get more info.

'''Previous meeting logs can be found here:''' http://eavesdrop.openstack.org/meetings/manila/

== Next meeting ==
'''NOTE:''' ''Include your IRC nickname next to agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. You might want to put this on your calendar if you are adding items.''

'''Scheduled for 10 Nov, 2016, 15:00 UTC'''
# Spec review & prioritization
# Topics not covered at design summit
#* ensure share
#* manage API requirements
#* manila-ui displaying only enabled share protocols
#* app-catalog use case
#* share replica quotas
# Open Discussion

== Previous meetings ==

'''Scheduled for 3 Nov, 2016, 15:00 UTC'''
# Spec review & prioritization
#* https://review.openstack.org/#/q/project:%22%255Eopenstack/manila-specs%22++status:open
#* https://etherpad.openstack.org/p/manila-ocata-spec-review-focus
#* List of open specs
#** manila-ipv6
#** add-db-manage-purge
#** create-share-from-snapshot-extra-spec
#** support-quota-usage-in-detail
#** ocata-migration-improvements
#** data-service-jobs-table
#** add-share-type-filter-to-get-pools
#** manila-share-revert-to-snapshot
#** scenario-tests
#** mountable_snapshots
#** share-network-multiple-subnets
#** share-retype
#** multiple_protocol_share
#** access-groups
#** backup_share
#** manila-share-groups
#** data-service-api
#* vponomaryov's prioritiesː https://docs.google.com/document/d/1zdMNuQTIqrtDtzbdcxyZVn8SoPN9pbtv_HSLVR3Eld0/edit

'''Scheduled for 20 Oct, 2016, 15:00 UTC'''
# Specs and Deadlines (bswartz)
#* https://review.openstack.org/#/c/374883/
# Adding invalid user to the access rule changes valid access rule state from 'active' to 'error' (ravichandran)
# Open Discussion

'''Scheduled for 13 Oct, 2016, 15:00 UTC'''
# Ocata Design Summit
#* https://www.openstack.org/summit/barcelona-2016/summit-schedule/global-search?t=Manila%3A
# PTG

'''Scheduled for 6 Oct, 2016, 15:00 UTC'''
# Microversion-ing addition or removal of transitional ('micro') states (gouthamr)
#* https://review.openstack.org/#/c/369668/

'''Scheduled for 29 Sep, 2016, 15:00 UTC'''
# Ocata Design Summit
#* https://etherpad.openstack.org/p/manila-ocata-design-summit-topics
# Open Discussion

'''Scheduled for 22 Sep, 2016, 15:00 UTC'''
# Specs process
#* https://review.openstack.org/#/c/374883

'''Scheduled for 8 Sep, 2016, 15:00 UTC'''
# RC1 status
#* https://launchpad.net/manila/+milestone/newton-rc1
# Project Logo
#* http://civs.cs.cornell.edu/cgi-bin/vote.pl?id=E_6f0d111cec78c5ef&akey=d34a751f2d084d79
# Container Driver Security (aovchinnikov)
#* https://review.openstack.org/#/c/353463/
# Security "vulnerability-managed" tag for manila (tbarron)
# Zanata translations for manila-client and manila-ui (tbarron)
# Tempest direction (tbarron)

'''Scheduled for 1 Sep, 2016, 15:00 UTC'''
# Mascot (bswartz)
# Vulnerability Management

'''Scheduled for 25 Aug, 2016, 15:00 UTC'''
# Huawei share replication and CI
# IPv6 (cknight)
# Open Discussion

'''Scheduled for 18 Aug, 2016, 15:00 UTC'''
# Driver private data admin API
#* Disagreement abounds on how this API is to be designed, tested, and presented.
#* https://review.openstack.org/#/c/315346/
# Scenario tests are broken
#* http://logs.openstack.org/86/309286/34/check/gate-manila-tempest-dsvm-scenario/610f96e/console.html
# Revert to snapshot
#* https://review.openstack.org/#/c/340502/
#* https://review.openstack.org/#/c/356682/

"Scheduled for 11 Aug, 2016, 15:00 UTC"
# None

'''Scheduled for 4 Aug, 2016, 15:00 UTC'''
# Migration APIs
#* http://lists.openstack.org/pipermail/openstack-dev/2016-August/100682.html
# Correct approach for avoiding races

'''Scheduled for 28 Jul, 2016, 15:00 UTC'''
# Stepping Down as Docs Liaison (dustins)
#* Current job responsibilities don't give me the time to devote to this that I'd like to have, so it should go to someone else with a passion for documentation and has the willingness and desire for taking up the mantle.
# N-3 Review focus
#* https://etherpad.openstack.org/p/manila-newton3-priorities

'''Scheduled for 21 Jul, 2016, 15:00 UTC'''
# Migrate API reference into tree (dgonzalez)
#* https://review.openstack.org/#/c/313874/

'''Scheduled for 14 Jul, 2016, 15:00 UTC'''
# N-2 milestone
# Managing initial security (of the root directory) for new shares
# Mascot/Logo discussion
# Tempest alternatives

'''Scheduled for 30 Jun, 2016, 15:00 UTC'''
# non-disruptive nfs-ganesha export modifications (rraja/tbarron)
#* https://sourceforge.net/p/nfs-ganesha/mailman/message/35173839
# share backup spec discussion (zhongjun)
#* https://review.openstack.org/#/c/330306

'''Scheduled for 23 Jun, 2016, 15:00 UTC'''
# Implement extend and shrink features for glusterfs driver (DaiDV)
# Do all drivers have to inherit from driver.ShareDriver? (mkoderer)
# Creating a second share instance for driver-assisted migration (ganso)

'''Scheduled for 16 Jun, 2016, 15:00 UTC'''
# A share support multiple protocols (zhongjun)
#* https://review.openstack.org/#/c/329392/
# Drivers with minimum size for shares (tpsilva)

'''Scheduled for 9 Jun, 2016, 15:00 UTC'''
# HPB feature (mkoderer)
# the fate of nova-network support in manila (gouthamr)
# multi-AZ tests in the gate

'''Scheduled for 2 Jun, 2016, 15:00 UTC'''
# N-1 milestone (bswartz)
# Decision on APIv1 removal for newton (bswartz)
# Model updates from drivers (bswartz)
# Midcycle meetup
# Spec update
#* New spec - Extending share networks to span subnets - https://review.openstack.org/#/c/323646/ (gouthamr)

'''Scheduled for 26 May, 2016, 15:00 UTC'''
# N-1 milestone
# Update access remaining work
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* Nova-network removal
#* APIv1 removal
#* Model updates from the driver

'''Scheduled for 19 May, 2016, 15:00 UTC'''
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* UI Customization
# Share Backup(zhongjun)
# Snapshot Restore (cknight)
# Specs (ganso)

'''Scheduled for 12 May, 2016, 15:00 UTC'''
# Driver private storage: https://review.openstack.org/#/c/315346/ (xyang)
#* Admin wants to check driver private storage before doing some actions on a resource, i.e., force delete a share or unmanage a share.
#* The patch proposes to add admin API to retrieve driver private storage data. Alternative approach proposed by Valeriy to use manila-manage (i.e., manila-manage driver-private-storage <uuid of a resource>). Which one should we choose?
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* Py3 functional tests
#* Replication with share networks
#* Service-check
#* Container driver

'''Scheduled for 5 May, 2016, 15:00 UTC'''
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* Documentation update (dustins/gouthamr)
#* User messages
#* Experimental features in UI
#* Share/snapshot size mismatch
# BP Specs (tbarron)
#* Need to ensure that the repo doesn't become "a wasteland of partial ideas where hope goes to die." (See http://lists.openstack.org/pipermail/openstack-dev/2016-May/094026.html)

'''Scheduled for 14 Apr, 2016, 15:00 UTC'''
# Manage API retrying (gouthamr/nidhimittalhada)
# Tempest test structuring - resource sharing and cleanup issues (akerr)
# Snapshot support common capability in the documentation (gouthamr)

'''Scheduled for 7 Apr, 2016, 15:00 UTC'''
# Access rules (bswartz)
# Design summit planning (bswartz)
#* Link: https://etherpad.openstack.org/p/manila-newton-summit-topics
# Release notes, continued (gouthamr)
#* Link: https://review.openstack.org/#/c/300656/
# Midcycle meetup
# Access Groups
#* https://wiki.openstack.org/wiki/Manila/design/access_groups
# Revert to snapshot

'''Scheduled for 31 Mar, 2016, 15:00 UTC'''
# Mitaka release status
#* RC1 expected to be the Mitaka release build
#* Last day for a re-spin is today
#** http://lists.openstack.org/pipermail/openstack-dev/2016-March/090877.html
# Design summit planning
#* Final schedule confirms 2 fishbowls, 4 working sessions, 1 half-day meetup
#** http://lists.openstack.org/pipermail/openstack-dev/2016-March/091032.html
#* Voting open now for summit topics
#** https://etherpad.openstack.org/p/manila-newton-summit-topics
# Manila Documentation (gouthamr)
#* Where do our docs live? how do we maintain them? (In Tree: devref, adminref | out of tree: install-guide, config-ref, cloud-admin guide)
#* Need a docs liaison
#** https://wiki.openstack.org/wiki/CrossProjectLiaisons#Documentation
#* Release Notes
#** http://docs.openstack.org/developer/reno/usage.html
# Managing/unmanaging replicated shares and snapshots (gouthamr)

'''Scheduled for 24 Mar, 2016, 15:00 UTC'''
# Design summit planning
#* https://etherpad.openstack.org/p/manila-newton-summit-topics
# Question about share servers (Jay.Xu)
#* Is there a way to get gateway IP address when the API setup_server() is invoked. I did not see this attribute in the input parameter network_info. Gateway IP is mandatory to create the interface for EMC storage which will be release coming soon, so it is expected that we can add the gateway IP into NetworkAllocation model if not.

'''Scheduled for 10 Mar, 2016, 15:00 UTC'''
# Common Capabilities Matrix (dustins)
#* https://review.openstack.org/#/c/288504/
#* I'm adding a table in the Developer Reference documentation to show which common capabilities are supported by which backends and when that support started.
#* I've got most drivers covered now.
#* If you want me to add your capabilities, leave a review on the patch and I'll update the table!

'''Scheduled for 25 Feb, 2016, 15:00 UTC'''

# OpenStack bug smash event (toabctl, mkoderer)
## link: https://etherpad.openstack.org/p/OpenStack-Bug-Smash-Mitaka
# Status of new drivers
## M.3 deadline is a week away.
## Only a few things have merged in the last couple days. At this rate, many things won't make it.
## Zuul is busy and will only get worse until the deadline.
## Multiple Manila gate breakages have occurred recently, and more are likely.
## The Manila tests are very unreliable at concurrency > 1.
## The generic driver is too slow to pass the gate at concurrency = 1.
## The new drivers *are* fast enough to run at concurrency = 1.
## The new drivers may not be 100% done, but they are done enough to pass reliably at concurrency = 1.
## We should switch voting to the new drivers NOW, or start throwing stuff overboard for the release.
# Next Manila mid-cycle location
## Proposal is to do it in Germany
## Need to know who could attend at that location.
## Would it make a difference if the Cinder mid-cycle happens in Dublin, and the Manila meeting is immediately before or after the Cinder one?

'''Scheduled for 18 Feb, 2016, 15:00 UTC'''

# Feature Proposal Freeze (bswartz)
# Open Discussion

'''Scheduled for 11 Feb, 2016, 15:00 UTC'''

# Hierarchical port binding support (mkoderer)
#* https://wiki.openstack.org/wiki/Manila/design/manila-newton-hpb-support
#* https://blueprints.launchpad.net/manila/+spec/manila-hpb-support
#* https://review.openstack.org/#/c/277731/
# Next Manila mid-cycle (mkoderer)
#* https://etherpad.openstack.org/p/manila-n-release-midcycle
# Replication + ZFS Driver (bswartz)
#* https://review.openstack.org/#/c/238572
# Driver FPF (bswartz)
#* https://review.openstack.org/#/c/279090/
#* https://review.openstack.org/#/c/278169/
# HDFS CI (bswartz)
'''Scheduled for 4 Feb, 2016, 15:00 UTC'''
# Data service handling protocol mount commands (ganso)
#* https://etherpad.openstack.org/p/manila-data-service-mount-meeting
# Separate process for copy on data service (tpsilva)
# ZFS driver (bswartz)
# First party drivers share migration support cooperation (ganso)

'''Scheduled for 28 Jan, 2016, 15:00 UTC'''
# Hierarchical port binding support (mkoderer)
#* https://etherpad.openstack.org/p/manila-hierarchical-port-binding
# Core reviewer team and other manila-related projects (bswartz)
# Replication status (bswartz)

'''Scheduled for 21 Jan, 2016, 15:00 UTC'''
# Update_access backwards compatibility approach discussion. (ganso)
# Exposing the Data Service to drivers. (ganso)
# Snapshot manage/unmanage (bswartz)

'''Scheduled for 17 Dec, 2015, 15:00 UTC'''
# Proposal for share groups (cknight)
# Liberty.1 stable release (bswartz)

'''Scheduled for 10 Dec, 2015, 15:00 UTC'''
# The most of the driver already report the "'QoS_support': False" to scheduler. Does we need to remove it or save it as Qos support flat? and write 'QoS_support' to capabilities_and_extra_specs.rst?(zhongjun)
##https://github.com/openstack/manila/blob/master/manila/share/drivers/hpe/hpe_3par_driver.py#L391
##https://github.com/openstack/manila/blob/master/manila/share/drivers/hitachi/hds_hnas.py#L354
##https://github.com/openstack/manila/blob/master/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py#L257
##https://github.com/openstack/manila/blob/master/manila/scheduler/host_manager.py#L111
##https://review.openstack.org/#/c/247286/4/doc/source/devref/capabilities_and_extra_specs.rst
# Architectural concerns and interoperability of new features (cknight)

'''Scheduled for 3 Dec, 2015, 15:00 UTC'''
# Midcycle meetup
# Add "data loss" flag in DB migrations which lead to losing of existing data and update "manila-manage db sync" command to forbid applying such migrations without "--force" flag (u_glide)
# Reorganization of Manila scheduler to merge oslo-incubator code and simplify future maintenance (cknight)
## https://review.openstack.org/#/c/252060/
## Mostly for awareness, this is now merged and should be backwards compatible.
## Cinder could benefit from a similar treatment. And with the incubator gone, we could explore a common library with the Cinder team, since the heretofore shared components are certain to diverge over time, but I'm not sure how receptive they would be. Xing?
# CI reliability
## Why should we merge patches for drivers whose CI systems consistently fail? This negates the value of CI. I think that accepting patches that aren't proven by CI does a disservice to everyone, most of all to the driver owners themselves. Refusing to merge driver patches without a good CI result for that driver seems like a gentler and more uniform-over-time approach than threatening to throw out a driver near the end of a release. And why make our illustrious and overworked PTL be the CI policeman when all core reviewers could simply glance at the corresponding CI result before giving +A? (cknight)
## http://ci-watch.tintri.com/project?project=manila
## https://review.openstack.org/#/c/232293/
## https://review.openstack.org/#/c/235522/
## https://review.openstack.org/#/c/235726/
## https://review.openstack.org/#/c/249999/
## https://review.openstack.org/#/c/237454/

'''Scheduled for 19 Nov, 2015, 15:00 UTC'''
# GB --> GiB (markstur)
# OpenStack Manuals and Devref (markstur)
#* Config Ref (and others) are moving to .RST!
#* How should we document prior releases in devref?
# Midcycle Meetup Date (bswartz)
# Deadlines for features (bswartz)
#* Feature proposal freeze vs Big Features
#* New drivers
# Data Copy Service network approach discussion (ganso)
#* https://drive.google.com/file/d/0B30HPZmaePCHMlAzcjJlYm5rRTA/view?usp=sharing

'''Scheduled for 12 Nov, 2015, 15:00 UTC'''
# Which method is better for Manila QoS(independent way or just use extra specs ) (zhongjun)
#* link:http://paste.openstack.org/show/477677/
#*If use extra specs to config QoS, Does we first need add “QoS_support=true/false” in share_type. Does we need to add QoS_support flag in common capabilities
#**https://github.com/openstack/manila/blob/master/doc/source/devref/capabilities_and_extra_specs.rst)?
#** eg: share_type: qos_support='<is> True' or '<is> False'
#** netapp:max_iops=100 or qos:netapp:max_iops=100 or qos:max_iops=100
#** driver update: qos_support=true
#** An alternative could be to have the driver report a range
# Manila DR update (ameade)
#* Still no reviews on https://review.openstack.org/#/c/238572/
#* First-party driver implementation? Which driver, current generic or new solution? When do we need this implemented and who wants to do it?
#* Replication_change share state - This is the state of the share when any of it's replicas are being promoted. Is 'replication_change' good or is there a better phrase?
# Manila Driver minimum requirements document update (ganso)
# Manila Data Copy service name (bswartz)
#*https://review.openstack.org/#/c/244286/
# Upcoming change for Manila CI hooks (vponomaryov)
#*https://review.openstack.org/#/c/243233/
# Manilaclient enhancement to provide request_id when set http_log_debug is True. It will co-ordinate the request and its id, so it will be helpful to filter the log and the failed request. (Jay Xu)
#*http://paste.openstack.org/show/478675/

'''Scheduled for 5 Nov, 2015, 15:00 UTC'''
# Decide on read-only access rules as a required feature
# Manila DR - Design/Code feedback (gouthamr)
#* [https://wiki.openstack.org/wiki/Manila/design/manila-mitaka-data-replication Design Document]
#* [https://review.openstack.org/#/c/238572/ Core Implementation on Gerrit]
#* [https://review.openstack.org/#/c/235448/ Client Implementation on Gerrit]

'''Scheduled for 1 Oct, 2015, 15:00 UTC'''
# Liberty RC2 status
#* https://launchpad.net/manila/+milestone/liberty-rc2
#* https://review.openstack.org/#/q/project:openstack/manila+branch:stable/liberty,n,z
# Mitaka Design Summit
#* https://etherpad.openstack.org/p/manila-mitaka-summit-topics
# Vendor driver docs
#* https://github.com/openstack/manila/blob/master/doc/source/devref/share_back_ends_feature_support_mapping.rst
# Manila Client Nova Style Microversion Support (cfouts)
#* https://review.openstack.org/#/c/229142/
#* http://docs.openstack.org/developer/manila/devref/api_microversion_dev.html

'''Scheduled for 24 Sep, 2015, 15:00 UTC'''
# 3rd-party CI status
#* http://ec2-54-67-102-119.us-west-1.compute.amazonaws.com:5000/?project=openstack/manila&user=&timeframe=72&start=&end=&page_size=500
#* http://ci-watch.tintri.com/project?project=manila
# Liberty RC status
# Support for QoS in Manila (zhongjun2)
#* https://blueprints.launchpad.net/manila/+spec/manila-support-qos
#* https://wiki.openstack.org/wiki/Manila/QoS
# Reminder for backend driver maintainers to update feature support table (toabctl)
#* http://docs.openstack.org/developer/manila/devref/share_back_ends_feature_support_mapping.html
# Regression when using NetApp - license issue with netapp_lib (toabctl)
#* https://bugs.launchpad.net/cinder/+bug/1499334
# Mitaka Design Summit
#* Propose topics - https://etherpad.openstack.org/p/manila-mitaka-summit-topics
# Vendor driver docs
# Open Discussion

Manila/Meetings

2016-11-03T13:52:09Z

Vponomaryov: /* Next meeting */

= Weekly Manila team meeting =
'''NOTE MEETING TIME: Thursday at 15:00 UTC'''

If you're interested in management of shared filesystems for OpenStack, we have a weekly meetings in <code><nowiki>#openstack-meeting-alt</nowiki></code>, on Thursdays at 15:00 UTC. Please feel free to add items to the agenda below. NOTE: When adding topics please include your IRC name so we know who's topic it is and how to get more info.

'''Previous meeting logs can be found here:''' http://eavesdrop.openstack.org/meetings/manila/

== Next meeting ==
'''NOTE:''' ''Include your IRC nickname next to agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. You might want to put this on your calendar if you are adding items.''

'''Scheduled for 3 Nov, 2016, 15:00 UTC'''
# Spec review & prioritization
#* https://review.openstack.org/#/q/project:%22%255Eopenstack/manila-specs%22++status:open
#** vponomaryov's prioritiesː https://docs.google.com/document/d/1zdMNuQTIqrtDtzbdcxyZVn8SoPN9pbtv_HSLVR3Eld0/edit
# Open Discussion

== Previous meetings ==

'''Scheduled for 20 Oct, 2016, 15:00 UTC'''
# Specs and Deadlines (bswartz)
#* https://review.openstack.org/#/c/374883/
# Adding invalid user to the access rule changes valid access rule state from 'active' to 'error' (ravichandran)
# Open Discussion

'''Scheduled for 13 Oct, 2016, 15:00 UTC'''
# Ocata Design Summit
#* https://www.openstack.org/summit/barcelona-2016/summit-schedule/global-search?t=Manila%3A
# PTG

'''Scheduled for 6 Oct, 2016, 15:00 UTC'''
# Microversion-ing addition or removal of transitional ('micro') states (gouthamr)
#* https://review.openstack.org/#/c/369668/

'''Scheduled for 29 Sep, 2016, 15:00 UTC'''
# Ocata Design Summit
#* https://etherpad.openstack.org/p/manila-ocata-design-summit-topics
# Open Discussion

'''Scheduled for 22 Sep, 2016, 15:00 UTC'''
# Specs process
#* https://review.openstack.org/#/c/374883

'''Scheduled for 8 Sep, 2016, 15:00 UTC'''
# RC1 status
#* https://launchpad.net/manila/+milestone/newton-rc1
# Project Logo
#* http://civs.cs.cornell.edu/cgi-bin/vote.pl?id=E_6f0d111cec78c5ef&akey=d34a751f2d084d79
# Container Driver Security (aovchinnikov)
#* https://review.openstack.org/#/c/353463/
# Security "vulnerability-managed" tag for manila (tbarron)
# Zanata translations for manila-client and manila-ui (tbarron)
# Tempest direction (tbarron)

'''Scheduled for 1 Sep, 2016, 15:00 UTC'''
# Mascot (bswartz)
# Vulnerability Management

'''Scheduled for 25 Aug, 2016, 15:00 UTC'''
# Huawei share replication and CI
# IPv6 (cknight)
# Open Discussion

'''Scheduled for 18 Aug, 2016, 15:00 UTC'''
# Driver private data admin API
#* Disagreement abounds on how this API is to be designed, tested, and presented.
#* https://review.openstack.org/#/c/315346/
# Scenario tests are broken
#* http://logs.openstack.org/86/309286/34/check/gate-manila-tempest-dsvm-scenario/610f96e/console.html
# Revert to snapshot
#* https://review.openstack.org/#/c/340502/
#* https://review.openstack.org/#/c/356682/

"Scheduled for 11 Aug, 2016, 15:00 UTC"
# None

'''Scheduled for 4 Aug, 2016, 15:00 UTC'''
# Migration APIs
#* http://lists.openstack.org/pipermail/openstack-dev/2016-August/100682.html
# Correct approach for avoiding races

'''Scheduled for 28 Jul, 2016, 15:00 UTC'''
# Stepping Down as Docs Liaison (dustins)
#* Current job responsibilities don't give me the time to devote to this that I'd like to have, so it should go to someone else with a passion for documentation and has the willingness and desire for taking up the mantle.
# N-3 Review focus
#* https://etherpad.openstack.org/p/manila-newton3-priorities

'''Scheduled for 21 Jul, 2016, 15:00 UTC'''
# Migrate API reference into tree (dgonzalez)
#* https://review.openstack.org/#/c/313874/

'''Scheduled for 14 Jul, 2016, 15:00 UTC'''
# N-2 milestone
# Managing initial security (of the root directory) for new shares
# Mascot/Logo discussion
# Tempest alternatives

'''Scheduled for 30 Jun, 2016, 15:00 UTC'''
# non-disruptive nfs-ganesha export modifications (rraja/tbarron)
#* https://sourceforge.net/p/nfs-ganesha/mailman/message/35173839
# share backup spec discussion (zhongjun)
#* https://review.openstack.org/#/c/330306

'''Scheduled for 23 Jun, 2016, 15:00 UTC'''
# Implement extend and shrink features for glusterfs driver (DaiDV)
# Do all drivers have to inherit from driver.ShareDriver? (mkoderer)
# Creating a second share instance for driver-assisted migration (ganso)

'''Scheduled for 16 Jun, 2016, 15:00 UTC'''
# A share support multiple protocols (zhongjun)
#* https://review.openstack.org/#/c/329392/
# Drivers with minimum size for shares (tpsilva)

'''Scheduled for 9 Jun, 2016, 15:00 UTC'''
# HPB feature (mkoderer)
# the fate of nova-network support in manila (gouthamr)
# multi-AZ tests in the gate

'''Scheduled for 2 Jun, 2016, 15:00 UTC'''
# N-1 milestone (bswartz)
# Decision on APIv1 removal for newton (bswartz)
# Model updates from drivers (bswartz)
# Midcycle meetup
# Spec update
#* New spec - Extending share networks to span subnets - https://review.openstack.org/#/c/323646/ (gouthamr)

'''Scheduled for 26 May, 2016, 15:00 UTC'''
# N-1 milestone
# Update access remaining work
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* Nova-network removal
#* APIv1 removal
#* Model updates from the driver

'''Scheduled for 19 May, 2016, 15:00 UTC'''
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* UI Customization
# Share Backup(zhongjun)
# Snapshot Restore (cknight)
# Specs (ganso)

'''Scheduled for 12 May, 2016, 15:00 UTC'''
# Driver private storage: https://review.openstack.org/#/c/315346/ (xyang)
#* Admin wants to check driver private storage before doing some actions on a resource, i.e., force delete a share or unmanage a share.
#* The patch proposes to add admin API to retrieve driver private storage data. Alternative approach proposed by Valeriy to use manila-manage (i.e., manila-manage driver-private-storage <uuid of a resource>). Which one should we choose?
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* Py3 functional tests
#* Replication with share networks
#* Service-check
#* Container driver

'''Scheduled for 5 May, 2016, 15:00 UTC'''
# Topics not covered at design summit https://etherpad.openstack.org/p/newton-manila-contributor-meetup
#* Documentation update (dustins/gouthamr)
#* User messages
#* Experimental features in UI
#* Share/snapshot size mismatch
# BP Specs (tbarron)
#* Need to ensure that the repo doesn't become "a wasteland of partial ideas where hope goes to die." (See http://lists.openstack.org/pipermail/openstack-dev/2016-May/094026.html)

'''Scheduled for 14 Apr, 2016, 15:00 UTC'''
# Manage API retrying (gouthamr/nidhimittalhada)
# Tempest test structuring - resource sharing and cleanup issues (akerr)
# Snapshot support common capability in the documentation (gouthamr)

'''Scheduled for 7 Apr, 2016, 15:00 UTC'''
# Access rules (bswartz)
# Design summit planning (bswartz)
#* Link: https://etherpad.openstack.org/p/manila-newton-summit-topics
# Release notes, continued (gouthamr)
#* Link: https://review.openstack.org/#/c/300656/
# Midcycle meetup
# Access Groups
#* https://wiki.openstack.org/wiki/Manila/design/access_groups
# Revert to snapshot

'''Scheduled for 31 Mar, 2016, 15:00 UTC'''
# Mitaka release status
#* RC1 expected to be the Mitaka release build
#* Last day for a re-spin is today
#** http://lists.openstack.org/pipermail/openstack-dev/2016-March/090877.html
# Design summit planning
#* Final schedule confirms 2 fishbowls, 4 working sessions, 1 half-day meetup
#** http://lists.openstack.org/pipermail/openstack-dev/2016-March/091032.html
#* Voting open now for summit topics
#** https://etherpad.openstack.org/p/manila-newton-summit-topics
# Manila Documentation (gouthamr)
#* Where do our docs live? how do we maintain them? (In Tree: devref, adminref | out of tree: install-guide, config-ref, cloud-admin guide)
#* Need a docs liaison
#** https://wiki.openstack.org/wiki/CrossProjectLiaisons#Documentation
#* Release Notes
#** http://docs.openstack.org/developer/reno/usage.html
# Managing/unmanaging replicated shares and snapshots (gouthamr)

'''Scheduled for 24 Mar, 2016, 15:00 UTC'''
# Design summit planning
#* https://etherpad.openstack.org/p/manila-newton-summit-topics
# Question about share servers (Jay.Xu)
#* Is there a way to get gateway IP address when the API setup_server() is invoked. I did not see this attribute in the input parameter network_info. Gateway IP is mandatory to create the interface for EMC storage which will be release coming soon, so it is expected that we can add the gateway IP into NetworkAllocation model if not.

'''Scheduled for 10 Mar, 2016, 15:00 UTC'''
# Common Capabilities Matrix (dustins)
#* https://review.openstack.org/#/c/288504/
#* I'm adding a table in the Developer Reference documentation to show which common capabilities are supported by which backends and when that support started.
#* I've got most drivers covered now.
#* If you want me to add your capabilities, leave a review on the patch and I'll update the table!

'''Scheduled for 25 Feb, 2016, 15:00 UTC'''

# OpenStack bug smash event (toabctl, mkoderer)
## link: https://etherpad.openstack.org/p/OpenStack-Bug-Smash-Mitaka
# Status of new drivers
## M.3 deadline is a week away.
## Only a few things have merged in the last couple days. At this rate, many things won't make it.
## Zuul is busy and will only get worse until the deadline.
## Multiple Manila gate breakages have occurred recently, and more are likely.
## The Manila tests are very unreliable at concurrency > 1.
## The generic driver is too slow to pass the gate at concurrency = 1.
## The new drivers *are* fast enough to run at concurrency = 1.
## The new drivers may not be 100% done, but they are done enough to pass reliably at concurrency = 1.
## We should switch voting to the new drivers NOW, or start throwing stuff overboard for the release.
# Next Manila mid-cycle location
## Proposal is to do it in Germany
## Need to know who could attend at that location.
## Would it make a difference if the Cinder mid-cycle happens in Dublin, and the Manila meeting is immediately before or after the Cinder one?

'''Scheduled for 18 Feb, 2016, 15:00 UTC'''

# Feature Proposal Freeze (bswartz)
# Open Discussion

'''Scheduled for 11 Feb, 2016, 15:00 UTC'''

# Hierarchical port binding support (mkoderer)
#* https://wiki.openstack.org/wiki/Manila/design/manila-newton-hpb-support
#* https://blueprints.launchpad.net/manila/+spec/manila-hpb-support
#* https://review.openstack.org/#/c/277731/
# Next Manila mid-cycle (mkoderer)
#* https://etherpad.openstack.org/p/manila-n-release-midcycle
# Replication + ZFS Driver (bswartz)
#* https://review.openstack.org/#/c/238572
# Driver FPF (bswartz)
#* https://review.openstack.org/#/c/279090/
#* https://review.openstack.org/#/c/278169/
# HDFS CI (bswartz)
'''Scheduled for 4 Feb, 2016, 15:00 UTC'''
# Data service handling protocol mount commands (ganso)
#* https://etherpad.openstack.org/p/manila-data-service-mount-meeting
# Separate process for copy on data service (tpsilva)
# ZFS driver (bswartz)
# First party drivers share migration support cooperation (ganso)

'''Scheduled for 28 Jan, 2016, 15:00 UTC'''
# Hierarchical port binding support (mkoderer)
#* https://etherpad.openstack.org/p/manila-hierarchical-port-binding
# Core reviewer team and other manila-related projects (bswartz)
# Replication status (bswartz)

'''Scheduled for 21 Jan, 2016, 15:00 UTC'''
# Update_access backwards compatibility approach discussion. (ganso)
# Exposing the Data Service to drivers. (ganso)
# Snapshot manage/unmanage (bswartz)

'''Scheduled for 17 Dec, 2015, 15:00 UTC'''
# Proposal for share groups (cknight)
# Liberty.1 stable release (bswartz)

'''Scheduled for 10 Dec, 2015, 15:00 UTC'''
# The most of the driver already report the "'QoS_support': False" to scheduler. Does we need to remove it or save it as Qos support flat? and write 'QoS_support' to capabilities_and_extra_specs.rst?(zhongjun)
##https://github.com/openstack/manila/blob/master/manila/share/drivers/hpe/hpe_3par_driver.py#L391
##https://github.com/openstack/manila/blob/master/manila/share/drivers/hitachi/hds_hnas.py#L354
##https://github.com/openstack/manila/blob/master/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py#L257
##https://github.com/openstack/manila/blob/master/manila/scheduler/host_manager.py#L111
##https://review.openstack.org/#/c/247286/4/doc/source/devref/capabilities_and_extra_specs.rst
# Architectural concerns and interoperability of new features (cknight)

'''Scheduled for 3 Dec, 2015, 15:00 UTC'''
# Midcycle meetup
# Add "data loss" flag in DB migrations which lead to losing of existing data and update "manila-manage db sync" command to forbid applying such migrations without "--force" flag (u_glide)
# Reorganization of Manila scheduler to merge oslo-incubator code and simplify future maintenance (cknight)
## https://review.openstack.org/#/c/252060/
## Mostly for awareness, this is now merged and should be backwards compatible.
## Cinder could benefit from a similar treatment. And with the incubator gone, we could explore a common library with the Cinder team, since the heretofore shared components are certain to diverge over time, but I'm not sure how receptive they would be. Xing?
# CI reliability
## Why should we merge patches for drivers whose CI systems consistently fail? This negates the value of CI. I think that accepting patches that aren't proven by CI does a disservice to everyone, most of all to the driver owners themselves. Refusing to merge driver patches without a good CI result for that driver seems like a gentler and more uniform-over-time approach than threatening to throw out a driver near the end of a release. And why make our illustrious and overworked PTL be the CI policeman when all core reviewers could simply glance at the corresponding CI result before giving +A? (cknight)
## http://ci-watch.tintri.com/project?project=manila
## https://review.openstack.org/#/c/232293/
## https://review.openstack.org/#/c/235522/
## https://review.openstack.org/#/c/235726/
## https://review.openstack.org/#/c/249999/
## https://review.openstack.org/#/c/237454/

'''Scheduled for 19 Nov, 2015, 15:00 UTC'''
# GB --> GiB (markstur)
# OpenStack Manuals and Devref (markstur)
#* Config Ref (and others) are moving to .RST!
#* How should we document prior releases in devref?
# Midcycle Meetup Date (bswartz)
# Deadlines for features (bswartz)
#* Feature proposal freeze vs Big Features
#* New drivers
# Data Copy Service network approach discussion (ganso)
#* https://drive.google.com/file/d/0B30HPZmaePCHMlAzcjJlYm5rRTA/view?usp=sharing

'''Scheduled for 12 Nov, 2015, 15:00 UTC'''
# Which method is better for Manila QoS(independent way or just use extra specs ) (zhongjun)
#* link:http://paste.openstack.org/show/477677/
#*If use extra specs to config QoS, Does we first need add “QoS_support=true/false” in share_type. Does we need to add QoS_support flag in common capabilities
#**https://github.com/openstack/manila/blob/master/doc/source/devref/capabilities_and_extra_specs.rst)?
#** eg: share_type: qos_support='<is> True' or '<is> False'
#** netapp:max_iops=100 or qos:netapp:max_iops=100 or qos:max_iops=100
#** driver update: qos_support=true
#** An alternative could be to have the driver report a range
# Manila DR update (ameade)
#* Still no reviews on https://review.openstack.org/#/c/238572/
#* First-party driver implementation? Which driver, current generic or new solution? When do we need this implemented and who wants to do it?
#* Replication_change share state - This is the state of the share when any of it's replicas are being promoted. Is 'replication_change' good or is there a better phrase?
# Manila Driver minimum requirements document update (ganso)
# Manila Data Copy service name (bswartz)
#*https://review.openstack.org/#/c/244286/
# Upcoming change for Manila CI hooks (vponomaryov)
#*https://review.openstack.org/#/c/243233/
# Manilaclient enhancement to provide request_id when set http_log_debug is True. It will co-ordinate the request and its id, so it will be helpful to filter the log and the failed request. (Jay Xu)
#*http://paste.openstack.org/show/478675/

'''Scheduled for 5 Nov, 2015, 15:00 UTC'''
# Decide on read-only access rules as a required feature
# Manila DR - Design/Code feedback (gouthamr)
#* [https://wiki.openstack.org/wiki/Manila/design/manila-mitaka-data-replication Design Document]
#* [https://review.openstack.org/#/c/238572/ Core Implementation on Gerrit]
#* [https://review.openstack.org/#/c/235448/ Client Implementation on Gerrit]

'''Scheduled for 1 Oct, 2015, 15:00 UTC'''
# Liberty RC2 status
#* https://launchpad.net/manila/+milestone/liberty-rc2
#* https://review.openstack.org/#/q/project:openstack/manila+branch:stable/liberty,n,z
# Mitaka Design Summit
#* https://etherpad.openstack.org/p/manila-mitaka-summit-topics
# Vendor driver docs
#* https://github.com/openstack/manila/blob/master/doc/source/devref/share_back_ends_feature_support_mapping.rst
# Manila Client Nova Style Microversion Support (cfouts)
#* https://review.openstack.org/#/c/229142/
#* http://docs.openstack.org/developer/manila/devref/api_microversion_dev.html

'''Scheduled for 24 Sep, 2015, 15:00 UTC'''
# 3rd-party CI status
#* http://ec2-54-67-102-119.us-west-1.compute.amazonaws.com:5000/?project=openstack/manila&user=&timeframe=72&start=&end=&page_size=500
#* http://ci-watch.tintri.com/project?project=manila
# Liberty RC status
# Support for QoS in Manila (zhongjun2)
#* https://blueprints.launchpad.net/manila/+spec/manila-support-qos
#* https://wiki.openstack.org/wiki/Manila/QoS
# Reminder for backend driver maintainers to update feature support table (toabctl)
#* http://docs.openstack.org/developer/manila/devref/share_back_ends_feature_support_mapping.html
# Regression when using NetApp - license issue with netapp_lib (toabctl)
#* https://bugs.launchpad.net/cinder/+bug/1499334
# Mitaka Design Summit
#* Propose topics - https://etherpad.openstack.org/p/manila-mitaka-summit-topics
# Vendor driver docs
# Open Discussion

Meetings/QATeamMeeting

2016-07-28T06:58:14Z

Vponomaryov: /* Agenda for July 28th 2016 (0900 UTC) */

= Weekly QA Team meeting =
The OpenStack QA Team holds public weekly meetings in <code><nowiki>#openstack-meeting</nowiki></code>, '''Thursday alternating every week between 17:00 UTC and 09:00 UTC''. 
Everyone interested in testing, quality assurance, performance engineering, etc, should attend!

The meeting minutes of can be found here: [http://eavesdrop.openstack.org/meetings/qa meeting minutes on eavesdrop]

''Rules for adding to the agenda'':
* anyone is free to add to proposed agenda, please do so by 1 hr before the meeting
* if you add an item to the agenda, please include your ircnick (i.e. sdague) to the end of your item, so we know who did, and who should lead the discussion

=== Agenda for July 28th 2016 (0900 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
* DevStack + Grenade
* openstack-health
* Critical Reviews
* Possibility to enable/disable external plugins [vponomaryov]
** https://review.openstack.org/#/c/343830/
* Open Discussion

== Previous meetings ==
All previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/qa/.
Old agendas are stored on [[Meetings/QATeamMeeting/OldAgendas]]

===Agenda for July 21th 2016 (1700 UTC)===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities

* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
* DevStack + Grenade
* openstack-health
* Critical Reviews
** https://review.openstack.org/#/c/341083/
* Open Discussion

=== Agenda for July 7th 2016 (1700 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
* DevStack + Grenade
* openstack-health
* Critical Reviews
* Open Discussion

=== Agenda for June 30th 2016 (0900 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
* DevStack + Grenade
* openstack-health
* Critical Reviews
* Open Discussion

=== Agenda for June 23th 2016 (1700 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
*Specs Reviews
**https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
*Tempest
*DevStack + Grenade
*openstack-health
*Critical Reviews
*Open Discussion

=== Agenda for June 16th 2016 (0900 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
** Remote client debuggability (andreaf)
** Test class hierarchy (dmellado)
** Neutron OO wrappers (jlwhite, hockeynut)
** Documentation cleanup (andreaf) - All is done?
** Tempest CLI (mtreinish, dpaterson, dmellado)
** Negative test guideline (hogepodge, oomichi)
** tempest-lib/plugin (gmann)
** microversions testing (gmann)
* DevStack + Grenade
** Neutron Cleanup (sc68cal)
** Heat devstack plugin (dixiaoli, oomichi)
* openstack-health (mtreinish, masayukig)
* Critical Reviews
* QA mid-cycle meetup
* Open Discussion

=== Agenda for June 9th 2016 (1700 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
* DevStack + Grenade
* openstack-health
* Critical Reviews
* Open Discussion

=== Agenda for June 2nd 2016 (0900 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
** Remote client debuggability (andreaf)
** Test class hierarchy (dmellado)
** Neutron OO wrappers (jlwhite, hockeynut)
** Documentation cleanup (andreaf) - All is done?
** Tempest CLI (mtreinish, dpaterson, dmellado)
** Negative test guideline (hogepodge, oomichi)
** tempest-lib/plugin (gmann)
** microversions testing (gmann)
* DevStack + Grenade
** Neutron Cleanup (sc68cal)
** Heat devstack plugin (dixiaoli, oomichi)
* openstack-health (mtreinish, masayukig)
* Critical Reviews
* QA mid-cycle meetup
* Open Discussion

=== Agenda for May 26th 2016 (1700 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
** Remote client debuggability (andreaf)
** Test class hierarchy (dmellado)
** Neutron OO wrappers (jlwhite, hockeynut)
** Documentation cleanup (andreaf) - All is done?
** Tempest CLI (mtreinish, dpaterson, dmellado)
** Negative test guideline (hogepodge, oomichi)
** tempest-lib/plugin (gmann)
** microversions testing (gmann)
** filtering by name support in auth catalog: https://review.openstack.org/#/c/284273 (fnaval)
** refactor or move some proposed DefCore Interoperability tests that require admin creds http://paste.openstack.org/show/505463/ (hogepodge)
** service clients method naming - see https://review.openstack.org/#/c/307789/ (andreaf)
* DevStack + Grenade
** Neutron Cleanup (sc68cal)
** Heat devstack plugin (dixiaoli, oomichi)
* openstack-health (mtreinish, masayukig)
* Critical Reviews
* QA mid-cycle meetup
* Open Discussion

=== Agenda for May 12th 2016 (1700 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
** Remote client debuggability (andreaf)
** Test class hierarchy (dmellado)
** Neutron OO wrappers (jlwhite, hockeynut)
** Documentation cleanup (andreaf) - All is done?
** Tempest CLI (mtreinish, dpaterson, dmellado)
** Negative test guideline (chrishoge, oomichi)
** tempest-lib/plugin (gmann)
** microversions testing (gmann)
* DevStack + Grenade
** Neutron Cleanup (sc68cal)
** Heat devstack plugin (dixiaoli, oomichi)
** MariaDB on Ubuntu: https://review.openstack.org/#/c/314087/ (rfolco)
* openstack-health (mtreinish, masayukig)
* Critical Reviews
* Open Discussion

=== Agenda for April 21st 2016 (0900 UTC) ===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
* DevStack + Grenade
* Austin Summit
** https://etherpad.openstack.org/p/newton-qa-summit-topics
* Critical Reviews
* Open Discussion

=== Agenda for April 14th 2016 (1700 UTC)===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Priority Items
** https://etherpad.openstack.org/p/mitaka-qa-priorities
* Tempest
* DevStack + Grenade
* Austin Summit
** https://etherpad.openstack.org/p/newton-qa-summit-topics
* Critical Reviews
* Open Discussion

=== Agenda for April 7th 2016 (0900 UTC) ===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Priority Items
** https://etherpad.openstack.org/p/mitaka-qa-priorities
* Tempest
* DevStack + Grenade
* Austin Summit
** https://etherpad.openstack.org/p/newton-qa-summit-topics
* Critical Reviews
* Open Discussion

=== Agenda for March 31th 2016 (1700 UTC) ===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Priority Items
** https://etherpad.openstack.org/p/mitaka-qa-priorities
* Tempest
* DevStack + Grenade
* Austin Summit
** https://etherpad.openstack.org/p/newton-qa-summit-topics
** Quality Assurance: 4fb, 4wr
* Critical Reviews
* Open Discussion

=== Agenda for March 24th 2016 (0900 UTC) ===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Priority Items
** https://etherpad.openstack.org/p/mitaka-qa-priorities
* Tempest
** Propose to remove negative tests
* DevStack + Grenade
* Austin Summit
** https://etherpad.openstack.org/p/newton-qa-summit-topics
** Quality Assurance: 4fb, 4wr
* Critical Reviews
* Open Discussion

=== Agenda for March 17th 2016 (1700 UTC) ===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Priority Items
** https://etherpad.openstack.org/p/mitaka-qa-priorities
* Tempest
** Propose change gate periodic-tempest-dsvm-neutron-identity-v3-only-full-master to be voting https://review.openstack.org/#/c/292894/ <raildo>
** Propose to remove negative tests
* DevStack + Grenade
* Austin Summit
** https://etherpad.openstack.org/p/newton-qa-summit-topics
** Quality Assurance: 4fb, 4wr
* Critical Reviews

=== Agenda for March 10th 2016 (0900 UTC) ===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Priority Items
** https://etherpad.openstack.org/p/mitaka-qa-priorities
* Tempest
* DevStack + Grenade
* Critical Reviews

Meetings/QATeamMeeting

2016-07-28T06:57:42Z

Vponomaryov: /* Agenda for July 28th 2016 (0900 UTC) */

= Weekly QA Team meeting =
The OpenStack QA Team holds public weekly meetings in <code><nowiki>#openstack-meeting</nowiki></code>, '''Thursday alternating every week between 17:00 UTC and 09:00 UTC''. 
Everyone interested in testing, quality assurance, performance engineering, etc, should attend!

The meeting minutes of can be found here: [http://eavesdrop.openstack.org/meetings/qa meeting minutes on eavesdrop]

''Rules for adding to the agenda'':
* anyone is free to add to proposed agenda, please do so by 1 hr before the meeting
* if you add an item to the agenda, please include your ircnick (i.e. sdague) to the end of your item, so we know who did, and who should lead the discussion

=== Agenda for July 28th 2016 (0900 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
* DevStack + Grenade
* openstack-health
* Critical Reviews
* Possibility to enable/disable externl plugins [vponomaryov]
** https://review.openstack.org/#/c/343830/
* Open Discussion

== Previous meetings ==
All previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/qa/.
Old agendas are stored on [[Meetings/QATeamMeeting/OldAgendas]]

===Agenda for July 21th 2016 (1700 UTC)===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities

* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
* DevStack + Grenade
* openstack-health
* Critical Reviews
** https://review.openstack.org/#/c/341083/
* Open Discussion

=== Agenda for July 7th 2016 (1700 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
* DevStack + Grenade
* openstack-health
* Critical Reviews
* Open Discussion

=== Agenda for June 30th 2016 (0900 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
* DevStack + Grenade
* openstack-health
* Critical Reviews
* Open Discussion

=== Agenda for June 23th 2016 (1700 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
*Specs Reviews
**https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
*Tempest
*DevStack + Grenade
*openstack-health
*Critical Reviews
*Open Discussion

=== Agenda for June 16th 2016 (0900 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
** Remote client debuggability (andreaf)
** Test class hierarchy (dmellado)
** Neutron OO wrappers (jlwhite, hockeynut)
** Documentation cleanup (andreaf) - All is done?
** Tempest CLI (mtreinish, dpaterson, dmellado)
** Negative test guideline (hogepodge, oomichi)
** tempest-lib/plugin (gmann)
** microversions testing (gmann)
* DevStack + Grenade
** Neutron Cleanup (sc68cal)
** Heat devstack plugin (dixiaoli, oomichi)
* openstack-health (mtreinish, masayukig)
* Critical Reviews
* QA mid-cycle meetup
* Open Discussion

=== Agenda for June 9th 2016 (1700 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
* DevStack + Grenade
* openstack-health
* Critical Reviews
* Open Discussion

=== Agenda for June 2nd 2016 (0900 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
** Remote client debuggability (andreaf)
** Test class hierarchy (dmellado)
** Neutron OO wrappers (jlwhite, hockeynut)
** Documentation cleanup (andreaf) - All is done?
** Tempest CLI (mtreinish, dpaterson, dmellado)
** Negative test guideline (hogepodge, oomichi)
** tempest-lib/plugin (gmann)
** microversions testing (gmann)
* DevStack + Grenade
** Neutron Cleanup (sc68cal)
** Heat devstack plugin (dixiaoli, oomichi)
* openstack-health (mtreinish, masayukig)
* Critical Reviews
* QA mid-cycle meetup
* Open Discussion

=== Agenda for May 26th 2016 (1700 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
** Remote client debuggability (andreaf)
** Test class hierarchy (dmellado)
** Neutron OO wrappers (jlwhite, hockeynut)
** Documentation cleanup (andreaf) - All is done?
** Tempest CLI (mtreinish, dpaterson, dmellado)
** Negative test guideline (hogepodge, oomichi)
** tempest-lib/plugin (gmann)
** microversions testing (gmann)
** filtering by name support in auth catalog: https://review.openstack.org/#/c/284273 (fnaval)
** refactor or move some proposed DefCore Interoperability tests that require admin creds http://paste.openstack.org/show/505463/ (hogepodge)
** service clients method naming - see https://review.openstack.org/#/c/307789/ (andreaf)
* DevStack + Grenade
** Neutron Cleanup (sc68cal)
** Heat devstack plugin (dixiaoli, oomichi)
* openstack-health (mtreinish, masayukig)
* Critical Reviews
* QA mid-cycle meetup
* Open Discussion

=== Agenda for May 12th 2016 (1700 UTC) ===
Newton Priorities: https://etherpad.openstack.org/p/newton-qa-newton-priorities
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
** Remote client debuggability (andreaf)
** Test class hierarchy (dmellado)
** Neutron OO wrappers (jlwhite, hockeynut)
** Documentation cleanup (andreaf) - All is done?
** Tempest CLI (mtreinish, dpaterson, dmellado)
** Negative test guideline (chrishoge, oomichi)
** tempest-lib/plugin (gmann)
** microversions testing (gmann)
* DevStack + Grenade
** Neutron Cleanup (sc68cal)
** Heat devstack plugin (dixiaoli, oomichi)
** MariaDB on Ubuntu: https://review.openstack.org/#/c/314087/ (rfolco)
* openstack-health (mtreinish, masayukig)
* Critical Reviews
* Open Discussion

=== Agenda for April 21st 2016 (0900 UTC) ===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Tempest
* DevStack + Grenade
* Austin Summit
** https://etherpad.openstack.org/p/newton-qa-summit-topics
* Critical Reviews
* Open Discussion

=== Agenda for April 14th 2016 (1700 UTC)===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Priority Items
** https://etherpad.openstack.org/p/mitaka-qa-priorities
* Tempest
* DevStack + Grenade
* Austin Summit
** https://etherpad.openstack.org/p/newton-qa-summit-topics
* Critical Reviews
* Open Discussion

=== Agenda for April 7th 2016 (0900 UTC) ===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Priority Items
** https://etherpad.openstack.org/p/mitaka-qa-priorities
* Tempest
* DevStack + Grenade
* Austin Summit
** https://etherpad.openstack.org/p/newton-qa-summit-topics
* Critical Reviews
* Open Discussion

=== Agenda for March 31th 2016 (1700 UTC) ===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Priority Items
** https://etherpad.openstack.org/p/mitaka-qa-priorities
* Tempest
* DevStack + Grenade
* Austin Summit
** https://etherpad.openstack.org/p/newton-qa-summit-topics
** Quality Assurance: 4fb, 4wr
* Critical Reviews
* Open Discussion

=== Agenda for March 24th 2016 (0900 UTC) ===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Priority Items
** https://etherpad.openstack.org/p/mitaka-qa-priorities
* Tempest
** Propose to remove negative tests
* DevStack + Grenade
* Austin Summit
** https://etherpad.openstack.org/p/newton-qa-summit-topics
** Quality Assurance: 4fb, 4wr
* Critical Reviews
* Open Discussion

=== Agenda for March 17th 2016 (1700 UTC) ===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Priority Items
** https://etherpad.openstack.org/p/mitaka-qa-priorities
* Tempest
** Propose change gate periodic-tempest-dsvm-neutron-identity-v3-only-full-master to be voting https://review.openstack.org/#/c/292894/ <raildo>
** Propose to remove negative tests
* DevStack + Grenade
* Austin Summit
** https://etherpad.openstack.org/p/newton-qa-summit-topics
** Quality Assurance: 4fb, 4wr
* Critical Reviews

=== Agenda for March 10th 2016 (0900 UTC) ===
* Specs Reviews
** https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
* Priority Items
** https://etherpad.openstack.org/p/mitaka-qa-priorities
* Tempest
* DevStack + Grenade
* Critical Reviews

Manila/Meetings

2015-11-12T15:01:52Z

Vponomaryov: /* Weekly Manila team meeting */

Manila/Meetings

2015-11-12T14:53:20Z

Vponomaryov: /* Next meeting */

Python3

2015-07-29T14:37:17Z

Vponomaryov: /* OpenStack clients */

This page tracks the progress of Python 3 effort porting for OpenStack.

== Python 3 ==

[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''

== Python 2: Python 2.6 support dropped, Python 2.7 only ==

OpenStack Liberty targets Python 2.7 and 3.4.

Python 2.6 support is being dropped in OpenStack since OpenStack Juno for servers. Python 2.6 support is currently kept in Oslo libraries and clients. See [https://etherpad.openstack.org/p/juno-cross-project-future-of-python juno-cross-project-future-of-python] etherpad.

Python 3.3 support is being dropped since OpenStack Liberty.

Python 2.6 is slowly being dropped in the whole OpenStack project, ex: [https://review.openstack.org/#/c/201295/ Remove p26 job from DIB].

== Port Python 2 code to Python 3 ==

OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack supported Python 2.6 for RHEL up to Juno, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).

=== sixer tool ===

The [https://pypi.python.org/pypi/sixer sixer] tool helps to replace most basic patterns to add Python 3 compatibility and it respects OpenStack coding style.

=== Common patterns ===

* Replace dict.iteritems() with dict.items()
* Replace dict.itervalues() with dict.values()
* Replace "for key in dict.iterkeys()" with "for key in dict"
* Replace iterator.next() with next(iterator)
* Replace basestring with six.string_types
* Replace unicode with six.text_type
* Replace (str, unicode) with six.string_types
* Replace (int, long) with six.integer_types
* Replace text.encode('hex') with base64.b64encode(text) which requires to encode (Unicode) text to UTF-8 on Python 3

if six.PY3:
text = text.encode('utf-8')
text_b64 = base64.b64encode(text)

* Replace raw.decode('hex') with base64.b64decode(raw). The output may be decoded from UTF-8 to get back text (Unicode) on Python 3.

text = base64.b64decode(raw)
if six.PY3:
text = text.decode('utf-8')

Note: Replacing dict.iteritems() with six.iteritems(dict) was preferred in the past, but there was [http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html a discussion suggesting to avoid six for this].

=== bytes.decode and unicode.encode ===

Python has a notion of "default encoding": sys.getdefaultencoding(). On Python 2, the default encoding is ASCII, whereas it is UTF-8 on Python 3.

Don't write <code>data.decode()</code> or <code>text.encode()</code> without parameter, because you will use a different encoding on Python 2 and Python 3.

Use an explicit encoding instead. Example: <code>data.decode('utf-8')</code> or <code>text.encode('utf-8')</code>. The right encoding depends on the use case, but UTF-8 is usually a good candidate (it is a superset of ASCII).

=== safe_decode ===

Olso Incubator has a function '''safe_decode()''' which can be used to decode a bytes string and pass text strings unchanged.

The default encoding is <code>sys.stdin.encoding or sys.getdefaultencoding()</code>:
* Python 3: the locale encoding, or UTF-8 if sys.stdin is "mocked" (io.StringIO instance)
* Python 2: the locale encoding, or ASCII if stdin is not a TTY or if sys.stdin is "mocked" (StringIO.StringIO instance)

It's safer to explicit the encoding to not rely on the locale encoding and have the same behaviour even if sys.stdin is "mocked".

Safe usage:
* <code>safe_decode(data, 'utf-8')</code>: decode bytes from UTF-8 or returns data unchanged if it's already a text string

Unsafe usage:
* <code>safe_decode(data)</code>

By default, the decoder is strict. You can specify a different error handler using the optional <code>errors</code> parameter. Example: safe_decode(b'[\xff]', 'ascii', 'ignore') returns '[]'.

=== safe_encode ===

Olso Incubator has a function '''safe_encode()''' which can be used to encode a string. Its usage is tricky and you should understand how it works and which encodings are used.
* <code>safe_encode(text)</code> encodes text to the output encoding
* <code>safe_encode(bytes)</code> may decode the string and then reencode to a different encoding if input and output encodings are different

The default input encoding (<code>incomding</code> parameter) is <code>sys.stdin.encoding or sys.getdefaultencoding()</code>:
* Python 3: the locale encoding, or UTF-8 if sys.stdin is "mocked" (io.StringIO instance)
* Python 2: the locale encoding, or ASCII if stdin is not a TTY or if sys.stdin is "mocked" (StringIO.StringIO instance)

The default output encoding (<code>encoding</code> parameter) is UTF-8.

It's safer to explicit the input encoding to not rely on the locale encoding and have the same behaviour even if sys.stdin is "mocked".

Safe usage:
* <code>safe_encode(data, incoming='utf-8')</code>: encode text to UTF-8 or returns data unchanged if it's already a bytes string (since the input and output encoding are UTF-8)

Unsafe usage:
* <code>safe_encode(data)</code>

Example:
* <code>safe_encode(b'\xe9', incoming='latin-1')</code> returns <code>b'\xc3\xa9'</code>.

By default, the encoder and the decoder are strict. You can specify a different error handler using the optional <code>errors</code> parameter. Example: <code>safe_encode(b'[\xff]', incoming='ascii', errors='ignore')</code> returns <code>b'[]'</code>.

=== logging module and format exceptions ===

On Python 2, the logging module accepts bytes and text strings. On Python 3, it only accepts text strings. For example, logging.error(b'hello') logs <code>b'hello'</code> instead of <code>'hello'</code>.

There is no clear rule for format exceptions yet. There are different choices depending on the project:

* <code>str(exc)</code>: native string, so use bytes on Python 2
* <code>six.text_type(exc)</code>: always use Unicode. It may raise unicode error depending on the exception, be careful. Example of such error in python 2: <code>unicode(Exception("nonascii:\xe9"))</code>.
* <code>six.u(str(exc))</code>: unsafe on Python 2 if str(exc) contains non-ASCII bytes, ex: <code>unicode(str(Exception("\xff")))</code>
* <code>LOG.exception(_LE("... %(exc)s ..."), {"exc": exc, ...})</code>

Since logging functions expect text strings on Python 3, logged exceptions should be formatted using <code>str(exc)</code>. Example: <code>LOG.debug(str(exc))</code>.

=== HTTP ===

The HTTP protocol is based on '''bytes''':

* HTTP body contains '''bytes'''. For example, use io.BytesIO for a stream storing an HTTP body.
* HTTPConnection.getresponse().read() returns '''bytes''' (in Python 3, '''str''' which is bytes in Python 2)
* On Python 3, the http.client accepts text for HTTP headers: keys are encoded to ASCII and values to ISO 8859-1 (which is only a small subset of the Unicode charset)
* It looks like Swift encodes internally HTTP headers to UTF-8 (directly using the UTF-8 encoding, not using a MIME encoding like =?UTF-8?Q?...?=. See the HTTP [RFC 2047 http://www.ietf.org/rfc/rfc2047.txt] and [http://stackoverflow.com/questions/4400678/http-header-should-use-what-character-encoding HTTP header should use what character encoding?]

=== References to port Python 2 code to Python 3 ===
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.

=== Common pitfalls ===

==== What is a string ? ====
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:

Python 2:
>>> type('foo')
<type 'str'>
>>> type(u'foo')
<type 'unicode'>
>>> type(b'foo')
<type 'str'>
>>> isinstance('foo', six.text_type)
False
>>> isinstance(u'foo', six.text_type)
True
>>> bytes is str
True
>>> b'foo'[0]
'f'

Python 3:
>>> type('foo')
<class 'str'>
>>> type(u'foo')
<class 'str'>
>>> type(b'foo')
<class 'bytes'>
>>> isinstance('foo', six.text_type)
True
>>> isinstance(b'foo', six.text_type)
False
>>> bytes is str
False
>>> b'foo'[0]
102

==== tox/testr error: db type could not be determined ====

The "db type could not be determined" error comes from .testrepository/times.dbm used by testr.

Workaround: "rm -rf .testrepository/" and then run "tox -e py34" before running "tox -e py27". You only have to do this once. The problem only occurs with "tox -e py34" when .testrepository/ was created by Python 2.

== Python 3 Status of OpenStack projects ==

=== Common Libraries (Oslo Projects) ===

For the list of Common Libraries, see http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml#n160

{|class="wikitable sortable"
|-
! Project !! Python 3 compatibility !! Comment
|-
| [https://pypi.python.org/pypi/cliff cliff] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.concurrency oslo.concurrency] || style="background-color: lightgreen;" | Yes ||
|-/
| oslo-incubator || style="background-color: lightgreen;" | Yes || py34 gate is voting
|-
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.context oslo.context] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.db oslo.db] || style="background-color: lightgreen;" | Yes ||
PyMySQL driver is now used by default for MySQL. setup.cfg contains the Python 3 classifier.
|-
| [https://pypi.python.org/pypi/oslo.i18n oslo.i18n] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.log oslo.log] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color: orange;" | Yes, except of Qpid & AMQP drivers. py34 is not more voting because of [//bugs.launchpad.net/ubuntu/+source/python3.4/+bug/1367907 ubuntu still uses 3.4.0 which has a severe bug] || oslo.messaging 1.11.0 fully works on Python 3, except of Qpid and AMQP 1.0 transports which are Python 2 only. The py34 gate is voting.
|-
| [https://pypi.python.org/pypi/oslo.middleware oslo.middleware] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.serialization oslo.serialization] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || ? || The project only contains two short .py files, it looks to be Python 3 compatible. Is Sphinx Python 3 compatible?
|-
| [https://pypi.python.org/pypi/oslotest oslotest] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.versionedobjects oslo.versionedobjects] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color: lightgreen;" | Yes || Supports Python 3 since oslo.vmware 0.13.0
|-
| [https://pypi.python.org/pypi/oslo.utils oslo.utils] || style="background-color: lightgreen;" | Yes ||
|-
| pylockfile || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/stevedore stevedore] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color: lightgreen;" | Yes ||
|}

=== Development tools ===

{|class="wikitable sortable"
|-
! Project !! Python 3 compatibility !! Comment
|-
| [https://pypi.python.org/pypi/cookiecutter cookiecutter] || style="background-color: lightgreen;" | yes ||
|-
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color: lightgreen;" | yes || py33 gate is not voting
|-
| [https://pypi.python.org/pypi/pbr pbr] || style="background-color: lightgreen;" | yes ||
|-
| stackforge/python-jenkins || style="background-color: lightgreen;" | yes || py33 gate is voting
|-
| openstack-infra/jenkins-job-builder || style="background-color: orange;" | partial || https://review.openstack.org/172238
|}

=== OpenStack clients ===

{|class="wikitable sortable"
|-
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment
|-

| [https://pypi.python.org/pypi/keystonemiddleware keystonemiddleware] || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | Yes, but [https://review.openstack.org/206107 test using python-memcached are skipped on Python 3] || style="background-color: lightgreen;" | Yes || python-memcached || keystonemiddleware 1.6.1 supports Python 3. python-memcached 1.56 now supports Python 3.
|-

| [https://pypi.python.org/pypi/python-barbicanclient python-barbicanclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||
|-

| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color:lightgreen" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-manilaclient python-manilaclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || ||
|-

| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||
|-

| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPII || ||
|-

| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || ||
|-

| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: lightgreen" | OK || style="background-color: lightgreen" | Voting || style="background-color: lightgreen" | Yes || || As of 0.9
|-

| [https://pypi.python.org/pypi/python-saharaclient python-savannaaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || style="background-color: lightgreen" | Yes || || [https://review.openstack.org/#/c/73128/ python-savannaclient: enable the py33 gates] abandonned: "We haven't enough tests to ensure that savanna client works correctly on py33, so, it's kind of premature step. We already have py33 and pypy jobs in experimental pipeline."
|-

| [https://pypi.python.org/pypi/python-saharaclient python-saharaclient] || style="background-color: lightgreen;" | OK || style="background-color: lightgreen" | Voting || style="background-color: lightgreen" | On PyP || ||
|-

| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | [https://pypi.python.org/pypi/python-swiftclient/ On PyPI] || ||
|-

| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||

|-

| [https://pypi.python.org/pypi/python-designateclient python-designateclient] || style="background-color: orange;" | work in progress || style="background-color: orange;" | No || style="background-color: red;" | No || ||

|}

=== OpenStack Libraries ===

{|class="wikitable sortable"
|-
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Comment
|-
| [https://pypi.python.org/pypi/glance_store glance_store] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | voting || style="background-color: lightgreen;" | Yes || glance_store 0.7.0 added the Python 3 support.
|-
| [https://pypi.python.org/pypi/os-brick os-brick] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | voting || style="background-color: lightgreen;" | Yes || os-brick 0.3.2 added Python 3 support.

|-
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | voting || style="background-color:lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | voting || style="background-color: lightgreen;" | Yes ||
|-
|}

=== Dependencies ===

There are two ways to express dependencies that should differ between Python 2.x and 3.x. We currently support using multiple requirements files, and will shortly be supporting and recommending the use of environment markers. Recent releases of all the tools in our ecosystem are needed for environment markers.

==== requirements-py3.txt ====

This approach generates a different requirements list depending on what Python version is running when the egg info is created (which goes into wheels and sdists). As a consequence this can't work with tox, publishing sdists to PyPI, or wheels. It is however useful for anything installing from git. To use it, create two requirements files:

* requirements-py2.txt: all dependencies for Python 2 (not only dependencies specific to Python 2)
* requirements-py3.txt: all dependencies for Python 3 (not only dependencies specific to Python 3)
* (same for test-requirements.txt)

You have to edit tox.ini to specify the right requirements file. Extract of a tox.ini file:

<pre>
...
[testenv:py33]
deps = -r{toxinidir}/requirements-py3.txt
-r{toxinidir}/test-requirements-py3.txt
...
</pre>

==== Environment markers ====

Environment markers provide a mini-language to express when a dependency is relevant. We hope the next release of pbr will support them. Recent pip, setuptools and wheel versions do support them, but until pbr exports the requirements metadata with them intact and structured correctly for setuptools, we can't use them anywhere.

* PEP 0345 -- Metadata for Python Software Packages 1.2 (accepted): [https://www.python.org/dev/peps/pep-0345/#environment-markers Environment markers]
* PEP 0426 -- Metadata for Python Software Packages 2.0 (draft): [https://www.python.org/dev/peps/pep-0426/#environment-markers Environment markers]

When placed in a requirements.txt file, the markers are interpreted directly by pip in `pip install -r requirements.txt`, and indirectly via setuptools when installing an sdist or wheel (including `pip install .`. For instance:
<pre>
six
futures; python_version < '3.2'
</pre>

See also:
* [https://review.openstack.org/#/c/184328/ oslo.db: Refactor deps to use extras and env markers] (require pip 7, not released yet)
* pip: [https://github.com/pypa/pip/pull/1472 Fix issue #1433: parse requirements in markers #1472]
* setuptools: pkg_resources doesn't understand environment markers: [https://bitbucket.org/pypa/wheel/issue/139/wheel-should-support-pep-426-environment pkg_resources.parse_requirements() raises ValueError]
* pbr: [https://review.openstack.org/#/c/181795/ Teach pbr to read extras and env markers], [https://review.openstack.org/#/c/165015/ Handle PEP426 markers] (abandoned)
* wheel: [https://bitbucket.org/pypa/wheel/issue/139/wheel-should-support-pep-426-environment wheel should support PEP 426 environment markers] (closed), wheel supports environment markers in extra requirements
* tox: [https://bitbucket.org/hpk42/tox/issue/236/tox-must-create-the-source-distribution tox must create the source distribution with the Python of the virtual environment], [https://bitbucket.org/hpk42/tox/pull-request/142/issue-236-subcommand_test-rebuilds-sdist/ Issue #236: subcommand_test() rebuilds sdist with venv python] (declined)

Oslo Messaging issue:
* futures is not needed on Python 3, since concurrent.futures is now part of the Pyhon stdlib since Python 3.2
* [https://review.openstack.org/#/c/172135/ futures was added to requirements-py3.txt] because tox creates the source tarball using Python 2 which reads requirements.txt
* The correct fix would be to use environment markers on futures

==== Dependencies status ====

[https://caniusepython3.com/check/4fd5dda2-b1f1-4db4-a636-67cd3276cb6a Porting status] for [https://github.com/openstack/requirements/blob/master/global-requirements.txt global-requirement.txt].

{|class="wikitable sortable"
|-
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment
|-

| [https://pypi.python.org/pypi/boto boto] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || || See https://github.com/boto/boto3 (experimental) <- This seems dead, and https://github.com/boto/boto works with Python 3.x (since 2.32).
|-

| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || || Requirements upgraded: https://review.openstack.org/94357
|-

| [https://pypi.python.org/pypi/tablib tablib ] || style="background-color: lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || || setup.py installs tablib.packages.markup which is not compatible with Python 2. setup.py displays a warning on compiling the bytecode of markup.py.

tablib is needed by cliff-tablib, itself needed by python-neutronclient
|-

| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || ||
As of 1.1.6
|-

| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60
|-

| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)
|-

| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:lightgreen;" | Yes || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || || eventlet 0.17.3 now fully support Python 3
|-

| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(
|-

| [https://pypi.python.org/pypi/libvirt-python libvirt-python] || style="background-color: lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || ||
|-

| [https://pypi.python.org/pypi/mysql-python MySQL-Python] || style="background-color:orange;" | No, but mysqlclient or PyMySQL are compatible || N/A || style="background-color: orange;" | No || ||
Patches:
* [https://review.openstack.org/#/c/179745/ Add mysqlclient dependency]

MySQL-Python: [https://github.com/farcepest/MySQLdb1/pulls 2 pending pull requests for Python 3].

INADA Naoki, the maintainer of PyMySQL, wrote [https://pypi.python.org/pypi/mysqlclient mysqlclient]: fork of MySQL-Python using libmysqlclient.so which is Python 3 compatible. He recommends to use mysqlclient instead of MySQL-python even on Python 2. mysqlclient fixes some bugs, build issues and it support Python 3. For example: support MariaDB's libmysqlclient.so, support microsecond in TIME column.

PyMySQL is compatible with Python 3: see [https://wiki.openstack.org/wiki/PyMySQL_evaluation PyMySQL evaluation] discussion to eventually replace mysql-python with PyMySQL in OpenStack.
|-

| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too. Python 3 support as of 0.10.4. Pushed to requirements: https://review.openstack.org/94358 .
|-

| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3
|-

| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||
* nose-exclude (tests only)
* openstack.nose-plugin
||
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247
* https://review.openstack.org/#/c/80956/
|-
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9
|-
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||
|-

| [https://pypi.python.org/pypi/pam pam] || style="background-color:orange;" | No for pam, Yes for simplepam || NA || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3
|-

| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Requirements upgraded: https://review.openstack.org/#/c/81132/
|-

| [https://pypi.python.org/pypi/paste paste] || style="background-color:lightgreen;" | Yes || style="background-color: lightgrey;" | N/A || style="background-color: lightgreen;" | Yes || || Compatible with Python 3 since Paste 2.0
|-

| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:lightgreen;" | Yes || style="background-color:lightgreen;" | Yes || style="background-color:lightgreen;" | Yes || ||
|-

| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead. See [https://pypi.python.org/pypi/ldap3 ldap3] which is Python 3 compatible (license: LGPL, one of the optional deps is BSD-4-Clause).
|-

| [https://pypi.python.org/pypi/python-memcached python-memcached] || style="background-color:lightgreen;" | Yes || style="background-color:lightgreen;" | N/A || style="background-color:lightgreen;" | Yes || ||
python-memcached 1.56 includes pull request: [https://github.com/linsomniac/python-memcached/pull/67 Port memcache to Python 3 #67].

Julien Danjou ported [https://pypi.python.org/pypi/pymemcache pymemcache] to Python 3, another memcached client, he suggests to use this one instead
|-

| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Required if using oslo.messaging's qpid backend. That is the old pure python amqp 0-10 client library. That only speaks 0-10 and works with qpidd broker. It is used by the impl_qpid.py driver in oslo.messaging. That client is legacy - it isn't being ported to use AMQP 1.0.
|-

| [https://pypi.python.org/pypi/pyngus pyngus] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||
pyngus is required if using oslo.messaging's AMQP 1.0 driver. pyngus is a wrapper for proton and proton has not been ported to python 3 yet: see [https://issues.apache.org/jira/browse/PROTON-490 Proton issue #490].

[https://pypi.python.org/pypi/python-qpid-proton Proton] is a C library with various bindings in other languages. The language bindings are auto generated via the SWIG tool, plus a the python wrapper bits. All the python stuff is found under proton-c/bindings/python in the [https://git-wip-us.apache.org/repos/asf/qpid-proton.git git tree]. The first bit of work is to refactor the SWIG file cproton.i to work for both Python 2 and 3.
|-

| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||
|-
| [https://pypi.python.org/pypi/Routes Routes] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || || Routes 2.2 works Python 2 and 3 using a single code base.
|-
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI || ||
|-
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: red;" | No || ||
|-
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||
|-
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||
|-
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | N/A || style="background-color: lightgreen;" | Yes || || python3 works with the 0.7.0 version released on Jul 23, 2015
|-

|}

=== OpenStack applications ===

{| class="wikitable sortable"
|-
! Project !! Python 3 compatibility !! py34 gate !! Comments
|-

| [https://pypi.python.org/pypi/gnocchi gnocchi] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/ironic ironic] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes ||
[http://lists.openstack.org/pipermail/openstack-dev/2015-May/064049.html Python 3.4 unit tests are now being run for openstack/ironic. The unit tests are a voting job.]
Thanks to Victor Sergeyev for all of his work to update the Ironic code to make it pass the unit tests using Python 3.4: [https://review.openstack.org/#/c/156192/ Run tests in py34 environment]
|-
| [https://pypi.python.org/pypi/rally Rally] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | voting || A big thank to Andrey Kurilin for a lot of work in this direction
|-

| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
Victor Stinner is working on porting Ceilometer to Python 3. Patches:
* [https://review.openstack.org/#/q/status:open+project:openstack/ceilometer+branch:master+topic:py3,n,z Patches with py3 topic]
|-

| [https://pypi.python.org/pypi/cinder cinder] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
[https://blueprints.launchpad.net/cinder/+spec/cinder-python3 cinder-python3] blueprint (written by Victor Stinner) has been accepted for Liberty.

Patches: [https://review.openstack.org/#/q/project:openstack/cinder+topic:bp/cinder-python3,n,z bp/cinder-python3 topic].
|-

| [https://github.com/openstack/manila manila] || style="background-color: orange;" | work in progress || style="background-color: orange;" | non-voting ||
[https://blueprints.launchpad.net/manila/+spec/py3-compatibility py3-compatibility] blueprint.

non-voting py34 CI job runs in 'check' pipeline for each Manila gerrit commit. It fails because py3 support fixes are in progress [L-2].
Patches: [https://review.openstack.org/#/q/project:openstack/manila+topic:bp/py3-compatibility,n,z bp/py3-compatibility topic].
|-

| [https://pypi.python.org/pypi/glance glance] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
Victor Stinner is working on porting Glance to Python3.

Patches: [https://review.openstack.org/#/q/status:open+project:openstack/glance+branch:master+topic:py3,n,z Patches for glance (topic: py3)].

test_migrations is blocked by a bug in oslo.db ([https://review.openstack.org/#/c/206476/ Fix test_migrations on Python 3]) / sqlalchemy-migrate ([https://review.openstack.org/#/c/206478/ Add VerNum.__index__() for Python 3 support]).
|-

| [https://pypi.python.org/pypi/heat heat] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
[http://specs.openstack.org/openstack/heat-specs/specs/liberty/heat-python34-support.html Python34 Support] spec (by Sirushti Murugesan) accepted for Liberty.

Patches: [https://review.openstack.org/#/q/topic:bp/heat-python34-support,n,z bp/heat-python34-support topic].
|-

| [https://pypi.python.org/pypi/keystone keystone] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
The spec [http://specs.openstack.org/openstack/keystone-specs/specs/liberty/python3.html Add spec for python-3 compatibility] (by Morgan Fainberg) was accepted for Liberty. See also the blueprint [https://launchpad.net/keystone/+spec/python3 python3] started by Dolph Mathews on 2014-07-22; David Stanek is working on it.

Patches: [https://review.openstack.org/#/q/project:openstack/keystone+branch:master+topic:bp/python3,n,z bp/python3 topic].

[https://github.com/rohe/pysaml2/ pysaml2] 3.0 now supports Python 3.

Aslo blocked by [https://pypi.python.org/pypi/ldappool ldappool] and [https://pypi.python.org/pypi/python-ldap python-ldap]. [https://github.com/pyldap/pyldap/ pyldap] is a fork of python-ldap which adds Python 3 support. python-ldap may be replaced with [https://pypi.python.org/pypi/ldap3 ldap3], but it requires more work.
|-

| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:orange;" | work in progress || style="background-color: lightgreen;" | voting ||
[https://review.openstack.org/#/c/172962/ Porting to Python 3] spec (by Cyril Roelandt) accepted for Liberty, port in progress. Patches:
* [https://review.openstack.org/#/q/project:openstack/neutron+branch:master+topic:bp/neutron-python3,n,z Patches of the blueprint neutron-python3]
|-

| [https://pypi.python.org/pypi/nova nova] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
[http://specs.openstack.org/openstack/nova-specs/specs/liberty/approved/adding-python34-support-to-nova.html Adding Python 3.4 support to Nova] spec (by Victor Stinner) accepted for Liberty. Port in progress.

Patches:
* [https://review.openstack.org/#/q/project:openstack/nova+branch:master+topic:bp/nova-python3,n,z master (bp/nova-python3)].

See also [https://etherpad.openstack.org/p/liberty-nova-priorities-tracking Py3.4 Compatibility in Liberty nova priorities tracking].
|-

| [https://pypi.python.org//pypi/sahara sahara] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
|-

| [https://pypi.python.org/pypi/swift swift] || style="background-color: orange;" | work in progress || style="background-color: orange;" | non voting (need to [https://review.openstack.org/#/c/199034/ fix tox -e py34] ||
swift now depends on six.(Jaivish Kothari is also working on this)

Patches:
* [https://review.openstack.org/#/q/project:openstack/swift+branch:master+topic:py3,n,z Patches with the topic py3]

Dependencies:
* dnspython: use dnspython3 on Python 3 ([https://review.openstack.org/#/c/200416/ not in global requirements yet])
* PyEClib: [https://bitbucket.org/kmgreen2/pyeclib/issues/69/need-a-release-108-with-python-3-fixes waiting for the release 1.0.8]
|-

| [https://pypi.python.org/pypi/horizon horizon] || style="background-color: orange;" | work in progress || style="background-color: red;" | No ||
George Peristerakis wrote [https://blueprints.launchpad.net/horizon/+spec/porting-python3 Make Horizon python3.4 compatible].

Patches:
* [https://review.openstack.org/#/q/topic:bp/porting-python3+project:openstack/horizon,n,z bp/porting-python3 topic]
* [https://review.openstack.org/#/c/178342/ Add py34 env] by lokesh s
* [https://review.openstack.org/#/c/183365/ Made the class Column Python 3 compatible] by Emmanuel Rousselle
* [https://review.openstack.org/#/c/183367/ Made HorizonComponent class Python3 compatible] (merged) by Eric Lafontaine
|-

| [https://pypi.python.org/pypi/designate designate] || style="background-color: orange;" | work in progress || style="background-color: red;" | No || Pradeep is working on the Python 3 port.
|-

|}

=== Functional tests ===

Doug Hellmann wrote a cross-project specification to run integration tests on Python 3: [https://review.openstack.org/#/c/177375/ Enabling Python 3 for Application Integration Tests] (spec approved for Liberty).

Patch for DevStack: [https://review.openstack.org/#/c/181165/ Enable optional Python 3 support].

== Status of Python 3 in Linux distributions ==

* ArchLinux already [https://www.archlinux.org/news/python-is-now-python-3 switched to Python 3 by default] in 2011
* Ubuntu wants to remove Python 2 from the default installation: see [http://www.phoronix.com/scan.php?page=news_item&px=Ubuntu-16.04-Python-Plans Ubuntu Plans For Python 3 By Default For Ubuntu 16.04 LTS]
* Fedora schedules the switch in Fedora 23 (october 2015): [https://fedoraproject.org/wiki/Changes/Python_3_as_Default Python 3 as the Default Implementation]
* RHEL: [https://www.softwarecollections.org/en/scls/rhscl/python33/ Python 3.3] and [https://www.softwarecollections.org/en/scls/rhscl/rh-python34/ Python 3.4] are available on RHEL6 and RHEL7 using SCL.
* CentOS: [https://www.softwarecollections.org/en/scls/rhscl/python33/ Python 3.3] and [https://www.softwarecollections.org/en/scls/rhscl/rh-python34/ Python 3.4] are available on CentOS 6 and 7 using SCL (no Red Hat subscription needed, CentOS has its own flavor of SCL)
* In Debian, the plan is to deprecate Python 2 in Stretch (aka: Debian 9, the next Stable Debian after Jessie) and to completely remove Python 2 for the Buster release of Debian (aka: Debian 10, to be release in approximatively 2019).

== Reports at OpenStack Summits ==

* Liberty Summit
** Cross-project session on enabling app integration testing: https://etherpad.openstack.org/p/liberty-cross-project-python3
* Kilo Summit
** Doug Hellmann: "Moving our applications to Python 3" cross-project session
** Victor Stinner: "Port nova to Python 3", https://etherpad.openstack.org/p/liberty-nova-summit-ideas
* Juno summit notes: https://etherpad.openstack.org/p/juno-cross-project-future-of-python (Oslo) and https://etherpad.openstack.org/p/juno_swift_python3 (Swift)
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3

== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==

Enovance organized a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday) 2014. See the page [[Python3/SprintPycon2014]].

Python3

2015-07-29T14:32:57Z

Vponomaryov: /* OpenStack applications */

This page tracks the progress of Python 3 effort porting for OpenStack.

== Python 3 ==

[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''

== Python 2: Python 2.6 support dropped, Python 2.7 only ==

OpenStack Liberty targets Python 2.7 and 3.4.

Python 2.6 support is being dropped in OpenStack since OpenStack Juno for servers. Python 2.6 support is currently kept in Oslo libraries and clients. See [https://etherpad.openstack.org/p/juno-cross-project-future-of-python juno-cross-project-future-of-python] etherpad.

Python 3.3 support is being dropped since OpenStack Liberty.

Python 2.6 is slowly being dropped in the whole OpenStack project, ex: [https://review.openstack.org/#/c/201295/ Remove p26 job from DIB].

== Port Python 2 code to Python 3 ==

OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack supported Python 2.6 for RHEL up to Juno, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).

=== sixer tool ===

The [https://pypi.python.org/pypi/sixer sixer] tool helps to replace most basic patterns to add Python 3 compatibility and it respects OpenStack coding style.

=== Common patterns ===

* Replace dict.iteritems() with dict.items()
* Replace dict.itervalues() with dict.values()
* Replace "for key in dict.iterkeys()" with "for key in dict"
* Replace iterator.next() with next(iterator)
* Replace basestring with six.string_types
* Replace unicode with six.text_type
* Replace (str, unicode) with six.string_types
* Replace (int, long) with six.integer_types
* Replace text.encode('hex') with base64.b64encode(text) which requires to encode (Unicode) text to UTF-8 on Python 3

if six.PY3:
text = text.encode('utf-8')
text_b64 = base64.b64encode(text)

* Replace raw.decode('hex') with base64.b64decode(raw). The output may be decoded from UTF-8 to get back text (Unicode) on Python 3.

text = base64.b64decode(raw)
if six.PY3:
text = text.decode('utf-8')

Note: Replacing dict.iteritems() with six.iteritems(dict) was preferred in the past, but there was [http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html a discussion suggesting to avoid six for this].

=== bytes.decode and unicode.encode ===

Python has a notion of "default encoding": sys.getdefaultencoding(). On Python 2, the default encoding is ASCII, whereas it is UTF-8 on Python 3.

Don't write <code>data.decode()</code> or <code>text.encode()</code> without parameter, because you will use a different encoding on Python 2 and Python 3.

Use an explicit encoding instead. Example: <code>data.decode('utf-8')</code> or <code>text.encode('utf-8')</code>. The right encoding depends on the use case, but UTF-8 is usually a good candidate (it is a superset of ASCII).

=== safe_decode ===

Olso Incubator has a function '''safe_decode()''' which can be used to decode a bytes string and pass text strings unchanged.

The default encoding is <code>sys.stdin.encoding or sys.getdefaultencoding()</code>:
* Python 3: the locale encoding, or UTF-8 if sys.stdin is "mocked" (io.StringIO instance)
* Python 2: the locale encoding, or ASCII if stdin is not a TTY or if sys.stdin is "mocked" (StringIO.StringIO instance)

It's safer to explicit the encoding to not rely on the locale encoding and have the same behaviour even if sys.stdin is "mocked".

Safe usage:
* <code>safe_decode(data, 'utf-8')</code>: decode bytes from UTF-8 or returns data unchanged if it's already a text string

Unsafe usage:
* <code>safe_decode(data)</code>

By default, the decoder is strict. You can specify a different error handler using the optional <code>errors</code> parameter. Example: safe_decode(b'[\xff]', 'ascii', 'ignore') returns '[]'.

=== safe_encode ===

Olso Incubator has a function '''safe_encode()''' which can be used to encode a string. Its usage is tricky and you should understand how it works and which encodings are used.
* <code>safe_encode(text)</code> encodes text to the output encoding
* <code>safe_encode(bytes)</code> may decode the string and then reencode to a different encoding if input and output encodings are different

The default input encoding (<code>incomding</code> parameter) is <code>sys.stdin.encoding or sys.getdefaultencoding()</code>:
* Python 3: the locale encoding, or UTF-8 if sys.stdin is "mocked" (io.StringIO instance)
* Python 2: the locale encoding, or ASCII if stdin is not a TTY or if sys.stdin is "mocked" (StringIO.StringIO instance)

The default output encoding (<code>encoding</code> parameter) is UTF-8.

It's safer to explicit the input encoding to not rely on the locale encoding and have the same behaviour even if sys.stdin is "mocked".

Safe usage:
* <code>safe_encode(data, incoming='utf-8')</code>: encode text to UTF-8 or returns data unchanged if it's already a bytes string (since the input and output encoding are UTF-8)

Unsafe usage:
* <code>safe_encode(data)</code>

Example:
* <code>safe_encode(b'\xe9', incoming='latin-1')</code> returns <code>b'\xc3\xa9'</code>.

By default, the encoder and the decoder are strict. You can specify a different error handler using the optional <code>errors</code> parameter. Example: <code>safe_encode(b'[\xff]', incoming='ascii', errors='ignore')</code> returns <code>b'[]'</code>.

=== logging module and format exceptions ===

On Python 2, the logging module accepts bytes and text strings. On Python 3, it only accepts text strings. For example, logging.error(b'hello') logs <code>b'hello'</code> instead of <code>'hello'</code>.

There is no clear rule for format exceptions yet. There are different choices depending on the project:

* <code>str(exc)</code>: native string, so use bytes on Python 2
* <code>six.text_type(exc)</code>: always use Unicode. It may raise unicode error depending on the exception, be careful. Example of such error in python 2: <code>unicode(Exception("nonascii:\xe9"))</code>.
* <code>six.u(str(exc))</code>: unsafe on Python 2 if str(exc) contains non-ASCII bytes, ex: <code>unicode(str(Exception("\xff")))</code>
* <code>LOG.exception(_LE("... %(exc)s ..."), {"exc": exc, ...})</code>

Since logging functions expect text strings on Python 3, logged exceptions should be formatted using <code>str(exc)</code>. Example: <code>LOG.debug(str(exc))</code>.

=== HTTP ===

The HTTP protocol is based on '''bytes''':

* HTTP body contains '''bytes'''. For example, use io.BytesIO for a stream storing an HTTP body.
* HTTPConnection.getresponse().read() returns '''bytes''' (in Python 3, '''str''' which is bytes in Python 2)
* On Python 3, the http.client accepts text for HTTP headers: keys are encoded to ASCII and values to ISO 8859-1 (which is only a small subset of the Unicode charset)
* It looks like Swift encodes internally HTTP headers to UTF-8 (directly using the UTF-8 encoding, not using a MIME encoding like =?UTF-8?Q?...?=. See the HTTP [RFC 2047 http://www.ietf.org/rfc/rfc2047.txt] and [http://stackoverflow.com/questions/4400678/http-header-should-use-what-character-encoding HTTP header should use what character encoding?]

=== References to port Python 2 code to Python 3 ===
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.

=== Common pitfalls ===

==== What is a string ? ====
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:

Python 2:
>>> type('foo')
<type 'str'>
>>> type(u'foo')
<type 'unicode'>
>>> type(b'foo')
<type 'str'>
>>> isinstance('foo', six.text_type)
False
>>> isinstance(u'foo', six.text_type)
True
>>> bytes is str
True
>>> b'foo'[0]
'f'

Python 3:
>>> type('foo')
<class 'str'>
>>> type(u'foo')
<class 'str'>
>>> type(b'foo')
<class 'bytes'>
>>> isinstance('foo', six.text_type)
True
>>> isinstance(b'foo', six.text_type)
False
>>> bytes is str
False
>>> b'foo'[0]
102

==== tox/testr error: db type could not be determined ====

The "db type could not be determined" error comes from .testrepository/times.dbm used by testr.

Workaround: "rm -rf .testrepository/" and then run "tox -e py34" before running "tox -e py27". You only have to do this once. The problem only occurs with "tox -e py34" when .testrepository/ was created by Python 2.

== Python 3 Status of OpenStack projects ==

=== Common Libraries (Oslo Projects) ===

For the list of Common Libraries, see http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml#n160

{|class="wikitable sortable"
|-
! Project !! Python 3 compatibility !! Comment
|-
| [https://pypi.python.org/pypi/cliff cliff] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.concurrency oslo.concurrency] || style="background-color: lightgreen;" | Yes ||
|-/
| oslo-incubator || style="background-color: lightgreen;" | Yes || py34 gate is voting
|-
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.context oslo.context] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.db oslo.db] || style="background-color: lightgreen;" | Yes ||
PyMySQL driver is now used by default for MySQL. setup.cfg contains the Python 3 classifier.
|-
| [https://pypi.python.org/pypi/oslo.i18n oslo.i18n] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.log oslo.log] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color: orange;" | Yes, except of Qpid & AMQP drivers. py34 is not more voting because of [//bugs.launchpad.net/ubuntu/+source/python3.4/+bug/1367907 ubuntu still uses 3.4.0 which has a severe bug] || oslo.messaging 1.11.0 fully works on Python 3, except of Qpid and AMQP 1.0 transports which are Python 2 only. The py34 gate is voting.
|-
| [https://pypi.python.org/pypi/oslo.middleware oslo.middleware] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.serialization oslo.serialization] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || ? || The project only contains two short .py files, it looks to be Python 3 compatible. Is Sphinx Python 3 compatible?
|-
| [https://pypi.python.org/pypi/oslotest oslotest] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.versionedobjects oslo.versionedobjects] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color: lightgreen;" | Yes || Supports Python 3 since oslo.vmware 0.13.0
|-
| [https://pypi.python.org/pypi/oslo.utils oslo.utils] || style="background-color: lightgreen;" | Yes ||
|-
| pylockfile || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/stevedore stevedore] || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color: lightgreen;" | Yes ||
|}

=== Development tools ===

{|class="wikitable sortable"
|-
! Project !! Python 3 compatibility !! Comment
|-
| [https://pypi.python.org/pypi/cookiecutter cookiecutter] || style="background-color: lightgreen;" | yes ||
|-
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color: lightgreen;" | yes || py33 gate is not voting
|-
| [https://pypi.python.org/pypi/pbr pbr] || style="background-color: lightgreen;" | yes ||
|-
| stackforge/python-jenkins || style="background-color: lightgreen;" | yes || py33 gate is voting
|-
| openstack-infra/jenkins-job-builder || style="background-color: orange;" | partial || https://review.openstack.org/172238
|}

=== OpenStack clients ===

{|class="wikitable sortable"
|-
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment
|-

| [https://pypi.python.org/pypi/keystonemiddleware keystonemiddleware] || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | Yes, but [https://review.openstack.org/206107 test using python-memcached are skipped on Python 3] || style="background-color: lightgreen;" | Yes || python-memcached || keystonemiddleware 1.6.1 supports Python 3. python-memcached 1.56 now supports Python 3.
|-

| [https://pypi.python.org/pypi/python-barbicanclient python-barbicanclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||
|-

| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color:lightgreen" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || ||
|-

| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||
|-

| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPII || ||
|-

| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || ||
|-

| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: lightgreen" | OK || style="background-color: lightgreen" | Voting || style="background-color: lightgreen" | Yes || || As of 0.9
|-

| [https://pypi.python.org/pypi/python-saharaclient python-savannaaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || style="background-color: lightgreen" | Yes || || [https://review.openstack.org/#/c/73128/ python-savannaclient: enable the py33 gates] abandonned: "We haven't enough tests to ensure that savanna client works correctly on py33, so, it's kind of premature step. We already have py33 and pypy jobs in experimental pipeline."
|-

| [https://pypi.python.org/pypi/python-saharaclient python-saharaclient] || style="background-color: lightgreen;" | OK || style="background-color: lightgreen" | Voting || style="background-color: lightgreen" | On PyP || ||
|-

| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | [https://pypi.python.org/pypi/python-swiftclient/ On PyPI] || ||
|-

| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||
|-

| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||

|-

| [https://pypi.python.org/pypi/python-designateclient python-designateclient] || style="background-color: orange;" | work in progress || style="background-color: orange;" | No || style="background-color: red;" | No || ||

|}

=== OpenStack Libraries ===

{|class="wikitable sortable"
|-
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Comment
|-
| [https://pypi.python.org/pypi/glance_store glance_store] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | voting || style="background-color: lightgreen;" | Yes || glance_store 0.7.0 added the Python 3 support.
|-
| [https://pypi.python.org/pypi/os-brick os-brick] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | voting || style="background-color: lightgreen;" | Yes || os-brick 0.3.2 added Python 3 support.

|-
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | voting || style="background-color:lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | voting || style="background-color: lightgreen;" | Yes ||
|-
|}

=== Dependencies ===

There are two ways to express dependencies that should differ between Python 2.x and 3.x. We currently support using multiple requirements files, and will shortly be supporting and recommending the use of environment markers. Recent releases of all the tools in our ecosystem are needed for environment markers.

==== requirements-py3.txt ====

This approach generates a different requirements list depending on what Python version is running when the egg info is created (which goes into wheels and sdists). As a consequence this can't work with tox, publishing sdists to PyPI, or wheels. It is however useful for anything installing from git. To use it, create two requirements files:

* requirements-py2.txt: all dependencies for Python 2 (not only dependencies specific to Python 2)
* requirements-py3.txt: all dependencies for Python 3 (not only dependencies specific to Python 3)
* (same for test-requirements.txt)

You have to edit tox.ini to specify the right requirements file. Extract of a tox.ini file:

<pre>
...
[testenv:py33]
deps = -r{toxinidir}/requirements-py3.txt
-r{toxinidir}/test-requirements-py3.txt
...
</pre>

==== Environment markers ====

Environment markers provide a mini-language to express when a dependency is relevant. We hope the next release of pbr will support them. Recent pip, setuptools and wheel versions do support them, but until pbr exports the requirements metadata with them intact and structured correctly for setuptools, we can't use them anywhere.

* PEP 0345 -- Metadata for Python Software Packages 1.2 (accepted): [https://www.python.org/dev/peps/pep-0345/#environment-markers Environment markers]
* PEP 0426 -- Metadata for Python Software Packages 2.0 (draft): [https://www.python.org/dev/peps/pep-0426/#environment-markers Environment markers]

When placed in a requirements.txt file, the markers are interpreted directly by pip in `pip install -r requirements.txt`, and indirectly via setuptools when installing an sdist or wheel (including `pip install .`. For instance:
<pre>
six
futures; python_version < '3.2'
</pre>

See also:
* [https://review.openstack.org/#/c/184328/ oslo.db: Refactor deps to use extras and env markers] (require pip 7, not released yet)
* pip: [https://github.com/pypa/pip/pull/1472 Fix issue #1433: parse requirements in markers #1472]
* setuptools: pkg_resources doesn't understand environment markers: [https://bitbucket.org/pypa/wheel/issue/139/wheel-should-support-pep-426-environment pkg_resources.parse_requirements() raises ValueError]
* pbr: [https://review.openstack.org/#/c/181795/ Teach pbr to read extras and env markers], [https://review.openstack.org/#/c/165015/ Handle PEP426 markers] (abandoned)
* wheel: [https://bitbucket.org/pypa/wheel/issue/139/wheel-should-support-pep-426-environment wheel should support PEP 426 environment markers] (closed), wheel supports environment markers in extra requirements
* tox: [https://bitbucket.org/hpk42/tox/issue/236/tox-must-create-the-source-distribution tox must create the source distribution with the Python of the virtual environment], [https://bitbucket.org/hpk42/tox/pull-request/142/issue-236-subcommand_test-rebuilds-sdist/ Issue #236: subcommand_test() rebuilds sdist with venv python] (declined)

Oslo Messaging issue:
* futures is not needed on Python 3, since concurrent.futures is now part of the Pyhon stdlib since Python 3.2
* [https://review.openstack.org/#/c/172135/ futures was added to requirements-py3.txt] because tox creates the source tarball using Python 2 which reads requirements.txt
* The correct fix would be to use environment markers on futures

==== Dependencies status ====

[https://caniusepython3.com/check/4fd5dda2-b1f1-4db4-a636-67cd3276cb6a Porting status] for [https://github.com/openstack/requirements/blob/master/global-requirements.txt global-requirement.txt].

{|class="wikitable sortable"
|-
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment
|-

| [https://pypi.python.org/pypi/boto boto] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || || See https://github.com/boto/boto3 (experimental) <- This seems dead, and https://github.com/boto/boto works with Python 3.x (since 2.32).
|-

| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || || Requirements upgraded: https://review.openstack.org/94357
|-

| [https://pypi.python.org/pypi/tablib tablib ] || style="background-color: lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || || setup.py installs tablib.packages.markup which is not compatible with Python 2. setup.py displays a warning on compiling the bytecode of markup.py.

tablib is needed by cliff-tablib, itself needed by python-neutronclient
|-

| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || ||
As of 1.1.6
|-

| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60
|-

| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)
|-

| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:lightgreen;" | Yes || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || || eventlet 0.17.3 now fully support Python 3
|-

| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(
|-

| [https://pypi.python.org/pypi/libvirt-python libvirt-python] || style="background-color: lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || ||
|-

| [https://pypi.python.org/pypi/mysql-python MySQL-Python] || style="background-color:orange;" | No, but mysqlclient or PyMySQL are compatible || N/A || style="background-color: orange;" | No || ||
Patches:
* [https://review.openstack.org/#/c/179745/ Add mysqlclient dependency]

MySQL-Python: [https://github.com/farcepest/MySQLdb1/pulls 2 pending pull requests for Python 3].

INADA Naoki, the maintainer of PyMySQL, wrote [https://pypi.python.org/pypi/mysqlclient mysqlclient]: fork of MySQL-Python using libmysqlclient.so which is Python 3 compatible. He recommends to use mysqlclient instead of MySQL-python even on Python 2. mysqlclient fixes some bugs, build issues and it support Python 3. For example: support MariaDB's libmysqlclient.so, support microsecond in TIME column.

PyMySQL is compatible with Python 3: see [https://wiki.openstack.org/wiki/PyMySQL_evaluation PyMySQL evaluation] discussion to eventually replace mysql-python with PyMySQL in OpenStack.
|-

| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too. Python 3 support as of 0.10.4. Pushed to requirements: https://review.openstack.org/94358 .
|-

| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3
|-

| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||
* nose-exclude (tests only)
* openstack.nose-plugin
||
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247
* https://review.openstack.org/#/c/80956/
|-
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9
|-
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||
|-

| [https://pypi.python.org/pypi/pam pam] || style="background-color:orange;" | No for pam, Yes for simplepam || NA || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3
|-

| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Requirements upgraded: https://review.openstack.org/#/c/81132/
|-

| [https://pypi.python.org/pypi/paste paste] || style="background-color:lightgreen;" | Yes || style="background-color: lightgrey;" | N/A || style="background-color: lightgreen;" | Yes || || Compatible with Python 3 since Paste 2.0
|-

| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:lightgreen;" | Yes || style="background-color:lightgreen;" | Yes || style="background-color:lightgreen;" | Yes || ||
|-

| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead. See [https://pypi.python.org/pypi/ldap3 ldap3] which is Python 3 compatible (license: LGPL, one of the optional deps is BSD-4-Clause).
|-

| [https://pypi.python.org/pypi/python-memcached python-memcached] || style="background-color:lightgreen;" | Yes || style="background-color:lightgreen;" | N/A || style="background-color:lightgreen;" | Yes || ||
python-memcached 1.56 includes pull request: [https://github.com/linsomniac/python-memcached/pull/67 Port memcache to Python 3 #67].

Julien Danjou ported [https://pypi.python.org/pypi/pymemcache pymemcache] to Python 3, another memcached client, he suggests to use this one instead
|-

| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Required if using oslo.messaging's qpid backend. That is the old pure python amqp 0-10 client library. That only speaks 0-10 and works with qpidd broker. It is used by the impl_qpid.py driver in oslo.messaging. That client is legacy - it isn't being ported to use AMQP 1.0.
|-

| [https://pypi.python.org/pypi/pyngus pyngus] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||
pyngus is required if using oslo.messaging's AMQP 1.0 driver. pyngus is a wrapper for proton and proton has not been ported to python 3 yet: see [https://issues.apache.org/jira/browse/PROTON-490 Proton issue #490].

[https://pypi.python.org/pypi/python-qpid-proton Proton] is a C library with various bindings in other languages. The language bindings are auto generated via the SWIG tool, plus a the python wrapper bits. All the python stuff is found under proton-c/bindings/python in the [https://git-wip-us.apache.org/repos/asf/qpid-proton.git git tree]. The first bit of work is to refactor the SWIG file cproton.i to work for both Python 2 and 3.
|-

| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||
|-
| [https://pypi.python.org/pypi/Routes Routes] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || || Routes 2.2 works Python 2 and 3 using a single code base.
|-
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI || ||
|-
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: red;" | No || ||
|-
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||
|-
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||
|-
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | N/A || style="background-color: lightgreen;" | Yes || || python3 works with the 0.7.0 version released on Jul 23, 2015
|-

|}

=== OpenStack applications ===

{| class="wikitable sortable"
|-
! Project !! Python 3 compatibility !! py34 gate !! Comments
|-

| [https://pypi.python.org/pypi/gnocchi gnocchi] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes ||
|-
| [https://pypi.python.org/pypi/ironic ironic] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes ||
[http://lists.openstack.org/pipermail/openstack-dev/2015-May/064049.html Python 3.4 unit tests are now being run for openstack/ironic. The unit tests are a voting job.]
Thanks to Victor Sergeyev for all of his work to update the Ironic code to make it pass the unit tests using Python 3.4: [https://review.openstack.org/#/c/156192/ Run tests in py34 environment]
|-
| [https://pypi.python.org/pypi/rally Rally] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | voting || A big thank to Andrey Kurilin for a lot of work in this direction
|-

| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
Victor Stinner is working on porting Ceilometer to Python 3. Patches:
* [https://review.openstack.org/#/q/status:open+project:openstack/ceilometer+branch:master+topic:py3,n,z Patches with py3 topic]
|-

| [https://pypi.python.org/pypi/cinder cinder] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
[https://blueprints.launchpad.net/cinder/+spec/cinder-python3 cinder-python3] blueprint (written by Victor Stinner) has been accepted for Liberty.

Patches: [https://review.openstack.org/#/q/project:openstack/cinder+topic:bp/cinder-python3,n,z bp/cinder-python3 topic].
|-

| [https://github.com/openstack/manila manila] || style="background-color: orange;" | work in progress || style="background-color: orange;" | non-voting ||
[https://blueprints.launchpad.net/manila/+spec/py3-compatibility py3-compatibility] blueprint.

non-voting py34 CI job runs in 'check' pipeline for each Manila gerrit commit. It fails because py3 support fixes are in progress [L-2].
Patches: [https://review.openstack.org/#/q/project:openstack/manila+topic:bp/py3-compatibility,n,z bp/py3-compatibility topic].
|-

| [https://pypi.python.org/pypi/glance glance] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
Victor Stinner is working on porting Glance to Python3.

Patches: [https://review.openstack.org/#/q/status:open+project:openstack/glance+branch:master+topic:py3,n,z Patches for glance (topic: py3)].

test_migrations is blocked by a bug in oslo.db ([https://review.openstack.org/#/c/206476/ Fix test_migrations on Python 3]) / sqlalchemy-migrate ([https://review.openstack.org/#/c/206478/ Add VerNum.__index__() for Python 3 support]).
|-

| [https://pypi.python.org/pypi/heat heat] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
[http://specs.openstack.org/openstack/heat-specs/specs/liberty/heat-python34-support.html Python34 Support] spec (by Sirushti Murugesan) accepted for Liberty.

Patches: [https://review.openstack.org/#/q/topic:bp/heat-python34-support,n,z bp/heat-python34-support topic].
|-

| [https://pypi.python.org/pypi/keystone keystone] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
The spec [http://specs.openstack.org/openstack/keystone-specs/specs/liberty/python3.html Add spec for python-3 compatibility] (by Morgan Fainberg) was accepted for Liberty. See also the blueprint [https://launchpad.net/keystone/+spec/python3 python3] started by Dolph Mathews on 2014-07-22; David Stanek is working on it.

Patches: [https://review.openstack.org/#/q/project:openstack/keystone+branch:master+topic:bp/python3,n,z bp/python3 topic].

[https://github.com/rohe/pysaml2/ pysaml2] 3.0 now supports Python 3.

Aslo blocked by [https://pypi.python.org/pypi/ldappool ldappool] and [https://pypi.python.org/pypi/python-ldap python-ldap]. [https://github.com/pyldap/pyldap/ pyldap] is a fork of python-ldap which adds Python 3 support. python-ldap may be replaced with [https://pypi.python.org/pypi/ldap3 ldap3], but it requires more work.
|-

| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:orange;" | work in progress || style="background-color: lightgreen;" | voting ||
[https://review.openstack.org/#/c/172962/ Porting to Python 3] spec (by Cyril Roelandt) accepted for Liberty, port in progress. Patches:
* [https://review.openstack.org/#/q/project:openstack/neutron+branch:master+topic:bp/neutron-python3,n,z Patches of the blueprint neutron-python3]
|-

| [https://pypi.python.org/pypi/nova nova] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
[http://specs.openstack.org/openstack/nova-specs/specs/liberty/approved/adding-python34-support-to-nova.html Adding Python 3.4 support to Nova] spec (by Victor Stinner) accepted for Liberty. Port in progress.

Patches:
* [https://review.openstack.org/#/q/project:openstack/nova+branch:master+topic:bp/nova-python3,n,z master (bp/nova-python3)].

See also [https://etherpad.openstack.org/p/liberty-nova-priorities-tracking Py3.4 Compatibility in Liberty nova priorities tracking].
|-

| [https://pypi.python.org//pypi/sahara sahara] || style="background-color: orange;" | work in progress || style="background-color: lightgreen;" | voting ||
|-

| [https://pypi.python.org/pypi/swift swift] || style="background-color: orange;" | work in progress || style="background-color: orange;" | non voting (need to [https://review.openstack.org/#/c/199034/ fix tox -e py34] ||
swift now depends on six.(Jaivish Kothari is also working on this)

Patches:
* [https://review.openstack.org/#/q/project:openstack/swift+branch:master+topic:py3,n,z Patches with the topic py3]

Dependencies:
* dnspython: use dnspython3 on Python 3 ([https://review.openstack.org/#/c/200416/ not in global requirements yet])
* PyEClib: [https://bitbucket.org/kmgreen2/pyeclib/issues/69/need-a-release-108-with-python-3-fixes waiting for the release 1.0.8]
|-

| [https://pypi.python.org/pypi/horizon horizon] || style="background-color: orange;" | work in progress || style="background-color: red;" | No ||
George Peristerakis wrote [https://blueprints.launchpad.net/horizon/+spec/porting-python3 Make Horizon python3.4 compatible].

Patches:
* [https://review.openstack.org/#/q/topic:bp/porting-python3+project:openstack/horizon,n,z bp/porting-python3 topic]
* [https://review.openstack.org/#/c/178342/ Add py34 env] by lokesh s
* [https://review.openstack.org/#/c/183365/ Made the class Column Python 3 compatible] by Emmanuel Rousselle
* [https://review.openstack.org/#/c/183367/ Made HorizonComponent class Python3 compatible] (merged) by Eric Lafontaine
|-

| [https://pypi.python.org/pypi/designate designate] || style="background-color: orange;" | work in progress || style="background-color: red;" | No || Pradeep is working on the Python 3 port.
|-

|}

=== Functional tests ===

Doug Hellmann wrote a cross-project specification to run integration tests on Python 3: [https://review.openstack.org/#/c/177375/ Enabling Python 3 for Application Integration Tests] (spec approved for Liberty).

Patch for DevStack: [https://review.openstack.org/#/c/181165/ Enable optional Python 3 support].

== Status of Python 3 in Linux distributions ==

* ArchLinux already [https://www.archlinux.org/news/python-is-now-python-3 switched to Python 3 by default] in 2011
* Ubuntu wants to remove Python 2 from the default installation: see [http://www.phoronix.com/scan.php?page=news_item&px=Ubuntu-16.04-Python-Plans Ubuntu Plans For Python 3 By Default For Ubuntu 16.04 LTS]
* Fedora schedules the switch in Fedora 23 (october 2015): [https://fedoraproject.org/wiki/Changes/Python_3_as_Default Python 3 as the Default Implementation]
* RHEL: [https://www.softwarecollections.org/en/scls/rhscl/python33/ Python 3.3] and [https://www.softwarecollections.org/en/scls/rhscl/rh-python34/ Python 3.4] are available on RHEL6 and RHEL7 using SCL.
* CentOS: [https://www.softwarecollections.org/en/scls/rhscl/python33/ Python 3.3] and [https://www.softwarecollections.org/en/scls/rhscl/rh-python34/ Python 3.4] are available on CentOS 6 and 7 using SCL (no Red Hat subscription needed, CentOS has its own flavor of SCL)
* In Debian, the plan is to deprecate Python 2 in Stretch (aka: Debian 9, the next Stable Debian after Jessie) and to completely remove Python 2 for the Buster release of Debian (aka: Debian 10, to be release in approximatively 2019).

== Reports at OpenStack Summits ==

* Liberty Summit
** Cross-project session on enabling app integration testing: https://etherpad.openstack.org/p/liberty-cross-project-python3
* Kilo Summit
** Doug Hellmann: "Moving our applications to Python 3" cross-project session
** Victor Stinner: "Port nova to Python 3", https://etherpad.openstack.org/p/liberty-nova-summit-ideas
* Juno summit notes: https://etherpad.openstack.org/p/juno-cross-project-future-of-python (Oslo) and https://etherpad.openstack.org/p/juno_swift_python3 (Swift)
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3

== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==

Enovance organized a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday) 2014. See the page [[Python3/SprintPycon2014]].

Manila/Meetings

2015-05-13T17:11:18Z

Vponomaryov: /* Next meeting */

Manila/Meetings

2015-05-13T16:47:47Z

Vponomaryov: /* Next meeting */

Manila/docs/HOWTO use manila with horizon

2015-04-21T11:42:50Z

Vponomaryov: /* Get custom horizon */

= How to run horizon with manila =

== Kilo ==
The manila UI has been converted to horizon plugin in Kilo release. Follow the installation instructions at https://github.com/hp-storage/manila-ui/blob/master/README.rst to have Horizon support for Kilo version of Manila.

== Juno ==
If you use Juno version of Manila, then you can use only fork of Horizon with appropriate changes for Manila support.

=== Get custom horizon ===

$ git clone -b manila_juno https://github.com/NetApp/horizon.git # Horizon of Juno version with Manila support. Should be used only with Juno version of manila server and manilaclient of 1.0.1 version - https://github.com/openstack/python-manilaclient/releases/tag/1.0.1

and

$ cd horizon
and

$ cp openstack_dashboard/local/local_settings.py.example openstack_dashboard/local/local_settings.py

open file '''''openstack_dashboard/local/local_settings.py'''''

find var "OPENSTACK_HOST" and set here address of host where devstack was installed

save and exit

=== Be sure, all dependencies are installed ===

run in horizon's dir

$ sudo python setup.py install
Also, need to install python-manilaclient

$ sudo pip install python-manilaclient==1.0.1
Run horizon
$ sudo python manage.py runserver
it will be available on 127.0.0.1:8000

to specify all interfaces, use:

$ sudo python manage.py runserver 0.0.0.0:8000
port can be set to any not used valid port value.

Manila/docs/HOWTO use manila with horizon

2015-04-21T09:45:14Z

Vponomaryov: /* How to run horizon with manila */

= How to run horizon with manila =

== Kilo ==
The manila UI has been converted to horizon plugin in Kilo release. Follow the installation instructions at https://github.com/hp-storage/manila-ui/blob/master/README.rst to have Horizon support for Kilo version of Manila.

== Juno ==
If you use Juno version of Manila, then you can use only fork of Horizon with appropriate changes for Manila support.

=== Get custom horizon ===

$ git clone -b manila_juno https://github.com/NetApp/horizon.git # Horizon of Juno version with Manila support. Should be used only with Juno version of manila server and manilaclient of 1.0.1 version - https://github.com/openstack/python-manilaclient/releases/tag/1.0.1

and

$ cd horizon
and

$ cp openstack_dashboard/local/local_settings.py.example openstack_dashboard/local/local_settings.py

open file openstack_dashboard/local/local_settings.py

find var "OPENSTACK_HOST" and set here address of host where devstack was installed

save and exit

=== Be sure, all dependencies are installed ===

run in horizon's dir

$ sudo python setup.py install
Also, need to install python-manilaclient

$ sudo pip install python-manilaclient==1.0.1
Run horizon
$ sudo python manage.py runserver
it will be available on 127.0.0.1:8000

to specify all interfaces, use:

$ sudo python manage.py runserver 0.0.0.0:8000
port can be set to any not used valid port value.

Manila/docs/HOWTO use manila with horizon

2015-03-12T14:47:51Z

Vponomaryov: /* Get custom horizon */

= How to run custom horizon with manila =
This page describes how to try manila with horizon. It can be done on any host, not exactly the same as devstack installation.

==Get custom horizon==

$ git clone -b manila_kilo https://github.com/NetApp/horizon.git # Horizon with support of Manila KIlo version, this one is recommended. Should be used with latest manilaclient (changes after version 1.0.2)
or
$ git clone -b manila_juno https://github.com/NetApp/horizon.git # Horizon of Juno version with Manila support. Should be used only with Juno version of manila server and manilaclient of 1.0.1 version - https://github.com/openstack/python-manilaclient/releases/tag/1.0.1
or
$ git clone -b manila_icehouse https://github.com/NetApp/horizon.git # Horizon of Icehouse version with Manila support. Old, not recommended
and
$ cd horizon
and
$ cp openstack_dashboard/local/local_settings.py.example openstack_dashboard/local/local_settings.py

open file '''''openstack_dashboard/local/local_settings.py'''''

find var "''OPENSTACK_HOST''" and set here address of host where devstack was installed

save and exit

==Be sure, all dependencies are installed==

run in horizon's dir

$ sudo python setup.py install

Also, need to install python-manilaclient

$ sudo pip install git+https://github.com/openstack/python-manilaclient.git

==Run horizon==
$ sudo python manage.py runserver

it will be available on 127.0.0.1:8000

to specify all interfaces, use:

$ sudo python manage.py runserver 0.0.0.0:8000

port can be set to any not used valid port value.

Manila/KiloDevstack

2015-02-26T07:49:57Z

Vponomaryov: /* Procedure */

== Intro ==

Here are instructions on how to use Devstack with the latest code from Manila to begin using Manila.

This procedure was tested and working as of 25 Feb 2015.

== Procedure ==

1. Clone devstack

$ git clone https://github.com/openstack-dev/devstack.git

2. Copy the sample local.conf into the root devstack directory

$ cd devstack ; cp samples/local.conf .

3. Edit/append local.conf to enable manila, and neutron, and to disable horizon, heat, and nova-network
$ cat << EOF >> local.conf

disable_service horizon

disable_service heat
disable_service h-eng
disable_service h-api
disable_service h-api-cfn
disable_service h-api-cw

disable_service n-net

enable_service neutron
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta

enable_plugin manila https://github.com/openstack/manila

Q_PLUGIN=ml2
ENABLE_TENANT_VLANS=True
ML2_VLAN_RANGES=physnet1:100:200
PHYSICAL_NETWORK=physnet1
OVS_PHYSICAL_BRIDGE=br-eth1
Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch
Q_ML2_PLUGIN_TYPE_DRIVERS=vlan,vxlan
EOF

4. Run stack.sh as normal; When it is complete, the manila processes should be started.

Manila/KiloDevstack

2015-02-25T20:31:40Z

Vponomaryov: /* Procedure */

== Intro ==

Here are instructions on how to use Devstack with the latest code from Manila to begin using Manila.

This procedure was tested and working as of 25 Feb 2015.

== Procedure ==

1. Clone devstack

$ git clone https://github.com/openstack-dev/devstack.git

2. Copy the sample local.conf into the root devstack directory

$ cd devstack ; cp samples/local.conf .

3. Edit/append local.conf to enable manila, and neutron, and to disable horizon, heat, and
$ cat << EOF >> local.conf

disable_service horizon

disable_service heat
disable_service h-eng
disable_service h-api
disable_service h-api-cfn
disable_service h-api-cw

disable_service n-net

enable_service neutron
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta

enable_plugin manila https://github.com/openstack/manila

Q_PLUGIN=ml2
ENABLE_TENANT_VLANS=True
ML2_VLAN_RANGES=physnet1:100:200
PHYSICAL_NETWORK=physnet1
OVS_PHYSICAL_BRIDGE=br-eth1
Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch
Q_ML2_PLUGIN_TYPE_DRIVERS=vlan,vxlan
EOF

4. Run stack.sh as normal; When it is complete, the manila processes should be started.

Manila/specs/scenario-tests

2015-02-23T17:09:09Z

Vponomaryov: /* 3. Relations between source shares and child shares */

== Scenario test cases for Manila project ==

Blueprint: https://blueprints.launchpad.net/manila/+spec/scenario-tests

===Dictionary===

{| class="wikitable"
|-
! Definition !! Meaning
|-
! UVM !! User VM. It is used as client machine for mounting shares.
|-
! SSH !! Secure SHell. It is used for connection to UVMs.
|}

===Prerequisites===
- Depending on share driver mode, it can be required to create share-network w/o security-services.
- Depending on protocol, its versions and access type, “mount” operations should be defined explicitly, scenario tests assume it as predefined and known. Hence, scenarios do not include difference between access type (IP, User, Cert). Due to this scenario tests should be data driven by “access_type”, “access_proto”, “access_level” and “mount command with all expected options”.
- ''Italic texts'' depend on specific implementations and can be unsupported by share backends.
- All user VMs should be separate VMs from host of shares. They should have network connectivity with share host. User VMs should be built with image that has shared file systems clients.
- Share host should have open ports for SSH protocol and protocols of shared file systems.

===List of scenarious===
{| class="wikitable"
|-
! № !! Name !! Description
|-
! 1!! Share access and file operations !! This test verifies basic file operations with shares.
|-
! 2!! Single/Group access operations !! This test verifies basic single or group access operations.
|-
! 3!! Relations between source shares and child shares !! This test verifies that child share does not inherit access rules from parent and is different in general.
|}

==== 1. Share access and file operations ====
{| class="wikitable"
|-
! № !! Action !! Result
|-
! 1 !! Create UVM !! ok, created
|-
! 2 !! Create share S !! ok, created
|-
! 3 !! SSH to UVM !! ok, connected
|-
! 4 !! Try mount S to UVM !! fail, access denied
|-
! 5 !! Provide RO access !! ok
|-
! 6 !! Try mount S to UVM !! ok, mounted
|-
! 7 !! ''Try create files on S'' !! ''fail, access denied''
|-
! 8 !! ''Unmount S from UVM'' !! ''ok''
|-
! 9 !! ''Remove RO access'' !! ''ok, removed''
|-
! 10 !! ''Try mount S to UVM'' !! ''fail, access denied''
|-
! 11 !! Provide RW access !! ok, provided
|-
! 12 !! Try mount S to UVM !! ok, mounted
|-
! 13 !! Try write files to S !! ok, written
|-
! 14 !! Try read files from S !! ok, read
|-
! 15 !! Try delete files on S !! ok, deleted
|-
! 16 !! Unmount S from UVM !! ok, unmounted
|-
! 17 !! Delete S !! ok, deleted
|-
! 18 !! Try mount S !! fail, not found
|-
! 19 !! Delete UVM !! ok, deleted
|}

==== 2. Single/Group access operations ====
{| class="wikitable"
|-
! № !! Action !! Result
|-
! 1 !! Create user UVM1 using network1 !! ok, created
|-
! 2 !! Create user UVM2 using network2 !! ok, created
|-
! 3 !! Create share S !! ok, created
|-
! 4 !! Add access to network1 !! ok, added
|-
! 5 !! SSH to UVM1 !! ok, connected
|-
! 6 !! Try mount S from UVM1 !! ok, mounted
|-
! 7 !! SSH to UVM2 !! ok, connected
|-
! 8 !! Try mount S from UVM2 !! fail, access denied
|-
! 9 !! Add access to network2 !! ok, added
|-
! 10 !! Try mount S from UVM2 !! ok, mounted (Two VMs have it mounted at once)
|-
! 11 !! Unmount S on UVM1 !! ok, unmounted
|-
! 12 !! Unmount S on UVM2 !! ok, unmounted
|-
! 13 !! Delete UVM1 !! ok, deleted
|-
! 14 !! Delete UVM2 !! ok, deleted
|-
! 15 !! Delete S !! ok, deleted
|}

==== 3. Relations between source shares and child shares ====
{| class="wikitable"
|-
! № !! Action !! Result
|-
! 1 !! Create UVM !! ok, created
|-
! 2 !! Create share S1 !! ok, created
|-
! 3 !! Provide RW access to S1 !! ok, provided
|-
! 4 !! SSH to UVM !! ok, connected
|-
! 5 !! Try mount S1 to UVM !! ok, mounted
|-
! 6 !! Create “file1” !! ok, created
|-
! 7 !! Create snapshot from S, called SS1 !! ok, created
|-
! 8 !! Create share from SS1, called S2 !! ok, created
|-
! 9 !! Try mount S2 !! fail, access denied (Here we test that child share did not get access rules form parent share)
|-
! 10 !! Provide RW access to S2 !! ok, provided
|-
! 11 !! Try mount S2 !! ok, mounted
|-
! 12 !! List files on S2 !! only one file exist - “file1”
|-
! 13 !! Create file2 in S2 !! ok, file created
|-
! 14 !! List files on S1 !! only one file exist - “file1”
|-
! 15 !! List files on S2 !! two files exist - “file1” and “file2”
|-
! 16 !! Unmount S1 and S2 !! ok, unmounted
|-
! 17 !! Delete S2 !! ok, deleted
|-
! 18 !! Delete SS1 !! ok, deleted
|-
! 19 !! Delete S1 !! ok, deleted
|-
! 20 !! Delete UVM !! ok, deleted
|}

Manila/specs/scenario-tests

2015-02-23T12:15:33Z

Vponomaryov: /* List of scenarious */

== Scenario test cases for Manila project ==

Blueprint: https://blueprints.launchpad.net/manila/+spec/scenario-tests

===Dictionary===

{| class="wikitable"
|-
! Definition !! Meaning
|-
! UVM !! User VM. It is used as client machine for mounting shares.
|-
! SSH !! Secure SHell. It is used for connection to UVMs.
|}

===Prerequisites===
- Depending on share driver mode, it can be required to create share-network w/o security-services.
- Depending on protocol, its versions and access type, “mount” operations should be defined explicitly, scenario tests assume it as predefined and known. Hence, scenarios do not include difference between access type (IP, User, Cert). Due to this scenario tests should be data driven by “access_type”, “access_proto”, “access_level” and “mount command with all expected options”.
- ''Italic texts'' depend on specific implementations and can be unsupported by share backends.
- All user VMs should be separate VMs from host of shares. They should have network connectivity with share host. User VMs should be built with image that has shared file systems clients.
- Share host should have open ports for SSH protocol and protocols of shared file systems.

===List of scenarious===
{| class="wikitable"
|-
! № !! Name !! Description
|-
! 1!! Share access and file operations !! This test verifies basic file operations with shares.
|-
! 2!! Single/Group access operations !! This test verifies basic single or group access operations.
|-
! 3!! Relations between source shares and child shares !! This test verifies that child share does not inherit access rules from parent and is different in general.
|}

==== 1. Share access and file operations ====
{| class="wikitable"
|-
! № !! Action !! Result
|-
! 1 !! Create UVM !! ok, created
|-
! 2 !! Create share S !! ok, created
|-
! 3 !! SSH to UVM !! ok, connected
|-
! 4 !! Try mount S to UVM !! fail, access denied
|-
! 5 !! Provide RO access !! ok
|-
! 6 !! Try mount S to UVM !! ok, mounted
|-
! 7 !! ''Try create files on S'' !! ''fail, access denied''
|-
! 8 !! ''Unmount S from UVM'' !! ''ok''
|-
! 9 !! ''Remove RO access'' !! ''ok, removed''
|-
! 10 !! ''Try mount S to UVM'' !! ''fail, access denied''
|-
! 11 !! Provide RW access !! ok, provided
|-
! 12 !! Try mount S to UVM !! ok, mounted
|-
! 13 !! Try write files to S !! ok, written
|-
! 14 !! Try read files from S !! ok, read
|-
! 15 !! Try delete files on S !! ok, deleted
|-
! 16 !! Unmount S from UVM !! ok, unmounted
|-
! 17 !! Delete S !! ok, deleted
|-
! 18 !! Try mount S !! fail, not found
|-
! 19 !! Delete UVM !! ok, deleted
|}

==== 2. Single/Group access operations ====
{| class="wikitable"
|-
! № !! Action !! Result
|-
! 1 !! Create user UVM1 using network1 !! ok, created
|-
! 2 !! Create user UVM2 using network2 !! ok, created
|-
! 3 !! Create share S !! ok, created
|-
! 4 !! Add access to network1 !! ok, added
|-
! 5 !! SSH to UVM1 !! ok, connected
|-
! 6 !! Try mount S from UVM1 !! ok, mounted
|-
! 7 !! SSH to UVM2 !! ok, connected
|-
! 8 !! Try mount S from UVM2 !! fail, access denied
|-
! 9 !! Add access to network2 !! ok, added
|-
! 10 !! Try mount S from UVM2 !! ok, mounted (Two VMs have it mounted at once)
|-
! 11 !! Unmount S on UVM1 !! ok, unmounted
|-
! 12 !! Unmount S on UVM2 !! ok, unmounted
|-
! 13 !! Delete UVM1 !! ok, deleted
|-
! 14 !! Delete UVM2 !! ok, deleted
|-
! 15 !! Delete S !! ok, deleted
|}

==== 3. Relations between source shares and child shares ====
{| class="wikitable"
|-
! № !! Action !! Result
|-
! 1 !! Create UVM !! ok, created
|-
! 2 !! Create share S1 !! ok, created
|-
! 3 !! Provide RW access to S1 !! ok, provided
|-
! 4 !! SSH to UVM !! ok, connected
|-
! 5 !! Try mount S1 to UVM !! ok, mounted
|-
! 6 !! Create “file1” !! ok, created
|-
! 7 !! Create snapshot from S, called SS1 !! ok, created
|-
! 8 !! Create share from SS1, called S2 !! ok, created
|-
! 9 !! Try mount S2 !! fail, access denied (Here we test that child share did not get access rules form parent share)
|-
! 10 !! Provide RW access to S2 !! ok, provided
|-
! 11 !! Try mount S2 !! ok, mounted
|-
! 12 !! List files on S2 !! only one file exist - “file1”
|-
! 13 !! Create file2 in S2 !! ok, file created
|-
! 14 !! List files on S1 !! only one file exist - “file1”
|-
! 15 !! List files on S2 !! two files exist - “file1” and “file2”
|-
! 16 !! Unmount S1 and S2 !! ok, unmounted
|-
! 17 !! Delete S2 !! ok, deleted
|-
! 18 !! Delete S1 !! ok, deleted
|-
! 19 !! Delete S1 !! ok, deleted
|-
! 20 !! Delete UVM !! ok, deleted
|}

Manila/specs/scenario-tests

2015-02-23T12:11:46Z

Vponomaryov: /* 3. Relations between source shares and child shares */

== Scenario test cases for Manila project ==

Blueprint: https://blueprints.launchpad.net/manila/+spec/scenario-tests

===Dictionary===

{| class="wikitable"
|-
! Definition !! Meaning
|-
! UVM !! User VM. It is used as client machine for mounting shares.
|-
! SSH !! Secure SHell. It is used for connection to UVMs.
|}

===Prerequisites===
- Depending on share driver mode, it can be required to create share-network w/o security-services.
- Depending on protocol, its versions and access type, “mount” operations should be defined explicitly, scenario tests assume it as predefined and known. Hence, scenarios do not include difference between access type (IP, User, Cert). Due to this scenario tests should be data driven by “access_type”, “access_proto”, “access_level” and “mount command with all expected options”.
- ''Italic texts'' depend on specific implementations and can be unsupported by share backends.
- All user VMs should be separate VMs from host of shares. They should have network connectivity with share host. User VMs should be built with image that has shared file systems clients.
- Share host should have open ports for SSH protocol and protocols of shared file systems.

===List of scenarious===
{| class="wikitable"
|-
! № !! Name !! Description
|-
! 1!! Share access and file operations !!
|-
! 2!! Single/Group access operations !!
|-
! 3!! Relations between source shares and child shares !!
|}

==== 1. Share access and file operations ====
{| class="wikitable"
|-
! № !! Action !! Result
|-
! 1 !! Create UVM !! ok, created
|-
! 2 !! Create share S !! ok, created
|-
! 3 !! SSH to UVM !! ok, connected
|-
! 4 !! Try mount S to UVM !! fail, access denied
|-
! 5 !! Provide RO access !! ok
|-
! 6 !! Try mount S to UVM !! ok, mounted
|-
! 7 !! ''Try create files on S'' !! ''fail, access denied''
|-
! 8 !! ''Unmount S from UVM'' !! ''ok''
|-
! 9 !! ''Remove RO access'' !! ''ok, removed''
|-
! 10 !! ''Try mount S to UVM'' !! ''fail, access denied''
|-
! 11 !! Provide RW access !! ok, provided
|-
! 12 !! Try mount S to UVM !! ok, mounted
|-
! 13 !! Try write files to S !! ok, written
|-
! 14 !! Try read files from S !! ok, read
|-
! 15 !! Try delete files on S !! ok, deleted
|-
! 16 !! Unmount S from UVM !! ok, unmounted
|-
! 17 !! Delete S !! ok, deleted
|-
! 18 !! Try mount S !! fail, not found
|-
! 19 !! Delete UVM !! ok, deleted
|}

==== 2. Single/Group access operations ====
{| class="wikitable"
|-
! № !! Action !! Result
|-
! 1 !! Create user UVM1 using network1 !! ok, created
|-
! 2 !! Create user UVM2 using network2 !! ok, created
|-
! 3 !! Create share S !! ok, created
|-
! 4 !! Add access to network1 !! ok, added
|-
! 5 !! SSH to UVM1 !! ok, connected
|-
! 6 !! Try mount S from UVM1 !! ok, mounted
|-
! 7 !! SSH to UVM2 !! ok, connected
|-
! 8 !! Try mount S from UVM2 !! fail, access denied
|-
! 9 !! Add access to network2 !! ok, added
|-
! 10 !! Try mount S from UVM2 !! ok, mounted (Two VMs have it mounted at once)
|-
! 11 !! Unmount S on UVM1 !! ok, unmounted
|-
! 12 !! Unmount S on UVM2 !! ok, unmounted
|-
! 13 !! Delete UVM1 !! ok, deleted
|-
! 14 !! Delete UVM2 !! ok, deleted
|-
! 15 !! Delete S !! ok, deleted
|}

==== 3. Relations between source shares and child shares ====
{| class="wikitable"
|-
! № !! Action !! Result
|-
! 1 !! Create UVM !! ok, created
|-
! 2 !! Create share S1 !! ok, created
|-
! 3 !! Provide RW access to S1 !! ok, provided
|-
! 4 !! SSH to UVM !! ok, connected
|-
! 5 !! Try mount S1 to UVM !! ok, mounted
|-
! 6 !! Create “file1” !! ok, created
|-
! 7 !! Create snapshot from S, called SS1 !! ok, created
|-
! 8 !! Create share from SS1, called S2 !! ok, created
|-
! 9 !! Try mount S2 !! fail, access denied (Here we test that child share did not get access rules form parent share)
|-
! 10 !! Provide RW access to S2 !! ok, provided
|-
! 11 !! Try mount S2 !! ok, mounted
|-
! 12 !! List files on S2 !! only one file exist - “file1”
|-
! 13 !! Create file2 in S2 !! ok, file created
|-
! 14 !! List files on S1 !! only one file exist - “file1”
|-
! 15 !! List files on S2 !! two files exist - “file1” and “file2”
|-
! 16 !! Unmount S1 and S2 !! ok, unmounted
|-
! 17 !! Delete S2 !! ok, deleted
|-
! 18 !! Delete S1 !! ok, deleted
|-
! 19 !! Delete S1 !! ok, deleted
|-
! 20 !! Delete UVM !! ok, deleted
|}

Manila/specs/scenario-tests

2015-02-23T12:06:16Z

Vponomaryov: /* 2. Single/Group access operations */

== Scenario test cases for Manila project ==

Blueprint: https://blueprints.launchpad.net/manila/+spec/scenario-tests

===Dictionary===

{| class="wikitable"
|-
! Definition !! Meaning
|-
! UVM !! User VM. It is used as client machine for mounting shares.
|-
! SSH !! Secure SHell. It is used for connection to UVMs.
|}

===Prerequisites===
- Depending on share driver mode, it can be required to create share-network w/o security-services.
- Depending on protocol, its versions and access type, “mount” operations should be defined explicitly, scenario tests assume it as predefined and known. Hence, scenarios do not include difference between access type (IP, User, Cert). Due to this scenario tests should be data driven by “access_type”, “access_proto”, “access_level” and “mount command with all expected options”.
- ''Italic texts'' depend on specific implementations and can be unsupported by share backends.
- All user VMs should be separate VMs from host of shares. They should have network connectivity with share host. User VMs should be built with image that has shared file systems clients.
- Share host should have open ports for SSH protocol and protocols of shared file systems.

===List of scenarious===
{| class="wikitable"
|-
! № !! Name !! Description
|-
! 1!! Share access and file operations !!
|-
! 2!! Single/Group access operations !!
|-
! 3!! Relations between source shares and child shares !!
|}

==== 1. Share access and file operations ====
{| class="wikitable"
|-
! № !! Action !! Result
|-
! 1 !! Create UVM !! ok, created
|-
! 2 !! Create share S !! ok, created
|-
! 3 !! SSH to UVM !! ok, connected
|-
! 4 !! Try mount S to UVM !! fail, access denied
|-
! 5 !! Provide RO access !! ok
|-
! 6 !! Try mount S to UVM !! ok, mounted
|-
! 7 !! ''Try create files on S'' !! ''fail, access denied''
|-
! 8 !! ''Unmount S from UVM'' !! ''ok''
|-
! 9 !! ''Remove RO access'' !! ''ok, removed''
|-
! 10 !! ''Try mount S to UVM'' !! ''fail, access denied''
|-
! 11 !! Provide RW access !! ok, provided
|-
! 12 !! Try mount S to UVM !! ok, mounted
|-
! 13 !! Try write files to S !! ok, written
|-
! 14 !! Try read files from S !! ok, read
|-
! 15 !! Try delete files on S !! ok, deleted
|-
! 16 !! Unmount S from UVM !! ok, unmounted
|-
! 17 !! Delete S !! ok, deleted
|-
! 18 !! Try mount S !! fail, not found
|-
! 19 !! Delete UVM !! ok, deleted
|}

==== 2. Single/Group access operations ====
{| class="wikitable"
|-
! № !! Action !! Result
|-
! 1 !! Create user UVM1 using network1 !! ok, created
|-
! 2 !! Create user UVM2 using network2 !! ok, created
|-
! 3 !! Create share S !! ok, created
|-
! 4 !! Add access to network1 !! ok, added
|-
! 5 !! SSH to UVM1 !! ok, connected
|-
! 6 !! Try mount S from UVM1 !! ok, mounted
|-
! 7 !! SSH to UVM2 !! ok, connected
|-
! 8 !! Try mount S from UVM2 !! fail, access denied
|-
! 9 !! Add access to network2 !! ok, added
|-
! 10 !! Try mount S from UVM2 !! ok, mounted (Two VMs have it mounted at once)
|-
! 11 !! Unmount S on UVM1 !! ok, unmounted
|-
! 12 !! Unmount S on UVM2 !! ok, unmounted
|-
! 13 !! Delete UVM1 !! ok, deleted
|-
! 14 !! Delete UVM2 !! ok, deleted
|-
! 15 !! Delete S !! ok, deleted
|}

==== 3. Relations between source shares and child shares ====
{| class="wikitable"
|-
! № !! Action !! Result
|-
! 1!! TBD !!
|-
! 2!! TBD !!
|}

Manila/specs/scenario-tests

2015-02-23T12:01:57Z

Vponomaryov: /* Scenario tests for Manila */

Manila/docs/HOWTO use tempest with manila

2015-02-17T19:28:32Z

Vponomaryov: /* Where do Tempest tests for Manila live? */

=Foreword=
This page describes how Tempest project for Manila project testing can be used.

It is assumed that user who reads this is acquainted with, at least, common information about Tempest:
http://docs.openstack.org/developer/tempest/
https://github.com/openstack/tempest

Also, it is assumed, that the following projects has been installed in test environmentː
- cinder
- glance
- keystone
- manila
- neutron
- nova

=Where do Tempest tests for Manila live?=
Manila stores its tests for Tempest in its repo here:
https://github.com/openstack/manila/tree/master/contrib/tempest

These are pluggable-like files which do not require changing of core Tempest files.
But the way of its pluggability is implemented as work-around for Tempest, which does not have pluggable functionality for 'exceptions', 'config' and 'clients' modules.
It is compatible with master branch of Tempest.

Just copy these files to Tempest dirː
$ TEMPEST_DIR=/path/to/tempest
$ MANILA_DIR=/path/to/manila
$ cp -rv ${MANILA_DIR}/contrib/tempest/tempest/* ${TEMPEST_DIR}/tempest

=Configuring Tempest before running tests=
Any test running with Tempest requires update of Tempest's config file located in
etc/tempest.conf
If it does not exist, just copy template "tempest.conf.sample"
cp tempest.conf.sample tempest.conf

Manila's config extension is defined in
https://github.com/openstack/manila/blob/master/contrib/tempest/tempest/config_share.py

Subsections below describe sections and options that can be updated with Tempest's config file.

==Cluster related options in general==
Required options for Manila tests are described below.

===Section "cli"===
====Option "enabled"====
Set it to
True
value, to be able to run Manila cli tests.

===Section "identity"===

====Option "uri"====
If using localhost it should look likeː
http://127.0.0.1:5000/v2.0/
Used for all tests, if v2 auth enabled.

====Option "uri_v3"====
If using localhost it should look likeː
http://127.0.0.1:5000/v3/

====Option "username"====
If using devstack installation it should have the following valueː
demo
====Option "password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'demo' user.

====Option "tenant_name"====
If using devstack installation it should have the following valueː
demo

====Option "alt_username"====
If using devstack installation it should have the following valueː
alt_demo

====Option "alt_password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'alt_demo' user.

====Option "alt_tenant_name"====
If using devstack installation it should have the following valueː
alt_demo

====Option "admin_username"====
If using devstack installation it should have the following valueː
admin

====Option "admin_password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'admin' user.

====Option "admin_tenant_name"====
If using devstack installation it should have the following valueː
admin

===Section "network"===
====Option "public_network_id"====
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron net-list

====Option "public_router_id"====
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron router-list

==Manila related options==

===Section "service_available"===
This section is defined within vanila Tempest. But "manila" option is defined within Manila's extension.
====Option "manila"====
typeː boolean
default valueː True
descriptionː Whether or not manila is expected to be available

===Section "share"===
This section is defined with Manila's extension.

====Option "catalog_type"====
typeː str
default valueː share
descriptionː Catalog type of the Share service. Should be changed only if it is changed in Manila itself.

====Option "endpoint_type"====
typeː str
default valueː publicURL
descriptionː The endpoint type to use for the share service. Expected values are 'public', 'admin', 'internal', 'publicURL', 'adminURL' and 'internalURL'.

====Option "multitenancy_enabled"====
typeː boolean
default valueː True
descriptionː This option is used to determine backend driver type, multitenant driver uses share-networks, but single-tenant doesn't.

Current singletenant drivers: LVM, GlusterFS, NetApp 7-mode
Current multitenant drivers: Generic, NetApp Cluster-mode (cmode)
Default driver in manila is Generic driver, so "multitenancy_enabled" option is enabled by default.

====Option "enable_protocols"====
typeː list
default valueː nfs,cifs
descriptionː The first value of the list is protocol by default, items of the list show enabled protocols.

Expected values are:
nfs - will be tested only NFS type of shares and common actions
cifs - will be tested only CIFS type of shares and common actions
nfs,cifs - will be tested both share types and common actions, for common actions will be used nfs protocol
cifs,nfs - will be tested both share types and common actions, for common actions will be used cifs protocol

====Option "enable_ip_rules_for_protocols"====
typeː list
default valueː nfs,cifs
descriptionː Whether allow running of ip rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "enable_sid_rules_for_protocols"====
typeː list
default valueː (empty)
descriptionː Whether allow running of sid (username/usergroup) rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "username_for_sid_rules"====
typeː str
default valueː Administrator
descriptionː What name should be used with sid rule tests? Specify it if 'sid rule' tests are enabled.

====Option "storage_protocol"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "alt_share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for alternative member creds. At the moment there are no tests, that use share_network with creds for alternative user.

====Option "admin_share_network_id"====
typeː str
default valueː
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for admin creds.

====Option "multi_backend"====
typeː boolean
default valueː False
descriptionː Does your environment has more than 1 backend or not? If yes, than you can set it to True to enable multibackend tests in tempest.

====Option "backend_names"====
typeː list
default valueː (empty)
descriptionː names of backends in Manila, that Tempest should use. Tempest will use first two values from list and will use it only if option "multi_backend" is Enabled.

====Option "build_interval"====
typeː int
default valueː 3
descriptionː time to wait between resend of requests.

====Option "build_timeout"====
typeː int
default valueː 500
descriptionː time to wait until raise of Exception with timeout.

=How to run tests=
==Run only cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.cli.*manila*\"

==Run only api tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share*\"

==Run only api and cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila*\"

==Run only api and cli tests with specified amount of threads==
export TEMPEST_CONCURRENCY=4
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila* -- --concurrency=$TEMPEST_CONCURRENCY\"
Above action will try to run Tempest in 4 threads, if your host does have at least 4 cores. By default it uses amount of threads equal to amount of available cores.

Manila tests are OK using concurrent threads.

==If something went wrong with test launch==
Exception may be raised on step where tests are gathered and parsed and as a result we would see something similar to the followingː
running testr
Non-zero exit code (2) from test listing. stdout='\xb3)\x01@d@Ytempest.api.share.admin.test_admin_actions.AdminActionsTest.test_force_delete_share[gate]<Z\x90"\xb3)\x01@g@\\
...
tempest.scenario.test_volume_boot_pattern6B\x1e\xcc', stderr=None
error: testr failed (3)
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list

======
Totals
======
Run: 0 in 0.0 sec.
- Passed: 0
- Skipped: 0
- Failed: 0
ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh tempest/api/share/test_extensions.py'
__________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
ERROR: venv: commands failed

There could be different causes for that, in most cases - dependency issues, when some dependency is not installed and can not be imported. Or python syntax error, if something were changed in code, etc...

As we can see, it does not show us real problem and we need dig root cause up. It can be done by a lot of means, for example, try nextː
nosetests -sv path/to/tests
or
.tox/venv/bin/python -m testtools.run discover -t ./ path/to/tests

=How to write own tests=

==Where to put new tests==
All tests, that cover admin API are stored in
tempest/api/share/admin/*
All tests, that cover non-admin API are stored in
tempest/api/share/*

==Add method as API caller==
All such methods are stored in
tempest/services/share/json/shares_client.py
XML is considered as deprecated and tests for it were removed, so, only json "client" is expected.

==Positive/Negative==
Positive and negative tests are separated into different modules, module with "negative" tests should have "_negative" suffix.
Example of module with positive testsː
tempest/api/share/test_shares.py
Example of module with negative testsː
tempest/api/share/test_shares_negative.py

==Resolving concurrency issue==
If some test is sensible to changes that can be done by other tests, this test should use isolated tenant, example of it can be found in quota tests, located in
tempest/api/share/admin/test_quotas.py

==Reusing resources==
If several tests have same dependencies, they should use same resources if possible, it can be created in "setUpClass" method. Example can be found hereː
tempest/api/share/test_shares_actions.py

==Utilization of resources==
Utilization stuff is defined in the following moduleː
tempest/api/share/base.py

Basic actionsː
1) All setUpClass methods should be decorated with "@test.safe_setup" decorator.
Reasonː if we get Exception on some step in setUpClass, tearDownClass won't start. So, this decorator runs 'tearDownClass' in any case.

2) Create resources that should be deleted with methods from base class.
correctː self.create_share() - will add resources to deletion queue
incorrectː self.shares_client.create_share() - won't add resource to deletion queue

Common infoː
All resources in deletion queue will be deleted in LIFO sequence - "last in first out" or "last created first deleted".

There are two deletion queuesː
- runs after each test
- runs after each test-suite (class with tests)
Be attentive setting up it.

Manila/docs/HOWTO use tempest with manila

2015-02-17T19:28:15Z

Vponomaryov: /* Configuring Tempest before running tests */

=Foreword=
This page describes how Tempest project for Manila project testing can be used.

It is assumed that user who reads this is acquainted with, at least, common information about Tempest:
http://docs.openstack.org/developer/tempest/
https://github.com/openstack/tempest

Also, it is assumed, that the following projects has been installed in test environmentː
- cinder
- glance
- keystone
- manila
- neutron
- nova

=Where do Tempest tests for Manila live?=
Manila stores its tests for Tempest in its repo here:
https://github.com/stackforge/manila/tree/master/contrib/tempest

These are pluggable-like files which do not require changing of core Tempest files.
But the way of its pluggability is implemented as work-around for Tempest, which does not have pluggable functionality for 'exceptions', 'config' and 'clients' modules.
It is compatible with master branch of Tempest.

Just copy these files to Tempest dirː
$ TEMPEST_DIR=/path/to/tempest
$ MANILA_DIR=/path/to/manila
$ cp -rv ${MANILA_DIR}/contrib/tempest/tempest/* ${TEMPEST_DIR}/tempest

=Configuring Tempest before running tests=
Any test running with Tempest requires update of Tempest's config file located in
etc/tempest.conf
If it does not exist, just copy template "tempest.conf.sample"
cp tempest.conf.sample tempest.conf

Manila's config extension is defined in
https://github.com/openstack/manila/blob/master/contrib/tempest/tempest/config_share.py

Subsections below describe sections and options that can be updated with Tempest's config file.

==Cluster related options in general==
Required options for Manila tests are described below.

===Section "cli"===
====Option "enabled"====
Set it to
True
value, to be able to run Manila cli tests.

===Section "identity"===

====Option "uri"====
If using localhost it should look likeː
http://127.0.0.1:5000/v2.0/
Used for all tests, if v2 auth enabled.

====Option "uri_v3"====
If using localhost it should look likeː
http://127.0.0.1:5000/v3/

====Option "username"====
If using devstack installation it should have the following valueː
demo
====Option "password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'demo' user.

====Option "tenant_name"====
If using devstack installation it should have the following valueː
demo

====Option "alt_username"====
If using devstack installation it should have the following valueː
alt_demo

====Option "alt_password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'alt_demo' user.

====Option "alt_tenant_name"====
If using devstack installation it should have the following valueː
alt_demo

====Option "admin_username"====
If using devstack installation it should have the following valueː
admin

====Option "admin_password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'admin' user.

====Option "admin_tenant_name"====
If using devstack installation it should have the following valueː
admin

===Section "network"===
====Option "public_network_id"====
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron net-list

====Option "public_router_id"====
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron router-list

==Manila related options==

===Section "service_available"===
This section is defined within vanila Tempest. But "manila" option is defined within Manila's extension.
====Option "manila"====
typeː boolean
default valueː True
descriptionː Whether or not manila is expected to be available

===Section "share"===
This section is defined with Manila's extension.

====Option "catalog_type"====
typeː str
default valueː share
descriptionː Catalog type of the Share service. Should be changed only if it is changed in Manila itself.

====Option "endpoint_type"====
typeː str
default valueː publicURL
descriptionː The endpoint type to use for the share service. Expected values are 'public', 'admin', 'internal', 'publicURL', 'adminURL' and 'internalURL'.

====Option "multitenancy_enabled"====
typeː boolean
default valueː True
descriptionː This option is used to determine backend driver type, multitenant driver uses share-networks, but single-tenant doesn't.

Current singletenant drivers: LVM, GlusterFS, NetApp 7-mode
Current multitenant drivers: Generic, NetApp Cluster-mode (cmode)
Default driver in manila is Generic driver, so "multitenancy_enabled" option is enabled by default.

====Option "enable_protocols"====
typeː list
default valueː nfs,cifs
descriptionː The first value of the list is protocol by default, items of the list show enabled protocols.

Expected values are:
nfs - will be tested only NFS type of shares and common actions
cifs - will be tested only CIFS type of shares and common actions
nfs,cifs - will be tested both share types and common actions, for common actions will be used nfs protocol
cifs,nfs - will be tested both share types and common actions, for common actions will be used cifs protocol

====Option "enable_ip_rules_for_protocols"====
typeː list
default valueː nfs,cifs
descriptionː Whether allow running of ip rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "enable_sid_rules_for_protocols"====
typeː list
default valueː (empty)
descriptionː Whether allow running of sid (username/usergroup) rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "username_for_sid_rules"====
typeː str
default valueː Administrator
descriptionː What name should be used with sid rule tests? Specify it if 'sid rule' tests are enabled.

====Option "storage_protocol"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "alt_share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for alternative member creds. At the moment there are no tests, that use share_network with creds for alternative user.

====Option "admin_share_network_id"====
typeː str
default valueː
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for admin creds.

====Option "multi_backend"====
typeː boolean
default valueː False
descriptionː Does your environment has more than 1 backend or not? If yes, than you can set it to True to enable multibackend tests in tempest.

====Option "backend_names"====
typeː list
default valueː (empty)
descriptionː names of backends in Manila, that Tempest should use. Tempest will use first two values from list and will use it only if option "multi_backend" is Enabled.

====Option "build_interval"====
typeː int
default valueː 3
descriptionː time to wait between resend of requests.

====Option "build_timeout"====
typeː int
default valueː 500
descriptionː time to wait until raise of Exception with timeout.

=How to run tests=
==Run only cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.cli.*manila*\"

==Run only api tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share*\"

==Run only api and cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila*\"

==Run only api and cli tests with specified amount of threads==
export TEMPEST_CONCURRENCY=4
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila* -- --concurrency=$TEMPEST_CONCURRENCY\"
Above action will try to run Tempest in 4 threads, if your host does have at least 4 cores. By default it uses amount of threads equal to amount of available cores.

Manila tests are OK using concurrent threads.

==If something went wrong with test launch==
Exception may be raised on step where tests are gathered and parsed and as a result we would see something similar to the followingː
running testr
Non-zero exit code (2) from test listing. stdout='\xb3)\x01@d@Ytempest.api.share.admin.test_admin_actions.AdminActionsTest.test_force_delete_share[gate]<Z\x90"\xb3)\x01@g@\\
...
tempest.scenario.test_volume_boot_pattern6B\x1e\xcc', stderr=None
error: testr failed (3)
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list

======
Totals
======
Run: 0 in 0.0 sec.
- Passed: 0
- Skipped: 0
- Failed: 0
ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh tempest/api/share/test_extensions.py'
__________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
ERROR: venv: commands failed

There could be different causes for that, in most cases - dependency issues, when some dependency is not installed and can not be imported. Or python syntax error, if something were changed in code, etc...

As we can see, it does not show us real problem and we need dig root cause up. It can be done by a lot of means, for example, try nextː
nosetests -sv path/to/tests
or
.tox/venv/bin/python -m testtools.run discover -t ./ path/to/tests

=How to write own tests=

==Where to put new tests==
All tests, that cover admin API are stored in
tempest/api/share/admin/*
All tests, that cover non-admin API are stored in
tempest/api/share/*

==Add method as API caller==
All such methods are stored in
tempest/services/share/json/shares_client.py
XML is considered as deprecated and tests for it were removed, so, only json "client" is expected.

==Positive/Negative==
Positive and negative tests are separated into different modules, module with "negative" tests should have "_negative" suffix.
Example of module with positive testsː
tempest/api/share/test_shares.py
Example of module with negative testsː
tempest/api/share/test_shares_negative.py

==Resolving concurrency issue==
If some test is sensible to changes that can be done by other tests, this test should use isolated tenant, example of it can be found in quota tests, located in
tempest/api/share/admin/test_quotas.py

==Reusing resources==
If several tests have same dependencies, they should use same resources if possible, it can be created in "setUpClass" method. Example can be found hereː
tempest/api/share/test_shares_actions.py

==Utilization of resources==
Utilization stuff is defined in the following moduleː
tempest/api/share/base.py

Basic actionsː
1) All setUpClass methods should be decorated with "@test.safe_setup" decorator.
Reasonː if we get Exception on some step in setUpClass, tearDownClass won't start. So, this decorator runs 'tearDownClass' in any case.

2) Create resources that should be deleted with methods from base class.
correctː self.create_share() - will add resources to deletion queue
incorrectː self.shares_client.create_share() - won't add resource to deletion queue

Common infoː
All resources in deletion queue will be deleted in LIFO sequence - "last in first out" or "last created first deleted".

There are two deletion queuesː
- runs after each test
- runs after each test-suite (class with tests)
Be attentive setting up it.

Manila/Meetings

2015-01-05T14:58:02Z

Vponomaryov:

= Weekly Manila team meeting =
'''NOTE MEETING TIME: Thursday at 15:00 UTC'''

'''IMPORTANT: Meetings on 25 Dec 2014 and 1 Jan 2015 are CANCELLED due to holidays.'''

If you're interested in management of shared filesystems for OpenStack, we have a weekly meetings in <code><nowiki>#openstack-meeting-alt</nowiki></code>, on Thursdays at 15:00 UTC. Please feel free to add items to the agenda below. NOTE: When adding topics please include your IRC name so we know who's topic it is and how to get more info.

== Next meeting ==
'''NOTE:''' ''Include your IRC nickname next to agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. You might want to put this on your calendar if you are adding items.''

'''Scheduled for 8 Jan, 2015, 15:00 UTC'''
# Development status
# rename driver mode (chen)
# [https://blueprints.launchpad.net/manila/+spec/level-of-access-for-shares BP "level-of-access-for-shares"] (vponomaryov)
# Open Discussion

== Previous meetings ==

http://eavesdrop.openstack.org/meetings/manila/

Manila/Meetings

2015-01-05T14:57:44Z

Vponomaryov: /* Next meeting */

= Weekly Manila team meeting =
'''NOTE MEETING TIME: Thursday at 15:00 UTC'''

'''IMPORTANT: Meetings on 25 Dec 2014 and 1 Jan 2015 are CANCELLED due to holidays.'''

If you're interested in management of shared filesystems for OpenStack, we have a weekly meetings in <code><nowiki>#openstack-meeting-alt</nowiki></code>, on Thursdays at 15:00 UTC. Please feel free to add items to the agenda below. NOTE: When adding topics please include your IRC name so we know who's topic it is and how to get more info.

== Next meeting ==
'''NOTE:''' ''Include your IRC nickname next to agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. You might want to put this on your calendar if you are adding items.''

'''Scheduled for 8 Jan, 2015, 15:00 UTC'''
# Development status
# rename driver mode (chen)
# [https://blueprints.launchpad.net/manila/+spec/level-of-access-for-shares BP "level-of-access-for-shares"] (vponomaryov)ː
# Open Discussion

== Previous meetings ==

http://eavesdrop.openstack.org/meetings/manila/

Manila/specs/split-storage-actions-computing-and-networking

2014-11-13T07:36:01Z

Vponomaryov: Created page with "= Split storage actions, computing and networking = This page describes changes to concept of share provisioning, interfaces changes and required implementations. == Intro ==..."

= Split storage actions, computing and networking =
This page describes changes to concept of share provisioning, interfaces changes and required implementations.

== Intro ==

For the moment Manila has different share drivers that implement storage activities, computing and networking all in one module using only neutron network plugin in some cases.

We should split it to logical parts:
1) share driver (storage activities)
2) compute helper (share server supply)
3) networking helper (network related logic)

as interfaces that should be implemented.

Details:

(1) share driver will implement all main interfaces:
- create/delete share f/wo snapshot
- create/delete snapshot
- allow/deny access

For getting share server it will use one of provided "compute helpers".

(2) compute helper will implement interfaces of share server creation and deletion. It will use "networking helper"
Possible implementations:
- Nova and VMs
- Ironic and hardware hosts
- storage-based (vservers in NetApp cDOT, etc...)

(3) networking helper will handle all network-related stuff, like:
- no network creation, no share server creation, usage of predefined share server (analog of so called single-tenancy)
- no network creation, share server creation with address from predefined network (combination of single and multi tenancy)
- creation of network (using either neutron or no-network), creation of share server (analog of so called multi-tenancy)

See pic with share driver - compute helper cooperation:
https://docs.google.com/drawings/d/1qUCBzIBnxDSJElWHydLlMIQAZznJttrYjr0thP1O-N4/edit?usp=sharing

== Proposed changes ==

=== Share Manager ===

Create "ShareManagerV2" in parallel to "ShareManager", then remove usage of "ShareManager", then remove "ShareManager" at all and rename "ShareManagerV2" to "ShareManager".

=== Share Drivers ===

Interfaces:
Will be the same, except following:
"setup_server" and "teardown_server" will be changed to use compute helpers

=== Compute Helpers ===

interfaces:
a) get_share_server(self, context, share_network_id)
-- share server either will be created or reused existed one based on share network.
-- security services will be taken from DB using "share_network_id" if some exist
-- should return id of share server either explicitly or as exception object attribute

b) delete_share_server(self, context, share_server)
-- delete share server and release all allocated resources

Share Servers will be used by share manager and share drivers.

=== Network Helpers ===

Share Network Model should be extended with following new options:

1) --nova-network-id
will be used for getting all required info from nova-network. It is mutually exclusive with following options and existing neutron options.

2) --flat-network-ip-pool
it will expect CIDR , list of CIDRs , list of IPS, list of CIDRs and IPs as values to be used for IP provisioning. It is mutually exclusive with following and above options and existing neutron options.

3) --dhcp-endpoint
it wil be used to get IP address from. It is mutually exclusive with above options and existing neutron options.

Network helper will have defined three plugins for each of modes, here is description how it takes decision what should be used:

flat_network:
--dhcp-endpoint
--flat-network-ip-pool

nova-network:
--nova-network-id

neutron_network:
--neutron-net-id
--neutron-subnet-id

It should not be allowed to set up contradictive options to share-network.

interfaces:

a) allocate_network(self, context, share_server, share_network, amount_of_ports, **kwargs)
will get network data from share network, then will allocate required net resources for share server.

b) deallocate_network(self, context, share_server)
will read all allocated network data for share server and release it.

=== Modules ===
Modules will be stored as following:

==== Share Drivers ====

interface:
- manila/share/driver.py

implementations:
- manila/share/drivers/generic.py
- manila/share/drivers/netapp/cluster_mode.py
- manila/share/drivers/emc/driver.py
- manila/share/drivers/ibm/gpfs.py
- manila/share/drivers/glusterfs.py

==== Compute Helpers ====

interface:
- manila/share/network_helper.py

implementations:
- manila/share/compute_helpers/nova.py
- manila/share/compute_helpers/ironic.py
- manila/share/compute_helpers/netapp.py
- manila/share/compute_helpers/emc.py

==== Network Helpers ====

interface:
- manila/share/network_helper.py

implementations:
- manila/share/network_helpers/flat_network.py
- manila/share/network_helpers/nova_network.py
- manila/share/network_helpers/neutron.py

Manila/Kilo Network Changes

2014-10-23T14:59:12Z

Vponomaryov: /* Cross-tenant share servers */

== Introduction ==
This document is intended to outline the design changes around Manila networking in the Kilo release.

== Use Cases ==

=== Simple lab config ===
A manila developer wants to write a driver for a new hardware platform. The
hardware exists in a lab plugged into a network switch with no VLAN tagging.
The developer has no management access on the switch or router and while he
does have management access on the storage controller he doesn't have any
physical access to modify cabling. The storage controller is configured
with static IP addresses on its management and data interfaces. On the plus
side, the developer had gotten the lab administrators to set aside a pool
of IP addresses on the same subnet which he can use to create SVMs.

* Network: flat, single subnet, multiple SVMs
* Security: none
* Other considerations: none

=== POC installation at a customer site ===
A customer which already uses OpenStack with some commercial storage wants
to experiment with Manila to learn about it and evaluate how useful it might
be. The customer already owns several storage controllers but they are all in
use either in production or in other proofs of concept or test projects in
the eval lab. The customer has administrator access to and physical access to
all of the revelant equipment but doesn't wish to disturb the existing
environment just to do a quick POC. A storage controller will need to be
shared, and the shared controller is cabled to a network switch with no VLAN
tagging.

* Network: flat, single subnet, single SVM
* Security: none
* Other considerations: none

=== Enterprise private cloud ===
An IT department has built a moderately large private cloud to serve various
business units in one company. The business units all know about eachother
and rely one simple access control to protect data from prying eyes and
accidental damage. The main motivation for building a cloud is the "aaS"
model which allows IT to be more efficient and responsive. The company is a
large user of NAS storage (on commercial storage controllers) and IT wants
to roll out Manila in production to begin to offer NASaaS so its userbase
can eventually stop using traditional NAS. The IT department has complete
control of its network design, physically and logically. Because all of the
"tenants" of the private cloud exist within one company a significant amount
of infrastructure is shared, including Active Directory, LDAP, and Kerberos,
and tenants are not firewalled off from each other in any complete sense.

* Network: flat, multiple subnets, multiple SVMs
* Security: tenants should not be able to access shares of other tenants
* Other considerations: none

=== Public Cloud Service ===
A company that offers cloud services to the public, based on OpenStack, would
like to deploy Manila in production and begin to offer NASaaS to its
customers alongside block and object storage. Tenants expect complete privacy
for their data, both in motion and at rest. Some customers have VPN
connections between their corporate offices and the cloud which their expect
only their own VMs to be able to access.

* Network: segmented, multiple subnets, multiple SVMs
* Security: tenants should not be able to detect the existence of others tenants
* Other considerations: network segmentation may be VLAN, VXLAN, or GRE

=== Public cloud (Tenant Facilitated) ===
Tenants desire to fire up and deliver Manila wholly contained within their tenant context atop a public (or potentially private) cloud.

* Network: flat, VPC, multiple SVMs
* Security: defined by tenant owner and contained within the context
* Other considerations: tenant network may reside atop any number of SDN technologies.

=== Standalone installation for management of existing NAS ===

An IT department wishes to offer NASaaS to users but currently uses funky
proprietary cloud software written by engineers with bad taste. They decide
to use Manila but without the rest of the OpenStack. Commercial storage
controllers are dedicated specifically for this purpose.

* Network: flat, multiple subnets, multiple SVMs
* Security: tenants should not be able to access shares of other tenants
* Other considerations: none

=== Automated test system ===

A manila developer would like to be able to quickly setup Manila inside a
virtual environment for testing. Manila and all of the rest of OpenStack
will run inside a single VM with network connectivity to the outside world
provided by a virtual router. The storage controller to be tested is not
virtualized and is shared with multiple test nodes. The developer has access
to configure the storage controller but the VM where the tests are being
run is in a different locale, connected only by IP to the storage controller.

* Network: flat, single subnet, single SVM
* Security: none
* Other considerations: none

=== Triple-O "Under Cloud" ===

A deployer of OpenStack employing Triple-O employees Manila at the under cloud layer to spin up / down storage server instances to facilitate Manila service expansion / contraction.
Under cloud provisioning primitives (e.g. NFS export) for Cinder backends or Glance repositories are employed

* Network: flat, single subnet, multiple share servers
* Security: TBD
* Other considerations: TBD

== Design Changes ==

To accommodate the above use cases, the following changes to Manila will be neededː

=== Cross-tenant share servers ===

Administrators should have a way to make a single share server usable by multiple tenants, and disable the creation of new share servers. This enables cases where share servers cannot be created by Manila either because of lack of access or lack of networking resources.

Open questionsː
* How will the share server get imported into Manila? Conf file options? A new admin API?
* Is there any point in allowing a mix of manila-created share servers with administrator create share servers?

Proposalsː

1) Easy
1.1) Add support of single preconfigured share-server for each driver separately. - They will work as single-tenant drivers in case no share-network provided.
proː no need to create share-network
conː only one share-server will be used

2) Hard
2.1) Add new APIs for share-server creation and share-server-details update and then link all of it to some share-network and back-end
proː dynamic add of custom share-servers
conː hard to support driver-specific things

=== Cross-tenant share networks ===

Administrators should have a way to make a single share network usable by multiple tenants. This network is used to create new share servers when the tenant declines to specify a share network (acting as a "default share network").

Proposalsː
1) Allow creation of "public" and "private" share-networks
2) Handle possibility to create both types by policies
3) Add possibility to make one of "public" share networks as "default" that will be used if one did not provide share-network explicitly.

Manila/Kilo Network Changes

2014-10-23T14:58:51Z

Vponomaryov: /* Cross-tenant share servers */

Manila/Kilo Network Changes

2014-10-23T14:58:33Z

Vponomaryov: /* Cross-tenant share servers */

Manila/Meetings

2014-10-23T14:57:13Z

Vponomaryov: /* Next meeting */

Manila/Kilo Network Changes

2014-10-23T14:52:41Z

Vponomaryov: /* Cross-tenant share networks */

== Introduction ==
This document is intended to outline the design changes around Manila networking in the Kilo release.

== Use Cases ==

=== Simple lab config ===
A manila developer wants to write a driver for a new hardware platform. The
hardware exists in a lab plugged into a network switch with no VLAN tagging.
The developer has no management access on the switch or router and while he
does have management access on the storage controller he doesn't have any
physical access to modify cabling. The storage controller is configured
with static IP addresses on its management and data interfaces. On the plus
side, the developer had gotten the lab administrators to set aside a pool
of IP addresses on the same subnet which he can use to create SVMs.

* Network: flat, single subnet, multiple SVMs
* Security: none
* Other considerations: none

=== POC installation at a customer site ===
A customer which already uses OpenStack with some commercial storage wants
to experiment with Manila to learn about it and evaluate how useful it might
be. The customer already owns several storage controllers but they are all in
use either in production or in other proofs of concept or test projects in
the eval lab. The customer has administrator access to and physical access to
all of the revelant equipment but doesn't wish to disturb the existing
environment just to do a quick POC. A storage controller will need to be
shared, and the shared controller is cabled to a network switch with no VLAN
tagging.

* Network: flat, single subnet, single SVM
* Security: none
* Other considerations: none

=== Enterprise private cloud ===
An IT department has built a moderately large private cloud to serve various
business units in one company. The business units all know about eachother
and rely one simple access control to protect data from prying eyes and
accidental damage. The main motivation for building a cloud is the "aaS"
model which allows IT to be more efficient and responsive. The company is a
large user of NAS storage (on commercial storage controllers) and IT wants
to roll out Manila in production to begin to offer NASaaS so its userbase
can eventually stop using traditional NAS. The IT department has complete
control of its network design, physically and logically. Because all of the
"tenants" of the private cloud exist within one company a significant amount
of infrastructure is shared, including Active Directory, LDAP, and Kerberos,
and tenants are not firewalled off from each other in any complete sense.

* Network: flat, multiple subnets, multiple SVMs
* Security: tenants should not be able to access shares of other tenants
* Other considerations: none

=== Public Cloud Service ===
A company that offers cloud services to the public, based on OpenStack, would
like to deploy Manila in production and begin to offer NASaaS to its
customers alongside block and object storage. Tenants expect complete privacy
for their data, both in motion and at rest. Some customers have VPN
connections between their corporate offices and the cloud which their expect
only their own VMs to be able to access.

* Network: segmented, multiple subnets, multiple SVMs
* Security: tenants should not be able to detect the existence of others tenants
* Other considerations: network segmentation may be VLAN, VXLAN, or GRE

=== Public cloud (Tenant Facilitated) ===
Tenants desire to fire up and deliver Manila wholly contained within their tenant context atop a public (or potentially private) cloud.

* Network: flat, VPC, multiple SVMs
* Security: defined by tenant owner and contained within the context
* Other considerations: tenant network may reside atop any number of SDN technologies.

=== Standalone installation for management of existing NAS ===

An IT department wishes to offer NASaaS to users but currently uses funky
proprietary cloud software written by engineers with bad taste. They decide
to use Manila but without the rest of the OpenStack. Commercial storage
controllers are dedicated specifically for this purpose.

* Network: flat, multiple subnets, multiple SVMs
* Security: tenants should not be able to access shares of other tenants
* Other considerations: none

=== Automated test system ===

A manila developer would like to be able to quickly setup Manila inside a
virtual environment for testing. Manila and all of the rest of OpenStack
will run inside a single VM with network connectivity to the outside world
provided by a virtual router. The storage controller to be tested is not
virtualized and is shared with multiple test nodes. The developer has access
to configure the storage controller but the VM where the tests are being
run is in a different locale, connected only by IP to the storage controller.

* Network: flat, single subnet, single SVM
* Security: none
* Other considerations: none

=== Triple-O "Under Cloud" ===

A deployer of OpenStack employing Triple-O employees Manila at the under cloud layer to spin up / down storage server instances to facilitate Manila service expansion / contraction.
Under cloud provisioning primitives (e.g. NFS export) for Cinder backends or Glance repositories are employed

* Network: flat, single subnet, multiple share servers
* Security: TBD
* Other considerations: TBD

== Design Changes ==

To accommodate the above use cases, the following changes to Manila will be neededː

=== Cross-tenant share servers ===

Administrators should have a way to make a single share server usable by multiple tenants, and disable the creation of new share servers. This enables cases where share servers cannot be created by Manila either because of lack of access or lack of networking resources.

Open questionsː
* How will the share server get imported into Manila? Conf file options? A new admin API?
* Is there any point in allowing a mix of manila-created share servers with administrator create share servers?

=== Cross-tenant share networks ===

Administrators should have a way to make a single share network usable by multiple tenants. This network is used to create new share servers when the tenant declines to specify a share network (acting as a "default share network").

Proposalsː
1) Allow creation of "public" and "private" share-networks
2) Handle possibility to create both types by policies
3) Add possibility to make one of "public" share networks as "default" that will be used if one did not provide share-network explicitly.

Manila/Kilo Network Changes

2014-10-23T14:52:17Z

Vponomaryov: /* Cross-tenant share networks */

== Introduction ==
This document is intended to outline the design changes around Manila networking in the Kilo release.

== Use Cases ==

=== Simple lab config ===
A manila developer wants to write a driver for a new hardware platform. The
hardware exists in a lab plugged into a network switch with no VLAN tagging.
The developer has no management access on the switch or router and while he
does have management access on the storage controller he doesn't have any
physical access to modify cabling. The storage controller is configured
with static IP addresses on its management and data interfaces. On the plus
side, the developer had gotten the lab administrators to set aside a pool
of IP addresses on the same subnet which he can use to create SVMs.

* Network: flat, single subnet, multiple SVMs
* Security: none
* Other considerations: none

=== POC installation at a customer site ===
A customer which already uses OpenStack with some commercial storage wants
to experiment with Manila to learn about it and evaluate how useful it might
be. The customer already owns several storage controllers but they are all in
use either in production or in other proofs of concept or test projects in
the eval lab. The customer has administrator access to and physical access to
all of the revelant equipment but doesn't wish to disturb the existing
environment just to do a quick POC. A storage controller will need to be
shared, and the shared controller is cabled to a network switch with no VLAN
tagging.

* Network: flat, single subnet, single SVM
* Security: none
* Other considerations: none

=== Enterprise private cloud ===
An IT department has built a moderately large private cloud to serve various
business units in one company. The business units all know about eachother
and rely one simple access control to protect data from prying eyes and
accidental damage. The main motivation for building a cloud is the "aaS"
model which allows IT to be more efficient and responsive. The company is a
large user of NAS storage (on commercial storage controllers) and IT wants
to roll out Manila in production to begin to offer NASaaS so its userbase
can eventually stop using traditional NAS. The IT department has complete
control of its network design, physically and logically. Because all of the
"tenants" of the private cloud exist within one company a significant amount
of infrastructure is shared, including Active Directory, LDAP, and Kerberos,
and tenants are not firewalled off from each other in any complete sense.

* Network: flat, multiple subnets, multiple SVMs
* Security: tenants should not be able to access shares of other tenants
* Other considerations: none

=== Public Cloud Service ===
A company that offers cloud services to the public, based on OpenStack, would
like to deploy Manila in production and begin to offer NASaaS to its
customers alongside block and object storage. Tenants expect complete privacy
for their data, both in motion and at rest. Some customers have VPN
connections between their corporate offices and the cloud which their expect
only their own VMs to be able to access.

* Network: segmented, multiple subnets, multiple SVMs
* Security: tenants should not be able to detect the existence of others tenants
* Other considerations: network segmentation may be VLAN, VXLAN, or GRE

=== Public cloud (Tenant Facilitated) ===
Tenants desire to fire up and deliver Manila wholly contained within their tenant context atop a public (or potentially private) cloud.

* Network: flat, VPC, multiple SVMs
* Security: defined by tenant owner and contained within the context
* Other considerations: tenant network may reside atop any number of SDN technologies.

=== Standalone installation for management of existing NAS ===

An IT department wishes to offer NASaaS to users but currently uses funky
proprietary cloud software written by engineers with bad taste. They decide
to use Manila but without the rest of the OpenStack. Commercial storage
controllers are dedicated specifically for this purpose.

* Network: flat, multiple subnets, multiple SVMs
* Security: tenants should not be able to access shares of other tenants
* Other considerations: none

=== Automated test system ===

A manila developer would like to be able to quickly setup Manila inside a
virtual environment for testing. Manila and all of the rest of OpenStack
will run inside a single VM with network connectivity to the outside world
provided by a virtual router. The storage controller to be tested is not
virtualized and is shared with multiple test nodes. The developer has access
to configure the storage controller but the VM where the tests are being
run is in a different locale, connected only by IP to the storage controller.

* Network: flat, single subnet, single SVM
* Security: none
* Other considerations: none

=== Triple-O "Under Cloud" ===

A deployer of OpenStack employing Triple-O employees Manila at the under cloud layer to spin up / down storage server instances to facilitate Manila service expansion / contraction.
Under cloud provisioning primitives (e.g. NFS export) for Cinder backends or Glance repositories are employed

* Network: flat, single subnet, multiple share servers
* Security: TBD
* Other considerations: TBD

== Design Changes ==

To accommodate the above use cases, the following changes to Manila will be neededː

=== Cross-tenant share servers ===

Administrators should have a way to make a single share server usable by multiple tenants, and disable the creation of new share servers. This enables cases where share servers cannot be created by Manila either because of lack of access or lack of networking resources.

Open questionsː
* How will the share server get imported into Manila? Conf file options? A new admin API?
* Is there any point in allowing a mix of manila-created share servers with administrator create share servers?

=== Cross-tenant share networks ===

Administrators should have a way to make a single share network usable by multiple tenants. This network is used to create new share servers when the tenant declines to specify a share network (acting as a "default share network").

Proposalsː
1) Allow creation of "public" and "private" share-networks
2) Handle possibility to create both types by policies
3) Add possibility to make one of "public" share networks as "default", that will be used if one did not provide share-network explicitly.

Manila/Kilo Network Changes

2014-10-23T14:51:58Z

Vponomaryov: /* Cross-tenant share networks */

== Introduction ==
This document is intended to outline the design changes around Manila networking in the Kilo release.

== Use Cases ==

=== Simple lab config ===
A manila developer wants to write a driver for a new hardware platform. The
hardware exists in a lab plugged into a network switch with no VLAN tagging.
The developer has no management access on the switch or router and while he
does have management access on the storage controller he doesn't have any
physical access to modify cabling. The storage controller is configured
with static IP addresses on its management and data interfaces. On the plus
side, the developer had gotten the lab administrators to set aside a pool
of IP addresses on the same subnet which he can use to create SVMs.

* Network: flat, single subnet, multiple SVMs
* Security: none
* Other considerations: none

=== POC installation at a customer site ===
A customer which already uses OpenStack with some commercial storage wants
to experiment with Manila to learn about it and evaluate how useful it might
be. The customer already owns several storage controllers but they are all in
use either in production or in other proofs of concept or test projects in
the eval lab. The customer has administrator access to and physical access to
all of the revelant equipment but doesn't wish to disturb the existing
environment just to do a quick POC. A storage controller will need to be
shared, and the shared controller is cabled to a network switch with no VLAN
tagging.

* Network: flat, single subnet, single SVM
* Security: none
* Other considerations: none

=== Enterprise private cloud ===
An IT department has built a moderately large private cloud to serve various
business units in one company. The business units all know about eachother
and rely one simple access control to protect data from prying eyes and
accidental damage. The main motivation for building a cloud is the "aaS"
model which allows IT to be more efficient and responsive. The company is a
large user of NAS storage (on commercial storage controllers) and IT wants
to roll out Manila in production to begin to offer NASaaS so its userbase
can eventually stop using traditional NAS. The IT department has complete
control of its network design, physically and logically. Because all of the
"tenants" of the private cloud exist within one company a significant amount
of infrastructure is shared, including Active Directory, LDAP, and Kerberos,
and tenants are not firewalled off from each other in any complete sense.

* Network: flat, multiple subnets, multiple SVMs
* Security: tenants should not be able to access shares of other tenants
* Other considerations: none

=== Public Cloud Service ===
A company that offers cloud services to the public, based on OpenStack, would
like to deploy Manila in production and begin to offer NASaaS to its
customers alongside block and object storage. Tenants expect complete privacy
for their data, both in motion and at rest. Some customers have VPN
connections between their corporate offices and the cloud which their expect
only their own VMs to be able to access.

* Network: segmented, multiple subnets, multiple SVMs
* Security: tenants should not be able to detect the existence of others tenants
* Other considerations: network segmentation may be VLAN, VXLAN, or GRE

=== Public cloud (Tenant Facilitated) ===
Tenants desire to fire up and deliver Manila wholly contained within their tenant context atop a public (or potentially private) cloud.

* Network: flat, VPC, multiple SVMs
* Security: defined by tenant owner and contained within the context
* Other considerations: tenant network may reside atop any number of SDN technologies.

=== Standalone installation for management of existing NAS ===

An IT department wishes to offer NASaaS to users but currently uses funky
proprietary cloud software written by engineers with bad taste. They decide
to use Manila but without the rest of the OpenStack. Commercial storage
controllers are dedicated specifically for this purpose.

* Network: flat, multiple subnets, multiple SVMs
* Security: tenants should not be able to access shares of other tenants
* Other considerations: none

=== Automated test system ===

A manila developer would like to be able to quickly setup Manila inside a
virtual environment for testing. Manila and all of the rest of OpenStack
will run inside a single VM with network connectivity to the outside world
provided by a virtual router. The storage controller to be tested is not
virtualized and is shared with multiple test nodes. The developer has access
to configure the storage controller but the VM where the tests are being
run is in a different locale, connected only by IP to the storage controller.

* Network: flat, single subnet, single SVM
* Security: none
* Other considerations: none

=== Triple-O "Under Cloud" ===

A deployer of OpenStack employing Triple-O employees Manila at the under cloud layer to spin up / down storage server instances to facilitate Manila service expansion / contraction.
Under cloud provisioning primitives (e.g. NFS export) for Cinder backends or Glance repositories are employed

* Network: flat, single subnet, multiple share servers
* Security: TBD
* Other considerations: TBD

== Design Changes ==

To accommodate the above use cases, the following changes to Manila will be neededː

=== Cross-tenant share servers ===

Administrators should have a way to make a single share server usable by multiple tenants, and disable the creation of new share servers. This enables cases where share servers cannot be created by Manila either because of lack of access or lack of networking resources.

Open questionsː
* How will the share server get imported into Manila? Conf file options? A new admin API?
* Is there any point in allowing a mix of manila-created share servers with administrator create share servers?

=== Cross-tenant share networks ===

Administrators should have a way to make a single share network usable by multiple tenants. This network is used to create new share servers when the tenant declines to specify a share network (acting as a "default share network").

Proposalsː
1) Allow creation of "public" and "private" share-networks
2) Handle possibility to create both types by policies
3) Add possibility to make one of "public" share networks as "default", that will be used if one did not provide share-network explicitly.

CrossProjectLiaisons

2014-10-08T08:16:20Z

Vponomaryov: /* QA */

Many of our cross-project teams need focused help for communicating with the other project teams. This page lists the people who have volunteered for that work.

== Oslo ==

There are now more projects consuming code from the Oslo incubator than we have Oslo contributors. That means we are going to need your help to make these migrations happen. We are asking for one person from each project to serve as a liaison between the project and Oslo, and to assist with integrating changes as we move code out of the incubator into libraries.

* The liaison should be a core reviewer for the project, but does not need to be the PTL.
* The liaison should be prepared to assist with writing and reviewing patches in their project as libraries are adopted, and with discussions of API changes to the libraries to make them easier to use within the project.
* Liaisons should pay attention to [Oslo] tagged messages on the openstack-dev mailing list.
* It is also useful for liaisons to be able to attend the Oslo team meeting ([[Meetings/Oslo]]) to participate in discussions and raise issues for real-time discussion.

{| class="wikitable"
|-
! Project !! Liaison !! IRC Handle
|-
| Ceilometer || Julien Danjou || jd__
|-
| Keystone || Brant Knudson || bknudson
|-
| Nova || Joe Gordon || jogo
|-
| Neutron || Ihar Hrachyshka (not a core!) || ihrachyshka
|-
| [[Octavia]] || Doug Wiegley || dougwig
|-
| Horizon || Akihiro Motoki || amotoki
|-
|}

== QA ==

There are now more projects that are being tested by Tempest, and Grenade or a part deployable by Devstack than we have QA contributors. That means we are going to need your help to keep on top of everything. We are asking for one person from each project to serve as a liaison between the project and QA, and to assist with integrating changes as we move forward.

The liaison should be a core reviewer for the project, but does not need to be the PTL. The liaison should be prepared to assist with writing and reviewing patches that interact with their project, and with discussions of changes to the QA projects to make them easier to use within the project.

{| class="wikitable"
|-
! Project !! Liaison !! IRC Handle
|-
| Nova || ||
|-
| Cinder || ||
|-
| Swift || ||
|-
| Neutron || ||
|-
| Keystone || ||
|-
| Horizon || ||
|-
| Glance || ||
|-
| Ceilometer || ||
|-
| Heat || ||
|-
| Oslo || ||
|-
| Trove || ||
|-
| Sahara || ||
|-
| Ironic || ||
|-
| Zaqar || ||
|-
| Barbican || ||
|-
| Manila || Valeriy Ponomaryov || vponomaryov
|}

== Documentation ==

{| class="wikitable"
|-
! Project !! Liaison !! IRC Handle
|-
| Nova || Jog0 or mikal? ||
|-
| Cinder || thingee (Mike Perez) ||
|-
| Swift || koolhead (Atul Jha) ||
|-
| Neutron || emagana (Edgar Magana) ||
|-
| Keystone || ayoung (Adam Young) ||
|-
| Horizon || jpichon (Julie Pichon) ||
|-
| Glance || rosmaita (Brian Rosmaita) ||
|-
| Ceilometer || ||
|-
| Heat || Randall Burt? ||
|-
| Oslo || ||
|-
| Trove || laurelm ||
|-
| Sahara || ||
|-
| Ironic || ||
|-
| Zaqar || ||
|-
| Barbican || ||
|-
| Manila || ||
|}

Manila/docs/HOWTO use manila with horizon

2014-10-03T12:10:05Z

Vponomaryov: /* Get custom horizon */

= How to run custom horizon with manila =
This page describes how to try manila with horizon. It can be done on any host, not exactly the same as devstack installation.

==Get custom horizon==

$ git clone -b manila_juno https://github.com/NetApp/horizon.git # Horizon of Juno version with Manila support, recommended
or
$ git clone -b manila_icehouse https://github.com/NetApp/horizon.git # Horizon of Icehouse version with Manila support. Old, not recommended
and
$ cd horizon
and
$ cp openstack_dashboard/local/local_settings.py.example openstack_dashboard/local/local_settings.py

open file '''''openstack_dashboard/local/local_settings.py'''''

find var "''OPENSTACK_HOST''" and set here address of host where devstack was installed

save and exit

==Be sure, all dependencies are installed==

run in horizon's dir

$ sudo python setup.py install

Also, need to install python-manilaclient

$ sudo pip install git+https://github.com/openstack/python-manilaclient.git

==Run horizon==
$ sudo python manage.py runserver

it will be available on 127.0.0.1:8000

to specify all interfaces, use:

$ sudo python manage.py runserver 0.0.0.0:8000

port can be set to any not used valid port value.

Manila/docs/HOWTO use manila with horizon

2014-10-03T11:15:42Z

Vponomaryov: /* Get custom horizon */

Manila

2014-09-20T06:08:00Z

Vponomaryov:

== Introduction ==
This page documents the concept and vision for establishing a shared file system service for OpenStack. The development name for this project is Manila. We propose and are in the process of implementing a new OpenStack service (originally based on Cinder). Cinder presently functions as the canonical storage provisioning control plane in [[OpenStack]] for block storage as well as delivering a persistence model for instance storage. The File Share Service prototype, in a similar manner, provides coordinated access to shared or distributed file systems. While the primary consumption of file shares would be across OpenStack Compute instances, the service is also intended to be accessible as an independent capability in line with the modular design established by other OpenStack services. The design and prototype implementation provide extensibility for multiple backends (to support vendor or file system specific nuances / capabilities) but is intended to be sufficiently abstract to accommodate any of a variety of shared or distributed file system types. The team's intention is to introduce the capability as an OpenStack incubated project in the Juno timeframe, graduate it and submit for consideration as a core service as early as the as the Kilo release.

[[File:Shares_Service.png]]

According to IDC in its “[http://www.idc.com/getdoc.jsp?containerId=235910 Worldwide File-Based Storage 2012-2016 Forecast]", file-based storage continues to be a thriving market, with spending on File-based storage solutions to reach north of $34.6 billion in 2016. Of the 27 Exabyte’s (EB) of total disk capacity estimated to have shipped in 2012, IDC projected that nearly 18 EB were file-based capacity, accounting for over 65% of all disk shipped by capacity. A diversity of applications, from server virtualization to relational or distributed databases to collaborative content creation, often depend on the performance, scalability and simplicity of management associated with file-based systems, and the large ecosystem of supporting software products. OpenStack, as the leading open IaaS capability, is increasingly contemplated as an option for deploying classic infrastructure in an "as a Service" model, but without specific accommodation for shared file systems represents an incomplete solution.

== Projects==

=== Manila ===

{| border="1" cellpadding="2"
| Source code
| https://github.com/openstack/manila
|-
| Bug tracker
| https://bugs.launchpad.net/manila
|-
| Feature tracker
| https://blueprints.launchpad.net/manila
|}

=== Python Manila Client ===
{| border="1" cellpadding="2"
| Source code
| https://github.com/openstack/python-manilaclient
|-
| Bug tracker
| https://bugs.launchpad.net/python-manilaclient
|-
| Feature tracker
| https://blueprints.launchpad.net/python-manilaclient
|}

=== Meetings ===

[[Manila/Meetings|Project Agenda & Meetings]]

[[Manila/JunoSummitPresentation|Overview Presentation from Juno (Atlanta) Summit]]

=== Getting Started with Manila ===

Instructions for getting started with Manila using Devstack are available at: [[Manila/IcehouseDevstack|Manila on Devstack]]

Developer documentation is still in progress, but is available in a draft state: [[Manila/docs|Manila Documentation]]

=== Project Plan & Progress ===

[[Manila/ProjectPlan|Manila Project Plan]] 
Track progress towards [[Manila/Graduation|Manila Incubation & Graduation]].

== Design & Use Cases ==

[[Manila/design|Manila Component Design]] 
[[Manila/usecases|Manila Use Cases]]

== Known issues ==

[[Manila/issue-neutron-on-localhost-ipv6|Connection to neutron failed.]]

== Subpages ==

{{Special:PrefixIndex/:Manila/}}
[[Manila/ManilaWithGREtunnels]]

[[Manila/ManilaWithVXLANtunnels]]

[[Manila_devstack_script|Manila/Manila Devstack Script]]

[[Category:Manila]]

Manila/Incubation Application

2014-07-24T14:33:30Z

Vponomaryov: /* Valeriy Ponomaryov */

== Project codename==
Manila

== Trademarks ==
We're not aware of any trademarks conflicts with the name. The capital city of the Philippines is called Manila, making the name a proper noun. There are no other names used by the project with trademark concerns.

== Summary (one sentence abstract of the project) ==
The Manila project provides an API for management of shared filesystems with support for multiple protocols and backend implementations.

== Parent Program name and PTL ==
Program: [[Manila/Program_Application|Shared Filesystems]]

PTL: Ben Swartzlander

== Mission statement ==
Stated simply, the goal of the Manila project is to do for shared filesystem storage what Cinder has done for blocks storage.

We aim to provide a vendor neutral management interface that allows for provisioning and attaching shared filesystems such as NFS, CIFS, and more. To the extent possible we aim to mirror the architecture of Cinder, with support for a public REST API, multiple backends, and a scheduler that performs resource assignment decisions. When differences are unavoidable, we plan to design solutions that are compatible with the OpenStack ideals of modularity and scalability.

== Detailed Description ==
The basic assumption underpinning Manila is that shared filesystems provide some valuable features that cannot be obtained from either blocks storage or object storage, and that OpenStack is missing management features for this 3rd form of storage. The unique features afforded by shared filesystems are shared, fine-grained, read/write access to persistent data by multiple instances simultaneously. The NFS and CIFS protocols have been developed to provide these features and still prove popular after decades of use.

The implementation of Manila is actually a modified fork of the Cinder project. The concept for management of shared filesystems was originally proposed as an extension to Cinder (at the SF design summit in April 2012), under the theory that there would be a lot of common code between the implementations, and many of the same developers would be interested in working on both projects. Because of this, the initial implementation for what is now Manila was actually a large patch to the Cinder project submitted in August 2012. For a variety of reasons, we ultimately decided that a separate project would be a better way to deliver the features and the Manila project was born.

Manila consist of all of the code from Cinder with our shared filesystem management code added in and all the blocks-specific code removed. The API largely mirrors the existing Cinder APIs, except that "volumes" have been renamed to "shares" and the attachment procedure is somewhat different.

== Basic roadmap for the project ==
The initial implementation of Manila was a proof of concept that shared filesystem management can fit into the same architecture as Cinder. The main difference between blocks and shared filesystems however is how the storage system and the ultimate user of the storage communicate with one another. In particular, shared filesystems work best when instances are able to communicate directly with the storage backend over the network, and the storage backend is able to serve multiple tenants while maintaining secure separation between them. Block storage can simply be virtualized through a hypervisor, with far fewer requirements on the backend storage system. Because of these differences, additional work is needed to help automate the networking portion of attaching a shared filesystem to one or more instances in a tenant network, and to automate the setup of security domains and other features that exist in a NAS environment but not a SAN environment.

During Icehouse, our main goals are to define and implement these new APIs as well as expanding the backend driver interface to support true multitenancy (the kind with network segmentation) in Manila. We also aim to get as many backends implemented as possible, and to this end we will be improving the reference drivers and documentation for developing new ones.
̈
== Location of project source code ==
* Project code: https://github.com/stackforge/manila
* Python client: https://github.com/stackforge/python-manilaclient

== Programming language, required technology dependencies ==
=== Language: ===
Python

=== Dependencies ===
Message queue, database server, keystone, neutron
Optional parts of Manila depend on: nova, cinder

There is a plan to make the neutron dependency optional in the future.

== Is project currently open sourced? What license? ==
Yes - Licensed under the Apache License, Version 2.0

== Level of maturity of software and team ==
=== Software ===
Aside from the code inherited from Cinder, the new Manila code is a little more than a year old, and has been actively developed from then until now, mostly by developers from NetApp and Mirantis.

=== Team ===
The core team now consists of developers from NetApp and Mirantis, with significant community interest since the code was open sourced in August 2013.

== Project developers qualifications ==

=== Ben Swartzlander ===
'''NetApp - Software Architect, Manila - PTL'''

Ben Swartzlander has been the technical lead for the project since its conception 2 years ago, and plans to continue leading the project from a design and administrative standpoint. Ben has been working the storage industry as a software engineer for more than 13 years and has extensive experience with storage systems, network protocols, virtualization, and open source projects. Ben has been a contributor to the OpenStack project for nearly 3 years.

=== Yulia Portnova ===
'''Mirantis - Software Developer, Manila - Core Team'''

Yulia has been working in Mirantis with OpenStack since Essex release.
Has contributions to Nova, Glance, Cinder.

=== Valeriy Ponomaryov ===
'''Mirantis - QA Automation Engineer, Manila - Core Team'''

Valeriy has been working with OpenStack since Folsom release, except Manila project he has contributions to Tempest, CI Config and Oslo-Incubator, also he is one of the developers of Horizon modification and Devstack plugin for Manila project.

=== Xing Yang ===
'''EMC - Software Developer, Manila - Core Team (pending team vote)'''

=== Alex Meade ===
'''NetApp - Software Developer'''

Alex has been working on OpenStack for over 3 years and is a Core contributor for the Glance project. He previously worked on the Rackspace public cloud deployment and is interested in OpenStack at scale.

=== Rushil Chugh ===
'''NetApp - Software Developer'''

Rushil holds a Masters in Computer Networking and has been working with OpenStack since the Grizzly release. Rushil holds a keen interest in Enterprise Network Deployments, Data Storage and Virtualization.

=== Clinton Knight ===
'''NetApp - Sr. Software Developer'''

Clinton holds a doctorate in electrical & computer engineering, has worked in the storage industry for 15 years, and has extensive experience with storage management and protocols as well as virtualization.

=== Rushi Agrawal ===
'''Reliance Jio Cloud - Software Developer'''

=== Andrei Ostapenko ===
'''Mirantis - Software Developer'''

Andrei works with OpenStack since folsom release and is mostly involved in new stackforge components.

=== Vitaly Kostenko ===
'''Mirantis - Software Developer'''

=== Aleksandr Chirko ===
'''Mirantis - Software Developer'''

=== Vijay Bellur ===
'''Red Hat'''

=== Csaba Henk ===
'''Red Hat'''

=== Ramana Raja ===
'''Red Hat'''

=== Christian Berendt ===

=== Shamail Tahir ===
'''EMC'''

=== Scott D'Angelo ===
'''HP Public Cloud'''

=== Deepak C Shetty ===
'''Red Hat - OpenStack Developer'''

=== Julia Varlamova ===
'''Mirantis - Intern Software Developer'''

Julia has been working on OpenStack for 1.5 years. Has contributions to Glance, Cinder, Oslo.DB.

== Infrastructure requirements (testing, etc) ==
Manila does not require any infrastructure above and beyond what's already provided by devstack, gerrit, jenkins, and tempest today.

== Have all current contributors agreed to the OpenStack CLA? ==
All current contributors HAVE agreed to the OpenStack CLA!

Manila/Incubation Application

2014-07-24T11:52:21Z

Vponomaryov: /* Aleksandr Chirko */

== Project codename==
Manila

== Trademarks ==
We're not aware of any trademarks conflicts with the name. The capital city of the Philippines is called Manila, making the name a proper noun. There are no other names used by the project with trademark concerns.

== Summary (one sentence abstract of the project) ==
The Manila project provides an API for management of shared filesystems with support for multiple protocols and backend implementations.

== Parent Program name and PTL ==
Program: [[Manila/Program_Application|Shared Filesystems]]

PTL: Ben Swartzlander

== Mission statement ==
Stated simply, the goal of the Manila project is to do for shared filesystem storage what Cinder has done for blocks storage.

We aim to provide a vendor neutral management interface that allows for provisioning and attaching shared filesystems such as NFS, CIFS, and more. To the extent possible we aim to mirror the architecture of Cinder, with support for a public REST API, multiple backends, and a scheduler that performs resource assignment decisions. When differences are unavoidable, we plan to design solutions that are compatible with the OpenStack ideals of modularity and scalability.

== Detailed Description ==
The basic assumption underpinning Manila is that shared filesystems provide some valuable features that cannot be obtained from either blocks storage or object storage, and that OpenStack is missing management features for this 3rd form of storage. The unique features afforded by shared filesystems are shared, fine-grained, read/write access to persistent data by multiple instances simultaneously. The NFS and CIFS protocols have been developed to provide these features and still prove popular after decades of use.

The implementation of Manila is actually a modified fork of the Cinder project. The concept for management of shared filesystems was originally proposed as an extension to Cinder (at the SF design summit in April 2012), under the theory that there would be a lot of common code between the implementations, and many of the same developers would be interested in working on both projects. Because of this, the initial implementation for what is now Manila was actually a large patch to the Cinder project submitted in August 2012. For a variety of reasons, we ultimately decided that a separate project would be a better way to deliver the features and the Manila project was born.

Manila consist of all of the code from Cinder with our shared filesystem management code added in and all the blocks-specific code removed. The API largely mirrors the existing Cinder APIs, except that "volumes" have been renamed to "shares" and the attachment procedure is somewhat different.

== Basic roadmap for the project ==
The initial implementation of Manila was a proof of concept that shared filesystem management can fit into the same architecture as Cinder. The main difference between blocks and shared filesystems however is how the storage system and the ultimate user of the storage communicate with one another. In particular, shared filesystems work best when instances are able to communicate directly with the storage backend over the network, and the storage backend is able to serve multiple tenants while maintaining secure separation between them. Block storage can simply be virtualized through a hypervisor, with far fewer requirements on the backend storage system. Because of these differences, additional work is needed to help automate the networking portion of attaching a shared filesystem to one or more instances in a tenant network, and to automate the setup of security domains and other features that exist in a NAS environment but not a SAN environment.

During Icehouse, our main goals are to define and implement these new APIs as well as expanding the backend driver interface to support true multitenancy (the kind with network segmentation) in Manila. We also aim to get as many backends implemented as possible, and to this end we will be improving the reference drivers and documentation for developing new ones.
̈
== Location of project source code ==
* Project code: https://github.com/stackforge/manila
* Python client: https://github.com/stackforge/python-manilaclient

== Programming language, required technology dependencies ==
=== Language: ===
Python

=== Dependencies ===
Message queue, database server, keystone, neutron
Optional parts of Manila depend on: nova, cinder

There is a plan to make the neutron dependency optional in the future.

== Is project currently open sourced? What license? ==
Yes - Licensed under the Apache License, Version 2.0

== Level of maturity of software and team ==
=== Software ===
Aside from the code inherited from Cinder, the new Manila code is a little more than a year old, and has been actively developed from then until now, mostly by developers from NetApp and Mirantis.

=== Team ===
The core team now consists of developers from NetApp and Mirantis, with significant community interest since the code was open sourced in August 2013.

== Project developers qualifications ==

=== Ben Swartzlander ===
'''NetApp - Software Architect, Manila - PTL'''

Ben Swartzlander has been the technical lead for the project since its conception 2 years ago, and plans to continue leading the project from a design and administrative standpoint. Ben has been working the storage industry as a software engineer for more than 13 years and has extensive experience with storage systems, network protocols, virtualization, and open source projects. Ben has been a contributor to the OpenStack project for nearly 3 years.

=== Yulia Portnova ===
'''Mirantis - Software Developer, Manila - Core Team'''

Yulia has been working in Mirantis with OpenStack since Essex release.
Has contributions to Nova, Glance, Cinder.

=== Valeriy Ponomaryov ===
'''Mirantis - Software Developer, Manila - Core Team'''

=== Xing Yang ===
'''EMC - Software Developer, Manila - Core Team (pending team vote)'''

=== Alex Meade ===
'''NetApp - Software Developer'''

Alex has been working on OpenStack for over 3 years and is a Core contributor for the Glance project. He previously worked on the Rackspace public cloud deployment and is interested in OpenStack at scale.

=== Rushil Chugh ===
'''NetApp - Software Developer'''

Rushil holds a Masters in Computer Networking and has been working with OpenStack since the Grizzly release. Rushil holds a keen interest in Enterprise Network Deployments, Data Storage and Virtualization.

=== Clinton Knight ===
'''NetApp - Sr. Software Developer'''

Clinton holds a doctorate in electrical & computer engineering, has worked in the storage industry for 15 years, and has extensive experience with storage management and protocols as well as virtualization.

=== Rushi Agrawal ===
'''Reliance Jio Cloud - Software Developer'''

=== Andrei Ostapenko ===
'''Mirantis - Software Developer'''

=== Vitaly Kostenko ===
'''Mirantis - Software Developer'''

=== Aleksandr Chirko ===
'''Mirantis - Software Developer'''

=== Vijay Bellur ===
'''Red Hat'''

=== Csaba Henk ===
'''Red Hat'''

=== Ramana Raja ===
'''Red Hat'''

=== Christian Berendt ===

=== Shamail Tahir ===
'''EMC'''

=== Scott D'Angelo ===
'''HP Public Cloud'''

=== Deepak C Shetty ===
'''Red Hat - OpenStack Developer'''

=== Julia Varlamova ===
'''Mirantis - Intern Software Developer'''

Julia has been working on OpenStack for 1.5 years. Has contributions to Glance, Cinder, Oslo.DB.

== Infrastructure requirements (testing, etc) ==
Manila does not require any infrastructure above and beyond what's already provided by devstack, gerrit, jenkins, and tempest today.

== Have all current contributors agreed to the OpenStack CLA? ==
All current contributors HAVE agreed to the OpenStack CLA!

Manila/Incubation Application

2014-07-24T11:52:11Z

Vponomaryov: /* Vitaly Kostenko */

== Project codename==
Manila

== Trademarks ==
We're not aware of any trademarks conflicts with the name. The capital city of the Philippines is called Manila, making the name a proper noun. There are no other names used by the project with trademark concerns.

== Summary (one sentence abstract of the project) ==
The Manila project provides an API for management of shared filesystems with support for multiple protocols and backend implementations.

== Parent Program name and PTL ==
Program: [[Manila/Program_Application|Shared Filesystems]]

PTL: Ben Swartzlander

== Mission statement ==
Stated simply, the goal of the Manila project is to do for shared filesystem storage what Cinder has done for blocks storage.

We aim to provide a vendor neutral management interface that allows for provisioning and attaching shared filesystems such as NFS, CIFS, and more. To the extent possible we aim to mirror the architecture of Cinder, with support for a public REST API, multiple backends, and a scheduler that performs resource assignment decisions. When differences are unavoidable, we plan to design solutions that are compatible with the OpenStack ideals of modularity and scalability.

== Detailed Description ==
The basic assumption underpinning Manila is that shared filesystems provide some valuable features that cannot be obtained from either blocks storage or object storage, and that OpenStack is missing management features for this 3rd form of storage. The unique features afforded by shared filesystems are shared, fine-grained, read/write access to persistent data by multiple instances simultaneously. The NFS and CIFS protocols have been developed to provide these features and still prove popular after decades of use.

The implementation of Manila is actually a modified fork of the Cinder project. The concept for management of shared filesystems was originally proposed as an extension to Cinder (at the SF design summit in April 2012), under the theory that there would be a lot of common code between the implementations, and many of the same developers would be interested in working on both projects. Because of this, the initial implementation for what is now Manila was actually a large patch to the Cinder project submitted in August 2012. For a variety of reasons, we ultimately decided that a separate project would be a better way to deliver the features and the Manila project was born.

Manila consist of all of the code from Cinder with our shared filesystem management code added in and all the blocks-specific code removed. The API largely mirrors the existing Cinder APIs, except that "volumes" have been renamed to "shares" and the attachment procedure is somewhat different.

== Basic roadmap for the project ==
The initial implementation of Manila was a proof of concept that shared filesystem management can fit into the same architecture as Cinder. The main difference between blocks and shared filesystems however is how the storage system and the ultimate user of the storage communicate with one another. In particular, shared filesystems work best when instances are able to communicate directly with the storage backend over the network, and the storage backend is able to serve multiple tenants while maintaining secure separation between them. Block storage can simply be virtualized through a hypervisor, with far fewer requirements on the backend storage system. Because of these differences, additional work is needed to help automate the networking portion of attaching a shared filesystem to one or more instances in a tenant network, and to automate the setup of security domains and other features that exist in a NAS environment but not a SAN environment.

During Icehouse, our main goals are to define and implement these new APIs as well as expanding the backend driver interface to support true multitenancy (the kind with network segmentation) in Manila. We also aim to get as many backends implemented as possible, and to this end we will be improving the reference drivers and documentation for developing new ones.
̈
== Location of project source code ==
* Project code: https://github.com/stackforge/manila
* Python client: https://github.com/stackforge/python-manilaclient

== Programming language, required technology dependencies ==
=== Language: ===
Python

=== Dependencies ===
Message queue, database server, keystone, neutron
Optional parts of Manila depend on: nova, cinder

There is a plan to make the neutron dependency optional in the future.

== Is project currently open sourced? What license? ==
Yes - Licensed under the Apache License, Version 2.0

== Level of maturity of software and team ==
=== Software ===
Aside from the code inherited from Cinder, the new Manila code is a little more than a year old, and has been actively developed from then until now, mostly by developers from NetApp and Mirantis.

=== Team ===
The core team now consists of developers from NetApp and Mirantis, with significant community interest since the code was open sourced in August 2013.

== Project developers qualifications ==

=== Ben Swartzlander ===
'''NetApp - Software Architect, Manila - PTL'''

Ben Swartzlander has been the technical lead for the project since its conception 2 years ago, and plans to continue leading the project from a design and administrative standpoint. Ben has been working the storage industry as a software engineer for more than 13 years and has extensive experience with storage systems, network protocols, virtualization, and open source projects. Ben has been a contributor to the OpenStack project for nearly 3 years.

=== Yulia Portnova ===
'''Mirantis - Software Developer, Manila - Core Team'''

Yulia has been working in Mirantis with OpenStack since Essex release.
Has contributions to Nova, Glance, Cinder.

=== Valeriy Ponomaryov ===
'''Mirantis - Software Developer, Manila - Core Team'''

=== Xing Yang ===
'''EMC - Software Developer, Manila - Core Team (pending team vote)'''

=== Alex Meade ===
'''NetApp - Software Developer'''

Alex has been working on OpenStack for over 3 years and is a Core contributor for the Glance project. He previously worked on the Rackspace public cloud deployment and is interested in OpenStack at scale.

=== Rushil Chugh ===
'''NetApp - Software Developer'''

Rushil holds a Masters in Computer Networking and has been working with OpenStack since the Grizzly release. Rushil holds a keen interest in Enterprise Network Deployments, Data Storage and Virtualization.

=== Clinton Knight ===
'''NetApp - Sr. Software Developer'''

Clinton holds a doctorate in electrical & computer engineering, has worked in the storage industry for 15 years, and has extensive experience with storage management and protocols as well as virtualization.

=== Rushi Agrawal ===
'''Reliance Jio Cloud - Software Developer'''

=== Andrei Ostapenko ===
'''Mirantis - Software Developer'''

=== Vitaly Kostenko ===
'''Mirantis - Software Developer'''

=== Aleksandr Chirko ===

=== Vijay Bellur ===
'''Red Hat'''

=== Csaba Henk ===
'''Red Hat'''

=== Ramana Raja ===
'''Red Hat'''

=== Christian Berendt ===

=== Shamail Tahir ===
'''EMC'''

=== Scott D'Angelo ===
'''HP Public Cloud'''

=== Deepak C Shetty ===
'''Red Hat - OpenStack Developer'''

=== Julia Varlamova ===
'''Mirantis - Intern Software Developer'''

Julia has been working on OpenStack for 1.5 years. Has contributions to Glance, Cinder, Oslo.DB.

== Infrastructure requirements (testing, etc) ==
Manila does not require any infrastructure above and beyond what's already provided by devstack, gerrit, jenkins, and tempest today.

== Have all current contributors agreed to the OpenStack CLA? ==
All current contributors HAVE agreed to the OpenStack CLA!

Manila/Meetings

2014-07-10T11:09:08Z

Vponomaryov: /* Next meeting */

Manila/docs/HOWTO use tempest with manila

2014-07-07T12:04:00Z

Vponomaryov: /* Foreword */

=Foreword=
This page describes how Tempest project for Manila project testing can be used.

It is assumed that user who reads this is acquainted with, at least, common information about Tempest:
http://docs.openstack.org/developer/tempest/
https://github.com/openstack/tempest

Also, it is assumed, that the following projects has been installed in test environmentː
- cinder
- glance
- keystone
- manila
- neutron
- nova

=Where do Tempest tests for Manila live?=
Manila stores its tests for Tempest in its repo here:
https://github.com/stackforge/manila/tree/master/contrib/tempest

These are pluggable-like files which do not require changing of core Tempest files.
But the way of its pluggability is implemented as work-around for Tempest, which does not have pluggable functionality for 'exceptions', 'config' and 'clients' modules.
It is compatible with master branch of Tempest.

Just copy these files to Tempest dirː
$ TEMPEST_DIR=/path/to/tempest
$ MANILA_DIR=/path/to/manila
$ cp ${MANILA_DIR}/contrib/tempest/tempest/* ${TEMPEST_DIR}/tempest

=Configuring Tempest before running tests=
Any test running with Tempest requires update of Tempest's config file located in
etc/tempest.conf
If it does not exist, just copy template "tempest.conf.sample"
cp tempest.conf.sample tempest.conf

Manila's config extension is defined in
https://github.com/stackforge/manila/blob/master/contrib/tempest/tempest/config_share.py

Subsections below describe sections and options that can be updated with Tempest's config file.

==Cluster related options in general==
Required options for Manila tests are described below.

===Section "cli"===
====Option "enabled"====
Set it to
True
value, to be able to run Manila cli tests.

===Section "identity"===

====Option "uri"====
If using localhost it should look likeː
http://127.0.0.1:5000/v2.0/
Used for all tests, if v2 auth enabled.

====Option "uri_v3"====
If using localhost it should look likeː
http://127.0.0.1:5000/v3/

====Option "username"====
If using devstack installation it should have the following valueː
demo
====Option "password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'demo' user.

====Option "tenant_name"====
If using devstack installation it should have the following valueː
demo

====Option "alt_username"====
If using devstack installation it should have the following valueː
alt_demo

====Option "alt_password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'alt_demo' user.

====Option "alt_tenant_name"====
If using devstack installation it should have the following valueː
alt_demo

====Option "admin_username"====
If using devstack installation it should have the following valueː
admin

====Option "admin_password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'admin' user.

====Option "admin_tenant_name"====
If using devstack installation it should have the following valueː
admin

===Section "network"===
====Option "public_network_id"====
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron net-list

====Option "public_router_id"====
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron router-list

==Manila related options==

===Section "service_available"===
This section is defined within vanila Tempest. But "manila" option is defined within Manila's extension.
====Option "manila"====
typeː boolean
default valueː True
descriptionː Whether or not manila is expected to be available

===Section "share"===
This section is defined with Manila's extension.

====Option "catalog_type"====
typeː str
default valueː share
descriptionː Catalog type of the Share service. Should be changed only if it is changed in Manila itself.

====Option "endpoint_type"====
typeː str
default valueː publicURL
descriptionː The endpoint type to use for the share service. Expected values are 'public', 'admin', 'internal', 'publicURL', 'adminURL' and 'internalURL'.

====Option "multitenancy_enabled"====
typeː boolean
default valueː True
descriptionː This option is used to determine backend driver type, multitenant driver uses share-networks, but single-tenant doesn't.

Current singletenant drivers: LVM, GlusterFS, NetApp 7-mode
Current multitenant drivers: Generic, NetApp Cluster-mode (cmode)
Default driver in manila is Generic driver, so "multitenancy_enabled" option is enabled by default.

====Option "enable_protocols"====
typeː list
default valueː nfs,cifs
descriptionː The first value of the list is protocol by default, items of the list show enabled protocols.

Expected values are:
nfs - will be tested only NFS type of shares and common actions
cifs - will be tested only CIFS type of shares and common actions
nfs,cifs - will be tested both share types and common actions, for common actions will be used nfs protocol
cifs,nfs - will be tested both share types and common actions, for common actions will be used cifs protocol

====Option "enable_ip_rules_for_protocols"====
typeː list
default valueː nfs,cifs
descriptionː Whether allow running of ip rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "enable_sid_rules_for_protocols"====
typeː list
default valueː (empty)
descriptionː Whether allow running of sid (username/usergroup) rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "username_for_sid_rules"====
typeː str
default valueː Administrator
descriptionː What name should be used with sid rule tests? Specify it if 'sid rule' tests are enabled.

====Option "storage_protocol"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "alt_share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for alternative member creds. At the moment there are no tests, that use share_network with creds for alternative user.

====Option "admin_share_network_id"====
typeː str
default valueː
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for admin creds.

====Option "multi_backend"====
typeː boolean
default valueː False
descriptionː Does your environment has more than 1 backend or not? If yes, than you can set it to True to enable multibackend tests in tempest.

====Option "backend_names"====
typeː list
default valueː (empty)
descriptionː names of backends in Manila, that Tempest should use. Tempest will use first two values from list and will use it only if option "multi_backend" is Enabled.

====Option "build_interval"====
typeː int
default valueː 3
descriptionː time to wait between resend of requests.

====Option "build_timeout"====
typeː int
default valueː 500
descriptionː time to wait until raise of Exception with timeout.

=How to run tests=
==Run only cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.cli.*manila*\"

==Run only api tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share*\"

==Run only api and cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila*\"

==Run only api and cli tests with specified amount of threads==
export TEMPEST_CONCURRENCY=4
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila* -- --concurrency=$TEMPEST_CONCURRENCY\"
Above action will try to run Tempest in 4 threads, if your host does have at least 4 cores. By default it uses amount of threads equal to amount of available cores.

Manila tests are OK using concurrent threads.

==If something went wrong with test launch==
Exception may be raised on step where tests are gathered and parsed and as a result we would see something similar to the followingː
running testr
Non-zero exit code (2) from test listing. stdout='\xb3)\x01@d@Ytempest.api.share.admin.test_admin_actions.AdminActionsTest.test_force_delete_share[gate]<Z\x90"\xb3)\x01@g@\\
...
tempest.scenario.test_volume_boot_pattern6B\x1e\xcc', stderr=None
error: testr failed (3)
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list

======
Totals
======
Run: 0 in 0.0 sec.
- Passed: 0
- Skipped: 0
- Failed: 0
ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh tempest/api/share/test_extensions.py'
__________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
ERROR: venv: commands failed

There could be different causes for that, in most cases - dependency issues, when some dependency is not installed and can not be imported. Or python syntax error, if something were changed in code, etc...

As we can see, it does not show us real problem and we need dig root cause up. It can be done by a lot of means, for example, try nextː
nosetests -sv path/to/tests
or
.tox/venv/bin/python -m testtools.run discover -t ./ path/to/tests

=How to write own tests=

==Where to put new tests==
All tests, that cover admin API are stored in
tempest/api/share/admin/*
All tests, that cover non-admin API are stored in
tempest/api/share/*

==Add method as API caller==
All such methods are stored in
tempest/services/share/json/shares_client.py
XML is considered as deprecated and tests for it were removed, so, only json "client" is expected.

==Positive/Negative==
Positive and negative tests are separated into different modules, module with "negative" tests should have "_negative" suffix.
Example of module with positive testsː
tempest/api/share/test_shares.py
Example of module with negative testsː
tempest/api/share/test_shares_negative.py

==Resolving concurrency issue==
If some test is sensible to changes that can be done by other tests, this test should use isolated tenant, example of it can be found in quota tests, located in
tempest/api/share/admin/test_quotas.py

==Reusing resources==
If several tests have same dependencies, they should use same resources if possible, it can be created in "setUpClass" method. Example can be found hereː
tempest/api/share/test_shares_actions.py

==Utilization of resources==
Utilization stuff is defined in the following moduleː
tempest/api/share/base.py

Basic actionsː
1) All setUpClass methods should be decorated with "@test.safe_setup" decorator.
Reasonː if we get Exception on some step in setUpClass, tearDownClass won't start. So, this decorator runs 'tearDownClass' in any case.

2) Create resources that should be deleted with methods from base class.
correctː self.create_share() - will add resources to deletion queue
incorrectː self.shares_client.create_share() - won't add resource to deletion queue

Common infoː
All resources in deletion queue will be deleted in LIFO sequence - "last in first out" or "last created first deleted".

There are two deletion queuesː
- runs after each test
- runs after each test-suite (class with tests)
Be attentive setting up it.

Manila/docs/HOWTO use tempest with manila

2014-07-07T11:26:20Z

Vponomaryov: /* Where do Tempest tests for Manila live? */

=Foreword=
This page describes how Tempest project for Manila project testing can be used.

It is assumed that user who reads this is acquainted with, at least, common information about Tempest:
http://docs.openstack.org/developer/tempest/
https://github.com/openstack/tempest

Also, it is assumed, that the following projects has been installed in test environmentː
- cinder
- glance
- keystone
- manila
- neutron
- nova

=Where do Tempest tests for Manila live?=
Manila stores its tests for Tempest in its repo here:
https://github.com/stackforge/manila/tree/master/contrib/tempest

These are pluggable-like files which do not require changing of core Tempest files.
But the way of its pluggability is implemented as work-around for Tempest, which does not have pluggable functionality for 'exceptions', 'config' and 'clients' modules.
It is compatible with master branch of Tempest.

Just copy these files to Tempest dirː
$ TEMPEST_DIR=/path/to/tempest
$ MANILA_DIR=/path/to/manila
$ cp ${MANILA_DIR}/contrib/tempest/tempest/* ${TEMPEST_DIR}/tempest

=Configuring Tempest before running tests=
Any test running with Tempest requires update of Tempest's config file located in
etc/tempest.conf
If it does not exist, just copy template "tempest.conf.sample"
cp tempest.conf.sample tempest.conf

Manila's config extension is defined in
https://github.com/stackforge/manila/blob/master/contrib/tempest/tempest/config_share.py

Subsections below describe sections and options that can be updated with Tempest's config file.

==Cluster related options in general==
Required options for Manila tests are described below.

===Section "cli"===
====Option "enabled"====
Set it to
True
value, to be able to run Manila cli tests.

===Section "identity"===

====Option "uri"====
If using localhost it should look likeː
http://127.0.0.1:5000/v2.0/
Used for all tests, if v2 auth enabled.

====Option "uri_v3"====
If using localhost it should look likeː
http://127.0.0.1:5000/v3/

====Option "username"====
If using devstack installation it should have the following valueː
demo
====Option "password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'demo' user.

====Option "tenant_name"====
If using devstack installation it should have the following valueː
demo

====Option "alt_username"====
If using devstack installation it should have the following valueː
alt_demo

====Option "alt_password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'alt_demo' user.

====Option "alt_tenant_name"====
If using devstack installation it should have the following valueː
alt_demo

====Option "admin_username"====
If using devstack installation it should have the following valueː
admin

====Option "admin_password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'admin' user.

====Option "admin_tenant_name"====
If using devstack installation it should have the following valueː
admin

===Section "network"===
====Option "public_network_id"====
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron net-list

====Option "public_router_id"====
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron router-list

==Manila related options==

===Section "service_available"===
This section is defined within vanila Tempest. But "manila" option is defined within Manila's extension.
====Option "manila"====
typeː boolean
default valueː True
descriptionː Whether or not manila is expected to be available

===Section "share"===
This section is defined with Manila's extension.

====Option "catalog_type"====
typeː str
default valueː share
descriptionː Catalog type of the Share service. Should be changed only if it is changed in Manila itself.

====Option "endpoint_type"====
typeː str
default valueː publicURL
descriptionː The endpoint type to use for the share service. Expected values are 'public', 'admin', 'internal', 'publicURL', 'adminURL' and 'internalURL'.

====Option "multitenancy_enabled"====
typeː boolean
default valueː True
descriptionː This option is used to determine backend driver type, multitenant driver uses share-networks, but single-tenant doesn't.

Current singletenant drivers: LVM, GlusterFS, NetApp 7-mode
Current multitenant drivers: Generic, NetApp Cluster-mode (cmode)
Default driver in manila is Generic driver, so "multitenancy_enabled" option is enabled by default.

====Option "enable_protocols"====
typeː list
default valueː nfs,cifs
descriptionː The first value of the list is protocol by default, items of the list show enabled protocols.

Expected values are:
nfs - will be tested only NFS type of shares and common actions
cifs - will be tested only CIFS type of shares and common actions
nfs,cifs - will be tested both share types and common actions, for common actions will be used nfs protocol
cifs,nfs - will be tested both share types and common actions, for common actions will be used cifs protocol

====Option "enable_ip_rules_for_protocols"====
typeː list
default valueː nfs,cifs
descriptionː Whether allow running of ip rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "enable_sid_rules_for_protocols"====
typeː list
default valueː (empty)
descriptionː Whether allow running of sid (username/usergroup) rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "username_for_sid_rules"====
typeː str
default valueː Administrator
descriptionː What name should be used with sid rule tests? Specify it if 'sid rule' tests are enabled.

====Option "storage_protocol"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "alt_share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for alternative member creds. At the moment there are no tests, that use share_network with creds for alternative user.

====Option "admin_share_network_id"====
typeː str
default valueː
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for admin creds.

====Option "multi_backend"====
typeː boolean
default valueː False
descriptionː Does your environment has more than 1 backend or not? If yes, than you can set it to True to enable multibackend tests in tempest.

====Option "backend_names"====
typeː list
default valueː (empty)
descriptionː names of backends in Manila, that Tempest should use. Tempest will use first two values from list and will use it only if option "multi_backend" is Enabled.

====Option "build_interval"====
typeː int
default valueː 3
descriptionː time to wait between resend of requests.

====Option "build_timeout"====
typeː int
default valueː 500
descriptionː time to wait until raise of Exception with timeout.

=How to run tests=
==Run only cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.cli.*manila*\"

==Run only api tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share*\"

==Run only api and cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila*\"

==Run only api and cli tests with specified amount of threads==
export TEMPEST_CONCURRENCY=4
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila* -- --concurrency=$TEMPEST_CONCURRENCY\"
Above action will try to run Tempest in 4 threads, if your host does have at least 4 cores. By default it uses amount of threads equal to amount of available cores.

Manila tests are OK using concurrent threads.

==If something went wrong with test launch==
Exception may be raised on step where tests are gathered and parsed and as a result we would see something similar to the followingː
running testr
Non-zero exit code (2) from test listing. stdout='\xb3)\x01@d@Ytempest.api.share.admin.test_admin_actions.AdminActionsTest.test_force_delete_share[gate]<Z\x90"\xb3)\x01@g@\\
...
tempest.scenario.test_volume_boot_pattern6B\x1e\xcc', stderr=None
error: testr failed (3)
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list

======
Totals
======
Run: 0 in 0.0 sec.
- Passed: 0
- Skipped: 0
- Failed: 0
ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh tempest/api/share/test_extensions.py'
__________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
ERROR: venv: commands failed

There could be different causes for that, in most cases - dependency issues, when some dependency is not installed and can not be imported. Or python syntax error, if something were changed in code, etc...

As we can see, it does not show us real problem and we need dig root cause up. It can be done by a lot of means, for example, try nextː
nosetests -sv path/to/tests
or
.tox/venv/bin/python -m testtools.run discover -t ./ path/to/tests

=How to write own tests=

==Where to put new tests==
All tests, that cover admin API are stored in
tempest/api/share/admin/*
All tests, that cover non-admin API are stored in
tempest/api/share/*

==Add method as API caller==
All such methods are stored in
tempest/services/share/json/shares_client.py
XML is considered as deprecated and tests for it were removed, so, only json "client" is expected.

==Positive/Negative==
Positive and negative tests are separated into different modules, module with "negative" tests should have "_negative" suffix.
Example of module with positive testsː
tempest/api/share/test_shares.py
Example of module with negative testsː
tempest/api/share/test_shares_negative.py

==Resolving concurrency issue==
If some test is sensible to changes that can be done by other tests, this test should use isolated tenant, example of it can be found in quota tests, located in
tempest/api/share/admin/test_quotas.py

==Reusing resources==
If several tests have same dependencies, they should use same resources if possible, it can be created in "setUpClass" method. Example can be found hereː
tempest/api/share/test_shares_actions.py

==Utilization of resources==
Utilization stuff is defined in the following moduleː
tempest/api/share/base.py

Basic actionsː
1) All setUpClass methods should be decorated with "@test.safe_setup" decorator.
Reasonː if we get Exception on some step in setUpClass, tearDownClass won't start. So, this decorator runs 'tearDownClass' in any case.

2) Create resources that should be deleted with methods from base class.
correctː self.create_share() - will add resources to deletion queue
incorrectː self.shares_client.create_share() - won't add resource to deletion queue

Common infoː
All resources in deletion queue will be deleted in LIFO sequence - "last in first out" or "last created first deleted".

There are two deletion queuesː
- runs after each test
- runs after each test-suite (class with tests)
Be attentive setting up it.

Manila/docs/HOWTO use tempest with manila

2014-07-07T11:25:01Z

Vponomaryov: /* If something went wrong with test launch */

=Foreword=
This page describes how Tempest project for Manila project testing can be used.

It is assumed that user who reads this is acquainted with, at least, common information about Tempest:
http://docs.openstack.org/developer/tempest/
https://github.com/openstack/tempest

Also, it is assumed, that the following projects has been installed in test environmentː
- cinder
- glance
- keystone
- manila
- neutron
- nova

=Where do Tempest tests for Manila live?=
Manila stores its tests for Tempest in its repo here:
https://github.com/stackforge/manila/tree/master/contrib/tempest

These are pluggable-like files which do not require changing of core Tempest files.
But the way of its pluggability is implemented as work-around for Tempest, which does not have pluggable functionality except for config and clients modules.
It is compatible with master branch of Tempest.

Just copy these files to Tempest dirː
$ TEMPEST_DIR=/path/to/tempest
$ MANILA_DIR=/path/to/manila
$ cp ${MANILA_DIR}/contrib/tempest/tempest/* ${TEMPEST_DIR}/tempest

=Configuring Tempest before running tests=
Any test running with Tempest requires update of Tempest's config file located in
etc/tempest.conf
If it does not exist, just copy template "tempest.conf.sample"
cp tempest.conf.sample tempest.conf

Manila's config extension is defined in
https://github.com/stackforge/manila/blob/master/contrib/tempest/tempest/config_share.py

Subsections below describe sections and options that can be updated with Tempest's config file.

==Cluster related options in general==
Required options for Manila tests are described below.

===Section "cli"===
====Option "enabled"====
Set it to
True
value, to be able to run Manila cli tests.

===Section "identity"===

====Option "uri"====
If using localhost it should look likeː
http://127.0.0.1:5000/v2.0/
Used for all tests, if v2 auth enabled.

====Option "uri_v3"====
If using localhost it should look likeː
http://127.0.0.1:5000/v3/

====Option "username"====
If using devstack installation it should have the following valueː
demo
====Option "password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'demo' user.

====Option "tenant_name"====
If using devstack installation it should have the following valueː
demo

====Option "alt_username"====
If using devstack installation it should have the following valueː
alt_demo

====Option "alt_password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'alt_demo' user.

====Option "alt_tenant_name"====
If using devstack installation it should have the following valueː
alt_demo

====Option "admin_username"====
If using devstack installation it should have the following valueː
admin

====Option "admin_password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'admin' user.

====Option "admin_tenant_name"====
If using devstack installation it should have the following valueː
admin

===Section "network"===
====Option "public_network_id"====
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron net-list

====Option "public_router_id"====
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron router-list

==Manila related options==

===Section "service_available"===
This section is defined within vanila Tempest. But "manila" option is defined within Manila's extension.
====Option "manila"====
typeː boolean
default valueː True
descriptionː Whether or not manila is expected to be available

===Section "share"===
This section is defined with Manila's extension.

====Option "catalog_type"====
typeː str
default valueː share
descriptionː Catalog type of the Share service. Should be changed only if it is changed in Manila itself.

====Option "endpoint_type"====
typeː str
default valueː publicURL
descriptionː The endpoint type to use for the share service. Expected values are 'public', 'admin', 'internal', 'publicURL', 'adminURL' and 'internalURL'.

====Option "multitenancy_enabled"====
typeː boolean
default valueː True
descriptionː This option is used to determine backend driver type, multitenant driver uses share-networks, but single-tenant doesn't.

Current singletenant drivers: LVM, GlusterFS, NetApp 7-mode
Current multitenant drivers: Generic, NetApp Cluster-mode (cmode)
Default driver in manila is Generic driver, so "multitenancy_enabled" option is enabled by default.

====Option "enable_protocols"====
typeː list
default valueː nfs,cifs
descriptionː The first value of the list is protocol by default, items of the list show enabled protocols.

Expected values are:
nfs - will be tested only NFS type of shares and common actions
cifs - will be tested only CIFS type of shares and common actions
nfs,cifs - will be tested both share types and common actions, for common actions will be used nfs protocol
cifs,nfs - will be tested both share types and common actions, for common actions will be used cifs protocol

====Option "enable_ip_rules_for_protocols"====
typeː list
default valueː nfs,cifs
descriptionː Whether allow running of ip rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "enable_sid_rules_for_protocols"====
typeː list
default valueː (empty)
descriptionː Whether allow running of sid (username/usergroup) rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "username_for_sid_rules"====
typeː str
default valueː Administrator
descriptionː What name should be used with sid rule tests? Specify it if 'sid rule' tests are enabled.

====Option "storage_protocol"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "alt_share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for alternative member creds. At the moment there are no tests, that use share_network with creds for alternative user.

====Option "admin_share_network_id"====
typeː str
default valueː
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for admin creds.

====Option "multi_backend"====
typeː boolean
default valueː False
descriptionː Does your environment has more than 1 backend or not? If yes, than you can set it to True to enable multibackend tests in tempest.

====Option "backend_names"====
typeː list
default valueː (empty)
descriptionː names of backends in Manila, that Tempest should use. Tempest will use first two values from list and will use it only if option "multi_backend" is Enabled.

====Option "build_interval"====
typeː int
default valueː 3
descriptionː time to wait between resend of requests.

====Option "build_timeout"====
typeː int
default valueː 500
descriptionː time to wait until raise of Exception with timeout.

=How to run tests=
==Run only cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.cli.*manila*\"

==Run only api tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share*\"

==Run only api and cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila*\"

==Run only api and cli tests with specified amount of threads==
export TEMPEST_CONCURRENCY=4
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila* -- --concurrency=$TEMPEST_CONCURRENCY\"
Above action will try to run Tempest in 4 threads, if your host does have at least 4 cores. By default it uses amount of threads equal to amount of available cores.

Manila tests are OK using concurrent threads.

==If something went wrong with test launch==
Exception may be raised on step where tests are gathered and parsed and as a result we would see something similar to the followingː
running testr
Non-zero exit code (2) from test listing. stdout='\xb3)\x01@d@Ytempest.api.share.admin.test_admin_actions.AdminActionsTest.test_force_delete_share[gate]<Z\x90"\xb3)\x01@g@\\
...
tempest.scenario.test_volume_boot_pattern6B\x1e\xcc', stderr=None
error: testr failed (3)
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list

======
Totals
======
Run: 0 in 0.0 sec.
- Passed: 0
- Skipped: 0
- Failed: 0
ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh tempest/api/share/test_extensions.py'
__________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
ERROR: venv: commands failed

There could be different causes for that, in most cases - dependency issues, when some dependency is not installed and can not be imported. Or python syntax error, if something were changed in code, etc...

As we can see, it does not show us real problem and we need dig root cause up. It can be done by a lot of means, for example, try nextː
nosetests -sv path/to/tests
or
.tox/venv/bin/python -m testtools.run discover -t ./ path/to/tests

=How to write own tests=

==Where to put new tests==
All tests, that cover admin API are stored in
tempest/api/share/admin/*
All tests, that cover non-admin API are stored in
tempest/api/share/*

==Add method as API caller==
All such methods are stored in
tempest/services/share/json/shares_client.py
XML is considered as deprecated and tests for it were removed, so, only json "client" is expected.

==Positive/Negative==
Positive and negative tests are separated into different modules, module with "negative" tests should have "_negative" suffix.
Example of module with positive testsː
tempest/api/share/test_shares.py
Example of module with negative testsː
tempest/api/share/test_shares_negative.py

==Resolving concurrency issue==
If some test is sensible to changes that can be done by other tests, this test should use isolated tenant, example of it can be found in quota tests, located in
tempest/api/share/admin/test_quotas.py

==Reusing resources==
If several tests have same dependencies, they should use same resources if possible, it can be created in "setUpClass" method. Example can be found hereː
tempest/api/share/test_shares_actions.py

==Utilization of resources==
Utilization stuff is defined in the following moduleː
tempest/api/share/base.py

Basic actionsː
1) All setUpClass methods should be decorated with "@test.safe_setup" decorator.
Reasonː if we get Exception on some step in setUpClass, tearDownClass won't start. So, this decorator runs 'tearDownClass' in any case.

2) Create resources that should be deleted with methods from base class.
correctː self.create_share() - will add resources to deletion queue
incorrectː self.shares_client.create_share() - won't add resource to deletion queue

Common infoː
All resources in deletion queue will be deleted in LIFO sequence - "last in first out" or "last created first deleted".

There are two deletion queuesː
- runs after each test
- runs after each test-suite (class with tests)
Be attentive setting up it.

Manila/docs/HOWTO use tempest with manila

2014-07-07T11:20:38Z

Vponomaryov: /* Option "uri" */

=Foreword=
This page describes how Tempest project for Manila project testing can be used.

It is assumed that user who reads this is acquainted with, at least, common information about Tempest:
http://docs.openstack.org/developer/tempest/
https://github.com/openstack/tempest

Also, it is assumed, that the following projects has been installed in test environmentː
- cinder
- glance
- keystone
- manila
- neutron
- nova

=Where do Tempest tests for Manila live?=
Manila stores its tests for Tempest in its repo here:
https://github.com/stackforge/manila/tree/master/contrib/tempest

These are pluggable-like files which do not require changing of core Tempest files.
But the way of its pluggability is implemented as work-around for Tempest, which does not have pluggable functionality except for config and clients modules.
It is compatible with master branch of Tempest.

Just copy these files to Tempest dirː
$ TEMPEST_DIR=/path/to/tempest
$ MANILA_DIR=/path/to/manila
$ cp ${MANILA_DIR}/contrib/tempest/tempest/* ${TEMPEST_DIR}/tempest

=Configuring Tempest before running tests=
Any test running with Tempest requires update of Tempest's config file located in
etc/tempest.conf
If it does not exist, just copy template "tempest.conf.sample"
cp tempest.conf.sample tempest.conf

Manila's config extension is defined in
https://github.com/stackforge/manila/blob/master/contrib/tempest/tempest/config_share.py

Subsections below describe sections and options that can be updated with Tempest's config file.

==Cluster related options in general==
Required options for Manila tests are described below.

===Section "cli"===
====Option "enabled"====
Set it to
True
value, to be able to run Manila cli tests.

===Section "identity"===

====Option "uri"====
If using localhost it should look likeː
http://127.0.0.1:5000/v2.0/
Used for all tests, if v2 auth enabled.

====Option "uri_v3"====
If using localhost it should look likeː
http://127.0.0.1:5000/v3/

====Option "username"====
If using devstack installation it should have the following valueː
demo
====Option "password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'demo' user.

====Option "tenant_name"====
If using devstack installation it should have the following valueː
demo

====Option "alt_username"====
If using devstack installation it should have the following valueː
alt_demo

====Option "alt_password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'alt_demo' user.

====Option "alt_tenant_name"====
If using devstack installation it should have the following valueː
alt_demo

====Option "admin_username"====
If using devstack installation it should have the following valueː
admin

====Option "admin_password"====
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'admin' user.

====Option "admin_tenant_name"====
If using devstack installation it should have the following valueː
admin

===Section "network"===
====Option "public_network_id"====
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron net-list

====Option "public_router_id"====
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron router-list

==Manila related options==

===Section "service_available"===
This section is defined within vanila Tempest. But "manila" option is defined within Manila's extension.
====Option "manila"====
typeː boolean
default valueː True
descriptionː Whether or not manila is expected to be available

===Section "share"===
This section is defined with Manila's extension.

====Option "catalog_type"====
typeː str
default valueː share
descriptionː Catalog type of the Share service. Should be changed only if it is changed in Manila itself.

====Option "endpoint_type"====
typeː str
default valueː publicURL
descriptionː The endpoint type to use for the share service. Expected values are 'public', 'admin', 'internal', 'publicURL', 'adminURL' and 'internalURL'.

====Option "multitenancy_enabled"====
typeː boolean
default valueː True
descriptionː This option is used to determine backend driver type, multitenant driver uses share-networks, but single-tenant doesn't.

Current singletenant drivers: LVM, GlusterFS, NetApp 7-mode
Current multitenant drivers: Generic, NetApp Cluster-mode (cmode)
Default driver in manila is Generic driver, so "multitenancy_enabled" option is enabled by default.

====Option "enable_protocols"====
typeː list
default valueː nfs,cifs
descriptionː The first value of the list is protocol by default, items of the list show enabled protocols.

Expected values are:
nfs - will be tested only NFS type of shares and common actions
cifs - will be tested only CIFS type of shares and common actions
nfs,cifs - will be tested both share types and common actions, for common actions will be used nfs protocol
cifs,nfs - will be tested both share types and common actions, for common actions will be used cifs protocol

====Option "enable_ip_rules_for_protocols"====
typeː list
default valueː nfs,cifs
descriptionː Whether allow running of ip rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "enable_sid_rules_for_protocols"====
typeː list
default valueː (empty)
descriptionː Whether allow running of sid (username/usergroup) rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "username_for_sid_rules"====
typeː str
default valueː Administrator
descriptionː What name should be used with sid rule tests? Specify it if 'sid rule' tests are enabled.

====Option "storage_protocol"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "alt_share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for alternative member creds. At the moment there are no tests, that use share_network with creds for alternative user.

====Option "admin_share_network_id"====
typeː str
default valueː
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for admin creds.

====Option "multi_backend"====
typeː boolean
default valueː False
descriptionː Does your environment has more than 1 backend or not? If yes, than you can set it to True to enable multibackend tests in tempest.

====Option "backend_names"====
typeː list
default valueː (empty)
descriptionː names of backends in Manila, that Tempest should use. Tempest will use first two values from list and will use it only if option "multi_backend" is Enabled.

====Option "build_interval"====
typeː int
default valueː 3
descriptionː time to wait between resend of requests.

====Option "build_timeout"====
typeː int
default valueː 500
descriptionː time to wait until raise of Exception with timeout.

=How to run tests=
==Run only cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.cli.*manila*\"

==Run only api tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share*\"

==Run only api and cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila*\"

==Run only api and cli tests with specified amount of threads==
export TEMPEST_CONCURRENCY=4
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila* -- --concurrency=$TEMPEST_CONCURRENCY\"
Above action will try to run Tempest in 4 threads, if your host does have at least 4 cores. By default it uses amount of threads equal to amount of available cores.

Manila tests are OK using concurrent threads.

==If something went wrong with test launch==
Exception may be raised on step where tests are gathered and parsed and as a result we would see the followingː
running testr
Non-zero exit code (2) from test listing. stdout='\xb3)\x01@d@Ytempest.api.share.admin.test_admin_actions.AdminActionsTest.test_force_delete_share[gate]<Z\x90"\xb3)\x01@g@\\
...
tempest.scenario.test_volume_boot_pattern6B\x1e\xcc', stderr=None
error: testr failed (3)
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list

======
Totals
======
Run: 0 in 0.0 sec.
- Passed: 0
- Skipped: 0
- Failed: 0
ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh tempest/api/share/test_extensions.py'
__________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
ERROR: venv: commands failed

There could be different causes for that, in most cases - dependency issues, when some dependency is not installed and can not be imported. Or python syntax error, if something were changed in code, etc...

As we can see, it does not show us real problem and we need dig root cause up. It can be done by a lot of means, for example, try nextː
nosetests -sv path/to/tests
or
.tox/venv/bin/python -m testtools.run discover -t ./ path/to/tests

=How to write own tests=

==Where to put new tests==
All tests, that cover admin API are stored in
tempest/api/share/admin/*
All tests, that cover non-admin API are stored in
tempest/api/share/*

==Add method as API caller==
All such methods are stored in
tempest/services/share/json/shares_client.py
XML is considered as deprecated and tests for it were removed, so, only json "client" is expected.

==Positive/Negative==
Positive and negative tests are separated into different modules, module with "negative" tests should have "_negative" suffix.
Example of module with positive testsː
tempest/api/share/test_shares.py
Example of module with negative testsː
tempest/api/share/test_shares_negative.py

==Resolving concurrency issue==
If some test is sensible to changes that can be done by other tests, this test should use isolated tenant, example of it can be found in quota tests, located in
tempest/api/share/admin/test_quotas.py

==Reusing resources==
If several tests have same dependencies, they should use same resources if possible, it can be created in "setUpClass" method. Example can be found hereː
tempest/api/share/test_shares_actions.py

==Utilization of resources==
Utilization stuff is defined in the following moduleː
tempest/api/share/base.py

Basic actionsː
1) All setUpClass methods should be decorated with "@test.safe_setup" decorator.
Reasonː if we get Exception on some step in setUpClass, tearDownClass won't start. So, this decorator runs 'tearDownClass' in any case.

2) Create resources that should be deleted with methods from base class.
correctː self.create_share() - will add resources to deletion queue
incorrectː self.shares_client.create_share() - won't add resource to deletion queue

Common infoː
All resources in deletion queue will be deleted in LIFO sequence - "last in first out" or "last created first deleted".

There are two deletion queuesː
- runs after each test
- runs after each test-suite (class with tests)
Be attentive setting up it.

Manila/docs/HOWTO use tempest with manila

2014-07-03T16:28:43Z

Vponomaryov: /* Section "share" */

=Foreword=
This page describes how can be used Tempest project for testing purposes of Manila project.

It is assumed that person who reads this is acquainted with, at least, common information about Tempest:
http://docs.openstack.org/developer/tempest/
https://github.com/openstack/tempest

Also, it is assumed, that test environment has next projects installedː
cinder
glance
keystone
manila
neutron
nova

=Where does Tempest tests for Manila live?=
Manila stores its tests for Tempest in its repo here:
https://github.com/stackforge/manila/tree/master/contrib/tempest

It is pluggable-like files without requirements to change core Tempest files.
But the way of its pluggability is work-around for Tempest, which hasn't pluggable functionality for exceptions, config and clients modules.
It is compatible with master branch of Tempest.

Just copy these files to Tempest dirː
$ TEMPEST_DIR=/path/to/tempest
$ MANILA_DIR=/path/to/manila
$ cp ${MANILA_DIR}/contrib/tempest/tempest/* ${TEMPEST_DIR}/tempest

=Configuring Tempest before running tests=
Any test run with Tempest requires update of Tempest's config file located in
etc/tempest.conf
If it does not exist, just copy template "tempest.conf.sample"
cp tempest.conf.sample tempest.conf

Manila's config extension is defined in
https://github.com/stackforge/manila/blob/master/contrib/tempest/tempest/config_share.py

Subsections below describes sections and options that can be updated with Tempest's config file.

==Cluster related options in general==
Below described required options for Manila tests

===Section "cli"===
====Option "enabled"====
Set it to
True
value, to be able to run Manila cli tests.

===Section "identity"===

====Option "uri"====
Using localhost it looks like nextː
http://127.0.0.1:5000/v2.0/
Used for all tests, if v2 auth enabled.

====Option "uri_v3"====
Using localhost it looks like nextː
http://127.0.0.1:5000/v3/

====Option "username"====
Using devstack installation it should have next valueː
demo
====Option "password"====
Using devstack installation it should have next valueː
secrete
Password can be redefined, just put here password from 'demo' user.

====Option "tenant_name"====
Using devstack installation it should have next valueː
demo

====Option "alt_username"====
Using devstack installation it should have next valueː
alt_demo

====Option "alt_password"====
Using devstack installation it should have next valueː
secrete
Password can be redefined, just put here password from 'alt_demo' user.

====Option "alt_tenant_name"====
Using devstack installation it should have next valueː
alt_demo

====Option "admin_username"====
Using devstack installation it should have next valueː
admin

====Option "admin_password"====
Using devstack installation it should have next valueː
secrete
Password can be redefined, just put here password from 'admin' user.

====Option "admin_tenant_name"====
Using devstack installation it should have next valueː
admin

===Section "network"===
====Option "public_network_id"====
Used for creation of isolated tenants. Its value can be got by
$ neutron net-list

====Option "public_router_id"====
Used for creation of isolated tenants. Its value can be got by
$ neutron router-list

==Manila related options==

===Section "service_available"===
This section is defined withinin vanila Tempest. But "manila" option is defined within Manila's extension.
====Option "manila"====
typeː boolean
default valueː True
descriptionː Whether or not manila is expected to be available

===Section "share"===
This section is defined with Manila's extension.

====Option "catalog_type"====
typeː str
default valueː share
descriptionː Catalog type of the Share service. Should be changed only if it is changed in Manila itself.

====Option "endpoint_type"====
typeː str
default valueː publicURL
descriptionː The endpoint type to use for the share service. Expected values are 'public', 'admin', 'internal', 'publicURL', 'adminURL' and 'internalURL'.

====Option "multitenancy_enabled"====
typeː boolean
default valueː True
descriptionː This option used to determine backend driver type, multitenant driver uses share-networks, but single-tenant doesn't.

Current singletenant drivers: LVM, GlusterFS, NetApp 7-mode
Current multitenant drivers: Generic, NetApp Cluster-mode (cmode)
Default driver in manila is Generic driver, so "multitenancy_enabled" option is enabled by default.

====Option "enable_protocols"====
typeː list
default valueː nfs,cifs
descriptionː First value of list is protocol by default, items of list show enabled protocols at all.

Expected values are:
nfs - will be tested only NFS type of shares and common actions
cifs - will be tested only CIFS type of shares and common actions
nfs,cifs - will be tested both share types and common actions, for common actions will be used nfs protocol
cifs,nfs - will be tested both share types and common actions, for common actions will be used cifs protocol

====Option "enable_ip_rules_for_protocols"====
typeː list
default valueː nfs,cifs
descriptionː Whether allow run of ip rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "enable_sid_rules_for_protocols"====
typeː list
default valueː (empty)
descriptionː Whether allow run of sid (username/usergroup) rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "username_for_sid_rules"====
typeː str
default valueː Administrator
descriptionː What name should be used with sid rule tests? Specify it if 'sid rule' tests are enabled.

====Option "storage_protocol"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "alt_share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for alternative member creds. At the moment there are no tests, that use share_network with creds for alternative user.

====Option "admin_share_network_id"====
typeː str
default valueː
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for admin creds.

====Option "multi_backend"====
typeː boolean
default valueː False
descriptionː Does your environment has more than 1 backend or not? If yes, than you can set it to True to enable multibackend tests in tempest.

====Option "backend_names"====
typeː list
default valueː (empty)
descriptionː names of backends in Manila, that Tempest should use. Tempest will use first two values from list and will use it only if option "multi_backend" is Enabled.

====Option "build_interval"====
typeː int
default valueː 3
descriptionː time to wait between resend of requests.

====Option "build_timeout"====
typeː int
default valueː 500
descriptionː time to wait until raise of Exception with timeout.

=How to run tests=
==Run only cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.cli.*manila*\"

==Run only api tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share*\"

==Run only api and cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila*\"

==Run only api and cli tests with specified amount of threads==
export TEMPEST_CONCURRENCY=4
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila* -- --concurrency=$TEMPEST_CONCURRENCY\"
Above action will try to run Tempest in 4 threads, if your host does have at least 4 cores. By default it uses amount of threads equal to amount of available cores.

Manila tests are OK using concurrent threads.

==If something went wrong with test launch==
On step where tests are gathered and parsed can be raised Exception and as a result we would see something like nextː
running testr
Non-zero exit code (2) from test listing. stdout='\xb3)\x01@d@Ytempest.api.share.admin.test_admin_actions.AdminActionsTest.test_force_delete_share[gate]<Z\x90"\xb3)\x01@g@\\
...
tempest.scenario.test_volume_boot_pattern6B\x1e\xcc', stderr=None
error: testr failed (3)
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list

======
Totals
======
Run: 0 in 0.0 sec.
- Passed: 0
- Skipped: 0
- Failed: 0
ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh tempest/api/share/test_extensions.py'
__________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
ERROR: venv: commands failed

There could be different causes for that, in most cases - dependency issues, when some dependency is not installed and can not be imported. Or python syntax error, if something were changed in code, etc...

As we can see, it does not show us real problem and we need dig root cause up. It can be done by a lot of cases, for example, try nextː
nosetests -sv path/to/tests
or
.tox/venv/bin/python -m testtools.run discover -t ./ path/to/tests

=How to write own tests=

==Where to put new tests==
All tests, that cover admin API are stored in
tempest/api/share/admin/*
All tests, that cover non-admin API are stored in
tempest/api/share/*

==Add method as API caller==
All such methods are stored in
tempest/services/share/json/shares_client.py
XML is considered as deprecated and tests for it were removed, so, only json "client" is expected.

==Positive/Negative==
Positive and negative tests are separated into different modules, module with "negative" tests should have "_negative" suffix.
Example of module with positive testsː
tempest/api/share/test_shares.py
Example of module with negative testsː
tempest/api/share/test_shares_negative.py

==Resolving concurrency issue==
If some test is sensible to changes that can be done by other tests, this test should use isolated tenant, example of it can be found in quota tests, located in
tempest/api/share/admin/test_quotas.py

==Reusing resources==
If several tests have same dependencies, they should use same resources if possible, it can be created in "setUpClass" method. Example can be found hereː
tempest/api/share/test_shares_actions.py

==Utilization of resources==
Utilisation stuff defined in next moduleː
tempest/api/share/base.py

Basic actionsː
1) All setUpClass methods should be decorated with "@test.safe_setup" decorator.
Reasonː if we get Exception on some step in setUpClass, tearDownClass won't start, where created resources are deleted. So, this decorator makes 'tearDownClass' run in any case.

2) Create resources that should be deleted with methods from base class.
correctː self.create_share() - will add resources to deletion queue
incorrectː self.shares_client.create_share() - won't add resource to deletion queue

Common infoː
All resources in deletion queue will be deleted in LIFO sequence - "last in first out" or "last created first deleted".

There are two deletion queuesː
- runs after each test
- runs after each test-suite (class with tests)
Be attentive setting up it.

Manila/HOWTO use tempest with manila

2014-07-03T10:21:17Z

Vponomaryov: Vponomaryov moved page Manila/HOWTO use tempest with manila to Manila/docs/HOWTO use tempest with manila: Moved from root dir to docs subdir.

#REDIRECT [[Manila/docs/HOWTO use tempest with manila]]

Manila/docs/HOWTO use tempest with manila

2014-07-03T10:21:16Z

Vponomaryov: Vponomaryov moved page Manila/HOWTO use tempest with manila to Manila/docs/HOWTO use tempest with manila: Moved from root dir to docs subdir.

=Foreword=
This page describes how can be used Tempest project for testing purposes of Manila project.

It is assumed that person who reads this is acquainted with, at least, common information about Tempest:
http://docs.openstack.org/developer/tempest/
https://github.com/openstack/tempest

Also, it is assumed, that test environment has next projects installedː
cinder
glance
keystone
manila
neutron
nova

=Where does Tempest tests for Manila live?=
Manila stores its tests for Tempest in its repo here:
https://github.com/stackforge/manila/tree/master/contrib/tempest

It is pluggable-like files without requirements to change core Tempest files.
But the way of its pluggability is work-around for Tempest, which hasn't pluggable functionality for exceptions, config and clients modules.
It is compatible with master branch of Tempest.

Just copy these files to Tempest dirː
$ TEMPEST_DIR=/path/to/tempest
$ MANILA_DIR=/path/to/manila
$ cp ${MANILA_DIR}/contrib/tempest/tempest/* ${TEMPEST_DIR}/tempest

=Configuring Tempest before running tests=
Any test run with Tempest requires update of Tempest's config file located in
etc/tempest.conf
If it does not exist, just copy template "tempest.conf.sample"
cp tempest.conf.sample tempest.conf

Manila's config extension is defined in
https://github.com/stackforge/manila/blob/master/contrib/tempest/tempest/config_share.py

Subsections below describes sections and options that can be updated with Tempest's config file.

==Cluster related options in general==
Below described required options for Manila tests

===Section "cli"===
====Option "enabled"====
Set it to
True
value, to be able to run Manila cli tests.

===Section "identity"===

====Option "uri"====
Using localhost it looks like nextː
http://127.0.0.1:5000/v2.0/
Used for all tests, if v2 auth enabled.

====Option "uri_v3"====
Using localhost it looks like nextː
http://127.0.0.1:5000/v3/

====Option "username"====
Using devstack installation it should have next valueː
demo
====Option "password"====
Using devstack installation it should have next valueː
secrete
Password can be redefined, just put here password from 'demo' user.

====Option "tenant_name"====
Using devstack installation it should have next valueː
demo

====Option "alt_username"====
Using devstack installation it should have next valueː
alt_demo

====Option "alt_password"====
Using devstack installation it should have next valueː
secrete
Password can be redefined, just put here password from 'alt_demo' user.

====Option "alt_tenant_name"====
Using devstack installation it should have next valueː
alt_demo

====Option "admin_username"====
Using devstack installation it should have next valueː
admin

====Option "admin_password"====
Using devstack installation it should have next valueː
secrete
Password can be redefined, just put here password from 'admin' user.

====Option "admin_tenant_name"====
Using devstack installation it should have next valueː
admin

===Section "network"===
====Option "public_network_id"====
Used for creation of isolated tenants. Its value can be got by
$ neutron net-list

====Option "public_router_id"====
Used for creation of isolated tenants. Its value can be got by
$ neutron router-list

==Manila related options==

===Section "service_available"===
This section is defined withinin vanila Tempest. But "manila" option is defined within Manila's extension.
====Option "manila"====
typeː boolean
default valueː True
descriptionː Whether or not manila is expected to be available

===Section "share"===
This section is defined with Manila's extension.

====Option "catalog_type"====
typeː str
default valueː share
descriptionː Catalog type of the Share service. Should be changed only if it is changed in Manila itself.

====Option "endpoint_type"====
typeː str
default valueː publicURL
descriptionː The endpoint type to use for the share service. Expected values are 'public', 'admin', 'internal', 'publicURL', 'adminURL' and 'internalURL'.

====Option "multitenancy_enabled"====
typeː boolean
default valueː True
descriptionː This option used to determine backend driver type, multitenant driver uses share-networks, but single-tenant doesn't.

Current singletenant drivers: LVM, GlusterFS, NetApp 7-mode
Current multitenant drivers: Generic, NetApp Cluster-mode (cmode)
Default driver in manila is Generic driver, so "multitenancy_enabled" option is enabled by default.

====Option "enable_protocols"====
typeː list
default valueː nfs,cifs
descriptionː First value of list is protocol by default, items of list show enabled protocols at all.

Expected values are:
nfs - will be tested only NFS type of shares and common actions
cifs - will be tested only CIFS type of shares and common actions
nfs,cifs - will be tested both share types and common actions, for common actions will be used nfs protocol
cifs,nfs - will be tested both share types and common actions, for common actions will be used cifs protocol

====Option "enable_ip_rules_for_protocols"====
typeː list
default valueː nfs,cifs
descriptionː Whether allow run of ip rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "enable_sid_rules_for_protocols"====
typeː list
default valueː (empty)
descriptionː Whether allow run of sid (username/usergroup) rules tests or not, and which share protocols should be used.

Expected values are:
nfs - sid rules will be tested only for nfs protocol
cifs - sid rules will be tested only for cifs protocol
nfs,cifs - sid rules will be tested with both share protocols
cifs,nfs - sid rules will be tested with both share protocols

====Option "username_for_sid_rules"====
typeː str
default valueː Administrator
descriptionː What name should be used with sid rule tests? Specify it if 'sid rule' tests are enabled.

====Option "storage_protocol"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.

====Option "share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for alternative member creds. At the moment there are no tests, that use share_network with creds for alternative user.

====Option "alt_share_network_id"====
typeː str
default valueː "" (empty)
descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for admin creds.

====Option "admin_share_network_id"====
typeː str
default valueː
descriptionː

====Option "multi_backend"====
typeː boolean
default valueː False
descriptionː Does your environment has more than 1 backend or not? If yes, than you can set it to True to enable multibackend tests in tempest.

====Option "backend_names"====
typeː list
default valueː (empty)
descriptionː names of backends in Manila, that Tempest should use. Tempest will use first two values from list and will use it only if option "multi_backend" is Enabled.

====Option "build_interval"====
typeː int
default valueː 3
descriptionː time to wait between resend of requests.

====Option "build_timeout"====
typeː int
default valueː 500
descriptionː time to wait until raise of Exception with timeout.

=How to run tests=
==Run only cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.cli.*manila*\"

==Run only api tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share*\"

==Run only api and cli tests==
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila*\"

==Run only api and cli tests with specified amount of threads==
export TEMPEST_CONCURRENCY=4
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila* -- --concurrency=$TEMPEST_CONCURRENCY\"
Above action will try to run Tempest in 4 threads, if your host does have at least 4 cores. By default it uses amount of threads equal to amount of available cores.

Manila tests are OK using concurrent threads.

==If something went wrong with test launch==
On step where tests are gathered and parsed can be raised Exception and as a result we would see something like nextː
running testr
Non-zero exit code (2) from test listing. stdout='\xb3)\x01@d@Ytempest.api.share.admin.test_admin_actions.AdminActionsTest.test_force_delete_share[gate]<Z\x90"\xb3)\x01@g@\\
...
tempest.scenario.test_volume_boot_pattern6B\x1e\xcc', stderr=None
error: testr failed (3)
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list

======
Totals
======
Run: 0 in 0.0 sec.
- Passed: 0
- Skipped: 0
- Failed: 0
ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh tempest/api/share/test_extensions.py'
__________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
ERROR: venv: commands failed

There could be different causes for that, in most cases - dependency issues, when some dependency is not installed and can not be imported. Or python syntax error, if something were changed in code, etc...

As we can see, it does not show us real problem and we need dig root cause up. It can be done by a lot of cases, for example, try nextː
nosetests -sv path/to/tests
or
.tox/venv/bin/python -m testtools.run discover -t ./ path/to/tests

=How to write own tests=

==Where to put new tests==
All tests, that cover admin API are stored in
tempest/api/share/admin/*
All tests, that cover non-admin API are stored in
tempest/api/share/*

==Add method as API caller==
All such methods are stored in
tempest/services/share/json/shares_client.py
XML is considered as deprecated and tests for it were removed, so, only json "client" is expected.

==Positive/Negative==
Positive and negative tests are separated into different modules, module with "negative" tests should have "_negative" suffix.
Example of module with positive testsː
tempest/api/share/test_shares.py
Example of module with negative testsː
tempest/api/share/test_shares_negative.py

==Resolving concurrency issue==
If some test is sensible to changes that can be done by other tests, this test should use isolated tenant, example of it can be found in quota tests, located in
tempest/api/share/admin/test_quotas.py

==Reusing resources==
If several tests have same dependencies, they should use same resources if possible, it can be created in "setUpClass" method. Example can be found hereː
tempest/api/share/test_shares_actions.py

==Utilization of resources==
Utilisation stuff defined in next moduleː
tempest/api/share/base.py

Basic actionsː
1) All setUpClass methods should be decorated with "@test.safe_setup" decorator.
Reasonː if we get Exception on some step in setUpClass, tearDownClass won't start, where created resources are deleted. So, this decorator makes 'tearDownClass' run in any case.

2) Create resources that should be deleted with methods from base class.
correctː self.create_share() - will add resources to deletion queue
incorrectː self.shares_client.create_share() - won't add resource to deletion queue

Common infoː
All resources in deletion queue will be deleted in LIFO sequence - "last in first out" or "last created first deleted".

There are two deletion queuesː
- runs after each test
- runs after each test-suite (class with tests)
Be attentive setting up it.

Manila/docs

2014-07-03T10:17:29Z

Vponomaryov: /* Manila Documentation */

== Manila Documentation ==
 
[[Manila/API|API specification]]
 
[[Manila/docs/db|Database specification]]
 
[[Manila/docs/API-roadmap|APIs to be implemented]]
 
[[Manila/docs/HOWTO_use_manila_with_horizon|HOWTO use manila with horizon]]
 
[[Manila/HOWTO_use_tempest_with_manila|HOWTO use tempest with manila]]
 

== Setting up DevStack with Manila ==
 
[[Manila/docs/Setting_up_DevStack_with_Manila_on_Fedora_20|On Fedora 20]]