OpenStack - User contributions [en]

Stackforge Namespace Retirement

2015-09-28T05:31:20Z

Ramana Raja: /* Active Projects to Move */

=== Background ===

The stackforge/ git namespace is being retired, and active projects are being moved to the openstack/ namespace. See this mailing list post for [http://lists.openstack.org/pipermail/openstack-dev/2015-August/071816.html full background]. These changes are scheduled to occur on October 17, 2015.

There are two lists below, one for active projects that should be moved, and a second for inactive projects that should become read-only. Please update them to add your project to the correct list.

=== Active Projects to Move ===
Active stackforge projects that wish to move into the openstack/ namespace should be added to this list. Projects in this list will be automatically moved by the Infrastructure team to openstack/ on October 17. Please note that no other renames can happen during this move -- projects will strictly be moved from stackforge/ to openstack/ with their existing names.

* aeromancer
* anvil
* bansho
* blazar
* blazar-nova
* ceilometer-powervm
* ceilometer-zvm
* cl-openstack-client
* cloud-init
* cloudbase-init
* clouddocs-maven-plugin
* cloudkitty
* cloudkitty-dashboard
* cloudpulse
* cognitive
* compass-adapters
* compass-core
* compass-specs
* compass-web
* compute-hyperv
* devstack-plugin-glusterfs
* devstack-plugin-sheepdog
* doc8
* dox
* drbd-devstack
* driverlog
* ec2-api
* faafo
* freezer
* freezer-api
* freezer-web-ui
* fuel-agent
* fuel-astute
* fuel-dev-tools
* fuel-devops
* fuel-docs
* fuel-library
* fuel-main
* fuel-mirror
* fuel-nailgun-agent
* fuel-octane
* fuel-ostf
* fuel-ostf-plugin
* fuel-plugin-calamari
* fuel-plugin-calico
* fuel-plugin-ceilometer-redis
* fuel-plugin-cinder-netapp
* fuel-plugin-cisco-aci
* fuel-plugin-contrail
* fuel-plugin-dbaas-trove
* fuel-plugin-detach-database
* fuel-plugin-detach-keystone
* fuel-plugin-detach-rabbitmq
* fuel-plugin-elasticsearch-kibana
* fuel-plugin-external-emc
* fuel-plugin-external-glusterfs
* fuel-plugin-external-zabbix
* fuel-plugin-glance-nfs
* fuel-plugin-ha-fencing
* fuel-plugin-influxdb-grafana
* fuel-plugin-ironic
* fuel-plugin-ldap
* fuel-plugin-lma-collector
* fuel-plugin-lma-infrastructure-alerting
* fuel-plugin-mellanox
* fuel-plugin-midonet
* fuel-plugin-neutron-fwaas
* fuel-plugin-neutron-lbaas
* fuel-plugin-neutron-vpnaas
* fuel-plugin-nova-nfs
* fuel-plugin-nsxv
* fuel-plugin-opendaylight
* fuel-plugin-saltstack
* fuel-plugin-solidfire-cinder
* fuel-plugin-swiftstack
* fuel-plugin-tintri-cinder
* fuel-plugin-tls
* fuel-plugin-vmware-dvs
* fuel-plugin-vxlan
* fuel-plugin-zabbix-monitoring-emc
* fuel-plugin-zabbix-monitoring-extreme-networks
* fuel-plugin-zabbix-snmptrapd
* fuel-plugins
* fuel-provision
* fuel-qa
* fuel-specs
* fuel-stats
* fuel-tasklib
* fuel-upgrade
* fuel-web
* gce-api
* gerrit-dash-creator
* git-upstream
* golang-client
* group-based-policy
* group-based-policy-automation
* group-based-policy-specs
* group-based-policy-ui
* intel-nfv-ci-tests
* merlin
* monasca-agent
* monasca-api
* monasca-ceilometer
* monasca-common
* monasca-log-api
* monasca-notification
* monasca-persister
* monasca-statsd
* monasca-thresh
* monasca-ui
* monasca-vagrant
* monitoring-for-openstack
* namos
* nerd-reviewer
* networking-6wind
* networking-hyperv
* networking-ovs-dpdk
* networking-zvm
* nova-docker
* nova-powervm
* nova-solver-scheduler
* nova-zvm-virt-driver
* ops-tags-team
* osprofiler
* ospurge
* packstack
* puppet-autossh
* puppet-ceph
* puppet-n1k-vsm
* puppet-setproxy
* puppet-surveil
* python-blazarclient
* python-cloudkittyclient
* python-cloudpulseclient
* python-cognitiveclient
* python-fuelclient
* python-group-based-policy-client
* python-jenkins
* python-monascaclient
* python-openstacksdk
* python-rackclient
* python-senlinclient
* python-surveilclient
* python-tackerclient
* rack
* requests-mock
* senlin
* senlin-dashboard
* shaker
* sqlalchemy-migrate
* surveil
* surveil-specs
* stackalytics
* swift3
* swiftonfile
* tacker
* tacker-horizon
* tacker-specs
* tap-as-a-service
* telcowg-usecases
* terracotta
* third-party-ci-tools
* tricircle
* vmtp
* wsme
* xenapi-os-testing
* xstatic-d3
* xstatic-angular-sanitize
* xstatic-bootstrap-datepicker
* xstatic-angular-gettext
* xstatic-bootswatch
* xstatic-angular-cookies
* xstatic-bootstrap-scss
* xstatic-angular-smart-table
* xstatic-angular-fileupload
* xstatic-angular-bootstrap
* xstatic-angular
* xstatic-angular-mock
* xstatic-angular-lrdragndrop
* xstatic-roboto-fontface
* xstatic-jquery.quicksearch
* xstatic-mdi
* xstatic-rickshaw
* xstatic-magic-search
* xstatic-font-awesome
* xstatic-hogan
* xstatic-spin
* xstatic-jasmine
* xstatic-jquery-migrate
* xstatic-jquery-migrate
* xstatic-jsencrypt
* yaql

=== Inactive Projects to Retire ===
Inactive projects that should be retired should be added to this list. These projects will have a commit merged removing their content and replacing it with a message indicating the project is no longer maintained and will become read-only in Gerrit.

* compass-monit
* fuel-plugin-availability-zones
* mercador-pub
* mercador-sub
* MRaaS
* networking-portforwarding
* openstackdroid
* python-mercadorclient
* rubick
* sahara-guestagent
* libra
* logaas
* python-libraclient
* python-rallyclient
* cookbook-pacemaker
* puppet-openstack_dev_env
* puppet_openstack_builder
* puppet-openstack-cloud
* tripleo-ansible

Manila/design

2015-04-21T06:39:57Z

Ramana Raja:

== Components / Phases ==

The original master blueprint is here: [https://blueprints.launchpad.net/manila/+spec/file-shares-service File Shares Service]

=== 1) File Shares Service ===

The service was initially conceived as an addition of a separate File Share Service, albeit delivered within the Cinder (OpenStack Block Storage) project given the opportunity to make use of common code. In June of 2013, however, the decision was made to establish an independent development project to accommodate:

* Creation of file system shares (e.g. the create API needs to support a "protocol" & a permissions "mask" and "ownership" parameters)
* Deletion of file systems shares
* List, show, provide and deny access, & list share access rules to file system shares
* Create, list, and delete snapshots / clones of file systems shares
* Coordination of mounting file system shares
* Unmounting file systems shares

'''Implementation status''': development underway in [[https://github.com/stackforge/manila|Stackforge.]]

[[Manila/docs/API|Manila API Documentation]]
[[Manila/docs/API-roadmap|Manila API Roadmap]]
[[Manila/docs/db|Manila Database Specification]]

NOTE: Manila started as a fork of Cinder given that it provides facility for many of the concepts that a File Shares service would depend upon. The present Cinder concepts of capacity, target (server in NAS parlance), initiator (likewise the client when referring to shared file systems) are common conceptually (if not entirely semantically) and broadly applicable to both block and file-based storage. Specific Cinder capabilities (such as the filter scheduler, the notion of type, and extra specs) likewise apply to provisioning shared file systems as well. The initial prototype of the File Share Service is thus based on an evolution of Cinder. Manila, however, addresses a variety of additional concerns that aren't relevant to Cinder operation. After discussion within the Cinder community and consultation with members of the Technical Committee it was determined that establishing a separate project expressly designed and developed to deliver shared / distributed file systems as a service was the most viable approach. The intention is to move any commonality between Cinder and the File Share Service into Oslo where sensible.

=== 2) Manila Reference Provider(s) ===

Creation of a reference Manila provider (commonly also referred to as a driver) for shared file system use under the proposed expanded API. As an example, a [[NetApp]] provider for this would be able to advertise, accept, and respond to requests for NFSv3, NFSv4, NFSv4.1 (w/ pNFS), & contemporary CIFS / SMB protocols (eg versions 2, 2.1, 3). Additional modification of python-cinderclient will be necessary to provide for the expanded array of request parameters. Both a vendor independent reference and NetApp-specific backend are part of the aforementioned prototype and are part of the submission.
as
'''Implementation status''': completed.

=== 3) Intelligent scheduling of Shares using Filter Scheduler and Multi-backend support ===

Allowing one Manila node to manage multiple share backends. A backend can run either or both of share and volume services. Support for shares in filter scheduler allows the cloud administrator to manage large-scale share storage by filtering backends based on predefined parameters.

'''Implementation status''': completed.

=== 4) End to End Experience (Automated Mounting) ===

A proposal for handling injection / updates of mounts to instantiated guests operating in the Nova context. A listener / agent that could either be interacted directly or more likely poll or receive updates from instance metadata changes would represent one possible solution. The possible use of either cloud-init or VirtFS (which would attach shared file systems to instances in a manner similar to Cinder block storage) is also under consideration.

Additional discussion here: [[Manila_Storage_Integration_Patterns|Manila Storage Integration Patterns]]

'''Implementation status''': Scoping.

=== 5) The last mile problem ===

Accommodation for a variety of use cases / networking topologies (ranging from flat networks, to Neutron SDNs, to hypervisor mediated options) for connecting from shares to instances are discussed here: [[Manila/Networking|Manila Networking]]

'''Implementation status''': not started.

=== 6) Tenant and Administrative UI ===

Manila must expose both administrative and tenant Horizon interfaces.

'''Implementation status''': in design phase... the initial Horizon UI will borrow heavily from the existing Cinder tenant facing and administrative UI.

Manila/Program Application

2014-07-30T06:02:36Z

Ramana Raja:

== Official Title ==
Shared File Systems

== Mission Statement ==
The goal of the Shared File Systems program is to provide a set of services for management of shared file systems in a multitenant cloud environment, similar to how OpenStack provices for block-based storage management through the Block Storage program.

We aim to provide a vendor neutral management interface that allows for provisioning and attaching shared file systems such as NFS, CIFS, and more. To the extent possible we aim to mirror the architecture of Block Storage program, with support for a public REST API, multiple backends, and a scheduler that performs resource assignment decisions. When differences are unavoidable, we plan to design solutions that are compatible with the OpenStack ideals of modularity and scalability.

== Deliverables ==
* A set of services (API server, scheduler, and share manager) that provide an infrastructure for self-sevice provisioning and managing shared file systems in a multi-tenant cloud
* A CLI-based client for users and administrators to interact with the API
* A Horizon GUI plugin
* A "generic" sorfware-only driver that runs on any hardware, allowing people to use the service without specialized equipment
* An example image (cirros-based) that implements part of the "generic" storage service, with documentation on how admins can create their own
* A set of drivers for commercial NAS storage servers
* API documentation, Administrator documentation, and Developer documentation

== Repositories ==
* Project code: https://github.com/stackforge/manila
* Python client: https://github.com/stackforge/python-manilaclient

== ATC Status ==
The program grants Active Technical Contributor (ATC) status to anyone completing a commit to any program repository during the previous two release cycles. Core project team members may grant ATC to significant, non code-contributors for two cycles.

== Development Team ==
====PTLː====
* Ben Swartzlander

====The core teamː====
* Yulia Portnova
* Valeriy Ponomaryov
* Xing Yang

====Significant contributorsː====
* Alex Meade
* Rushil Chugh
* Clinton Knight
* Vitaly Kostenko
* Andrei Ostapenko
* Aleksandr Chirko
* Csaba Henk
* Ramana Raja
* Christian Berendt

=== Why a new program? ===
We started a new program for two main reasons:
* Initially we tried to address the above needs by expanding the scope of the Block Storage program, but the Cinder core team didn't want the additional responsibility. This left us with no option other than to start a new program.
* While there is some overlap between Block Storage and Shared File Systems, the technical implementation details and use cases are substantial enough to require a dedicated group of developers to solve the unique problems related to Shared File Systems. Keeping the programs separate has been greatly advantageous as it allows each group to focus on different problems and ultimately get more done.
 
'''NOTE''': Manila started as a fork of Cinder given that it provides facility for many of the concepts that a Shared File Systems service would depend upon. The present Cinder concepts of capacity, target (server in NAS parlance), initiator (likewise the client when referring to shared file systems) are common conceptually (if not entirely semantically) and broadly applicable to both block and file-based storage. Specific Cinder capabilities (such as the filter scheduler, the notion of type, and extra specs) likewise apply to provisioning shared file systems as well. Manila is thus based on an evolution of Cinder. Manila, however, addresses a variety of additional concerns that aren't relevant to Cinder operation (e.g. provisioning into tenant SDN, mapping to auth authority / user namespace, et cetera). After involved discussion within the Cinder community and consultation with members of the Technical Committee it was determined that establishing a separate project expressly designed and developed to deliver shared / distributed file systems as a service was the most viable approach. The intention is to move any remaining commonality between Cinder and the File Share Service into Oslo over time where sensible.

Manila Overview

2014-01-21T04:59:45Z

Ramana Raja: /* Developers */

== Codename ==

The project codename is Manila.

== Trademarks ==

We're not aware of any trademarks conflicts with the name. The capital city of the Philippines is called Manila, making the name a proper noun. There are no other names used by the project with trademark concerns.

== Summary ==

The Manila project provides an API for management of shared filesystems with support for multiple protocols and backend implementations.

== Mission Statement ==

Stated simply, the goal of the Manila project is to do for shared filesystem storage what Cinder has done for blocks storage.

We aim to provide a vendor neutral management interface that allows for provisioning and attaching shared filesystems such as NFS, CIFS, and more. To the extent possible we aim to mirror the architecture of Cinder, with support for a public REST API, multiple backends, and a scheduler that performs resource assignment decisions. When differences are unavoidable, we plan to design solutions that are compatible with the OpenStack ideals of modularity and scalability.

== Detailed Description ==

The basic assumption underpinning Manila is that shared filesystems provide some valuable features that cannot be obtained from either blocks storage or object storage, and that OpenStack is missing management features for this 3rd form of storage. The unique features afforded by shared filesystems are shared, fine-grained, read/write access to persistent data by multiple instances simultaneously. The NFS and CIFS protocols have been developed to provide these features and still prove popular after decades of use.

The implementation of Manila is actually a modified fork of the Cinder project. The concept for management of shared filesystems was originally proposed as an extension to Cinder (at the SF design summit in April 2012), under the theory that there would be a lot of common code between the implementations, and many of the same developers would be interested in working on both projects. Because of this, the initial implementation for what is now Manila was actually a large patch to the Cinder project submitted in August 2012. For a variety of reasons, we ultimately decided that a separate project would be a better way to deliver the features and the Manila project was born.

Manila consist of all of the code from Cinder with our shared filesystem management code added in and all the blocks-specific code removed. The API largely mirrors the existing Cinder APIs, except that "volumes" have been renamed to "shares" and the attachment procedure is somewhat different.

== Roadmap ==

The initial implementation of Manila was a proof of concept that shared filesystem management can fit into the same architecture as Cinder. The main difference between blocks and shared filesystems however is how the storage system and the ultimate user of the storage communicate with one another. In particular, shared filesystems work best when instances are able to communicate directly with the storage backend over the network, and the storage backend is able to serve multiple tenants while maintaining secure separation between them. Block storage can simply be virtualized through a hypervisor, with far fewer requirements on the backend storage system. Because of these differences, additional work is needed to help automate the networking portion of attaching a shared filesystem to one or more instances in a tenant network, and to automate the setup of security domains and other features that exist in a NAS environment but not a SAN environment.

During Icehouse, our main goals are to define and implement these new APIs as well as expanding the backend driver interface to support true multitenancy (the kind with network segmentation) in Manila. We also aim to get as many backends implemented as possible, and to this end we will be improving the reference drivers and documentation for developing new ones.

== Source Code ==

* Project code: https://github.com/stackforge/manila
* Python client: https://github.com/stackforge/python-manilaclient

We are waiting for official incubation status before pushing the devstack and tempest integrations upstream.
* Devstack integration: https://github.com/bswartz/devstack
* Tempest integration: https://github.com/bswartz/tempest/tree/manila

== Language and Dependencies ==

Manila is written in Python and depends on some core OpenStack projects such as Oslo and Keystone. Integration with other OpenStack projects is planned (most notably Horizon, Nova, and Neutron). The reference driver depends on standard Linux services such as nfsd and samba.

== License ==

The Manila project is open sourced under the Apache 2 license.

== Maturity ==

Aside from the code inherited from Cinder, the new Manila code is a little more than a year old, and has been actively developed from then until now, mostly by developers from NetApp and Mirantis. The core team now consists of developers from NetApp and Mirantis, with significant community interest since the code was open sourced in August 2013.

== Technical Lead ==

Ben Swartzlander from NetApp has been the technical lead for the project since its conception 18 months ago, and plans to continue leading the project from a design and administrative standpoint. Ben has been working the storage industry as a software engineer for nearly 13 years and has extensive experience with storage systems, network protocols, virtualization, and open source projects. Ben has been a contributor to the OpenStack project for a little over 2 years.

== Developers ==

Current:
* Yulia Portnova (Mirantis)
* Glenn Gobeli (NetApp)
* Navneet Singh (NetApp)
* Vasiliy Belokon (Mirantis)
* Valeriy Ponomaryov (Mirantis)
* Andrei Ostapenko (Mirantis)
* Rushi Agrawal (Reliance Jio Cloud)
* Vijay Bellur (Red Hat)
* Csaba Henk (Red Hat)
* Ramana Raja (Red Hat)

Future:
* Caitlin Bester (Nextenta)
* Bill Owen (IBM)
* David Noveck (EMC)
* Shamail Tahir (EMC)
* Alex Pecoraro (EMC/Isilon)

Sorry if I forgot anybody -- feel free to add yourself (it's a wiki!)

== Infrastructure Requirements ==

Manila does not require any infrastructure above and beyond what's already provided by devstack today.

== Contributor License Agreement ==

All current contributors HAVE agreed to the OpenStack CLA!

Manila/Meetings

2014-01-21T04:30:04Z

Ramana Raja: edited the hyperlink to the manila meeting logs

= Weekly Manila team meeting =
'''NOTE MEETING TIME: Thursday at 15:00 UTC'''

'''NOTE MEETING CANCELLATIONS: The meeting is cancelled on 26 Dec and 2 Jan due to holidays!'''

If you're interested in management of shared filesystems for OpenStack, we have a weekly meetings in <code><nowiki>#openstack-meeting-alt</nowiki></code>, on Thursdays at 15:00 UTC. Please feel free to add items to the agenda below. NOTE: When adding topics please include your IRC name so we know who's topic it is and how to get more info.

== Next meeting ==
'''NOTE:''' ''Include your IRC nickname next to agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. You might want to put this on your calendar if you are adding items.''

'''Scheduled for 9 Jan, 2014, 15:00 UTC'''
# Development Status
# Open Discussion

== Previous meetings ==

'''Oct 17, 2013, 15:00 UTC'''
# [https://blueprints.launchpad.net/manila/+spec/join-tenant-network Networking BP]

'''Sep 26, 2013, 15:00 UTC'''
# [[Manila Networking|Networking Wiki]] (bswartz)

'''Aug 8, 2013, 15:00 UTC'''
# [[ManilaProjectPlan|Project Plan]] -- bswartz

http://eavesdrop.openstack.org/meetings/manila/

Manila/Networking/Gateway mediated

2013-12-05T15:09:48Z

Ramana Raja:

=== Hypervisor mediated storage access for NFS shares and Multi-tenancy ===

=== Architecture ===
This document outlines an approach to achieving multi-tenancy using Manila OpenStack and the architecture/design considerations wrt enabling Multi-tenancy in the cloud using this approach. This model is based on the [[Manila_Networking#Hypervisor_Mediated|Hypervisor mediated approach to Manila Networking]]

The approach taken is a hypervisor mediated one where the decision to allow tenant access to the intended Manila share is made in the hypervisor. Typically, each hypervisor can have multiple guest instances each of which is considered a tenant. The proposed design involves running a user-space NFS server on the hypervisor. Tenants rely on NFS clients to connect to this NFS server and consume any exported shares. The NFS server in turn is assumed to be capable enough, to talk to different storage backends and serve requests made by its clients.

We propose to use nfs-ganesha[1][2] as the user-space NFS daemon in this architecture. This is based on the following:
nfs-ganesha is easily extensible to various filesystem backends and has a modular/plugin-based architecture. A plugin is commonly referred to as File System Abstraction Layer (FSAL) in NFS-Ganesha parlance. Currently, nfs-ganesha has support for a variety of FSALs including ones for CephFS, GlusterFS, Lustre, ZFS, VFS, GPFS etc. nfs-ganesha also supports various NFS protocol versions like NFSv2,v3,v4,v4.1,v4.2 and pNFS. This makes it ideal for future use-case extensibility and also from the point of view of accomodating the maximum possible filesystem backends, including any new vendor developed ones.

When a mount request is received by the nfs-ganesha server, it checks whether the client/tenant is allowed to access the related export or share. Based on the NFS share ACL configurations, the exported share operation will succeed or fail. If the mount succeeds, all further operations performed from the client will succeed and Ganesha would route such requests to the storage backend through the corresponding filesystem FSAL. So tenant share separation is made possible based on NFS ACLs and access permissions. Further, nfs-ganesha also supports Kerberos based authentication in case Manila shares need to be exported and consumed across nodes in an Openstack compute cluster when nfs-ganesha would be mounted on each hypervisor in the cluster in Multi-head configuration.

==== Advantages: ====

* Works with all guest operating systems that have a nfs client built in. Well standardized and widely available support.
* Does not require any VM/guest plugin installation or any Openstack components to be modified to work with this approach.
* Works across the majority of the prevalent filesystem backends and/or Storage Arrays without modifications.
* This approach most closely mirrors how guests interact with storage controllers in Cinder. The storage controller only needs to know about compute nodes and the hypervisor deals with the problem of presenting the storage to the guest.
* Security is enforced by the hypervisor. Backends need not support any security to protect tenant data from other tenants (or even enforce protection between instances of one tenant).
* Further tenant separation is based on share permissions based on NFS ACLs and authentication. This is easily enforced.
* No dependence or interaction with Neutron
* NFS server on the hypervisor performs caching. So not all requests from the instances need to be served by the storage layer. Thereby data access is sometimes faster, and the storage layer is less burdened.
* Being a user-space server, nfs-ganesha's cache-inode layer can grow arbitrarily large, limited mostly (only) by the underlying hardware. This supports greater scalability in terms of supporting larger number of compute instances and share operations/requests.
* Extensible to support other multi-tenant use-cases like share reservations, QoS requirements etc.
==== Disadvantages: ====
* Resources are consumed by nfs-ganesha as it runs on the hypervisor. Resource consumption by ganesha has been seen to be moderate across general workloads so we don't see this as a major impact to the overall architecture currently. Further tests need to be carried out to ascertain this impact if any.
* High Availability for tenant share access via nfs-ganesha could necessitate additional configuration and state maintenance. However, this availability support/infrastructure is mostly available as a separate Ganesha solution w.r.t. different filesystem backends.

==== References: ====
[1] https://github.com/nfs-ganesha/nfs-ganesha/wiki

[2] https://forge.gluster.org/nfs-ganesha-and-glusterfs-integration/pages/Home

[[File:Hypervisor-mediated.png]]

File:Hypervisor-mediated.png

2013-12-05T14:35:07Z

Ramana Raja:

File:Hypervisor-mediated-manila-access.svg

2013-12-05T11:26:08Z

Ramana Raja: Ramana Raja uploaded a new version of "File:Hypervisor-mediated-manila-access.svg"

Hypervisor mediated storage access for NFS shares

File:Hypervisor-mediated-manila-access.svg

2013-12-04T23:43:05Z

Ramana Raja: Hypervisor mediated storage access for NFS shares

Hypervisor mediated storage access for NFS shares

Manila/Networking/Gateway mediated

2013-12-04T19:52:37Z

Ramana Raja: Created page with "=== Hypervisor mediated storage access for NFS shares === This is one of the approaches to achieve multi-tenancy in Manila, where the decision to access a share is made in th..."

=== Hypervisor mediated storage access for NFS shares ===

This is one of the approaches to achieve multi-tenancy in Manila, where the decision to access a share is made in the hypervisor. Typically, each hypervisor can have multiple guest instances belonging to different tenants. The proposed scheme involves running an userspace NFS server on the hypervisors. Tenants can make use of NFS clients to connect to this NFS server. The NFS server in turn will talk to different storage backends to serve requests made by its clients. We propose to use NFS-Ganesha as the userspace NFS daemon in this scheme.

NFS-Ganesha is very extensible and has a plugin based architecture. A plugin is commonly referred to as File System Abstraction Layer (FSAL) in NFS-Ganesha parlance. Currently, NFS-Ganesha has support for a variety of FSALs including ones for CephFS, GlusterFS and GPFS.

When a mount request is received by the NFS Ganesha server, it checks whether the client/tenant is allowed to access the related export or share. This can be determined by looking at the tenant’s network endpoint. Based on the share’s ACL, the mount operation can succeed or fail. If the mount succeeds, all further operations performed from the client will succeed and Ganesha would route such requests to the storage backend through the appropriate FSAL.

==== Advantages: ====

* Works with all guest operating systems that have a nfs client built in.
* This approach most closely mirrors how guests interact with storage controllers in Cinder. The storage controller only needs to know about compute nodes and the hypervisor deals with the problem of presenting the storage to the guest.
* Security is enforced by the hypervisor. Backends need not support any security to protect tenant data from other tenants (or even enforce protection between instances of one tenant).
* No dependence or interaction with Neutron
* NFS server on the hypervisor performs caching. So not all requests from the instances need to be served by the storage layer. Thereby data access is sometimes faster, and the storage layer is less burdened.

==== Disadvantages: ====
* Resources are consumed by NFS-Ganesha on the hypervisor.
* HA for NFS Ganesha would necessitate additional configuration and state maintenance.

Manila/docs

2013-11-29T05:52:42Z

Ramana Raja: A quick start guide to begin working with Manila

== Manila Documentation ==
 
[[Manila/API|API specification]]
 
[[Manila/docs/db|Database specification]]
 
 
== Getting started with Manila ==
 
[[Manila/docs/Manila_Developer_Setup_Fedora19|Quick starting Manila!]]

Manila/docs/Manila Developer Setup Fedora19

2013-11-29T05:44:54Z

Ramana Raja:

= INSTALLING AND TESTING MANILA USING DEVSTACK ON A VM =

== Recommended VM requirements ==

* 4G RAM
* 20G of free disk space

The instructions were tested on a Fedora 19 (f19) VM. However, the following notes should work with a Ubuntu VM as well, but some of the instructions might not be required by Ubuntu users. So whenever the notes are suspected to be Fedora specific they'd be stated as being so.

== Installing DevStack ==

Below is the DevStack repo with Manila integrated. Besides installing other OpenStack services, it pulls and installs Manila from, https://github.com/stackforge/manila, the current upto-date Manila repo:

<pre>$ git clone git@github.com:bswartz/devstack.git
$ cd devstack</pre>
Note: Very recently, the Manila repo has been integrated with DevStack master, which you could also use. Hence, some of the below instructions might no longer be relevant. Expect appropriate changes in instructions soon.

You might need to make the following changes. The change in ''./lib/manila'' is needed if you use F19. Otherwise, the ''stack.sh'' script, which installs the OpenStack services, would install the manila scripts in ''/usr/bin'', but later try to execute the manila scripts from ''/usr/local/bin''.

<source lang="diff">diff --git a/lib/manila b/lib/manila
index abd0158..7f75ebc 100644
--- a/lib/manila
+++ b/lib/manila
@@ -48,7 +48,7 @@ MANILA_SERVICE_PROTOCOL=${MANILA_SERVICE_PROTOCOLS
if [[ -d $MANILA_DIR/bin ]]; then
MANILA_BIN_DIR=$MANILA_DIR/bin
else
- MANILA_BIN_DIR=/usr/local/bin
+ MANILA_BIN_DIR=/usr/bin
fi

SHARE_GROUP=${SHARE_GROUPtack-shares}
@@ -456,4 +456,4 @@ function stop_manila() {
}</source>
 
The changes in localrc:

* The change in password of Rabbit service to 'guest' is needed if you use F19. A fresh installation of rabbitmq creates a user 'guest' with the password 'guest'. And for some reason, DevStack fails to set the password of the user 'guest' as desired by the user.
* Extra variables can be set for logging the screens of the OpenStack services running. This is useful especially because the default screens have a very limited scrollback buffer.
* The size of the backing file for a Manila-LVM share can be reduced from 8G to 2G.
 
<source lang="diff">diff --git a/localrc b/localrc
index 05ea5cd..4474425 100644
--- a/localrc
+++ b/localrc
@@ -1,9 +1,15 @@
DATABASE_PASSWORD=rengen
-RABBIT_PASSWORD=rengen
+RABBIT_PASSWORD=guest
SERVICE_TOKEN=rengen
SERVICE_PASSWORD=rengen
ADMIN_PASSWORD=rengen

+# Enable Logging
+LOGFILE=/opt/stack/logs/stack.sh.log
+VERBOSE=True
+LOG_COLOR=True
+SCREEN_LOGDIR=/opt/stack/logs
+

@@ -66,7 +72,7 @@ MANILA_SERVICE_PORT_INT=18776

VOLUME_BACKING_FILE_SIZE=${VOLUME_BACKING_FILE_SIZE:-1048M}

-SHARE_BACKING_FILE_SIZE=${SHARE_BACKING_FILE_SIZE:-8400M}
+SHARE_BACKING_FILE_SIZE=${SHARE_BACKING_FILE_SIZE:-2048M}
MANILA_SECURE_DELETE=$CINDER_SECURE_DELETE</source>
 
Run ''stack.sh''.

<pre>$ ./stack.sh</pre>
Note that the above script sets SELinux to Permissive mode for Fedora users. This setting however doesn't survive a reboot. For a permanent change in SELinux setting, you need to edit the ''/etc/sysconfig/selinux'' file to be as below:

<pre># This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=permissive</pre>
Check the status of SELinux:

<pre>$ getenforce</pre>
== Launching a Guest instance using Nova ==

Make sure that the services, httpd, rabbitmq-server (or some other AMQP), mysqld (db) are running every time you begin (e.g. after a sytem reboot) using OpenStack. For F19, you could do this by:

<pre>$ systemctl status rabbitmq-server</pre>
If they aren't running you should get them up and running:

<pre># systemctl restart rabbitmq-server</pre>
And also don't forget to check whether SELinux is set to Permissive mode :).

Now you need to setup the keystone login credentials before you can use the CLI of the various OpenStack services. This can be easily done by sourcing ''openrc'' file with a user and a tenant name respectively. And ''stack.sh'' already has created users (admin and demo) and tenants (admin and demo), which you could use. So, you could do the following:

<pre>$ source openrc admin admin</pre>
Bring up (if you rebooted after running ''stack.sh'') and interact with the various OpenStack services by running ''rejoin-stack.sh'' script.

<pre>$ ./rejoin-stack.sh</pre>
You'd need Nova guests to test if the Manila shares are accessible. And it makes sense that you use a light weight image (CirrOS) to launch an instance. One such lightweight image with nfs kernel support, nfs-utils, rpcbind and strace was put together by [https://launchpad.net/~chenk Csaba Henk]: https://mega.co.nz/#!B91VUA6Z!eNB2BSW4p1o8lRqaPeaIek4g_J1LJL_SY2OymVnurlc

Create a glance-image using the image downloaded from the above link:

<pre>$ glance image-create --name="Cirros 0.3.1" --disk-format=qcow2 \
--container-format bare < cirros-git-disk.qcow2</pre>
List the glance images, and check if available:

<pre>$ glance image-list</pre>
Add a keypair list:

<pre>$ nova keypair-add --pub_key ~/.ssh/id_rsa.pub mykey</pre>
Set network rules to allow ssh and ping access to the guest VM:

<pre>$ nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
$ nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0</pre>
Boot a nova instance. Please use the image-id of the downloaded image from glance image-list:

<pre>$ nova boot --flavor m1.tiny --image 00aa37a3-0bec-4140-bbf1-617fc3b3da63 \
--key_name mykey --security_group default myvm0</pre>
Check the status of the guest instance:

<pre>$ nova list</pre>
Check by ssh-ing to the VM. Password of the guest VM's cirros is <code>cubswin:)</code>:

<pre>$ ssh cirros@<the ip of the vm, usually a private IP, unless setup></pre>
Inside the guest vm. If you need to be root:

<pre>$ sudo -s
$ id</pre>
== Creating a Manila LVM NFS share ==

Currently, Manila doesn't support multi-tenancy (the code to do so is under review as of today), and the networking model is the simple FlatNetwork model. For more info, see

* https://wiki.openstack.org/wiki/Shares_Service
* https://wiki.openstack.org/wiki/Manila_Networking

Check if the Manila share driver and it's helpers are as follows, else set them:

<pre>$ cat /etc/manila/manila.conf | grep share_driver
share_driver = manila.share.drivers.lvm.LVMShareDriver
$ cat /etc/manila/manila.conf | grep share_lvm_helpers
share_lvm_helpers = CIFS=manila.share.drivers.lvm.CIFSHelper,NFS=manila.share.drivers.lvm.NFSHelper</pre>
Creating a 1G LVM Manila NFS share by carving 1G logical volume from a volume group, which is by default stack-shares:

<pre>$ manila create --name myshare0 NFS 1</pre>
Check if the share is available:

<pre>$ manila list</pre>
Allow the share to be accessed by the running instance:

<pre>$ manila allow-access <share-id> ip <instance-id></pre>
You should now be able to ssh into your guest VM, and check for the NFS share:

<pre>$ showmount -e</pre>
If all's well, you can mount the NFS share on your guest. If not, the most probable reason could be that the firewall rules setup by Nova are working against you. The dirty way to work around this problem would be to stop and disable firewall daemon in your host VM, and restart the Nova services.

<pre># systemctl stop firewalld
# systemctl disable firewalld</pre>

Manila/docs/Manila Developer Setup Fedora19

2013-11-27T15:18:24Z

Ramana Raja: Instructions to setup and test Manila in a VM using DevStack.

= INSTALLING AND TESTING MANILA USING DEVSTACK ON A VM =

== Recommended VM requirements ==

* 4G RAM
* 20G of free disk space

The instructions were tested on a Fedora 19 (f19) VM. However, the following notes should work with Ubuntu VM as well, but some of the instructions might not be required by Ubuntu users. So whenever the notes are suspected to be Fedora specific they'd be stated as being so.

== Installing DevStack ==

DevStack repo with Manila integrated. Besides installing other OpenStack services, pulls and installs Manila from, https://github.com/stackforge/manila, the current upto-date Manila repo:

<pre>$ git clone git@github.com:bswartz/devstack.git
$ cd devstack</pre>
Note: Very recently, the Manila repo has been integrated with DevStack master, which you could also use. Hence, some of the below instructions might no longer be relevant. Expect appropriate changes in instructions soon.

You might need to make the following changes. The change in ''./lib/manila'' is needed if you use F19. Otherwise, the ''stack.sh'' script, which installs the OpenStackservices, would install the manila scripts in ''/usr/bin'', but later try to execute the manila scripts from ''/usr/local/bin''.

<source lang="diff">diff --git a/lib/manila b/lib/manila
index abd0158..7f75ebc 100644
--- a/lib/manila
+++ b/lib/manila
@@ -48,7 +48,7 @@ MANILA_SERVICE_PROTOCOL=${MANILA_SERVICE_PROTOCOLS
if [[ -d $MANILA_DIR/bin ]]; then
MANILA_BIN_DIR=$MANILA_DIR/bin
else
- MANILA_BIN_DIR=/usr/local/bin
+ MANILA_BIN_DIR=/usr/bin
fi

SHARE_GROUP=${SHARE_GROUPtack-shares}
@@ -456,4 +456,4 @@ function stop_manila() {
}</source>
The changes in localrc:

* The change in password of Rabbit service to 'guest' is needed if you use F19. A fresh installation of rabbitmq creates a user 'guest' with the password 'guest'. And for some reason, DevStack fails to set the password of the user 'guest' as desired by the user.
* Extra variables can be set for logging the screens of the OpenStack services running. This is useful especially because the default screens have a very limited scrollback buffer.
* The size of the backing file for a Manila-LVM share can be reduced from 8G to 2G.

<source lang="diff">diff --git a/localrc b/localrc
index 05ea5cd..4474425 100644
--- a/localrc
+++ b/localrc
@@ -1,9 +1,15 @@
DATABASE_PASSWORD=rengen
-RABBIT_PASSWORD=rengen
+RABBIT_PASSWORD=guest
SERVICE_TOKEN=rengen
SERVICE_PASSWORD=rengen
ADMIN_PASSWORD=rengen

+# Enable Logging
+LOGFILE=/opt/stack/logs/stack.sh.log
+VERBOSE=True
+LOG_COLOR=True
+SCREEN_LOGDIR=/opt/stack/logs
+

@@ -66,7 +72,7 @@ MANILA_SERVICE_PORT_INT=18776

VOLUME_BACKING_FILE_SIZE=${VOLUME_BACKING_FILE_SIZE:-1048M}

-SHARE_BACKING_FILE_SIZE=${SHARE_BACKING_FILE_SIZE:-8400M}
+SHARE_BACKING_FILE_SIZE=${SHARE_BACKING_FILE_SIZE:-2048M}
MANILA_SECURE_DELETE=$CINDER_SECURE_DELETE</source>
Run ''stack.sh''.

<pre>$ ./stack.sh</pre>
Note that the above script sets SELinux to Permissive mode for Fedora users. This setting however doesn't survive a reboot. For a permanent change in SELinux setting, you need to edit the ''/etc/sysconfig/selinux'' file to be as below:

<pre># This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=permissive</pre>
Check the status of SELinux:

<pre>$ getenforce</pre>
== Launching a Guest instance using Nova ==

Make sure that the services, httpd, rabbitmq-server (or some other AMQP), mysqld (db) are running every time you begin (e.g. after a sytem reboot) using OpenStack.For F19 you could do this by:

<pre>$ systemctl status rabbitmq-server</pre>
If they aren't running you should get them up and running:

<pre># systemctl restart rabbitmq-server</pre>
And also don't forget to check whether SELinux is set to Permissive mode :).

Now you need to setup the keystone login credentials before you can use the CLI of the various OpenStack services. This can be easily done by sourcing ''openrc'' file with a user and a tenant name respectively. And ''stack.sh'' already has created users (admin and demo) and tenants (admin and demo), which you could use. So, you could do the following:

<pre>$ source openrc admin admin</pre>
Bring up (if you rebooted after running ''stack.sh'') and interact with the various OpenStack services by running ''rejoin-stack.sh'' script.

<pre>$ ./rejoin-stack.sh</pre>
You'd need Nova guests to test if the Manila shares are accessible. And it makes sense that you use a light weight image (CirrOS) to launch an instance. One such lightweight image with nfs kernel support, nfs-utils, rpcbind and strace was put together by [https://launchpad.net/~chenk Csaba Henk]: https://mega.co.nz/#!B91VUA6Z!eNB2BSW4p1o8lRqaPeaIek4g_J1LJL_SY2OymVnurlc

Create a glance-image using the image downloaded from the above link:

<pre>$ glance image-create --name="Cirros 0.3.1" --disk-format=qcow2 \
--container-format bare < cirros-git-disk.qcow2</pre>
List the glance images, and check if available:

<pre>$ glance image-list</pre>
Add a keypair list:

<pre>$ nova keypair-add --pub_key ~/.ssh/id_rsa.pub mykey</pre>
Set network rules to allow ssh and ping access to the guest VM:

<pre>$ nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
$ nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0</pre>
Boot a nova instance. Please use the image-id of the downloaded image from glance image-list:

<pre>$ nova boot --flavor m1.tiny --image 00aa37a3-0bec-4140-bbf1-617fc3b3da63 \
--key_name mykey --security_group default myvm0</pre>
Check the status of the guest instance:

<pre>$ nova list</pre>
Check by ssh-ing to the VM. Password of the guest VM's cirros is <code>cubswin:)</code>:

<pre>$ ssh cirros@<the ip of the vm, usually a private IP, unless setup></pre>
Inside the guest vm. If you need to be root:

<pre>$ sudo -s
$ id</pre>
== Creating a Manila LVM NFS share ==

Currently Manila doesn't support multi-tenancy (the code to do so is under review as of today), and the networking model is the simple FlatNetwork model. For more info, see

* https://wiki.openstack.org/wiki/Shares_Service
* https://wiki.openstack.org/wiki/Manila_Networking

Check if the Manila share driver and it's helpers are as follows, else set them:

<pre>$ cat /etc/manila/manila.conf | grep share_driver
share_driver = manila.share.drivers.lvm.LVMShareDriver
$ cat /etc/manila/manila.conf | grep share_lvm_helpers
share_lvm_helpers = CIFS=manila.share.drivers.lvm.CIFSHelper,NFS=manila.share.drivers.lvm.NFSHelper</pre>
Creating a 1G LVM Manila NFS share by carving 1G logical volume from a volume group, which is by default stack-shares:

<pre>$ manila create --name myshare0 NFS 1</pre>
Check if the share is available:

<pre>$ manila list</pre>
Allow the share to be accessed by the running instance:

<pre>$ manila allow-access <share-id> ip <instance-id></pre>
You should now be able to ssh into your guest VM, and check for the NFS share:

<pre>$ showmount -e</pre>
If all's well, you can mount the NFS share on your guest. If not, the most probable reason could be that the firewall rules setup by Nova are working against you. The dirty way to work around this problem would be to stop and disable firewall daemon in your host VM, and restart the Nova services.

<pre># systemctl stop firewalld
# systemctl disable firewalld</pre>