OpenStack - User contributions [en]

Manila/Meetings

2015-09-03T15:22:54Z

Chenk: /* Next meeting */

= Weekly Manila team meeting =
'''NOTE MEETING TIME: Thursday at 15:00 UTC'''

If you're interested in management of shared filesystems for OpenStack, we have a weekly meetings in <code><nowiki>#openstack-meeting-alt</nowiki></code>, on Thursdays at 15:00 UTC. Please feel free to add items to the agenda below. NOTE: When adding topics please include your IRC name so we know who's topic it is and how to get more info.

== Next meeting ==
'''NOTE:''' ''Include your IRC nickname next to agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. You might want to put this on your calendar if you are adding items.''

'''Scheduled for 3 Sep, 2015, 15:00 UTC'''
# Manila gate issues
#* Liberty-3 content
#* Feature freeze exceptions & deferral candidates
# 3rd-party CI status
#* http://ec2-54-67-102-119.us-west-1.compute.amazonaws.com:5000/?project=openstack/manila&user=&timeframe=72&start=&end=&page_size=500
#* http://ci-watch.tintri.com/project?project=manila
# Manila devref update: (u_glide)
#* Move DHSS support to separate table from http://docs.openstack.org/developer/manila/devref/share_back_ends_feature_support_mapping.html#mapping-of-share-drivers-and-share-features-support
#* Removal of "supported operations" pages in driver doc sections
# Automount vs asset injection [csaba]
# Open Discussion

== Previous meetings ==

http://eavesdrop.openstack.org/meetings/manila/

Manila/Meetings

2015-07-09T14:46:11Z

Chenk: /* Next meeting */

Manila/Meetings

2015-04-23T15:00:06Z

Chenk: /* Next meeting */

= Weekly Manila team meeting =
'''NOTE MEETING TIME: Thursday at 15:00 UTC'''

If you're interested in management of shared filesystems for OpenStack, we have a weekly meetings in <code><nowiki>#openstack-meeting-alt</nowiki></code>, on Thursdays at 15:00 UTC. Please feel free to add items to the agenda below. NOTE: When adding topics please include your IRC name so we know who's topic it is and how to get more info.

== Next meeting ==
'''NOTE:''' ''Include your IRC nickname next to agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. You might want to put this on your calendar if you are adding items.''

'''Scheduled for 23 Apr, 2015, 15:00 UTC'''

# Doc reviews
# Liberty design summit
#* https://etherpad.openstack.org/p/manila-liberty-proposed-sessions
# Mount automation
#* Who suggested zeroconf? Should we be investigating that further?
#* Other options under consideration: SSH/PsExec, automounter+LDAP, custom agent, Cloud-Init
# fix for glusterfs_native breaking multibackend setups (csaba, u_glide)
# Open Discussion

== Previous meetings ==

http://eavesdrop.openstack.org/meetings/manila/

Manila/Provide private data storage API for drivers

2015-04-20T05:34:04Z

Chenk: fix Github commit reference

== Provide limited data API for drivers ==

===Problem ===
Drivers haven’t possibility to store key/value pairs in the database for each share (and probably for share snapshots too). These values are not visible to the tenants, they're just for the drivers.

===Use cases ===
'''Use case #1:''' Generic driver should store volume id instead of renaming volume (current behaviour) <br />
'''Use case #2:''' Some backends that can't create 32-character share names and need to maintain a mapping from the 128-bit UUID to something smaller.

===Concept===
Provide share data storage (key-value) for drivers.

Implement simple storage with following interface:
<source lang="python" >
class DriverShareData(object):
def __init__(self, context, db, backend_host):
pass

def get(self, share_id, key=None, default_value=None):
pass

def update(self, share_id, share_data, delete_existing=False):
pass

def delete(self, share_id, key):
pass
</source>

Provide this storage in the manager to all drivers:

<source lang="python">
class ShareManager(manager.SchedulerDependentManager):
# …
def __init__(self, share_driver=None, service_name=None, *args, **kwargs):
# …
self.share_data = DriverShareData(self.context, self.db, self.host)
self.driver = importutils.import_object(
share_driver,
self.share_data,
#...
)
# ...
</source>

This storage will allow to get/update/delete private data of any share managed by the driver. Drivers will be able to create mappings between data in manila (Share) and backends (NAS). Also, drivers could use this storage for caching purposes - to minimize an amount of requests to the backend.

Default implementation will store data in separate table in Manila database.

===This concept vs model updates in Share Manager===
As you can see in https://github.com/openstack/manila/blob/815e2c95acc402847bb497365ea5caba608fd8cc/manila/share/manager.py Share Manager is responsible to update information in DB. For example:
<source lang="python">
# ...
try:
if snapshot_ref:
export_locations = self.driver.create_share_from_snapshot(
context, share_ref, snapshot_ref,
share_server=share_server)
else:
export_locations = self.driver.create_share(
context, share_ref, share_server=share_server)

self.db.share_export_locations_update(context, share_id,
export_locations)
# ...
</source>

We could follow this rule and simply replace return value by dictionary with optional keys:

<source lang="python">
# ...
try:
if snapshot_ref:
data = self.driver.create_share_from_snapshot(
context, share_ref, snapshot_ref,
share_server=share_server)
else:
data = self.driver.create_share(
context, share_ref, share_server=share_server)

self.db.share_export_locations_update(context, share_id,
data['export_locations'])
# New code
private_share_data = data.get('private_share_data')
if private_share_data:
self.db.private_share_data_update(context, share_id, private_share_data)
# ...
</source>

But in this case we will be forced to do so in each method where drivers require private share data. Furthermore, we will be forced to retrieve and pass private share data to drivers through argument even if they don't require this data.
This will cause performance penalty and code will look like a mess :)

'''That's why my (u_glide) proposal is:'''
Leave current model updates in Share manager as is and provide DriverShareData interface to all drivers in the manager.

===DB scaling risks===
By default, all private share data will be stored in manila SQL DB and we have a risk that DB becomes a bottleneck in large deployments. This risk (in a scope of this feature) can be addressed by implementation of different storage backends for this interface. As we can see backends don’t share data, so storage can be easily horizontally scaled. We can move private share data to any distributed key-value storage.

Manila/Meetings

2014-12-18T14:56:58Z

Chenk: /* Next meeting */

= Weekly Manila team meeting =
'''NOTE MEETING TIME: Thursday at 15:00 UTC'''

If you're interested in management of shared filesystems for OpenStack, we have a weekly meetings in <code><nowiki>#openstack-meeting-alt</nowiki></code>, on Thursdays at 15:00 UTC. Please feel free to add items to the agenda below. NOTE: When adding topics please include your IRC name so we know who's topic it is and how to get more info.

== Next meeting ==
'''NOTE:''' ''Include your IRC nickname next to agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. You might want to put this on your calendar if you are adding items.''

'''Scheduled for 18 Dec, 2014, 15:00 UTC'''
# Development status
# Kilo-1 milestone review (bswartz)
#* https://launchpad.net/manila/+milestone/kilo-1
# Kilo-2 planning (bswartz)
#* https://launchpad.net/manila/+milestone/kilo-2
# NetApp cDOT driver refactoring (cknight)
#* https://blueprints.launchpad.net/manila/+spec/netapp-manila-cdot-driver-refactoring
# Holidays
# Syetem services (if by chance time remains) (csaba)
# Open Discussion

== Previous meetings ==

http://eavesdrop.openstack.org/meetings/manila/

Manila/Meetings

2014-12-04T14:57:14Z

Chenk: /* Next meeting */

= Weekly Manila team meeting =
'''NOTE MEETING TIME: Thursday at 15:00 UTC'''

If you're interested in management of shared filesystems for OpenStack, we have a weekly meetings in <code><nowiki>#openstack-meeting-alt</nowiki></code>, on Thursdays at 15:00 UTC. Please feel free to add items to the agenda below. NOTE: When adding topics please include your IRC name so we know who's topic it is and how to get more info.

== Next meeting ==
'''NOTE:''' ''Include your IRC nickname next to agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. You might want to put this on your calendar if you are adding items.''

'''IMPORTANT:''' The meeting on 27 Nov is CANCELLED due to a US holiday.

'''Scheduled for 4 Dec, 2014, 15:00 UTC'''
# Development status
# Kilo-1 milestone (bswartz)
# Ask Manila community: resolving a name clash (csaba)
# Show Manila community: git-review-branch (csaba)
# Open Discussion

== Previous meetings ==

http://eavesdrop.openstack.org/meetings/manila/

Manila/Meetings

2014-12-04T14:40:30Z

Chenk: /* Next meeting */

Manila/Meetings

2014-08-14T14:45:24Z

Chenk: /* Next meeting */

= Weekly Manila team meeting =
'''NOTE MEETING TIME: Thursday at 15:00 UTC'''

If you're interested in management of shared filesystems for OpenStack, we have a weekly meetings in <code><nowiki>#openstack-meeting-alt</nowiki></code>, on Thursdays at 15:00 UTC. Please feel free to add items to the agenda below. NOTE: When adding topics please include your IRC name so we know who's topic it is and how to get more info.

== Next meeting ==
'''NOTE:''' ''Include your IRC nickname next to agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. You might want to put this on your calendar if you are adding items.''

'''Scheduled for 15 Aug, 2014, 15:00 UTC'''
# Cirros status (csaba)

== Previous meetings ==

'''Feb 13, 2014 15:00 UTC'''
# [http://git.openstack.org/cgit/openstack/governance/tree/reference/incubation-integration-requirements Incubation] (bswartz)

'''Oct 17, 2013, 15:00 UTC'''
# [https://blueprints.launchpad.net/manila/+spec/join-tenant-network Networking BP]

'''Sep 26, 2013, 15:00 UTC'''
# [[Manila/Networking|Networking Wiki]] (bswartz)

'''Aug 8, 2013, 15:00 UTC'''
# [[Manila/ProjectPlan|Project Plan]] -- bswartz

http://eavesdrop.openstack.org/meetings/manila/

Manila/docs/

2014-06-25T11:38:06Z

Chenk: Redirected page to Manila/docs

#REDIRECT [[Manila/docs]]

Setting up DevStack with Manila on Fedora 20

2014-06-25T11:35:21Z

Chenk: Blanked the page

Manila/Meetings

2014-02-20T14:37:00Z

Chenk:

= Weekly Manila team meeting =
'''NOTE MEETING TIME: Thursday at 15:00 UTC'''

If you're interested in management of shared filesystems for OpenStack, we have a weekly meetings in <code><nowiki>#openstack-meeting-alt</nowiki></code>, on Thursdays at 15:00 UTC. Please feel free to add items to the agenda below. NOTE: When adding topics please include your IRC name so we know who's topic it is and how to get more info.

== Next meeting ==
'''NOTE:''' ''Include your IRC nickname next to agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. You might want to put this on your calendar if you are adding items.''

'''Scheduled for 20 Feb, 2014, 15:00 UTC'''
# Development Status
#* Cirros manila-service-generic update (csaba)
# Open Discussion
#* Discussion on service VM role for generic driver and other (hypothetic) multitenant drivers (csaba)

== Previous meetings ==

'''Feb 13, 2014 15:00 UTC'''
# [http://git.openstack.org/cgit/openstack/governance/tree/reference/incubation-integration-requirements Incubation] (bswartz)

'''Oct 17, 2013, 15:00 UTC'''
# [https://blueprints.launchpad.net/manila/+spec/join-tenant-network Networking BP]

'''Sep 26, 2013, 15:00 UTC'''
# [[Manila/Networking|Networking Wiki]] (bswartz)

'''Aug 8, 2013, 15:00 UTC'''
# [[Manila/ProjectPlan|Project Plan]] -- bswartz

http://eavesdrop.openstack.org/meetings/manila/

Manila/Networking/Gateway mediated

2014-02-05T20:14:44Z

Chenk: Chenk moved page Manila Networking/Hypervisor mediated to Manila Networking/Gateway mediated: calling the subject "Hypervisor mediated" was a misnomer.

= Gateway mediated storage access for NFS shares and Multi-tenancy =

== Architecture ==

This document outlines an approach to achieving multi-tenancy using Manila OpenStack and the architecture/design considerations wrt enabling Multi-tenancy in the cloud using this approach. This model is based on the [[Manila_Networking#Gateway_Mediated|Gateway mediated approach to Manila Networking]]. Here "gateway" is to be understood as the storage gateway, ie. the machine which is member a permanent network which contains the storage backends, and manages/mediates access from tenant machines to the storage backends.

The approach taken is a gateway mediated one where the decision to allow tenant access to the intended Manila share is made in the gateway. Typically, each gateway can have multiple guest instances each of which is considered a tenant. The proposed design involves running a user-space NFS server on the gateway. Tenants rely on NFS clients to connect to this NFS server and consume any exported shares. The NFS server in turn is assumed to be capable enough, to talk to different storage backends and serve requests made by its clients.

We propose to use nfs-ganesha[1][2] as the user-space NFS daemon in this architecture. This is based on the following: nfs-ganesha is easily extensible to various filesystem backends and has a modular/plugin-based architecture. A plugin is commonly referred to as File System Abstraction Layer (FSAL) in NFS-Ganesha parlance. Currently, nfs-ganesha has support for a variety of FSALs including ones for CephFS, GlusterFS, Lustre, ZFS, VFS, GPFS etc. nfs-ganesha also supports various NFS protocol versions like NFSv2,v3,v4,v4.1,v4.2 and pNFS. This makes it ideal for future use-case extensibility and also from the point of view of accomodating the maximum possible filesystem backends, including any new vendor developed ones.

When a mount request is received by the nfs-ganesha server, it checks whether the client/tenant is allowed to access the related export or share. Based on the NFS share ACL configurations, the exported share operation will succeed or fail. If the mount succeeds, all further operations performed from the client will succeed and Ganesha would route such requests to the storage backend through the corresponding filesystem FSAL. So tenant share separation is made possible based on NFS ACLs and access permissions. Further, nfs-ganesha also supports Kerberos based authentication in case Manila shares need to be exported and consumed across nodes in an Openstack compute cluster when nfs-ganesha would be mounted on each gateway in the cluster in Multi-head configuration.

== Advantages: ==

* Works with all guest operating systems that have a nfs client built in. Well standardized and widely available support.
* Does not require any VM/guest plugin installation or any Openstack components to be modified to work with this approach.
* Works across the majority of the prevalent filesystem backends and/or Storage Arrays without modifications.
* This approach most closely mirrors how guests interact with storage controllers in Cinder. The storage controller only needs to know about compute nodes and the gateway deals with the problem of presenting the storage to the guest.
* Security is enforced by the gateway. Backends need not support any security to protect tenant data from other tenants (or even enforce protection between instances of one tenant).
* Further tenant separation is based on share permissions based on NFS ACLs and authentication. This is easily enforced.
* No dependence or interaction with Neutron
* NFS server on the gateway performs caching. So not all requests from the instances need to be served by the storage layer. Thereby data access is sometimes faster, and the storage layer is less burdened.
* Being a user-space server, nfs-ganesha's cache-inode layer can grow arbitrarily large, limited mostly (only) by the underlying hardware. This supports greater scalability in terms of supporting larger number of compute instances and share operations/requests.
* Extensible to support other multi-tenant use-cases like share reservations, QoS requirements etc.

== Disadvantages: ==

* Resources are consumed by nfs-ganesha as it runs on the gateway. Resource consumption by ganesha has been seen to be moderate across general workloads so we don't see this as a major impact to the overall architecture currently. Further tests need to be carried out to ascertain this impact if any.
* High Availability for tenant share access via nfs-ganesha could necessitate additional configuration and state maintenance. However, this availability support/infrastructure is mostly available as a separate Ganesha solution w.r.t. different filesystem backends.

== References: ==

[1] https://github.com/nfs-ganesha/nfs-ganesha/wiki

[2] https://forge.gluster.org/nfs-ganesha-and-glusterfs-integration/pages/Home

[[Image:Hypervisor-mediated.png|frame|none]]

Manila/Networking/Hypervisor mediated

2014-02-05T20:14:44Z

Chenk: Chenk moved page Manila Networking/Hypervisor mediated to Manila Networking/Gateway mediated: calling the subject "Hypervisor mediated" was a misnomer.

#REDIRECT [[Manila Networking/Gateway mediated]]

Manila/Networking/Gateway mediated

2014-02-05T20:13:46Z

Chenk:

Manila/Networking

2014-02-04T16:04:32Z

Chenk: formatting cleanup

= Manila =

This is a page about networking for Manila (OpenStack Shared Filesystem Service)

= Protocols =

Currently we support:
* NFS
* CIFS

We intend to support more though. Possible future protocols include:
* GlusterFS
* GPFS
* Ceph FS
* Dropbox (or similar)

= Backends =

Currently we've implemented two backends:
* Linux LVM
* NetApp (7-mode and C-mode)

We're looking to other vendors to implement backends for their hardware/software.

= Frontends =

How the end user actually interacts with the filesystems provided for Manila is an interesting question. Consider the following approaches:

=== Network Plumbing ===

The approach here is to interact with Neutron to ensure that the server(s) providing the storage are connected to the tenant's private network. This means that by the time an attach operation completes, a network link has been configured so that the client(s) specified in the attach operation can send packets to the server(s) returned hosting the storage.

In this mode, clients can directly interact with the storage server using the storage protocol that they intended (i.e. CIFS or NFS). The storage server must be responsible for enforcing security such that clients are not able to see resources that haven't been attached to them, and more importantly, tenants must not see data owned by other tenants. On the controller side this may be accomplished by virtualizing the storage server (The NetApp vserver approach), by using firewalls between the clients and server, or by using rules built into the storage server itself (rules in /etc/exports or /etc/smb.conf, etc).

Advantages:
* No performance bottlenecks introduced
* It feels natural to the end user, and applications qualified against CIFS/NFS in the physical work are more likely to work without issues in the cloud
* Users can take advantage of special CIFS/NFS features if they are available
* It pushes the hard security problems down into the drivers
* It does not depend on Nova
* It can work with bare metal clients

Disadvantages:
* It relies on Neutron, creating a (possibly unwanted) dependency
* It pushes the hard security problems down into the drivers

For security to work in this approach, each tenant will have to provide some details before they can connect to any shares. Each tenant must provide these details a minimum of one time, but possibly more than once (if the tenant has multiple security domains). The details will depend on the protocols being used. Some examples:
* Active Directory domain and credentials (CIFS)
* LDAP and/or Kerberos domain and credentials (NFS)

=== Hypervisor Mediated ===

This approach involves using some special technology to virtualize the filesystem through to the guest. The one we've looked at most closely is VirtFS.

An attach operation under this approach would simply involve attaching the storage to the compute node rather than the ultimate user (as far as the backend is concerned). An additional sequence of actions would have to take place in the Manila manager to complete the attachment all the way through to the guest, which would include callbacks to nova to configure the needed VirtFS steps.

Advantages:
* This approach most closely mirrors how guests interact with storage controllers in Cinder. The storage controller only needs to know about compute nodes and the hypervisor deals with the problem of presenting the storage to the guest.
* Security is enforced by the hypervisor. Backends need not support any security to protect tenant data from other tenants (or even enforce protection between instances of one tenant).
* No dependence or interaction with Neutron

Disadvantages:
* Requires a hypervisor (will not work w/ bare metal)
* Hypervisor support limited (only KVM and Xen)
* Guest OS support limited (Only Linux and BSD)
* User sees a Plan9 filesystem instead of a NFS or CIFS filesystem which may prevent access to features
* Depends on adding some new features to Nova
* It may be hard to preserve ACLs on the files if people care about that (for example UID/GID/mode bits to enforce filesystem security in multiuser environments)
* VirtFS may be immature and not well maintained (this is an opinion)
* Possible performance bottleneck in VirtFS itself, although benchmarks suggest that VirtFS can be even faster than running NFS over a virtual NIC

=== Gateway Mediated ===

The theory of the gateway mediated approach is that the hypervisor mediated (VirtFS) approach is desirable, but we want to overcome the limitations of VirtFS itself. One way to do this is to take advantage of the hypervisor to add an additional virtual NIC through which the guest OS may access its shared filesystems.

The virtual NIC would implement half of a point to point tunnel, the other half being at the storage gateway. Under normal circumstances, the storage gateway would run on the compute node alongside nova. In this configuration, backends would still attach storage to the compute node just like the VirtFS case, but the gateway would simply tunnel filesystem access protocols straight through, such that the guest could use a regular NFS or CIFS client, connecting to special tunnel-specific IP addresses, and the gateway would do any firewalling and NAT necessary to connect the client with the real backend while preventing the client from seeing data it shouldn't.

Advantages:
* This approach also closely mirrors how guests interact with storage controllers in Cinder
* Security is enforced by the gateway, in the form of spoofing-prevention and firewalling of packets from the client. Backends could implement only IP-based security.
* No dependence or interaction with Neutron
* Works with any hypervisor (unlike VirtFS)
* Work with any guest (unlike VirtFS)
* Users can take advantage of special CIFS/NFS features if they are available (with some exceptions)
* It feels natural to the end user, and applications qualified against CIFS/NFS in the physical work are more likely to work without issues in the cloud (with some exceptions)

Disadvantages:
* Requires a hypervisor (will not work w/ bare metal)
* Depends on adding some new features to Nova
* The gateway can become a performance bottleneck
* The extra NIC may seem strange to end users, especially if it uses magic IP for the 2 ends of the tunnel
* The NAT aspect of the gateway could interfere with protocol features
* Security is limited to methods that can work with 1-way communication (UNIX/system security for NFS, workgroup security for CIFS)

=== Flat Network ===

Included for completeness, this is what Manila implements today. There will always be scenarios where the administrator is either doing something way more complicated that what we can imagine, or the administrator is happy with a simple flat network and minimal security. In this case, connectivity between the ultimate users of the storage and the backends is left up to somebody else, and Manila is only responsible for provisioning and exporting storage to a client without regard to what's on the other end.

Advantages:
* It's very simple from an implementation standpoint (it's done)
* It allows compatibility with unanticipated future designs
* It may be the preferred approach for single tenant scenarios like private clouds

Disadvantages:
* Cannot support multi-tenant scenarios like public cloud

== Plugins ==

Given the advantages and disadvantages of the different approaches above, it seems clear that no one approach will be ideal for all circumstances. I believe a plugin system is needed so that the administrator can choose what he wants to use in his own environment.

= Security =

As I mentioned in the network plumbing section, using filesystem security requires some configuration work up-front.

In the of CIFS, the user needs to tell Manila what domain (active directory) the CIFS share should be part of, if any. The backend may need to create a new virtual storage controller and join it to the domain, which means the user needs to provide relevant information, such as DNS server IP, domain controller IP, domain administrator account name and password, etc. In theory this could be a one time operation, separate from the storage provisioning operation, but in the case where a tenant has 2 different domains and wants to provision into both of them, then individual provisioning requests need to either include all of the above details or a reference to a previously created object. In both cases, Manila will need to store the details so it can tell whether new requests can be placed on a controller that's already joined the domain, or whether a new controller needs to be set up.

NFS has similar requirements, if Kerberos is used. The user needs to provide Manila with enough details that the backend can join the Kerberos and LDAP domains, which means IP addresses of those services, or DNS and hostnames for those services, as well as credentials for an administrative account so the controller can join the domains.

= Comments =
== Caitlin Bestler ==
My analysis is similar, but structured a bit differently.

There are three fundamental objectives that must be achieved by any mediation scheme:
* Disambiguation of User IDs (as well as Roles/Groups/etc.), Security Domain X UID 500 must never be confused with Security Domain Y UID 500.
* Restrict mounting of shares to being within Security Domains. All access to Security Domain X shares must be approved by a Security Domain X specified authentication server (probably LDAP or AD).
* Disallow cross-security domain access, even with forged mount handles. A would-be pirate within Security Domain X must not be able to access Security Domain Y file servers by just faking an IP address. It is not enough to have a Security Domain X login, the VM or port you are using must be enabled to access Security Domain X.

The different solutions can be summarized as:

* Network mediation: Each security domain is placed on an isolated logical network (whether by tunneling or VLANs). The NAS servers are expected to provide a virtual interface for each security domain network.
** Disambiguation of User identifiers is the responsibility of the virtual server. Manila does not care how the server appliance does this.
** A virtual server associated with Security Domain X can only export file shares owned by Security Domain X.
** Neutron will only allow ports that are admitted to Security Domain X to use that logical network.
** The NAS Client is in the Guest OS (or user space).
* Hypervisor-mediated with virtFS
** Virtual File System operations are serialized over the virtualization bus, so the file system is really running in the Hypervisor or a designated domain.
** This file system acts as a proxy for accessing the true storage servers.
** It must map Security Domain specific user identifiers to universal identifiers.
** It may only allow same-domain mounts.
** This requires adding functionality to the Guest. Therefore it cannot be the ''only'' solution.
* Hypervisor-mediated: with proxy
** The Hypervisor runs NAS proxies (either itself or in designated domains)
** A NAS client runs in each Guest (OS or user).
** The proxy otherwise must do the same translations as with the Hyerpvisor-mediated-virtFS solution.
** At worst case the Guest must be configured to access a different file server, therefore this can work without modifying Guest code (only guest configuration).
*** Transparent proxying can eliminate even that, but might not be worth the effort.

Manila/Networking/Gateway mediated

2013-12-17T09:16:13Z

Chenk: sanitize section levels

= Hypervisor mediated storage access for NFS shares and Multi-tenancy =

== Architecture ==
This document outlines an approach to achieving multi-tenancy using Manila OpenStack and the architecture/design considerations wrt enabling Multi-tenancy in the cloud using this approach. This model is based on the [[Manila_Networking#Hypervisor_Mediated|Hypervisor mediated approach to Manila Networking]]

The approach taken is a hypervisor mediated one where the decision to allow tenant access to the intended Manila share is made in the hypervisor. Typically, each hypervisor can have multiple guest instances each of which is considered a tenant. The proposed design involves running a user-space NFS server on the hypervisor. Tenants rely on NFS clients to connect to this NFS server and consume any exported shares. The NFS server in turn is assumed to be capable enough, to talk to different storage backends and serve requests made by its clients.

We propose to use nfs-ganesha[1][2] as the user-space NFS daemon in this architecture. This is based on the following:
nfs-ganesha is easily extensible to various filesystem backends and has a modular/plugin-based architecture. A plugin is commonly referred to as File System Abstraction Layer (FSAL) in NFS-Ganesha parlance. Currently, nfs-ganesha has support for a variety of FSALs including ones for CephFS, GlusterFS, Lustre, ZFS, VFS, GPFS etc. nfs-ganesha also supports various NFS protocol versions like NFSv2,v3,v4,v4.1,v4.2 and pNFS. This makes it ideal for future use-case extensibility and also from the point of view of accomodating the maximum possible filesystem backends, including any new vendor developed ones.

When a mount request is received by the nfs-ganesha server, it checks whether the client/tenant is allowed to access the related export or share. Based on the NFS share ACL configurations, the exported share operation will succeed or fail. If the mount succeeds, all further operations performed from the client will succeed and Ganesha would route such requests to the storage backend through the corresponding filesystem FSAL. So tenant share separation is made possible based on NFS ACLs and access permissions. Further, nfs-ganesha also supports Kerberos based authentication in case Manila shares need to be exported and consumed across nodes in an Openstack compute cluster when nfs-ganesha would be mounted on each hypervisor in the cluster in Multi-head configuration.

== Advantages: ==

* Works with all guest operating systems that have a nfs client built in. Well standardized and widely available support.
* Does not require any VM/guest plugin installation or any Openstack components to be modified to work with this approach.
* Works across the majority of the prevalent filesystem backends and/or Storage Arrays without modifications.
* This approach most closely mirrors how guests interact with storage controllers in Cinder. The storage controller only needs to know about compute nodes and the hypervisor deals with the problem of presenting the storage to the guest.
* Security is enforced by the hypervisor. Backends need not support any security to protect tenant data from other tenants (or even enforce protection between instances of one tenant).
* Further tenant separation is based on share permissions based on NFS ACLs and authentication. This is easily enforced.
* No dependence or interaction with Neutron
* NFS server on the hypervisor performs caching. So not all requests from the instances need to be served by the storage layer. Thereby data access is sometimes faster, and the storage layer is less burdened.
* Being a user-space server, nfs-ganesha's cache-inode layer can grow arbitrarily large, limited mostly (only) by the underlying hardware. This supports greater scalability in terms of supporting larger number of compute instances and share operations/requests.
* Extensible to support other multi-tenant use-cases like share reservations, QoS requirements etc.
== Disadvantages: ==
* Resources are consumed by nfs-ganesha as it runs on the hypervisor. Resource consumption by ganesha has been seen to be moderate across general workloads so we don't see this as a major impact to the overall architecture currently. Further tests need to be carried out to ascertain this impact if any.
* High Availability for tenant share access via nfs-ganesha could necessitate additional configuration and state maintenance. However, this availability support/infrastructure is mostly available as a separate Ganesha solution w.r.t. different filesystem backends.

== References: ==
[1] https://github.com/nfs-ganesha/nfs-ganesha/wiki

[2] https://forge.gluster.org/nfs-ganesha-and-glusterfs-integration/pages/Home

[[File:Hypervisor-mediated.png]]

File:Hypervisor-mediated-manila-access-2.svg

2013-12-05T14:07:32Z

Chenk: Chenk uploaded a new version of "File:Hypervisor-mediated-manila-access-2.svg"

Hypervisor mediated manila access

File:Hypervisor-mediated-manila-access-2.svg

2013-12-05T12:53:33Z

Chenk: Chenk uploaded a new version of "File:Hypervisor-mediated-manila-access-2.svg"

Hypervisor mediated manila access

File:Hypervisor-mediated-manila-access-2.svg

2013-12-05T12:50:58Z

Chenk: Chenk uploaded a new version of "File:Hypervisor-mediated-manila-access-2.svg"

Hypervisor mediated manila access

File:Images layouts-2.svg

2013-12-05T12:47:33Z

Chenk: test

test

File:Hypervisor-mediated-manila-access-2.svg

2013-12-05T12:20:19Z

Chenk: Chenk uploaded a new version of "File:Hypervisor-mediated-manila-access-2.svg"

Hypervisor mediated manila access

File:Hypervisor-mediated-manila-access-2.svg

2013-12-05T12:01:06Z

Chenk: Hypervisor mediated manila access

Hypervisor mediated manila access

Manila/docs/Manila Developer Setup Fedora19

2013-12-01T20:26:45Z

Chenk: decrease header levels by one to avoid superfluous nesting (makes quite a difference in mobil view)

This document describes how to install devstack with Manila support on a virtual machine.

= Recommended VM requirements =

* 4G RAM
* 20G of free disk space

The instructions were tested on a Fedora 19 (f19) VM. However, the following notes should work with a Ubuntu VM as well, but some of the instructions might not be required by Ubuntu users. So whenever the notes are suspected to be Fedora specific they'd be stated as being so.

= Installing DevStack =

Below is the DevStack repo with Manila integrated. Besides installing other OpenStack services, it pulls and installs Manila from, https://github.com/stackforge/manila, the current upto-date Manila repo:

<pre>$ git clone git@github.com:bswartz/devstack.git
$ cd devstack</pre>
Note: Very recently, the Manila repo has been integrated with DevStack master, which you could also use. Hence, some of the below instructions might no longer be relevant. Expect appropriate changes in instructions soon.

You might need to make the following changes. The change in ''./lib/manila'' is needed if you use F19. Otherwise, the ''stack.sh'' script, which installs the OpenStack services, would install the manila scripts in ''/usr/bin'', but later try to execute the manila scripts from ''/usr/local/bin''.

<source lang="diff">diff --git a/lib/manila b/lib/manila
index abd0158..7f75ebc 100644
--- a/lib/manila
+++ b/lib/manila
@@ -48,7 +48,7 @@ MANILA_SERVICE_PROTOCOL=${MANILA_SERVICE_PROTOCOLS
if [[ -d $MANILA_DIR/bin ]]; then
MANILA_BIN_DIR=$MANILA_DIR/bin
else
- MANILA_BIN_DIR=/usr/local/bin
+ MANILA_BIN_DIR=/usr/bin
fi

SHARE_GROUP=${SHARE_GROUPtack-shares}
@@ -456,4 +456,4 @@ function stop_manila() {
}</source>
<br />
The changes in localrc:

* The change in password of Rabbit service to 'guest' is needed if you use F19. A fresh installation of rabbitmq creates a user 'guest' with the password 'guest'. And for some reason, DevStack fails to set the password of the user 'guest' as desired by the user.
* Extra variables can be set for logging the screens of the OpenStack services running. This is useful especially because the default screens have a very limited scrollback buffer.
* The size of the backing file for a Manila-LVM share can be reduced from 8G to 2G.
<br />
<source lang="diff">diff --git a/localrc b/localrc
index 05ea5cd..4474425 100644
--- a/localrc
+++ b/localrc
@@ -1,9 +1,15 @@
DATABASE_PASSWORD=rengen
-RABBIT_PASSWORD=rengen
+RABBIT_PASSWORD=guest
SERVICE_TOKEN=rengen
SERVICE_PASSWORD=rengen
ADMIN_PASSWORD=rengen

+# Enable Logging
+LOGFILE=/opt/stack/logs/stack.sh.log
+VERBOSE=True
+LOG_COLOR=True
+SCREEN_LOGDIR=/opt/stack/logs
+

@@ -66,7 +72,7 @@ MANILA_SERVICE_PORT_INT=18776

VOLUME_BACKING_FILE_SIZE=${VOLUME_BACKING_FILE_SIZE:-1048M}

-SHARE_BACKING_FILE_SIZE=${SHARE_BACKING_FILE_SIZE:-8400M}
+SHARE_BACKING_FILE_SIZE=${SHARE_BACKING_FILE_SIZE:-2048M}
MANILA_SECURE_DELETE=$CINDER_SECURE_DELETE</source>
<br />
Run ''stack.sh''.

<pre>$ ./stack.sh</pre>
Note that the above script sets SELinux to Permissive mode for Fedora users. This setting however doesn't survive a reboot. For a permanent change in SELinux setting, you need to edit the ''/etc/sysconfig/selinux'' file to be as below:

<pre># This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=permissive</pre>
Check the status of SELinux:

<pre>$ getenforce</pre>
= Launching a Guest instance using Nova =

Make sure that the services, httpd, rabbitmq-server (or some other AMQP), mysqld (db) are running every time you begin (e.g. after a sytem reboot) using OpenStack. For F19, you could do this by:

<pre>$ systemctl status rabbitmq-server</pre>
If they aren't running you should get them up and running:

<pre># systemctl restart rabbitmq-server</pre>
And also don't forget to check whether SELinux is set to Permissive mode :).

Now you need to setup the keystone login credentials before you can use the CLI of the various OpenStack services. This can be easily done by sourcing ''openrc'' file with a user and a tenant name respectively. And ''stack.sh'' already has created users (admin and demo) and tenants (admin and demo), which you could use. So, you could do the following:

<pre>$ source openrc admin admin</pre>
Bring up (if you rebooted after running ''stack.sh'') and interact with the various OpenStack services by running ''rejoin-stack.sh'' script.

<pre>$ ./rejoin-stack.sh</pre>
You'd need Nova guests to test if the Manila shares are accessible. And it makes sense that you use a light weight image (CirrOS) to launch an instance. One such lightweight image with nfs kernel support, nfs-utils, rpcbind and strace was put together by [https://launchpad.net/~chenk Csaba Henk]: https://mega.co.nz/#!B91VUA6Z!eNB2BSW4p1o8lRqaPeaIek4g_J1LJL_SY2OymVnurlc

Create a glance-image using the image downloaded from the above link:

<pre>$ glance image-create --name="Cirros 0.3.1" --disk-format=qcow2 \
--container-format bare < cirros-git-disk.qcow2</pre>
List the glance images, and check if available:

<pre>$ glance image-list</pre>
Add a keypair list:

<pre>$ nova keypair-add --pub_key ~/.ssh/id_rsa.pub mykey</pre>
Set network rules to allow ssh and ping access to the guest VM:

<pre>$ nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
$ nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0</pre>
Boot a nova instance. Please use the image-id of the downloaded image from glance image-list:

<pre>$ nova boot --flavor m1.tiny --image 00aa37a3-0bec-4140-bbf1-617fc3b3da63 \
--key_name mykey --security_group default myvm0</pre>
Check the status of the guest instance:

<pre>$ nova list</pre>
Check by ssh-ing to the VM. Password of the guest VM's cirros is <code>cubswin:)</code>:

<pre>$ ssh cirros@<the ip of the vm, usually a private IP, unless setup></pre>
Inside the guest vm. If you need to be root:

<pre>$ sudo -s
$ id</pre>
= Creating a Manila LVM NFS share =

Currently, Manila doesn't support multi-tenancy (the code to do so is under review as of today), and the networking model is the simple FlatNetwork model. For more info, see

* https://wiki.openstack.org/wiki/Shares_Service
* https://wiki.openstack.org/wiki/Manila_Networking

Check if the Manila share driver and it's helpers are as follows, else set them:

<pre>$ cat /etc/manila/manila.conf | grep share_driver
share_driver = manila.share.drivers.lvm.LVMShareDriver
$ cat /etc/manila/manila.conf | grep share_lvm_helpers
share_lvm_helpers = CIFS=manila.share.drivers.lvm.CIFSHelper,NFS=manila.share.drivers.lvm.NFSHelper</pre>
Creating a 1G LVM Manila NFS share by carving 1G logical volume from a volume group, which is by default stack-shares:

<pre>$ manila create --name myshare0 NFS 1</pre>
Check if the share is available:

<pre>$ manila list</pre>
Allow the share to be accessed by the running instance:

<pre>$ manila allow-access <share-id> ip <instance-id></pre>
You should now be able to ssh into your guest VM, and check for the NFS share:

<pre>$ showmount -e</pre>
If all's well, you can mount the NFS share on your guest. If not, the most probable reason could be that the firewall rules setup by Nova are working against you. The dirty way to work around this problem would be to stop and disable firewall daemon in your host VM, and restart the Nova services.

<pre># systemctl stop firewalld
# systemctl disable firewalld</pre>