Jump to: navigation, search


What is Mercador?

Mercador is a system for federating independent OpenStack clouds. The name comes from the Portuguese word for merchant.

The basic consumption model for cloud services is a simple two-party interaction between a Cloud Service User (CSU) and a Cloud Service Provider (CSP). The CSP deploys a set of services and publishes them though a standard interface supporting UI, CLI, and API. The CSU subscribes to this bundle of services (using a mechanism that is generally out of scope for OpenStack), and consumes the services via the published interfaces. No matter how complex or geographically distributed the services, they are all owned and operated by a single legal entity, which has complete control over how they are configured and managed.

A number of use cases are emerging which require us to extend this model. CSPs operating private clouds wish to provide uniform access for their users to both private and public cloud services. A CSP operating in one geography wants to offer services hosted in a different country (for data sovereignty reasons); rather than setting up an OpenStack cloud in that country, it elects to source those services from another CSP. A telco decides that it wants to offer cloud services to its customers, but doesn't have the expertise to do so; instead it prefers to resell services from one or more CSPs under its own brand. A research consortium wants to aggregate cloud services from several university CSPs and present the aggregated services to its users through a "single pane of glass".

The common pattern in all of these cases is that we need to be able to take a bundle of resources and services from one CSP and make them available to a second CSP in such a way that they can be treated as part of the second CSPs service offering. A CSU should be unaware whether the services being consumed are implemented by the CSP with which they have a contractual relationship or by a third party. In the extreme case of a CSP acting as a pure reseller, they may not operate any services for themselves.


There are five important principles which affect the way in which Mercador is being developed.

  • First, we assume that the various actors - providers, consumers, resellers, brokers, etc. - are independent agents; there is no single authority that can impose and manage an overarching configuration. Each CSP will continue to manage its own policies for user on boarding, identity management, and so forth.
  • Second, we assume that the various relationships are dynamic, but not ephemeral. This means that the system must support both the programmatic construction of a publish-subscribe binding as well as the long-term management of the relationship.
  • Third, we assume that the life cycle of a publish-subscribe binding is governed by business and operational interactions that are out of scope for OpenStack, and that the system must be governed by workflows associated with these interactions.
  • Fourth, we want Mercador to be minimally intrusive. Ideally, a CSP should be able to publish resources by deploying a single package - the Mercador publisher - which will add two REST API services to the Service Catalog. A CSP should be able to discover and subscribe to third-party services by deploying a Mercador subscriber package, which adds one REST service. Both services are administered using a common CLI. The CSU should not see any changes.
  • Fifth, a CSP must be able to configure the collection of services and resources which it publishes. It must be able to restrict the set of OpenStack services, impose quotas and other policies, control the visibility of resources such as images, volumes and networks. It must also be able to configure metadata about the service which can be consumed by the subscriber.


The following terms are used in this document:

  • Region: A collection of OpenStack services and resources that are accessed via a single token (service catalog plus identity information). Almost all OpenStack resources and name spaces are scoped to a single region. An OpenStack Keystone service may be shared by multiple regions, and will supply a distinct token for each region. An OpenStack Horizon UI may provide access to multiple regions by being paired with a shared Keystone, from which it determines the accessible regions. (A second Horizon mode, based on the AVAILABLE_REGIONS configuration, is not supported by Mercador.)
  • Project: An OpenStack project. All OpenStack resource allocation operations take place in the context of a specific project, and all must be initiated by users who are authorized (via RBAC) to perform the operation within the project. All allocated resources are "owned" by a project.
  • Hierarchical Multitenancy (HMT): The system whereby a project can be "owned" by another project in an arbitrary hierarchy. At the present time, most OpenStack services are not HMT aware, and so they operate within the context of a single project without regard to hierarchical context. In the future, it is anticipated that various OpenStack functions will be enhanced to support HMT; for example, the quota system may allow the resource usage of different projects to be checked against an aggregate quota for the project subtree.
  • Domain: Any project may be identified as a domain. A domain may be associated with a specific (non-default) identity management policy, in which case this policy is applied to all sub-projects (unless over-ridden in a subordinate domain). The OpenStack RBAC framework allows domain-local administrative rights to be granted to users.
  • Virtual Region: A virtual region defines a set of services and resources published by a CSP as part of Mercador. The CSP defines the virtual region by specifying a domain within an existing region, together with a set of constraints, policies, and metadata. When a second CSP subscribes to the virtual region, it is able to make it available to its own CSUs as though it was just another region; the CSU should not be able to distinguish between a region operated by a CSP and one which the CSP has subscribed to.
  • Publisher: The CSP making available a virtual region; also, the Mercador service which implements this function.
  • Subscriber: The CSP consuming and "reselling" a virtual region; also the Mercador service which implements this function.
  • Service Aggregator: A specialized OpenStack configuration in which a CSP deploys no local resource services -- Nova, Neutron, Cinder, etc. All OpenStack services provided by the Service Aggregator are sourced as virtual regions from third party CSPs.

User stories


  • Cumulus, Cirrus and Nimbus are three OpenStack IaaS operators (Cloud Service Providers,or CSPs). Cumulus and Cirrus offer public cloud services. Nimbus is primarily in the business of supporting its SaaS activities.
  • Chris and Pat are two developers working for Widget Corp.
  • Acme Corp is a reseller of OpenStack cloud services.

User Story #1:

Widget wishes to provide OpenStack services to its developers to build in-house applications. For business reasons, it wants to use multiple cloud providers, but it wants to provide its developers with a single, consistent user experience. It contracts with Cumulus and Cirrus to provide OpenStack services. It deploys a Service Aggregator which allows Chris and Pat to use a single Horizon and Keystone to access the two clouds.

User Story #2:

Nimbus decides to raise revenue by reselling some of its IaaS capacity. However it does not want the complications of sales, support and billing to end users. Instead it contracts with Acme to resell its OpenStack services.

User Story #3:

Cirrus wishes to add a new OpenStack service, BDaaS (using Sahara). However it has no in-house expertise in supporting big data applications. Cirrus decides to subcontract the resale of OpenStack services with BDaaS to Acme, while preventing its existing customers like Widget from having access to BDaaS. It does this by creating two Virtual Regions. One Virtual Region includes BDaaS, and is made available to Acme; the other does not, and is provided to Widget.

User Story #4:

Cumulus suffers a major outage, and Widget decides that it needs to switch to another provider. It contracts with Acme to provide OpenStack services. Acme satisfies this contract using services from Nimbus (Story #2). Widget reconfigures its Service Aggregator, replacing the Cumulus configuration with one for Acme.

User Story #5:

The outage at Cumulus continues, and Cumulus decides that they need to add capacity from a third party so that they can continue to service their customers. Cumulus contracts with Cirrus to provide OpenStack services. Cirrus creates a Virtual Region, and makes it available to Cumulus. Cumulus configures this Virtual Region to use its existing IDM solution, so that its existing customers can log in using their regular credentials.

Use cases



A user (CSU) normally interacts with a cloud service provider (CSP) by logging in to the Horizon user interface, and accessing compute, network, and storage resources in an OpenStack region. The Keystone service handles identity management (IDM) functions, and provides the endpoints to interact with the services in the region. (In these diagram, dashed lines denote independent actors.)


A CSP may deploy more than one region, and share the Horizon and Keystone services across all regions. The user logs in to Horizon, selects a region to work in, and then obtains a Keystone token for the selected region. This provides single sign-on access. However, all regions are managed by a single CSP, and the configuration is static. (Note that this depends on a Horizon configuration in which the list of regions is obtained from Keystone. It assumes that all regions share the same Keystone. This is incompatible with the Horizon AVAILABLE_REGIONS mechanism.)


A user can work with with two different (independent) CSPs, but there is no shared context; she has to log in to each Horizon as a distinct session.


Let us suppose that Region B1 (operated by CSP B) has some attractive feature – geographical location, or connectivity – and CSP A would like to offer this service to its customers. Ideally, it would like region B1 to appear as just another region in Horizon, with the same SSO as for regions A1 and A2.

Setting this up is obviously non-trivial. Negotiating resource quotas. Coordinating project and user namespaces. Configuring IDM. And the OSS and BSS systems (out of scope for OpenStack) must be connected. Ideally, we want to be able to set things up dynamically: to discover a suitable region, and include it in the available resources.

In addition, CSP B is unlikely to be willing to provide unrestricted access (including full admin rights) for region B1. Instead B will want to provide access to a subset of resources, services, and administrative privileges. We call this package a virtual region


To manage the process of providing access from one CSP to the resources of another, we introduce two new services:

  • The Mercado Publisher (M-pub) sets up virtual regions and publishes them to other CSPs.
  • The Mercado Subscriber (M-sub) discovers virtual regions and makes them available to users.

The primary workflow proceeds as follows:

  • The administrator of CSP B accesses M-pub B to set up a virtual region. To do this, she uses a Python CLI client which talks to the REST API implemented by M-pub. M-pub B stores the virtual region configuration in a local database.
  • The administrator of CSP A accesses M-sub A to discover available virtual regions. She provides a list of Keystone URLs of CSPs which offer federation services. M-sub A invokes the REST API for M-pub in each of the designated regions and retrieves the description of each available virtual region. (TBD: authentication and authorization for this process; one approach is to transfer a Keystone token via an out-of-band business process.)
  • The administrator of CSP A instructs M-sub A to bind virtual region B1 from CSP B.
    • M-sub A talks to M-pub B to reserve the virtual region, and provides the URL of Keystone for CSP A.
    • M-pub B records the reservation, and configures the local Keystone so that all authorization for the virtual region will be handled by CSP A's Keystone (using Keystone-to-Keystone).
    • M-pub B creates a local user with admin rights over the domain which scopes the virtual region.
    • M-pub B constructs a Keystone token for this admin user enumerating the set of OpenStack services which are being provided, and returns this to M-sub A.
    • M-sub A configures its local Keystone to make the new virtual region available.
  • CSP A's Horizon discovers the new region from Keystone, and presents it to the user in the "Regions" drop-down.

With this mechanism, there’s no reason that a “CSP” has to actually provide any local resources at all. Here we have a service aggregator -- a reseller, or broker -- which sources all of its regions from other CSPs. To the user, it looks just like any other CSP – UI, SSO, APIs. All applications are required to run without any change.


The goal is that any OpenStack CSP can take part in this system by deploying the M-pub and/or M-sub services, and that the results will pass RefStack. It must be emphasized that there will be various OSS/BSS requirements to be satisfied for federated operations. This project simply addresses OpenStack.


The CSPs in the Mercador architecture are independent entities - typically distinct legal entities, possibly subject to different legal and compliance regimes. They also have business relationships with their existing customers which cannot be compromised.

Identity management


A user of CSP A expects that their personal information is held only by CSP A, and not shared with CSP B. She will sign in to CSP A, using whatever IdP CSP A supports. If CSP A chooses to support the user with resources hosted an a virtual region operated by CSP B, it is important that the information shared with CSP B is (a) clearly identified, (b) minimized, and (c) potentially anonymized. In particular, CSP B must not require access (or even know about) the IDM context in CSP A. We assume that CSP B will use Keystone-to-Keystone to handle this, and that M-sub will supply the necessary information to M-pub in order to set up Keystone A.

Domains and projects

The intention is a virtual region will behave the same - functionally and operationally - as a regular region. Today this isn't possible, because of domain and project name space limitations.

Remote administration

An administrator for CSP A needs to be able to manage the resources of her local users whether they are hosted in a "regular" region or a virtual region. The subscription process for a virtual region has to include steps to create the identity and roles necessary to provide this capability. One approach is to use nested domains. The vregion B1 is defined in terms of a domain, B1-D1. CSP B's M-pub can create a local (CSP B) user, and give this user admin rights in this region. It can then create a subordinate domain, B1-D1-Da, for which the IdP is set to CSP A's Keystone. This ensures that the CSP A admin can log in to CSP B and administer the vregion even if K2K has failed for some reason.

Mercador architecture


Interaction sequences

Defining a vregion

Listing available vregions

Establishing a vregion subscription

Creating a subdomain in a subscribed vregion

Creating a project in a subdomain of a subscribed vregion

User listing available regions (inc. vregions)

User selecting a project in a vregion

User creating a Nova instance in a project in a vregion

Refreshing a vregion subscription

Canceling a vregion subscription (Subscriber initiated)

Canceling a vregion subscription (Publisher initiated)

Data models

Publisher data model

Subscriber data model


Publisher API

Admin functions (CLI-Pub)
Subscriber functions (Sub-Pub)

Subscriber API

Admin functions (CLI-Sub)

Mercador context

Preconditions for vregion publication

OSS/BSS call-outs

Operational issues



Development (Blueprints, Roadmap, Design...)


Links & IRC

The IRC channel for Mercador discussions is #openstack-mercador



There are three Stackforge repositories, corresponding to the publisher, subscriber, and CLI components:





For various reasons, we've decided that the proof-of-concept implementation of Mercador is going to be based on forked copies of Keystone and the CLI clients openstackclient and keystoneclient The repository can be found here:


The following Trello page is used for project task tracking:



Meetings are held on IRC every Friday at 1700UTC on Freenode channel #openstack-meeting

The agenda and other information can be found here.


Geoff Arnold David Cheperdak Sean Perry Guang Yee




API design notes (PDF)