Jump to: navigation, search

Difference between revisions of "Cue/api"

< Cue
(System Context Diagram)
(Functional Tests)
 
(77 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Openstack Cue:  Messaging as a Service (MSGaaS) ==
+
== Cue API Design ==
 
=== Acronyms ===
 
=== Acronyms ===
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
! Acronym !! Definition
+
! '''Acronym''' !! '''Definition'''
 
|-
 
|-
 
| SSL || Secure Sockets Layer
 
| SSL || Secure Sockets Layer
|-
 
| ALS || Application Life-cycle Service
 
 
|-
 
|-
 
| REST || Representational State Transfer
 
| REST || Representational State Transfer
Line 18: Line 16:
 
|}
 
|}
  
=== Introduction ===
+
=== Requirements ===
  
Messaging systems enable software applications to communicate and scale.  Many of today's enterprise systems make use of messaging systems as the glue between the different components or applications.  These components depend on messaging to interact with each other to provide end-to-end functionality.  This Openstack stackforge project is intended to solve the problem in the provisioning, deployment and management of messaging systems in a seamless automated fashion for applications deployed on the Openstack cloud operating system.
+
API Requirements for Cue - Kilo timeframe.
  
There are several off-the-shelf messaging systems that implement messaging and queuing semantics such as RabbitMQ, Qpid and ActiveMQ.  These proven systems have wide acceptance, usage and testing.  The goal of this project is not to implement another messaging system for the Openstack platform, but to implement an automated messaging provisioning, deployment and management system.  This system will simplify the application development life-cycle to allow the developer to focus on their application instead of the back-end storage services.
+
* Keystone integration
 +
* CRUD on Cluster
 +
* Cluster Management – Scale up/down
 +
* Devstack integration
 +
* Gate Tests
  
=== Goals ===
+
=== System Architecture Diagram ===
  
The focus of the project is to provide high-availability, auto-healing and tenant level isolation to the end userThe initial goal for version 1.0 is to use RabbitMQ as the supported messaging systemThe overall goals for this effort are as follows.
+
[[File:Cue-arch.png]]
 
+
{| class="wikitable"
# Messaging queuing provisioning service for OpenStack built with community support
+
|-
# Messaging queuing provisioning service for HP Helion Development Platform
+
! '''Component''' !! '''Description'''
 +
|-
 +
| User || Direct customer of Cue.
 +
|-
 +
| Horizon || Cue functionality will be added to Horizon, which will provide a web-based portal for Cue control.
 +
|-
 +
| CLI || Command Line Interface to Cue, provides user access to provisioning and deploying messaging clusters.
 +
|-
 +
| REST_API || Provides user access to provisioning and deploying messaging clusters through REST interface.  This is a light-weight interface, provisioning and configuration of clusters/nodes is delegated to the TaskWorker process.
 +
|-
 +
| TaskScheduler || Used to synchronize work tasks between the REST_API and TaskWorker processes.
 +
|-
 +
| TaskWorker || Carries out work associated with all provisioning, configuration and management of RabbitMQ clusters and nodesMakes use of heat for initial provisioning/deployment.
 +
|-
 +
| DB || Database to store information on clusters and nodes. Example, in cluster creation, when request is initially received through REST_API, this DB is updated accordingly and the work is delegated to TaskWorker.  TaskWorker then updates this DB as provisioning and configuration takes placeSubsequent calls to check on status of cluster creation, will return updated information from this DB.
 +
|-
 +
| Heat || Used cloud instance orchestration in deploying RabbitMQ cluster images.
 +
|}
  
=== Requirements ===
+
=== REST API ===
  
The requirements for the MSGaaS Queue v1.0 project are listed in priority order below.
+
General requirement, REST API must respond within 500ms.
 +
==== List Clusters ====
  
    *  Community Building (Rackspace involvement)
+
'''''GET /v1/clusters'''''
    *  Control plane
 
    *  Rabbit Management Console
 
    *  Keystone integration
 
    *  Command-Line – Create/Delete Cluster
 
    *  Horizon Integration - Create/Delete Cluster
 
    *  Development Platform Installer Integration
 
    *  Queue Control Plane Patching
 
    *  RabbitMQ Instance Patching
 
    *  Helion – Control Plane SSL
 
    *  Cluster Management – Scale up/down
 
    *  Devstack integration
 
    *  Gate Tests
 
    *  Queue Control Plane Upgrade
 
    *  RabbitMQ Instance Upgrade
 
  
Post v1.0 requirements.
+
This operation synchronously returns all clusters provisioned within the associated project id.
  
    *    Highly Available RabbitMQCluster (client must handle fail-over scenario)
+
'''Request Parameters'''
    *    ALS integration
+
N/A
    *    Helion – RabbitMQ SSL
 
    *    Availability Zone support
 
    *    Cluster Telemetry/Monitoring
 
    *    Logging
 
    *    Health checks
 
    *    Self-healing
 
    *    RabbitMQ Management
 
    *    Create RabbitMQ user
 
    *    Create Queue
 
    *    Backup configuration
 
    *    Update RabbitQM version
 
    *    Patch instance
 
    *    Plug-in management
 
    *    RabbitMQ Configuration management
 
    *    Performance/Scale Benchmarks
 
    *    Openstack CI integration
 
    *    Tempest Tests
 
    *    Helion monitoring
 
    *    Central logging
 
    *    Volume support
 
    *    Multiple Messaging Systems (beyond RabbitMQ)
 
  
=== System Context Diagram ===
+
{| class="wikitable"
 +
|-
 +
! Response !! Code(s)
 +
|-
 +
| Normal || ok (200)
 +
|-
 +
| Error || unauthorized (401)
 +
|}
  
[[File:https://wiki.openstack.org/w/images/1/13/Scd.png|framed|left|System Context Diagram]]
+
'''Response Parameters'''
  
Component Description
+
{| class="wikitable"
User Direct customer of MSGaaS.
+
|-
Horizon MSGaaS functionality will be added to Horizon, which will provide a web-based portal for MSGaaS control.
+
! Parameter !! Type !! Description
 
+
|-
CLI
+
| cluster(list) || string || List of clusters, detailing respective cluster id, name, status, created time stamp and updated time stamp.
Command Line Interface to MSGaaS, provides user access to provisioning and deploying messaging clusters.
+
|}
  
REST_API
 
 
 
  
Provides user access to provisioning and deploying messaging clusters through REST interface.  This is a light-weight interface, provisioning and configuration of clusters/nodes is delegated to the TaskWorker process.
+
'''JSON Request'''
RabbitMQ Message broker used as the communication pathways between the REST_API and TaskWorker.
 
TaskWorker Carries out work associated with all provisioning, configuration and management of RabbitMQ clusters and nodes.  Makes use of heat for initial provisioning/deployment.
 
DB Database to store information on clusters and nodes.  Example, in cluster creation, when request is initially received through REST_API, this DB is updated accordingly and the work is delegated to TaskWorker.  TaskWorker then updates this DB as provisioning and configuration takes place.  Subsequent calls to check on status of cluster creation, will return updated information from this DB.
 
Heat Used cloud instance orchestration in deploying RabbitMQ cluster images.
 
 
 
=== REST API ===
 
 
 
General requirement, REST API must respond within 500ms.
 
List Clusters
 
 
 
GET /v1/{tenant_id}/clusters
 
 
 
This operation synchronously returns all clusters provisioned within the provided tenant.
 
 
 
Request Parameters
 
Parameter Type Description
 
tenant_id (URI) UUID The tenant ID in a multi-tenancy cloud.
 
 
 
Response Codes
 
Response Code(s)
 
Normal ok (200)
 
Error unauthorized (401)
 
 
 
Response Parameters
 
Parameter Type Description
 
cluster(list) string List of clusters, detailing respective cluster id, name, status, created time stamp and updated time stamp.
 
 
 
JSON Request
 
  
 
N/A
 
N/A
  
JSON Response
+
'''JSON Response'''
  
 
{
 
{
     "clusters": [
+
     "clusters": <nowiki>[</nowiki>
 
         {
 
         {
 
             "cluster_id": "b51948c9-1ac5-4c28-a580-6f7c500d82f8",
 
             "cluster_id": "b51948c9-1ac5-4c28-a580-6f7c500d82f8",
 +
            "network_id": "d32019d3-bc6e-4319-9c1d-6722fc136a22",
 
             "name": "Message Cluster 1",
 
             "name": "Message Cluster 1",
 
             "status": "ACTIVE",
 
             "status": "ACTIVE",
Line 136: Line 100:
 
         {
 
         {
 
             "cluster_id": "13c456c9-bbfc-4c31-b26d-3ae5c3cd7a77",
 
             "cluster_id": "13c456c9-bbfc-4c31-b26d-3ae5c3cd7a77",
 +
            "network_id": "d32019d3-bc6e-4319-9c1d-6722fc136a22",
 
             "name": "Message Cluster 2",
 
             "name": "Message Cluster 2",
 
             "status": "ACTIVE",
 
             "status": "ACTIVE",
Line 144: Line 109:
 
}
 
}
  
Create Cluster
+
==== Create Cluster ====
  
POST /v1/{tenant_id}/clusters
+
'''''POST /v1/clusters'''''
  
This operation asynchronously creates a new cluster of Nova instances provisioned with the required message brokers in a central tenant.  
+
This operation asynchronously creates a new cluster of Nova instances provisioned with the required message brokers in a central project id.  
  
Request Parameters
+
'''Request Parameters'''
Parameter Type Description
 
tenant_id (URI) UUID The tenant ID in a multi-tenancy cloud.
 
nic UUID Network Identification for a Neutron network where cluster will be created in.
 
name string Name of cluster.
 
volume_size int Optional parameter to indicate size of volume for node instance.  If volumes are supported, then this parameter will be used.  If ephmeral disk are not supported, volume support will be required.
 
flavor (list) string
 
  
List of node flavors, which specify VM type in terms of CPU/memory/disk resources (e.g. small, medium, large). Size of list denotes the number of nodes in cluster.
+
{| class="wikitable"
 +
|-
 +
! Parameter !! Type !! Description
 +
|-
 +
| nic || UUID || Network Identification for a Neutron network where cluster will be created in.
 +
|-
 +
| name || string || Name of cluster.
 +
|-
 +
| volume_size || int || Optional parameter to indicate size of volume for node instance.  If volumes are supported, then this parameter will be used.  If ephmeral disk are not supported, volume support will be required.
 +
|-
 +
| flavor (list) || string || List of node flavors, which specify VM type in terms of CPU/memory/disk resources. See http://docs.openstack.org/openstack-ops/content/flavors.html for more information.
 +
|}
  
small:  1 GHz dual-core CPU; 512 MB memory; 250 GB disk
+
'''Response Codes'''
  
medium: 2.8 GHz dual-core CPU; 4 GB memory; 1 TB disk
+
{| class="wikitable"
 +
|-
 +
! Response !! Code(s)
 +
|-
 +
| Normal || 202 (accepted)
 +
|-
 +
| Error || badRequest (400), unauthorized (401), itemNotFound (404)
 +
|}
  
large: 3.6 GHz quad-core CPU; 32 GB memory; 5 TB disk
+
'''Response Parameters'''
  
Response Codes
+
{| class="wikitable"
Response Code(s)
+
|-
Normal 202 (accepted)
+
! Parameter !! Type !! Description
Error badRequest (400), unauthorized (401), itemNotFound (404)
+
|-
 +
| cluster_id || UUID || ID of cluster to be created.
 +
|-
 +
| name || string || Name of cluster (same as provided name in request parameters).
 +
|-
 +
| status || string || Current status of cluster.
  
Response Parameters
+
BUILDING: Cluster is in progress of being provisioned.
Parameter Type Description
 
cluster_id UUID ID of cluster to be created.
 
name string Name of cluster (same as provided name in request parameters).
 
status string
 
 
 
Current status of cluster.
 
  
BUILDING: Cluster is in progress of being provisioned.
+
UPDATING: Cluster in process of being updated.
  
 
ACTIVE:  Cluster is running.
 
ACTIVE:  Cluster is running.
  
 
ERROR: Provisioning error(s) encountered.
 
ERROR: Provisioning error(s) encountered.
 +
 +
DELETING: Cluster is in process of being deleted.
  
 
DELETED:  Cluster has been deleted.
 
DELETED:  Cluster has been deleted.
 +
|}
  
JSON Request
+
'''JSON Request'''
  
 
{
 
{
     "nic": "d32019d3-bc6e-4319-9c1d-6722fc136a22",
+
     "network_id": "d32019d3-bc6e-4319-9c1d-6722fc136a22",
     "name": "Message Cluster 1",
+
     "name": "MessageCluster1",
     "volume_size": 100
+
     "volume_size": "100",
 
     "nodes": [
 
     "nodes": [
 
         {
 
         {
Line 206: Line 185:
 
}
 
}
  
JSON Response
+
'''JSON Response'''
 +
 
  
 
{
 
{
     "cluster_id": "dd745f4a-9333-417e-bb89-9c989c84c068",
+
     "cluster": {
    "name": "Message Cluster 1",
+
        "cluster_id": "dd745f4a-9333-417e-bb89-9c989c84c068",
    "status": "BUILDING",
+
        "network_id": "d32019d3-bc6e-4319-9c1d-6722fc136a22",
    "created": "2014-11-11T01:02:03Z",
+
        "name": "Message Cluster 1",
    "updated": "2014-11-11T01:02:03Z",
+
        "status": "BUILDING",
    "nodes": [
+
        "created": "2014-11-11T01:02:03Z",
        {
+
        "updated": "2014-11-11T01:02:03Z",
            "node_id": "616fb98f-46ca-475e-917e-2563e5a8cd19",
+
        "nodes": [
             "status": "BUILDING",
+
            {
             "flavor": "large",
+
                "node_id": "616fb98f-46ca-475e-917e-2563e5a8cd19",
         },
+
                "status": "BUILDING",
        {
+
                "flavor": "large"
 +
            },
 +
             {
 +
                "node_id": "e90c9d13-c4b8-4a08-992a-dad6109b8ac2",
 +
                "status": "BUILDING",
 +
                "flavor": "large"
 +
            },
 +
             {
 +
                "node_id": "372f8f47-6818-4d83-aa42-8744c0e689b8",
 +
                "status": "BUILDING",
 +
                "flavor": "large"
 +
            }
 +
         ]
 +
    }
 +
}
  
            "node_id": "e90c9d13-c4b8-4a08-992a-dad6109b8ac2",
+
==== Show Cluster ====
            "status": "BUILDING",
 
            "flavor": "large",
 
        },
 
        {
 
  
            "node_id": "372f8f47-6818-4d83-aa42-8744c0e689b8",
+
'''''GET /v1/clusters/{cluster_id}'''''
            "status": "BUILDING",           
 
            "flavor": "large",
 
        }
 
}
 
  
Show Cluster
+
This operation synchronously returns the status and information on the specified cluster within the provided project id.
  
GET /v1/{tenant_id}/clusters/{cluster_id}
+
'''Request Parameters'''
  
This operation synchronously returns the status and information on the specified cluster within the provided tenant.
+
{| class="wikitable"
 +
|-
 +
! Parameter !! Type !! Description
 +
|-
 +
| cluster_id (URI) || UUID || Cluster ID.  This value is returned when a new cluster is created.
 +
|}
  
Request Parameters
+
'''Response Codes'''
Parameter Type Description
 
tenant_id (URI) UUID The tenant ID in a multi-tenancy cloud.
 
cluster_id (URI) UUID Cluster ID.  This value is returned when a new cluster is created.
 
  
Response Codes
+
{| class="wikitable"
Response Code(s)
+
|-
Normal ok (200)
+
! Response !! Code(s)
Error unauthorized (401), itemNotFound (404)
+
|-
 +
| Normal || ok (200)
 +
|-
 +
| Error || unauthorized (401), itemNotFound (404)
 +
|}
  
Response Parameters
+
'''Response Parameters'''
Parameter Type Description
 
cluster_id UUID ID of requested cluster.
 
name string Name of cluster.
 
status string
 
  
Current status of cluster.
+
{| class="wikitable"
 +
|-
 +
! Parameter !! Type !! Description
 +
|-
 +
| cluster_id || UUID || ID of requested cluster.
 +
|-
 +
| name || string || Name of cluster.
 +
|-
 +
| status || string || Current status of cluster.
  
 
BUILDING: Cluster is in progress of being provisioned.
 
BUILDING: Cluster is in progress of being provisioned.
 +
 +
UPDATING:  Cluster in process of being updated.
  
 
ACTIVE:  Cluster is running.
 
ACTIVE:  Cluster is running.
  
 
ERROR: Provisioning error(s) encountered.
 
ERROR: Provisioning error(s) encountered.
 +
 +
DELETING: Cluster is in process of being deleted.
  
 
DELETED:  Cluster has been deleted.
 
DELETED:  Cluster has been deleted.
created string Created time stamp in format: yyyy-mm-ddThh:mm:ssZ
+
|-
updated string Last updated time stamp in format: yyyy-mm-ddThh:mm:ssZ
+
| created || string || Created time stamp in format: yyyy-mm-ddThh:mm:ssZ
nodes(list) node List of nodes, which includes node id, instance state, and flavor.
+
|-
 +
| updated || string || Last updated time stamp in format: yyyy-mm-ddThh:mm:ssZ
 +
|-
 +
| nodes(list) || node || List of nodes, which includes node id, instance state, and flavor.
 +
|}
  
JSON Request
+
'''JSON Request'''
  
 
N/A
 
N/A
  
JSON Response
+
'''JSON Response'''
  
 
{
 
{
     "cluster_id": "dd745f4a-9333-417e-bb89-9c989c84c068",
+
     "cluster": {
    "name": "Message Cluster 1",
+
        "cluster_id": "dd745f4a-9333-417e-bb89-9c989c84c068",
    "status": "ACTIVE",
+
        "network_id": "d32019d3-bc6e-4319-9c1d-6722fc136a22",
    "created": "2014-11-11T01:02:03Z",
+
        "name": "Message Cluster 1",
    "updated": "2014-11-11T01:02:03Z",
+
        "status": "ACTIVE",
    "nodes": [
+
        "created": "2014-11-11T01:02:03Z",
        {
+
        "updated": "2014-11-11T01:02:03Z",
            "node_id": "616fb98f-46ca-475e-917e-2563e5a8cd19",
+
        "nodes": [
            "status": "ACTIVE",
+
            {
            "flavor": "large",
+
                "node_id": "616fb98f-46ca-475e-917e-2563e5a8cd19",
            "endpoints": [
+
                "status": "ACTIVE",
                {
+
                "flavor": "large",
                    "type": "AMQP",
+
                "endpoints": [
                    "value": "amqp://10.20.30.40:10000"
+
                    {
                },
+
                        "type": "AMQP",
                {
+
                        "value": "amqp://10.20.30.40:10000"
                    "type": "console",
+
                    },
                    "value": "http://10.20.30.40:5672"
+
                    {
                }
+
                        "type": "console",
            ],
+
                        "value": "http://10.20.30.40:5672"
            "created": "2014-11-11T01:06:03Z",
+
                    }
            "updated": "2014-11-11T01:06:03Z"
+
                ],
        },
+
                "created": "2014-11-11T01:06:03Z",
        {
+
                "updated": "2014-11-11T01:06:03Z"
 
+
            },
            "node_id": "e90c9d13-c4b8-4a08-992a-dad6109b8ac2",
+
            {
            "status": "ACTIVE",
+
                "node_id": "e90c9d13-c4b8-4a08-992a-dad6109b8ac2",
            "flavor": "large",
+
                "status": "ACTIVE",
            "endpoints": [
+
                "flavor": "large",
                {
+
                "endpoints": [
                    "type": "AMQP",
+
                    {
                    "value": "amqp://10.20.30.41:10000"
+
                        "type": "AMQP",
                },
+
                        "value": "amqp://10.20.30.41:10000"
                {
+
                    },
                    "type": "console",
+
                    {
                    "value": "http://10.20.30.41:5672"
+
                        "type": "console",
                }
+
                        "value": "http://10.20.30.41:5672"
            ],
+
                    }
            "created": "2014-11-11T01:08:03Z",
+
                ],
            "updated": "2014-11-11T01:08:03Z"
+
                "created": "2014-11-11T01:08:03Z",
        },
+
                "updated": "2014-11-11T01:08:03Z"
        {
+
            },
 
+
            {
            "node_id": "372f8f47-6818-4d83-aa42-8744c0e689b8",
+
                "node_id": "372f8f47-6818-4d83-aa42-8744c0e689b8",
            "status": "ACTIVE",
+
                "status": "ACTIVE",
            "flavor": "large",
+
                "flavor": "large",
            "endpoints": [
+
                "endpoints": [
                {
+
                    {
                    "type": "AMQP",
+
                        "type": "AMQP",
                    "value": "amqp://10.20.30.42:10000"
+
                        "value": "amqp://10.20.30.42:10000"
                },
+
                    },
                {
+
                    {
                    "type": "console",
+
                        "type": "console",
                    "value": "http://10.20.30.42:5672"
+
                        "value": "http://10.20.30.42:5672"
                }
+
                    }
            ],
+
                ],
            "created": "2014-11-11T01:12:03Z",
+
                "created": "2014-11-11T01:12:03Z",
            "updated": "2014-11-11T01:12:03Z"
+
                "updated": "2014-11-11T01:12:03Z"
        }
+
            }
    ]
+
        ]
 
+
    }
 
 
 
}
 
}
  
Delete a Cluster
+
==== Delete Cluster ====
  
DELETE /v1/{tenant_id}/clusters/{cluster_id}
+
'''''DELETE /v1/clusters/{cluster_id}'''''''
  
This operation will asynchronously delete the indicated cluster within the provided tenant.
+
This operation will asynchronously delete the indicated cluster within the provided project id.
  
Request Parameters
+
'''Request Parameters'''
Parameter Type Description
 
tenant_id (URI) UUID The tenant ID in a multi-tenancy cloud.
 
  
Response Codes
+
{| class="wikitable"
Response Code(s)
+
|-
Normal accepted (202)
+
! Parameter !! Type !! Description
Error unauthorized (401), itemNotFound (404)
+
|-
 +
| cluster_id (URI) || UUID || Cluster ID.  This value is returned when a new cluster is created.
 +
|}
  
Response Parameters
+
'''Response Codes'''
  
N/A
+
{| class="wikitable"
 +
|-
 +
! Response !! Code(s)
 +
|-
 +
| Normal || accepted (202)
 +
|-
 +
| Error || unauthorized (401), itemNotFound (404)
 +
|}
  
JSON Request
+
'''Response Parameters'''
  
 
N/A
 
N/A
  
JSON Response
+
'''JSON Request'''
  
 
N/A
 
N/A
List Nodes
 
  
GET /v1/{tenant_id}/clusters/{cluster_id}/nodes
+
'''JSON Response'''
 
 
This operation synchronously returns the status and information on all nodes within the specified cluster/tenant.
 
 
 
Request Parameters
 
Parameter Type Description
 
tenant_id (URI) UUID The tenant ID in a multi-tenancy cloud.
 
cluster_id (URI) UUID Cluster ID.
 
 
 
Response Codes
 
Response Code(s)
 
Normal ok (200)
 
Error unauthorized (401), itemNotFound (404)
 
 
 
Response Parameters
 
Parameter Type Description
 
cluster_id UUID ID of requested cluster.
 
nodes(list) node List of nodes, which includes node id, instance state, flavor, endpoint(s), created time stamp and updated time stamp.
 
 
 
JSON Request
 
  
 
N/A
 
N/A
  
JSON Response
+
===Testing===
 
 
{
 
    "cluster_id": "dd745f4a-9333-417e-bb89-9c989c84c068",
 
    "name": "Message Cluster 1",
 
    "status": "ACTIVE",
 
    "created": "2014-11-11T01:02:03Z",
 
    "updated": "2014-11-11T01:02:03Z",
 
    "nodes": [
 
        {
 
            "node_id": "616fb98f-46ca-475e-917e-2563e5a8cd19",
 
            "status": "ACTIVE",
 
            "flavor": "large",
 
            "endpoints": [
 
                {
 
                    "type": "AMQP",
 
                    "value": "amqp://10.20.30.40:10000"
 
                },
 
                {
 
                    "type": "console",
 
                    "value": "http://10.20.30.40:5672"
 
                }
 
            ],
 
            "created": "2014-11-11T01:06:03Z",
 
            "updated": "2014-11-11T01:06:03Z"
 
        },
 
        {
 
  
            "node_id": "e90c9d13-c4b8-4a08-992a-dad6109b8ac2",
+
Cue API testing will verify the expected functionality of the Cue user interface with both positive/negative tests. The overall scope will cover testing from the HTTP REST request to required database interactions and work-flow task submission for RPC workers.
            "status": "ACTIVE",
 
            "flavor": "large",
 
            "endpoints": [
 
                {
 
                    "type": "AMQP",
 
                    "value": "amqp://10.20.30.41:10000"
 
                },
 
                {
 
                    "type": "console",
 
                    "value": "http://10.20.30.41:5672"
 
                }
 
            ],
 
            "created": "2014-11-11T01:08:03Z",
 
            "updated": "2014-11-11T01:08:03Z"
 
        },
 
        {
 
  
            "node_id": "372f8f47-6818-4d83-aa42-8744c0e689b8",
+
====Unit Tests====
            "status": "ACTIVE",
 
            "flavor": "large",
 
            "endpoints": [
 
                {
 
                    "type": "AMQP",
 
                    "value": "amqp://10.20.30.42:10000"
 
                },
 
                {
 
                    "type": "console",
 
                    "value": "http://10.20.30.42:5672"
 
                }
 
            ],
 
            "created": "2014-11-11T01:12:03Z",
 
            "updated": "2014-11-11T01:12:03Z"
 
        }
 
    ]
 
}
 
  
Add Node
+
Unit tests will verify the resulting function calls for each REST-ful URI and action(s).  The Python Mock library will be used to replace external system dependencies with placeholder objects.
  
POST /v1/{tenant_id}/clusters/{cluster_id}/nodes
+
{| class="wikitable"
 +
|-
 +
! Function !! Tests !! Input Data !! Expected Result(s)
 +
|-
 +
| List Clusters ||
 +
# Create 'n' cluster(s), then call list clusters and verify expected return object
 +
# Call list clusters when no clusters exist within the project id and verify expected return object
 +
||
 +
* n = # of clusters to create
 +
||
 +
# List of 'n' clusters is returned with appropriate fields, HTTP Ok (200)
 +
# Empty list of clusters is returned, HTTP Ok (200)
 +
|-
 +
| Create Cluster ||
 +
# Create standard cluster 'c' and verify expected return object
 +
# Create cluster with invalid flavor and verify expected return value
 +
# Create cluster with invalid volume size and verify expected return value
 +
||
 +
* n = cluster size (number of nodes)
 +
* m = cluster name
 +
* f = flavor
 +
* s = volume size
 +
* nic = Network UUID
 +
||
 +
# Cluster ID, name and status for newly created cluster is returned, HTTP Accepted (202)
 +
# HTTP Bad request is returned (400)
 +
# HTTP Bad request is returned (400)
 +
|-
 +
| Show Cluster ||
 +
# Create cluster, then call get cluster and verify expected return object
 +
# Get cluster with invalid cluster_id
 +
||
 +
* cluster_id = cluster ID
 +
||
 +
# Cluster ID, name, status, created/updated date stamp and list of nodes is returned, HTTP Ok (200).
 +
# HTTP Bad request is returned (400)
 +
|-
 +
| Delete Cluster ||
 +
# Create a cluster, then call delete cluster and verify expected return value
 +
# Delete cluster with invalid cluster_id
 +
||
 +
* cluster_id = cluster ID
 +
||
 +
# HTTP accepted is returned (202)
 +
# HTTP Bad request is returned (400)
 +
|}
  
This operation asynchronously creates a new node within the indicated cluster.
+
====Functional Tests====
  
Request Parameters
+
The functional tests will verify the HTTP REST URI request lifecycle from controller routing to HTTP responseThese tests will make use of Pecan's testing utility; pecan.testing module.
Parameter Type Description
 
tenant_id (URI) UUID The tenant ID in a multi-tenancy cloud.
 
cluster_id (URI) UUID Cluster ID.
 
volume_size int Optional parameter to indicate size of volume for node instanceIf volumes are supported, then this parameter will be used. If ephmeral disk are not supported, volume support will be required.
 
flavor string
 
  
Node flavors, which specify VM type in terms of CPU/memory/disk resources (e.g. small, medium, large).
+
{| class="wikitable"
 
+
|-
small:  1 GHz dual-core CPU; 512 MB memory; 250 GB disk
+
! Operation !! URI !! Tests !! Input Data !! Expected Data (JSON)
 
+
|-
medium: 2.8 GHz dual-core CPU; 4 GB memory; 1 TB disk
+
| GET || /v1/clusters ||
 
+
# Call when clusters exist
large: 3.6 GHz quad-core CPU; 32 GB memory; 5 TB disk
+
# Call when no clusters exist
 
+
||
*Note:  If messaging system being provisioned requires the same flavor, this parameter will be ignored if cluster already has provisioned node(s)
+
* n/a
 
+
||
Response Codes
+
# List of 'n' clusters is returned with appropriate fields, HTTP Ok (200)
Response Code(s)
+
# Empty list of clusters is returned, HTTP Ok (200)
Normal 202 (accepted)
+
|-
Error badRequest (400), unauthorized (401), itemNotFound (404)
+
| POST || /v1/clusters ||
 
+
# Create standard cluster 'c' and verify expected return object
Response Parameters
+
# Create cluster with invalid flavor and verify expected return value
Parameter Type Description
+
# Create cluster with invalid volume size and verify expected return value
cluster_id UUID ID of cluster where node will be created in.
+
||
node_id UUID ID of newly created node.
+
* n = cluster size (number of nodes)
status string
+
* m = cluster name
 
+
* f = flavor
Current state of node.
+
* s = volume size
 
+
* nic = Network UUID
BUILDING: Cluster is in progress of being provisioned.
+
||
 
+
# Cluster ID, name and status for newly created cluster is returned, HTTP Accepted (202)
ACTIVE:  Cluster is running.
+
# HTTP Bad request is returned (400)
 
+
# HTTP Bad request is returned (400)
ERROR: Provisioning error(s) encountered.
+
|-
 
+
| GET || /v1/clusters/{cluster_id} ||
DELETED:  Cluster has been deleted.
+
# Create cluster, then call get cluster and verify expected return object
 
+
# Get cluster with invalid cluster_id
JSON Request
+
||
 
+
* cluster_id = cluster
{
+
||
    "flavor": "medium"
+
# Cluster ID, name, status, created/updated date stamp and list of nodes is returned, HTTP Ok (200).
}
+
# HTTP Bad request is returned (400)
 
+
|-
JSON Response
+
| DELETE || /v1/clusters/{cluster_id} ||
 
+
# Create a cluster, then call delete cluster and verify expected return value
{
+
# Delete cluster with invalid cluster_id
    "cluster_id": "dd745f4a-9333-417e-bb89-9c989c84c068",
+
||
    "node_id": "8408211b-b720-4ddc-bc62-f1d474ed2658",
+
* cluster_id = cluster ID
    "status": "BUILDING"
+
||
}
+
# HTTP accepted is returned (202)
 
+
# HTTP Bad request is returned (400)
Show Node
+
|}
 
 
GET /v1/{tenant_id}/clusters/{cluster_id}/nodes/{node_id}
 
 
 
This operation synchronously displays details associated with the specified node.
 
 
 
Request Parameters
 
Parameter Type Description
 
tenant_id (URI) UUID The tenant ID in a multi-tenancy cloud.
 
cluster_id (URI) UUID Cluster ID.
 
node_id (URI) UUID Node ID.
 
 
 
Response Codes
 
Response Code(s)
 
Normal ok (200)
 
Error badRequest (400), unauthorized (401), itemNotFound (404)
 
 
 
Response Parameters
 
Parameter Type Description
 
cluster_id UUID ID of cluster where node will be created in.
 
node_id UUID ID of newly created node.
 
status string Current state of node (i.e. BUILDING, ACTIVE, ERROR ...).
 
flavor string Node flavor (i.e. small, medium or large)
 
endpoint (list) string List of endpoints associated with accessing this node.
 
created time stamp string Time stamp when build processes started on this node.
 
updated time stamp string Time stamp when latest change was started on this node.
 
 
 
JSON Request
 
  
N/A
+
====Integration Tests====
 
 
JSON Response
 
 
 
{
 
    "cluster_id": "dd745f4a-9333-417e-bb89-9c989c84c068",
 
    "node": {
 
        "node_id": "616fb98f-46ca-475e-917e-2563e5a8cd19",
 
        "status": "ACTIVE",
 
        "flavor": "medium",
 
        "endpoints": [
 
            {
 
                "type": "AMQP",
 
                "value": "amqp://10.20.30.40:10000"
 
            },
 
            {
 
                "type": "console",
 
                "value": "http://10.20.30.40:5672"
 
            }
 
        ],
 
        "created": "2014-11-11T01:06:03Z",
 
        "updated": "2014-11-11T01:06:03Z"
 
    }
 
}
 
  
Delete Node
+
The integration tests will cover API functionality from HTTP request to database access and task submission for RPC workersThe Pecan test utility will be used to route test request URI to appropriate controllers, then database record(s) will be verified for applicable change. Finally the creation of task objections will be verified to ensure valid task flows.
 
 
DELETE /v1/{tenant_id}/clusters/{cluster_id}/nodes/{node_id}
 
 
 
This operation asynchronously deletes the indicated nodeNote, this operation will depend on the provisioned messaging system, e.g. even number of nodes required, etc.
 
 
 
Request Parameters
 
Parameter Type Description
 
tenant_id (URI) UUID The tenant ID in a multi-tenancy cloud.
 
cluster_id (URI) UUID Cluster ID.
 
node_id (URI) UUID Node ID.
 
 
 
Response Codes
 
Response Code(s)
 
Normal accepted (202)
 
Error badRequest (400), unauthorized (401), itemNotFound (404)
 
 
 
Response Parameters
 
 
 
N/A
 
 
 
JSON Request
 
 
 
N/A
 
 
 
JSON Response
 
 
 
N/A
 

Latest revision as of 16:51, 11 December 2015

Cue API Design

Acronyms

Acronym Definition
SSL Secure Sockets Layer
REST Representational State Transfer
URI Uniform Resource Identifier
UUID Universally Unique Identifier
AMQP Advanced Messaging Queuing Protocol

Requirements

API Requirements for Cue - Kilo timeframe.

  • Keystone integration
  • CRUD on Cluster
  • Cluster Management – Scale up/down
  • Devstack integration
  • Gate Tests

System Architecture Diagram

Cue-arch.png

Component Description
User Direct customer of Cue.
Horizon Cue functionality will be added to Horizon, which will provide a web-based portal for Cue control.
CLI Command Line Interface to Cue, provides user access to provisioning and deploying messaging clusters.
REST_API Provides user access to provisioning and deploying messaging clusters through REST interface. This is a light-weight interface, provisioning and configuration of clusters/nodes is delegated to the TaskWorker process.
TaskScheduler Used to synchronize work tasks between the REST_API and TaskWorker processes.
TaskWorker Carries out work associated with all provisioning, configuration and management of RabbitMQ clusters and nodes. Makes use of heat for initial provisioning/deployment.
DB Database to store information on clusters and nodes. Example, in cluster creation, when request is initially received through REST_API, this DB is updated accordingly and the work is delegated to TaskWorker. TaskWorker then updates this DB as provisioning and configuration takes place. Subsequent calls to check on status of cluster creation, will return updated information from this DB.
Heat Used cloud instance orchestration in deploying RabbitMQ cluster images.

REST API

General requirement, REST API must respond within 500ms.

List Clusters

GET /v1/clusters

This operation synchronously returns all clusters provisioned within the associated project id.

Request Parameters N/A

Response Code(s)
Normal ok (200)
Error unauthorized (401)

Response Parameters

Parameter Type Description
cluster(list) string List of clusters, detailing respective cluster id, name, status, created time stamp and updated time stamp.


JSON Request

N/A

JSON Response

{

   "clusters": [
       {
           "cluster_id": "b51948c9-1ac5-4c28-a580-6f7c500d82f8",
           "network_id": "d32019d3-bc6e-4319-9c1d-6722fc136a22",
           "name": "Message Cluster 1",
           "status": "ACTIVE",
           "created": "2014-11-11T01:02:03Z",
           "updated": "2014-11-11T01:02:03Z"
       },
       {
           "cluster_id": "13c456c9-bbfc-4c31-b26d-3ae5c3cd7a77",
           "network_id": "d32019d3-bc6e-4319-9c1d-6722fc136a22",
           "name": "Message Cluster 2",
           "status": "ACTIVE",
           "created": "2014-11-12T13:23:54Z",
           "updated": "2014-11-13T19:55:01Z"
       }
   ]

}

Create Cluster

POST /v1/clusters

This operation asynchronously creates a new cluster of Nova instances provisioned with the required message brokers in a central project id.

Request Parameters

Parameter Type Description
nic UUID Network Identification for a Neutron network where cluster will be created in.
name string Name of cluster.
volume_size int Optional parameter to indicate size of volume for node instance. If volumes are supported, then this parameter will be used. If ephmeral disk are not supported, volume support will be required.
flavor (list) string List of node flavors, which specify VM type in terms of CPU/memory/disk resources. See http://docs.openstack.org/openstack-ops/content/flavors.html for more information.

Response Codes

Response Code(s)
Normal 202 (accepted)
Error badRequest (400), unauthorized (401), itemNotFound (404)

Response Parameters

Parameter Type Description
cluster_id UUID ID of cluster to be created.
name string Name of cluster (same as provided name in request parameters).
status string Current status of cluster.

BUILDING: Cluster is in progress of being provisioned.

UPDATING: Cluster in process of being updated.

ACTIVE: Cluster is running.

ERROR: Provisioning error(s) encountered.

DELETING: Cluster is in process of being deleted.

DELETED: Cluster has been deleted.

JSON Request

{

   "network_id": "d32019d3-bc6e-4319-9c1d-6722fc136a22",
   "name": "MessageCluster1",
   "volume_size": "100",
   "nodes": [
       {
           "flavor": "large"
       },
       {
           "flavor": "large"
       },
       {
           "flavor": "large"
       }
   ]

}

JSON Response


{

   "cluster": {
       "cluster_id": "dd745f4a-9333-417e-bb89-9c989c84c068",
       "network_id": "d32019d3-bc6e-4319-9c1d-6722fc136a22",
       "name": "Message Cluster 1",
       "status": "BUILDING",
       "created": "2014-11-11T01:02:03Z",
       "updated": "2014-11-11T01:02:03Z",
       "nodes": [
           {
               "node_id": "616fb98f-46ca-475e-917e-2563e5a8cd19",
               "status": "BUILDING",
               "flavor": "large"
           },
           {
               "node_id": "e90c9d13-c4b8-4a08-992a-dad6109b8ac2",
               "status": "BUILDING",
               "flavor": "large"
           },
           {
               "node_id": "372f8f47-6818-4d83-aa42-8744c0e689b8",
               "status": "BUILDING",
               "flavor": "large"
           }
       ]
   }

}

Show Cluster

GET /v1/clusters/{cluster_id}

This operation synchronously returns the status and information on the specified cluster within the provided project id.

Request Parameters

Parameter Type Description
cluster_id (URI) UUID Cluster ID. This value is returned when a new cluster is created.

Response Codes

Response Code(s)
Normal ok (200)
Error unauthorized (401), itemNotFound (404)

Response Parameters

Parameter Type Description
cluster_id UUID ID of requested cluster.
name string Name of cluster.
status string Current status of cluster.

BUILDING: Cluster is in progress of being provisioned.

UPDATING: Cluster in process of being updated.

ACTIVE: Cluster is running.

ERROR: Provisioning error(s) encountered.

DELETING: Cluster is in process of being deleted.

DELETED: Cluster has been deleted.

created string Created time stamp in format: yyyy-mm-ddThh:mm:ssZ
updated string Last updated time stamp in format: yyyy-mm-ddThh:mm:ssZ
nodes(list) node List of nodes, which includes node id, instance state, and flavor.

JSON Request

N/A

JSON Response

{

   "cluster": {
       "cluster_id": "dd745f4a-9333-417e-bb89-9c989c84c068",
       "network_id": "d32019d3-bc6e-4319-9c1d-6722fc136a22",
       "name": "Message Cluster 1",
       "status": "ACTIVE",
       "created": "2014-11-11T01:02:03Z",
       "updated": "2014-11-11T01:02:03Z",
       "nodes": [
           {
               "node_id": "616fb98f-46ca-475e-917e-2563e5a8cd19",
               "status": "ACTIVE",
               "flavor": "large",
               "endpoints": [
                   {
                       "type": "AMQP",
                       "value": "amqp://10.20.30.40:10000"
                   },
                   {
                       "type": "console",
                       "value": "http://10.20.30.40:5672"
                   }
               ],
               "created": "2014-11-11T01:06:03Z",
               "updated": "2014-11-11T01:06:03Z"
           },
           {
               "node_id": "e90c9d13-c4b8-4a08-992a-dad6109b8ac2",
               "status": "ACTIVE",
               "flavor": "large",
               "endpoints": [
                   {
                       "type": "AMQP",
                       "value": "amqp://10.20.30.41:10000"
                   },
                   {
                       "type": "console",
                       "value": "http://10.20.30.41:5672"
                   }
               ],
               "created": "2014-11-11T01:08:03Z",
               "updated": "2014-11-11T01:08:03Z"
           },
           {
               "node_id": "372f8f47-6818-4d83-aa42-8744c0e689b8",
               "status": "ACTIVE",
               "flavor": "large",
               "endpoints": [
                   {
                       "type": "AMQP",
                       "value": "amqp://10.20.30.42:10000"
                   },
                   {
                       "type": "console",
                       "value": "http://10.20.30.42:5672"
                   }
               ],
               "created": "2014-11-11T01:12:03Z",
               "updated": "2014-11-11T01:12:03Z"
           }
       ]
   }

}

Delete Cluster

DELETE /v1/clusters/{cluster_id}''

This operation will asynchronously delete the indicated cluster within the provided project id.

Request Parameters

Parameter Type Description
cluster_id (URI) UUID Cluster ID. This value is returned when a new cluster is created.

Response Codes

Response Code(s)
Normal accepted (202)
Error unauthorized (401), itemNotFound (404)

Response Parameters

N/A

JSON Request

N/A

JSON Response

N/A

Testing

Cue API testing will verify the expected functionality of the Cue user interface with both positive/negative tests. The overall scope will cover testing from the HTTP REST request to required database interactions and work-flow task submission for RPC workers.

Unit Tests

Unit tests will verify the resulting function calls for each REST-ful URI and action(s). The Python Mock library will be used to replace external system dependencies with placeholder objects.

Function Tests Input Data Expected Result(s)
List Clusters
  1. Create 'n' cluster(s), then call list clusters and verify expected return object
  2. Call list clusters when no clusters exist within the project id and verify expected return object
  • n = # of clusters to create
  1. List of 'n' clusters is returned with appropriate fields, HTTP Ok (200)
  2. Empty list of clusters is returned, HTTP Ok (200)
Create Cluster
  1. Create standard cluster 'c' and verify expected return object
  2. Create cluster with invalid flavor and verify expected return value
  3. Create cluster with invalid volume size and verify expected return value
  • n = cluster size (number of nodes)
  • m = cluster name
  • f = flavor
  • s = volume size
  • nic = Network UUID
  1. Cluster ID, name and status for newly created cluster is returned, HTTP Accepted (202)
  2. HTTP Bad request is returned (400)
  3. HTTP Bad request is returned (400)
Show Cluster
  1. Create cluster, then call get cluster and verify expected return object
  2. Get cluster with invalid cluster_id
  • cluster_id = cluster ID
  1. Cluster ID, name, status, created/updated date stamp and list of nodes is returned, HTTP Ok (200).
  2. HTTP Bad request is returned (400)
Delete Cluster
  1. Create a cluster, then call delete cluster and verify expected return value
  2. Delete cluster with invalid cluster_id
  • cluster_id = cluster ID
  1. HTTP accepted is returned (202)
  2. HTTP Bad request is returned (400)

Functional Tests

The functional tests will verify the HTTP REST URI request lifecycle from controller routing to HTTP response. These tests will make use of Pecan's testing utility; pecan.testing module.

Operation URI Tests Input Data Expected Data (JSON)
GET /v1/clusters
  1. Call when clusters exist
  2. Call when no clusters exist
  • n/a
  1. List of 'n' clusters is returned with appropriate fields, HTTP Ok (200)
  2. Empty list of clusters is returned, HTTP Ok (200)
POST /v1/clusters
  1. Create standard cluster 'c' and verify expected return object
  2. Create cluster with invalid flavor and verify expected return value
  3. Create cluster with invalid volume size and verify expected return value
  • n = cluster size (number of nodes)
  • m = cluster name
  • f = flavor
  • s = volume size
  • nic = Network UUID
  1. Cluster ID, name and status for newly created cluster is returned, HTTP Accepted (202)
  2. HTTP Bad request is returned (400)
  3. HTTP Bad request is returned (400)
GET /v1/clusters/{cluster_id}
  1. Create cluster, then call get cluster and verify expected return object
  2. Get cluster with invalid cluster_id
  • cluster_id = cluster
  1. Cluster ID, name, status, created/updated date stamp and list of nodes is returned, HTTP Ok (200).
  2. HTTP Bad request is returned (400)
DELETE /v1/clusters/{cluster_id}
  1. Create a cluster, then call delete cluster and verify expected return value
  2. Delete cluster with invalid cluster_id
  • cluster_id = cluster ID
  1. HTTP accepted is returned (202)
  2. HTTP Bad request is returned (400)

Integration Tests

The integration tests will cover API functionality from HTTP request to database access and task submission for RPC workers. The Pecan test utility will be used to route test request URI to appropriate controllers, then database record(s) will be verified for applicable change. Finally the creation of task objections will be verified to ensure valid task flows.