Jump to: navigation, search

Difference between revisions of "MagnetoDB"

m (→‎Authorization)
 
(57 intermediate revisions by 8 users not shown)
Line 1: Line 1:
−
=== Status ===
 
  
−
* Inital draft --[[User:Isviridov|isviridov]] ([[User talk:Isviridov|talk]]) 17:40, 16 October 2013 (UTC)
+
== /!\ Project is not maintained anymore ==
−
=== Scope ===
 
−
Document describes the idea and technical concept of DB service for OpenStack like [http://aws.amazon.com/dynamodb Amazon DynamoDB].
 
−
There is no implementation yet.
 
  
−
== What is MagnetoDB? ==
+
'''MagnetoDB''' is a key-value store service for OpenStack. It provides horizontally scalable, queryable storage, accessible via REST API. MagnetoDB supports Amazon DynamoDB API as well. MagnetoDB has been designed and developed in order to provide:
−
MagnetoDB is [http://aws.amazon.com/dynamodb Amazon DynamoDB] [http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/API.html API]  implementation for OpenStack world.
+
* ''Easy integration'': REST API. Support Amazon DynamoDB API
 +
* ''OpenStack interoperability'': integrated with Keystone. Follows OpenStack design tenets, packaging and distribution
 +
* ''Database pluggability'': supports Cassandra,any other databases like MongoDB, HBase could be easily added
 +
* ''Horizontal scalability'': MagnetoDB is scalable lineary by amount of data and requests
 +
* ''High availability'': just one working API node is enough to continue handling requests
  
−
=== What is DynamoDB? ===
+
== Project status ==
−
DynamoDB is a fast, fully managed NoSQL database service that makes it simple and cost-effective to store and retrieve any amount of data, and serve any level of request traffic.
+
Project is not maintained anymore.
−
With DynamoDB, you can offload the administrative burden of operating and scaling a highly available distributed database cluster
 
  
−
DynamoDB tables do not have fixed schemas, and each item may have a different number of attributes. Multiple data types add richness to the data model. Local secondary indexes add flexibility to the queries you can perform, without impacting performance. [http://aws.amazon.com/dynamodb/ [1<nowiki>]</nowiki>]
+
== Resources ==
 +
* [http://magnetodb.readthedocs.org/ User/Developer Documentation ]
 +
* [https://github.com/zaletniy/magnetodb-specs Design Documentation ]
 +
* [https://github.com/zaletniy/magnetodb MagnetoDB at GitHub]
 +
* [https://github.com/AlexanderChudnovets/magnetodb-test-bench Performace test bench]
 +
* [https://launchpad.net/magnetodb Launchpad]
 +
* [[MagnetoDB/FAQ|FAQ]]
 +
* [https://webchat.freenode.net/?channels=magnetodb #magnetodb at FreeNode]
 +
* [[MagnetoDB/WeeklyMeetingAgenda|Weekly IRC meetings]]
  
−
=== Why it is needed? ===
+
== Core Team ==
−
In order to provide hight level database service with DynamoDB API interface for OpenStack users.
+
* Andrei V. Ostapenko (aostapenko), Mirantis
 +
* Charles Wang (charlesw), Symantec
 +
* Dmitriy Ukhlov (dukhlov), Mirantis
 +
* Ilia Khudoshyn (ikhudoshyn), Mirantis
 +
* [PTL] Ilya Sviridov (isviridov), Mirantis
  
−
Just simply create the table and start working with your data. No vm provisioning, database software setup, configuration, patching, monitoring and other administrative routine.
 
  
−
=== Integration with Trove? ===
+
90 days project statisics http://stackalytics.com/report/contribution/magnetodb/90
−
Initially suggested as [https://blueprints.launchpad.net/trove/+spec/http-accessible-storage blueprint] for Trove  it was rejected as not following the programm mission.
 
  
−
MagnetoDB is not one more database, which can be provisioned and managed by Trove. It is one more OpenStack service with own API, keystone based authentication, quota management.
+
== Getting started ==
 +
[[MagnetoDB/How_To_Contribute|How To Contribute]]
  
−
However, the underlaying database will be provisioned and managed by Trove, in order to reuse existing Openstack components and avoid duplication of functionality.
+
== Presentations & blogposts ==
 +
[https://www.mirantis.com/blog/introducing-magnetodb-nosql-database-service-openstack/ Introducing MagnetoDB]
 +
== Sub pages ==
  
−
== Overall architecture ==
+
{{Special:PrefixIndex/:MagnetoDB/}}
−
The shape of the architecture based on Cassandra storage is below. It is the classic 2 layer web application, where each layer is horizontally scalable.
 
−
Requests are routed by load balancer to web application layer, and processed there.
 
−
 
 
−
[[File:magnetodb_architecture.png|800px|frameless|center]]
 
−
 
 
−
In projection to existing OpenStack components
 
−
[[File:magnetodb_components_by_layers.png|1000px|frameless|center]]
 
−
 
 
−
== API ==
 
−
The DynamoDB API documented very well here  [http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/operationlist.html Operations in Amazon DynamoDB]
 
−
However, the key aspects are mentioned below.
 
−
 
 
−
The API present HTTP request/responce, where JSON  is used as data transfer model. The request body contains the payload, and HTTP headers are used for metadata.
 
−
 
 
−
Example of such request
 
−
 
 
−
POST / HTTP/1.1
 
−
host: dynamodb.us-west-2.amazonaws.com
 
−
x-amz-date: 20130112T092034Z
 
−
x-amz-target: DynamoDB_20120810.CreateTable
 
−
Authorization: AWS4-HMAC-SHA256 Credential=AccessKeyID/20130112/us-west-2/dynamodb/aws4_request,SignedHeaders=host;x-amz-date;x-amz-target,Signature=145b1567ab3c50d929412f28f52c45dbf1e63ec5c66023d232a539a4afd11fd9
 
−
content-type: application/x-amz-json-1.0
 
−
content-length: 23
 
−
connection: Keep-Alive
 
−
 
 
−
As you can see the operation name and authorization data is sent in HTTP request headers.
 
−
 
 
−
=== Authorization ===
 
−
It is HMAC based on [http://docs.openstack.org/trunk/openstack-ops/content/get_creds.html  EC2 Credentials], which is sent in a header of HTTP request as in example above.
 
−
 
 
−
=== Table CRUD operations ===
 
−
* [http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_ListTables.html ListTables]
 
−
* [http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_DescribeTable.html DescribeTable]
 
−
* [http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_CreateTable.html CreateTable]
 
−
* [http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_UpdateTable.html UpdateTable]
 
−
* [http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_DeleteTable.html DeleteTable]
 
−
 
 
−
=== Item CRUD operations ===
 
−
* [http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html PutItem]
 
−
* [http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_UpdateItem.html UpdateItem]
 
−
* [http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_DeleteItem.html DeleteItem]
 
−
* [http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_GetItem.html GetItem]
 
−
 
 
−
=== Data quering ===
 
−
* [http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html Query]
 
−
* [http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html Scan]
 
−
 
 
−
=== Batch operations ===
 
−
* [http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchGetItem.html BatchGetItem]
 
−
* [http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchWriteItem.html BatchWriteItem]
 
−
 
 
−
== Implementation ==
 
−
 
 
−
=== Pluggability ===
 
−
The most suitable data source is [http://cassandra.apache.org/ Cassandra].
 
−
But it is better to make datasource plugguble and give the framework for implementation of interface for any other underlying storage.
 
−
 
 
−
=== OpenStack component reusage ===
 
−
{| class="wikitable"
 
−
|-
 
−
! Purpose !! OS component
 
−
|-
 
−
| Provisioning of web tire || [[Heat]], Mistral (Convection)
 
−
|-
 
−
| Load balancer || [[Neutron/LBaaS|LBaaS, Neutron]]
 
−
|-
 
−
| Autoscaling of web tier || Heat autoscaling, [[Ceilometer]]
 
−
|-
 
−
| Data Source provisioning and management || [[Trove]]
 
−
|-
 
−
| Monitoring ||  [[Ceilometer]]
 
−
|-
 
−
| Authentication || [[Keystone]]
 
−
|}
 
−
=== Autoscaling ===
 
−
* Web autoscaling is suggested to implement with HEAT autoscaling group resource
 
−
* Datasource scalability based on future Trove autoscaling
 

Latest revision as of 23:32, 8 August 2016

/!\ Project is not maintained anymore

MagnetoDB is a key-value store service for OpenStack. It provides horizontally scalable, queryable storage, accessible via REST API. MagnetoDB supports Amazon DynamoDB API as well. MagnetoDB has been designed and developed in order to provide:

  • Easy integration: REST API. Support Amazon DynamoDB API
  • OpenStack interoperability: integrated with Keystone. Follows OpenStack design tenets, packaging and distribution
  • Database pluggability: supports Cassandra,any other databases like MongoDB, HBase could be easily added
  • Horizontal scalability: MagnetoDB is scalable lineary by amount of data and requests
  • High availability: just one working API node is enough to continue handling requests

Project status

Project is not maintained anymore.

Resources

Core Team

  • Andrei V. Ostapenko (aostapenko), Mirantis
  • Charles Wang (charlesw), Symantec
  • Dmitriy Ukhlov (dukhlov), Mirantis
  • Ilia Khudoshyn (ikhudoshyn), Mirantis
  • [PTL] Ilya Sviridov (isviridov), Mirantis


90 days project statisics http://stackalytics.com/report/contribution/magnetodb/90

Getting started

How To Contribute

Presentations & blogposts

Introducing MagnetoDB

Sub pages

Retrieved from "https://wiki.openstack.org/w/index.php?title=MagnetoDB&oldid=129930"