Difference between revisions of "Obsolete:ArchitecturalOverview"
| Line 3: | Line 3: | ||
Live Notes may be taken for this topic at: http://etherpad.openstack.org/Architecture | Live Notes may be taken for this topic at: http://etherpad.openstack.org/Architecture | ||
| + | |||
| + | [[Image:ArchitecturalOverview$overview.png]] | ||
| + | |||
| + | === “Small” components, loosely coupled === | ||
| + | |||
| + | * Queue based (currently AMQP/RabbitMQ) | ||
| + | * Flexible schema for datastore (currently Redis) | ||
| + | * LDAP (allows for integration with MS Active Directory via translucent proxy) | ||
| + | * Workers & Web hooks (be of the web) | ||
| + | * Asynchronous everything (don't block) | ||
| + | |||
| + | === Development goals === | ||
| + | |||
| + | * Testing & Continuous Integration | ||
| + | * Fakes (allows development on a laptop) | ||
| + | * Adaptable (goal is to make integration with existing resources at organization easier) | ||
| + | |||
| + | === Queue === | ||
| + | |||
| + | * Each worker/agent listens on a general topic, and a subtopic for that node. Example would be "compute" & "compute:hostname" | ||
| + | * Messages in the queue are currently '''Topic''', '''Method''', '''Arguments''' - which maps to a method in the python class for the worker | ||
| + | * exposed via method calls | ||
| + | ** '''rpc.cast''' to broadcast the message and not wait for a response | ||
| + | ** '''rpc.call''' to send a message and wait for the response | ||
| + | |||
| + | === Datastore === | ||
| + | |||
| + | * Currently data is stored in Redis 2.0 (RC) | ||
| + | * Do the work on write - make reads FAST | ||
| + | ** maintain indexes / lists of common subsets | ||
| + | ** use pools (SETs in redis) that are drained for IPs instead of tracking what is allocated | ||
| + | |||
| + | === Delta === | ||
| + | |||
| + | * Scheduler does not exist (instances are distributed via the queue to the first worker that consumes the message) | ||
| + | * Object store in Nova is a naive stub which would be replaced with Cloud Files in Production (a simple object store that mimics Cloud Files might be good for development) | ||
| + | * Tornado should be phased out for WSGI-based web framework | ||
Revision as of 11:16, 13 July 2010
Architectural Overview
Live Notes may be taken for this topic at: http://etherpad.openstack.org/Architecture
File:ArchitecturalOverview$overview.png
“Small” components, loosely coupled
- Queue based (currently AMQP/RabbitMQ)
- Flexible schema for datastore (currently Redis)
- LDAP (allows for integration with MS Active Directory via translucent proxy)
- Workers & Web hooks (be of the web)
- Asynchronous everything (don't block)
Development goals
- Testing & Continuous Integration
- Fakes (allows development on a laptop)
- Adaptable (goal is to make integration with existing resources at organization easier)
Queue
- Each worker/agent listens on a general topic, and a subtopic for that node. Example would be "compute" & "compute:hostname"
- Messages in the queue are currently Topic, Method, Arguments - which maps to a method in the python class for the worker
- exposed via method calls
- rpc.cast to broadcast the message and not wait for a response
- rpc.call to send a message and wait for the response
Datastore
- Currently data is stored in Redis 2.0 (RC)
- Do the work on write - make reads FAST
- maintain indexes / lists of common subsets
- use pools (SETs in redis) that are drained for IPs instead of tracking what is allocated
Delta
- Scheduler does not exist (instances are distributed via the queue to the first worker that consumes the message)
- Object store in Nova is a naive stub which would be replaced with Cloud Files in Production (a simple object store that mimics Cloud Files might be good for development)
- Tornado should be phased out for WSGI-based web framework
