Jump to: navigation, search

Difference between revisions of "ReddwarfUsingHeat"

(Reddwarf using heat)
(Risks)
 
(12 intermediate revisions by 2 users not shown)
Line 8: Line 8:
  
 
====Benefits====
 
====Benefits====
* Reddwarf instrumenting on create/resize
+
* Reddwarf instrumenting on create
 
** Currently we are responsible for making sure the instance is prov'd and services are installed. this can be mitigated by using heat
 
** Currently we are responsible for making sure the instance is prov'd and services are installed. this can be mitigated by using heat
 
** package installation becomes a moot point. no need to write code to manage apt dependencies
 
** package installation becomes a moot point. no need to write code to manage apt dependencies
 +
** Clustering becomes easier as heat takes care of it for us
 
* Configuration is more flexible
 
* Configuration is more flexible
 
** If a company wants a particular image.. done.
 
** If a company wants a particular image.. done.
Line 17: Line 18:
  
 
====Risks====
 
====Risks====
* Heat is going thru a refactor in the next cycle
+
* Keep a stable reddwarf api without many architecture changes
** There is much talk of a dsl, and many improvements to heat in the next cycle
+
** The guest install flow will be different, but not drastically. Also the same w/ the creation of the instance.
** Its understood that heat is stable enough to be put in now
 
* Keep a stable reddwarf api w/o many architecture changes
 
 
* Make it optional for time being
 
* Make it optional for time being
 
** need to be agile enough to still rely on the original provisioning architecture
 
** need to be agile enough to still rely on the original provisioning architecture
Line 26: Line 25:
 
** flags for all new builds require heat, and eventually deprecate / remove non heat support  
 
** flags for all new builds require heat, and eventually deprecate / remove non heat support  
 
*** n+1?, or to graduate incubation?
 
*** n+1?, or to graduate incubation?
 +
* Not every deployed cloud is using heat
 +
** We need to make sure that an installer can stand up reddwarf w/o a hard dependency on heat
 +
*** For instance, we could point reddwarf to rax or hp cloud today and it would work.
  
====Explainings====
+
====Explanation====
  
The way i see it, reddwarf will talk to heat to provision single instances and clusters.
+
Reddwarf will talk to heat to provision single instances and clusters. Below is the only flow that will change for reddwarf -- ''viz.'' the create (orchestrate) instance. There is no api change from the customer perspective; they are abstracted from the provisioning work required to give them a working database instance. Heat, does, however, vastly simplify the installation process of the database (eg. mysql), and lets our guest focus on what it needs to -- securing the db.
 +
 
 +
The only touch points from reddwarf into nova itself are 1) create instance, 2) resize instance, 3) check status, 4) delete instance. This is a very minimal set of interactions.  
  
 
=====Current high level provisioning=====
 
=====Current high level provisioning=====
1) api calls task manager
+
# api calls task manager
2) task manager creates instance and volume (if volume support enabled)
+
# '''''ASYNC''''' task manager creates instance and volume (if volume support enabled)
3) task manager puts a message in the guest queue
+
# instance is booted, guest is either 1) installed in some way [1], or 2) already on image
4) task manager waits for guest to finish "prepare"
+
## guest receives message saying it is a ______ database (in this case its MySQL)
5) instance is booted, guest is either 1) installed in some way [1], or 2) already on image
+
## guest installs db software, generates a user, os_admin, w root privs
5.1) guest receives message saying it is a ______ database (in this case its MySQL)
+
## guest secures db in whatever way it deems necessary for security (queries, privs, etc, etc..) [2]
5.2) guest installs db software, generates a user, os_admin, w root privs
+
## guest informs TM that its done
5.3) guest secures db in whatever way it deems necessary for security (queries, privs, etc, etc..) [2]
+
# task manager puts a message in the guest queue
5.4) guest informs TM that its done
+
# task manager waits for guest to finish "prepare"
6) TM changes status of instance to active
+
# TM changes status of instance to active
 +
 
  
 
[1] we do not mandate _how_ the guest is installed. you can rsync, cloud-init package install, etc etc etc...
 
[1] we do not mandate _how_ the guest is installed. you can rsync, cloud-init package install, etc etc etc...
[2] we dont claim this is the best solution for 102% of people, but it is a solution that works for most people right out of the gate. Its a secure instance.
+
 
 +
[2] we dont claim this is the best solution for 102% of people, but it is a solution that works for most people right out of the gate. It's a secure instance.
  
 
=====With Heat high level provisioning=====
 
=====With Heat high level provisioning=====
1) api calls task manager
+
# api calls task manager
2) task manager instructs to heat to create instance w/ or w/o a volume
+
# '''''ASYNC''''' task manager instructs to heat to create instance w/ or w/o a volume
2.1) heat creates volume/instance
+
# heat creates instance and volume as necessary
2.2) heat installs db software
+
## heat installs db software
2.3) heat sets os_admin user/pass (if possible)  
+
## heat sets os_admin user/pass (if possible)  
2.4) heat installs guest, heat signals to task manager its finished
+
## heat installs guest, heat signals to task manager its finished
2.5) guest takes over
+
## guest takes over
2.5.1) guest secures db in whatever way it deems necessary for security (queries, privs, etc, etc..) [ see 2 above ]
+
### guest secures db in whatever way it deems necessary for security (queries, privs, etc, etc..) [ see 2 above ]
2.5.2) guest informs TM that its done
+
### guest informs TM that its done
3) task manager puts message in guest queue
+
# task manager puts message in guest queue
4) task manager waits for heat to finish installation
+
# task manager waits for heat to finish installation
5) task manager gets message heat is done, waits for guest to finish securing mysql
+
# task manager gets message heat is done, waits for guest to finish securing mysql
6) TM changes status of instance to active
+
# TM changes status of instance to active
 
 
Heat vastly simplifies the installation process of mysql, and lets our guest focus on what it needs to, 2.5.1, securing mysql.
 

Latest revision as of 18:38, 7 May 2013

Reddwarf using heat

Main requirements

  • Not modifying the api
  • Minimizing architecture differences
  • Make it optional
  • Make sure that the nova uuid is accessible, or that all state interaction to nova is managed by heat

Benefits

  • Reddwarf instrumenting on create
    • Currently we are responsible for making sure the instance is prov'd and services are installed. this can be mitigated by using heat
    • package installation becomes a moot point. no need to write code to manage apt dependencies
    • Clustering becomes easier as heat takes care of it for us
  • Configuration is more flexible
    • If a company wants a particular image.. done.
    • If a company wants "extra" bits, like a log collection agent in the image.. done.
    • You get the picture

Risks

  • Keep a stable reddwarf api without many architecture changes
    • The guest install flow will be different, but not drastically. Also the same w/ the creation of the instance.
  • Make it optional for time being
    • need to be agile enough to still rely on the original provisioning architecture
    • need to have legacy instance support, as in, if a instance was created w/o heat, it needs to stay managed w/o heat
    • flags for all new builds require heat, and eventually deprecate / remove non heat support
      • n+1?, or to graduate incubation?
  • Not every deployed cloud is using heat
    • We need to make sure that an installer can stand up reddwarf w/o a hard dependency on heat
      • For instance, we could point reddwarf to rax or hp cloud today and it would work.

Explanation

Reddwarf will talk to heat to provision single instances and clusters. Below is the only flow that will change for reddwarf -- viz. the create (orchestrate) instance. There is no api change from the customer perspective; they are abstracted from the provisioning work required to give them a working database instance. Heat, does, however, vastly simplify the installation process of the database (eg. mysql), and lets our guest focus on what it needs to -- securing the db.

The only touch points from reddwarf into nova itself are 1) create instance, 2) resize instance, 3) check status, 4) delete instance. This is a very minimal set of interactions.

Current high level provisioning
  1. api calls task manager
  2. ASYNC task manager creates instance and volume (if volume support enabled)
  3. instance is booted, guest is either 1) installed in some way [1], or 2) already on image
    1. guest receives message saying it is a ______ database (in this case its MySQL)
    2. guest installs db software, generates a user, os_admin, w root privs
    3. guest secures db in whatever way it deems necessary for security (queries, privs, etc, etc..) [2]
    4. guest informs TM that its done
  4. task manager puts a message in the guest queue
  5. task manager waits for guest to finish "prepare"
  6. TM changes status of instance to active


[1] we do not mandate _how_ the guest is installed. you can rsync, cloud-init package install, etc etc etc...

[2] we dont claim this is the best solution for 102% of people, but it is a solution that works for most people right out of the gate. It's a secure instance.

With Heat high level provisioning
  1. api calls task manager
  2. ASYNC task manager instructs to heat to create instance w/ or w/o a volume
  3. heat creates instance and volume as necessary
    1. heat installs db software
    2. heat sets os_admin user/pass (if possible)
    3. heat installs guest, heat signals to task manager its finished
    4. guest takes over
      1. guest secures db in whatever way it deems necessary for security (queries, privs, etc, etc..) [ see 2 above ]
      2. guest informs TM that its done
  4. task manager puts message in guest queue
  5. task manager waits for heat to finish installation
  6. task manager gets message heat is done, waits for guest to finish securing mysql
  7. TM changes status of instance to active