Difference between revisions of "StructuredStateManagement"

Revision as of 05:33, 26 April 2013

Structured state management

Please note that this is a PROPOSAL ONLY. This is not yet 100% implemented.

Goal: Move away from ad-hoc states and state transitions for resource acquisition and modification to a more concrete structured & organized state management system in nova. This new state management system will have advanced new & shiny features such as greater stability, automatic recover mechanisms and greater scalability than what currently exists in nova (and more!).

Definitions

State: The particular condition that someone or something is in at a specific time: "the state of the instance request".
State transition: Altering a state by applying a function on-top of that state (of which said function may take inputs and provide outputs) resulting in a new state.
Task: The application of a state transition on a given state.
Workflow: The sequence of administrative/other processes through which a piece of work passes from initiation to completion.
Orchestrator: An individual or entity that arranges or control the elements of, as to achieve a desired overall effect.

What problems does this solve in general

Increases the [stability, extendability, reliability, recoverability] of states and state transitions in nova.
Makes it easier to [debug, test, understand, verify, review] states and state transitions in nova.
Removes hard to discover state+transition dependencies and interactions.
Ensures state transitions are done reliably and correctly by isolating those transitions to a single place/entity.
Fixes a variety of problems that previously had piecemeal like patches applied to attempt to solve them.
Eliminates the inherent fragility of the current ad-hoc workflows that exist in nova.
- They are by there ad-hoc nature hard to debug, hard to verify, hard to adjust, hard to understand (just hard in general)...
Makes it possible to audit & track the state transitions performed on a given resource.
- This kind of functionality has started to appear in nova, but the ad-hoc nature was preserved :-(
Addresses the underlying key point of http://www.slideshare.net/harlowja/nova-states-summit/9 where states will now be fully recovered from on cutting.

What problems does this solve in nova (on-top of the general ones)

Removes the need for periodic tasks to cleanup garbage (orphaned instances, orphaned resources...) left by nova's ad-hoc states.
Creates the path for smart resource scheduling.
Makes it possible to do [resizing, live migration] in a more secure and manageable manner.
- Discussion about how this can be done correctly require a intermediary to orchestrate this ownership transfer.
Makes it possible for nova to have multi-stage booting where an instances and its dependent resources are first reserved, the resources configured, the instance configured, and then finally the instance is powered-on (thus completing the instance provisioning process).

Issues that would likely not have happened with a better state management system

Plan of record

Step 1

Create prototype
1. Create library and use said library in nova for run_instance action in nova.

Step 2

Get feedback on prototype.
1. Get feedback from summit session.
2. Get more feedback from email list + other interested parties.

Step 3

Adjust nova prototype as needed from feedback.
Split nova prototype into small chunks.
Adjust tests for each small chunks (depending on what it changes).
Submit chunks into http://review.openstack.org (disabling whole/pieces component until ready to turn on?).

Step 4

Pick another nova action and refactor it to use design from prototype.
Split this other action (refactored) into small chunks.
Adjust tests for each small chunks (depending on what it changes).
Submit chunks into http://review.openstack.org (disabling whole/pieces component until ready to turn on?).
Repeat

Prototype

https://github.com/Yahoo/NovaOrc

Design

Details

See: StructuredStateManagementDetails

@@ Line 87: / Line 87: @@
 ==== Step 4 ====
-# Pick another nova action and refactor it to use design from prototype
+# Pick another nova action and refactor it to use design from prototype.
 # Split this other action (refactored) into small chunks.
 # Adjust tests for each small chunks (depending on what it changes).
 # Submit chunks into http://review.openstack.org (disabling whole/pieces component until ready to turn on?).
 # Repeat
 === Prototype ===