Jump to: navigation, search

Difference between revisions of "PolicyGuidedFulfillmentLibertyPlanning"

 
(4 intermediate revisions by one other user not shown)
Line 39: Line 39:
 
* '''''Policy based monitoring enabling (Ceilometer/Monasca) during provisioning and subsequent monitoring and enforcement'''''
 
* '''''Policy based monitoring enabling (Ceilometer/Monasca) during provisioning and subsequent monitoring and enforcement'''''
 
** Goal is to be able to simply control application scalability via Congress policies using telemetry (e.g., Ceilometer, Monasca) data.  
 
** Goal is to be able to simply control application scalability via Congress policies using telemetry (e.g., Ceilometer, Monasca) data.  
** There will be two phases
+
* see [[PolicyGuidedFulfillmentLibertyPlanning_PolicyBasedMonitoring]]
*** environment modification at predeploy time
 
**** prior predeploy_error evaluation, there will be policy ''monitoring'' evaluated (via simulation). It will contain rules which execute actions modifying environment in Murano by adding (via Murano REST) monitoring components. Such monitoring components are responsible for configuration of corresponding monitoring system (e.g., Ceilometer's alarms,....). User can create component for any monitoring system (e.g., watch dogs to check running Tomcat,DB,...)
 
*** public action invocation at runtime
 
**** it defines binding between monitoring system and Murano. If rule detects monitoring event, it executes corresponding action in Murano (e.g., invocation of public action "scaleOut" on scalable component, ...)
 
Example
 
component type - Ceilometer , its input parameters are 'relation to Instance', and  'alarm definition'. On deploy it creates alarm in Ceilometer.
 
  
Policy rules:
 
 
execute[ add-object-to-murano(envid, name, type, instance=instance-id, alarm_expression='cpu_util>50') ] :- murano:object(instance-id,parent-id,'Instance'), murano:connected(envid, instrance-id), murano:relationships(tomcat-id,instance-id, 'instance'), murano:objects(tomcat-id, parent-id2, 'Tomcat'),....
 
 
execute[murano-action(envid, scalableObject, 'scale-out' ) :- ceilometer:alarm(...instance-id,...), murano:object(instance-id,parent-id,'Instance'), murano:connected(env-id, instance-id),murano:relationships(tomcat-id,instance-id, 'instance'), murano:objects(tomcat-id, parent-id2, 'Tomcat')
 
  
 
* '''''Support any workflow engine in Murano''''' //[avigail] seems that this already exist in Murano. we are currently verifying it.
 
* '''''Support any workflow engine in Murano''''' //[avigail] seems that this already exist in Murano. we are currently verifying it.
Line 59: Line 48:
 
* '''''Remediation via Mistral workflows'''''  
 
* '''''Remediation via Mistral workflows'''''  
 
** Congress' action execution feature will trigger Mistral workflow implementing remediation for given situation/breach. Mistral workflow can use Murano components action, any O~S API, ...
 
** Congress' action execution feature will trigger Mistral workflow implementing remediation for given situation/breach. Mistral workflow can use Murano components action, any O~S API, ...
** Example:
+
* see [[PolicyGuidedFulfillmentLibertyPlanning_Remediation]]
  
execute[murano:muranoaction(env-id, obj-id, 'restart')] :- watchdog:not-responding(server-uuid), murano:properties(vmid,'server',server-uuid), murano:objects(vmid, parent, 'Instance'), murano:connected(env-id, vm-id), murano:objects(env-id, p2, 'Environment')
+
* Enable datasources to contribute policy statements
 +
** Remove the manual creation of murano_system policy for simulate calls
 +
** https://blueprints.launchpad.net/congress/+spec/datasource-rule-contributions
 +
** The datasource driver knows of some convenient helper tables that can be defined on top of its explicit tables and should therefore be able to insert them into its policy. The presence of these rules only serves to extend the tables provided by the datasource; it does not require the policy writer to use them or even know about them.
  
  
 
* '''''Improve the basic action execution based on usage'''''
 
* '''''Improve the basic action execution based on usage'''''
 
** see above remediation story.
 
** see above remediation story.
 
  
 
* '''''Accelerate the Datalog Horizon UI work started in Kilo'''''
 
* '''''Accelerate the Datalog Horizon UI work started in Kilo'''''
 
+
** Try to make more user friendly
 +
** Able to define complex rules
 +
** Ability to defines rules without datalog syntax knowledge
  
 
* '''''Performance validation of Congress with large data sets'''''
 
* '''''Performance validation of Congress with large data sets'''''
 
+
** Some of the items like multi-threaded data log from : https://github.com/openstack/congress/blob/d1ef962a7e6e1a55537d50ebb3604ade73ba2588/future-features.txt
 +
** How congress behaves with large data sets
 +
** Deploy on real world openstack deployment vs devstack based tests
  
 
* '''''Follow the delegation PoC work and updates at summit demo'''''
 
* '''''Follow the delegation PoC work and updates at summit demo'''''
 
+
** Track the progress and understand the implementation
  
 
* '''''Add new Monasca data source driver'''''
 
* '''''Add new Monasca data source driver'''''
 
+
** How Monasca monitoring works with Ceilometer monitoring
  
 
* '''''Evaluate/contribute to Heat data source driver'''''
 
* '''''Evaluate/contribute to Heat data source driver'''''
 +
** Use Heat id from Murano for associating to Nova/Nuetron for post deployment policies

Latest revision as of 16:29, 15 May 2015

Policy Guided Fulfillment - Liberty Planning

Policy Guided Fulfillment Main Page

Completed work in Kilo

  • Murano - Congress integration
    • Murano updated to check with Congress policy as part of provisioning steps
      • predeployment enforcement
    • New Murano data source in Congress
    • Contributed to adding reactive policy enforcement using action execution in Congress
  • Murano - Mistral Integration
    • Added the option to invoke a Mistral workflow as part of Murano application model


Check topic PolicyGuidedFulfillmentDemo for demo of features.

Planning topics for Liberty

Murano

  • Improving Murano API/core-model for better integration with Congress datasource
    • Goal is to improve Murano Congress Datasource
    • API: listing of environments from all tenants, more topo/type info in responses
    • Core topology model: provide more details on provisioned core components - e.g., server id and heat id in Instance,...
  • see PolicyGuidedFulfillmentLibertyPlanning_MuranoAPI


  • Workflow embracement
    • Currently it is possible to use Mistral workflow in Murano component action. The next step is to create component completely from the Mistral workflow, so any component action will be implemented by an action. During embracement (i.e., process of component creation from workflows) it should be possible to specify a) dependencies of the component on other components (e.g., Instance); b) mapping of workflow's input and output from/to component properties.
  • see PolicyGuidedFulfillmentLibertyPlanning_WorkflowEmbracement


  • Workflow based deployment
    • When all components in an environment will be built from workflows, it will be possible to create master workflow which orchestrate deployment of whole environment. Complete deployment execution on workflow engine will be scalable, reliable and robust.
    • To provide this, we need to have dual (supports both MuranoPL and Workflow components) or alternative implementation of core classes (e.g., Environment, Instance, Neutron network, ...).
  • see PolicyGuidedFulfillmentLibertyPlanning_WorkflowBasedDeployment


  • Policy based monitoring enabling (Ceilometer/Monasca) during provisioning and subsequent monitoring and enforcement
    • Goal is to be able to simply control application scalability via Congress policies using telemetry (e.g., Ceilometer, Monasca) data.
  • see PolicyGuidedFulfillmentLibertyPlanning_PolicyBasedMonitoring


  • Support any workflow engine in Murano //[avigail] seems that this already exist in Murano. we are currently verifying it.
    • [Radek] Avigail/Natasha will provide more info latter

Congress

  • Remediation via Mistral workflows
    • Congress' action execution feature will trigger Mistral workflow implementing remediation for given situation/breach. Mistral workflow can use Murano components action, any O~S API, ...
  • see PolicyGuidedFulfillmentLibertyPlanning_Remediation
  • Enable datasources to contribute policy statements
    • Remove the manual creation of murano_system policy for simulate calls
    • https://blueprints.launchpad.net/congress/+spec/datasource-rule-contributions
    • The datasource driver knows of some convenient helper tables that can be defined on top of its explicit tables and should therefore be able to insert them into its policy. The presence of these rules only serves to extend the tables provided by the datasource; it does not require the policy writer to use them or even know about them.


  • Improve the basic action execution based on usage
    • see above remediation story.
  • Accelerate the Datalog Horizon UI work started in Kilo
    • Try to make more user friendly
    • Able to define complex rules
    • Ability to defines rules without datalog syntax knowledge
  • Follow the delegation PoC work and updates at summit demo
    • Track the progress and understand the implementation
  • Add new Monasca data source driver
    • How Monasca monitoring works with Ceilometer monitoring
  • Evaluate/contribute to Heat data source driver
    • Use Heat id from Murano for associating to Nova/Nuetron for post deployment policies