Jump to: navigation, search

Difference between revisions of "Senlin"

(Created page with "__NOTOC__ = OpenStack Clustering = The mission of the OpenStack Clustering project is to create a service for managing the homogeneous objects exposed by other services in a...")
 
(Get Involved)
 
(39 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
__NOTOC__
 
__NOTOC__
  
= OpenStack Clustering =
+
[[File:OpenStack_Project_Senlin_Horizontal.png|right]]
  
The mission of the OpenStack Clustering project is to create a service for managing the homogeneous objects exposed by other services in an OpenStack cloud.
+
== Overview ==
  
= Senlin =
+
Senlin is a clustering service for OpenStack clouds. It creates and operates clusters of homogeneous objects exposed by other OpenStack services. The goal is to make orchestration of collections of similar objects easier.
  
Senlin provides a clustering service to create and manage objects of the same nature, e.g. Nova servers, Heat stacks, Cinder volumes, etc. The collection of these objects is referred to as a cluster. Senlin abstracts the physical objects as nodes, which can belong to any cluster of the same type. Senlin supports object creation, deletion and update via a concept called Profile. Each profile is in essential a driver to communicate with certain services for object manipulation.
+
Senlin interacts with other OpenStack services so that clusters of resources exposed by those services can be created and operated. These interactions are mostly done through the via profile plugins. Each profile type implementation enable Senlin to create, update, delete a specific type of resources.
  
With Senlin, user can attach a policy to a cluster and enforce it, or detach it from a cluster. Some built-in policies are provided to meet requirements such as auto-scaling, load-balancing, high-availability etc.  
+
A Cluster can be associated with different Policy objects that can be checked/enforced at varying enforcement levels. Through service APIs, a user can dynamically add Node to and remove node from a cluster, attach and detach policies, such as creation policy, deletion policy, load-balancing policy, scaling policy, health policy etc. Through integration with other OpenStack projects, users will be enabled to manage deployments and orchestrations large-scale resource pools much easier.
  
Senlin provides an OpenStack-native ReST API and a command line interface tool to other services and the users.
+
Senlin is designed to be capable of managing different types of objects. An object’s lifecycle is managed using Profile Type implementations, which are plugins that can be dynamically loaded by the service engine.
  
== How it works ==
+
=== Features Highlight ===
  
* A Senlin profile describes the information needed to create a node (or a cluster). A user will create a profile via Senlin first. Different profile types talk to different backend drivers.
+
The primary features of the Senlin service are listed below:
* The profile then can be used to create a node or a cluster.
+
 
* The cluster membership can be changed by explicit adding or removal of nodes, or by a scale-in/out operation.
+
* A generic clustering/collection service for managing groups of homogeneous cloud objects on OpenStack.
* User can instruct a node to join a cluster or leave its current cluster.
+
* A set of [http://developer.openstack.org/api-ref/clustering APIs] for managing cluster membership, e.g. add/remove nodes.
* A Senlin policy describes certain rules that will be checked and enforced.
+
* A plugin-based object profile management enabling the creation and management of any object pools.
* (TBC)
+
* A plugin-based policy enforcement framework featuring flexible policy customization for cluster management.
 +
* A plugin-based event notification that allows for dumping or pumping cluster actions to storage or downstream software.
 +
* A asynchronous execution engine for ensuring the state consistency of clusters and nodes.
  
 
== Architecture ==
 
== Architecture ==
  
Senlin comprises a number of Python applications:
+
The following picture depicts the high-level system architecture of the Senlin service:
  
=== senlin ===
+
[[File:senlin-architecture.png]]
  
The senlin tool is a CLI tool.
+
The main components in the picture are:
 +
* The ''senlinclient'' component provides the command line interface (CLI) for users to interact with the Senlin service;
 +
* The ''senlin-api'' service provides the RESTful [http://developer.openstack.org/api-ref-clustering-v1.html APIs] for the ''senlinclient'' component or other services;
 +
* The ''senlin-engine'' service sits behind the ''senlin-api'' service and operates on the clusters/nodes, using ''profiles'' and ''policies'' that are loaded as plugins.
  
=== senlin-api ===
+
== Usage Scenarios ==
  
The senlin-api component provides an OpenStack-native ReST API.
+
Below are some typical usage scenarios of the Senlin service. More scenarios will be added later.
  
=== senlin-engine ===
+
* [http://docs.openstack.org/developer/senlin/scenarios/affinity.html Managing Node Affinity]
 +
* [http://docs.openstack.org/developer/senlin/scenarios/autoscaling_ceilometer.html AutoScaling using Ceilometer/Aodh]
 +
* [http://docs.openstack.org/developer/senlin/scenarios/autoscaling_heat.html AutoScaling using Heat Resource Types]
  
The senlin engine does the main work of managing clusters, nodes, profiles, policies, actions, events and webhooks.
+
== Documentation ==
  
== Development ==
+
* [http://docs.openstack.org/developer/senlin/#install-and-configuration Installation Guide]
 +
* [http://docs.openstack.org/developer/senlin/#tutorial Tutorial]
 +
* [http://docs.openstack.org/developer/senlin/#user-references User References]
 +
* [http://docs.openstack.org/developer/senlin/#developer-s-guide Developer Documents]
 +
* [http://developer.openstack.org/api-ref/clustering API Version 1.0 Reference]
  
[TBD]
+
== Related Presentations ==
  
= Get involved =
+
* [http://www.slideshare.net/cliffton75/high-availability-in-openstack-cloud Enhancing high availability in the context of OpenStack], Atlanta Summit, 2014
== IRC ==
+
  - A taxonomy of architecting high-availability into an OpenStack cloud.
 +
  - The then current status of OpenStack support to HA.
 +
  - A proposed service (HAaaS) which layed the foundation for Senlin's cluster design.
 +
* [http://www.slideshare.net/cliffton75/suning-openstack-cloud-and-heat Suning cloud and Heat], Paris Summit, 2014
 +
  - Experience sharing on using OpenStack for an online retailer
 +
  - Requirements on orchestrating enterprise applications: clustering, scaling, HA ...
 +
  - Gaps identified in the then current status of OpenStack services
 +
* [http://www.slideshare.net/cliffton75/vm-ha-and-crossregion-scaling VM HA and cross-region autoscaling], Paris Summit, 2014
 +
  - How VM HA can be achieved by integrating OpenStack services.
 +
  - How to scale a VM cluster across multiple regions.
 +
* [http://www.slideshare.net/cliffton75/senlin-deep-dive-2015-0520 Senlin design discussion], Vancouver Design Summit, 2015
 +
  - A discussion on Heat design summit about senlin service design
 +
  - How the service is architected, the terminology behind it
 +
  - How auto-scaling can be achieved in this design
 +
* [http://www.slideshare.net/cliffton75/autoscaling-with-magnum-and-senlin Integration Magnum and Senlin for container autoscaling], Tokyo Summit, 2015
 +
  - Experience sharing on integrating Magnum and Senlin
 +
  - Magnum provisions the container orchestration engines (COEs)
 +
  - Senlin is responsible to scale the container clusters and the underlying VM clusters.
 +
* [http://www.slideshare.net/cliffton75/senlin-deep-dive-2016 Senlin deep dive for v1.0.0 release], Austin Summit, 2016
 +
  - An overview of the Senlin service design as of the 1.0.0 release.
 +
  - A summary of progress made during Mitaka cycle.
 +
  - An overlook of features planned for Newton cycle.
 +
* [http://www.slideshare.net/cliffton75/managing-container-clusters-in-openstack-native-way OpenStack native container clustering], Austin Summit, 2016
 +
  - A technology preview of the container clustering support in Senlin
 +
  - The discussion focuses on the on-going effort that builds and manages a docker cluster on OpenStack
 +
* [http://www.slideshare.net/cliffton75/deploy-an-elastic-resilient-loadbalanced-cluster-in-5-minutes-with-senlin Elastic, resilient and load-balanced cluster in 5 mins], Austin Summit, 2016
 +
  - Experiences sharing how elasticity, resiliency and load-balancing can be achieved using Senlin
 +
  - The demo shows how the above requirements are met using a Heat template consisting of Senlin resource types.
 +
* on-building-an-auto-healing-resource-cluster-using-senlin [https://www.openstack.org/videos/barcelona-2016/on-building-an-auto-healing-resource-cluster-using-senlin], Barcelona, 2016
 +
* managing-clusters-of-thousands-of-vms-using-senlin [https://www.openstack.org/videos/barcelona-2016/managing-clusters-of-thousands-of-vms-using-senlin], Barcelona, 2016
 +
* integration-of-enterprise-monitoring-product-senlin-and-mistral-for-auto-healing [https://www.openstack.org/videos/boston-2017/integration-of-enterprise-monitoring-product-senlin-and-mistral-for-auto-healing], Boston, 2017
  
The developers use IRC in <code>#senlin</code> on [http://freenode.net/ Freenode] for development discussion.
+
*high-availablity-and-scalability-management-of-vnf [https://www.openstack.org/videos/boston-2017/high-availablity-and-scalability-management-of-vnf], Boston, 2017
  
== Meetings ==
+
* senlin-an-ideal-bridge-between-nfv-orchestrator-and-openstack[https://www.openstack.org/videos/boston-2017/senlin-an-ideal-bridge-between-nfv-orchestrator-and-openstack], Boston, 2017
  
[TBD]
+
* standing-up-kubernetes-on-openstack-is-cool-then-what [https://www.openstack.org/videos/sydney-2017/standing-up-kubernetes-on-openstack-is-cool-then-what], Sydeny Summit, 2017
  
== Mailing list ==
+
== Get Involved ==
  
Discussions about Senlin happens on the [http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev openstack-dev] mailing list.
+
{| class="wikitable"
Please use the tag <code>[Senlin]</code> in the subject line for new threads.
+
|-
 +
| Launchpad Projects || [https://launchpad.net/senlin Senlin] [https://launchpad.net/python-senlinclient SenlinClient] [https://launchpad.net/senlin-dashboard SenlinDashboard]
 +
|-
 +
| Code Repositories || [https://git.openstack.org/cgit/openstack/senlin Senlin] [http://git.openstack.org/cgit/openstack/python-senlinclient SenlinClient] [http://git.openstack.org/cgit/openstack/senlin-dashboard/ SenlinDashboard]
 +
|-
 +
| Blueprints || [https://blueprints.launchpad.net/senlin Senlin Blueprints]
 +
|-
 +
| Code Review || [https://review.openstack.org/#/q/project:openstack/senlin,n,z Senlin] [https://review.openstack.org/#/q/project:openstack/python-senlinclient,n,z SenlinClient] [https://review.openstack.org/#/q/project:openstack/senlin-dashboard,n,z SenlinDashboard]
 +
|-
 +
| Bug Tracking || [https://bugs.launchpad.net/senlin Senlin] [https://bugs.launchpad.net/python-senlinclient SenlinClient]
 +
|-
 +
| IRC Channel || <code>#senlin</code> on freenode.net
 +
|-
 +
| Mailinglist || Drop emails to [http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev openstack-dev] with tag <code>[Senlin]</code> in the subject.
 +
|-
 +
| Weekly Meetings || 0530UTC biweekly-even:Friday on #openstack-meeting channel. <br />
 +
[https://wiki.openstack.org/wiki/Meetings/SenlinAgenda Agenda] [http://eavesdrop.openstack.org/meetings/senlin/2015/ 2015 Minutes Archive] [http://eavesdrop.openstack.org/meetings/senlin/2016/ 2016 Minutes Archive]
 +
|}

Latest revision as of 05:52, 6 May 2019


OpenStack Project Senlin Horizontal.png

Overview

Senlin is a clustering service for OpenStack clouds. It creates and operates clusters of homogeneous objects exposed by other OpenStack services. The goal is to make orchestration of collections of similar objects easier.

Senlin interacts with other OpenStack services so that clusters of resources exposed by those services can be created and operated. These interactions are mostly done through the via profile plugins. Each profile type implementation enable Senlin to create, update, delete a specific type of resources.

A Cluster can be associated with different Policy objects that can be checked/enforced at varying enforcement levels. Through service APIs, a user can dynamically add Node to and remove node from a cluster, attach and detach policies, such as creation policy, deletion policy, load-balancing policy, scaling policy, health policy etc. Through integration with other OpenStack projects, users will be enabled to manage deployments and orchestrations large-scale resource pools much easier.

Senlin is designed to be capable of managing different types of objects. An object’s lifecycle is managed using Profile Type implementations, which are plugins that can be dynamically loaded by the service engine.

Features Highlight

The primary features of the Senlin service are listed below:

  • A generic clustering/collection service for managing groups of homogeneous cloud objects on OpenStack.
  • A set of APIs for managing cluster membership, e.g. add/remove nodes.
  • A plugin-based object profile management enabling the creation and management of any object pools.
  • A plugin-based policy enforcement framework featuring flexible policy customization for cluster management.
  • A plugin-based event notification that allows for dumping or pumping cluster actions to storage or downstream software.
  • A asynchronous execution engine for ensuring the state consistency of clusters and nodes.

Architecture

The following picture depicts the high-level system architecture of the Senlin service:

Senlin-architecture.png

The main components in the picture are:

  • The senlinclient component provides the command line interface (CLI) for users to interact with the Senlin service;
  • The senlin-api service provides the RESTful APIs for the senlinclient component or other services;
  • The senlin-engine service sits behind the senlin-api service and operates on the clusters/nodes, using profiles and policies that are loaded as plugins.

Usage Scenarios

Below are some typical usage scenarios of the Senlin service. More scenarios will be added later.

Documentation

Related Presentations

 - A taxonomy of architecting high-availability into an OpenStack cloud.
 - The then current status of OpenStack support to HA.
 - A proposed service (HAaaS) which layed the foundation for Senlin's cluster design.
 - Experience sharing on using OpenStack for an online retailer
 - Requirements on orchestrating enterprise applications: clustering, scaling, HA ...
 - Gaps identified in the then current status of OpenStack services
 - How VM HA can be achieved by integrating OpenStack services.
 - How to scale a VM cluster across multiple regions.
 - A discussion on Heat design summit about senlin service design
 - How the service is architected, the terminology behind it
 - How auto-scaling can be achieved in this design
 - Experience sharing on integrating Magnum and Senlin
 - Magnum provisions the container orchestration engines (COEs)
 - Senlin is responsible to scale the container clusters and the underlying VM clusters.
 - An overview of the Senlin service design as of the 1.0.0 release.
 - A summary of progress made during Mitaka cycle.
 - An overlook of features planned for Newton cycle.
 - A technology preview of the container clustering support in Senlin
 - The discussion focuses on the on-going effort that builds and manages a docker cluster on OpenStack
 - Experiences sharing how elasticity, resiliency and load-balancing can be achieved using Senlin
 - The demo shows how the above requirements are met using a Heat template consisting of Senlin resource types.
  • on-building-an-auto-healing-resource-cluster-using-senlin [1], Barcelona, 2016
  • managing-clusters-of-thousands-of-vms-using-senlin [2], Barcelona, 2016
  • integration-of-enterprise-monitoring-product-senlin-and-mistral-for-auto-healing [3], Boston, 2017
  • high-availablity-and-scalability-management-of-vnf [4], Boston, 2017
  • senlin-an-ideal-bridge-between-nfv-orchestrator-and-openstack[5], Boston, 2017
  • standing-up-kubernetes-on-openstack-is-cool-then-what [6], Sydeny Summit, 2017

Get Involved

Launchpad Projects Senlin SenlinClient SenlinDashboard
Code Repositories Senlin SenlinClient SenlinDashboard
Blueprints Senlin Blueprints
Code Review Senlin SenlinClient SenlinDashboard
Bug Tracking Senlin SenlinClient
IRC Channel #senlin on freenode.net
Mailinglist Drop emails to openstack-dev with tag [Senlin] in the subject.
Weekly Meetings 0530UTC biweekly-even:Friday on #openstack-meeting channel.

Agenda 2015 Minutes Archive 2016 Minutes Archive