PolicyGuidedFulfillmentDemo

Policy Guided Fulfillment Demo
This topic provides information on demo of Policy Guided Fulfillment.

Application Predeployment Scenario
In this scenario we will use predeployment and postdeployment enforcement to define business policies for application:
 * predeployment
 * controls what application can be deployed in tenant
 * postdeployment
 * detects if VM instance exists for instance defined in an environment

Scenario Steps:
 * The cloud admin(s) sets 2 policy rules:
 * Tenant Demo can only use MySQL as a DB vendor
 * Tenant QA can only use PostgreSQL as a DB vendor
 * The admin wishes to model a Tomcat application that depends on a DB
 * Tenant Demo admin models it with a MySQL DB
 * Tenant QA admin models it with a PostgreSQL DB
 * Application developer want to add a workflow to the application deployment that
 * populates DB with data
 * verifies the Tomcat is up and running (by sending a GET request) and will send an email notifying whether the deployment has completed successfully or not.
 * The admins deploy the application models to Murano. The package will also include the Mistral workflow (so we will not depend on it being pre-deployed to Mistral)
 * End user of tenant Demo wishes to deploy the Tomcat application that relies on PostgreSQL
 * Deployment fails since the policy rules restrict this DB for tenant Demo
 * End user of tenant Demo asks Murano to deploy tomcat with MySQL DB
 * Application is deployed
 * At the end of deployment phase, the workflow is deployed to Mistral (programmatically, by this specific application deployment logic)
 * The workflow is triggered by the application class
 * data are populated into DB
 * mail will be sent to the end user indicating whether Tomcat has been deployed and whether it is responsive, if yes then init data will be inserted by the flow.

Application Postdeployment Scenario

 * When application is deployed, admin user creates error rules to check if deployed application's infrastructure is correct
 * rule will check if VMs created by Murano instances exist or are running
 * Admin user pauses VM instance of deployed application using Horizon UI
 * Admin user then will execute the policy rule to see the invalid application
 * Admin user resumes VM instance of deployed application using Horizon UI
 * Admin user then will execute the policy rule to check that instances are running

Automatic Remediation Scenario

 * When application is deployed, admin user creates rule which unpauses VM instances which is paused
 * Admin user pauses VM instance of deployed application using Horizon UI
 * Congress will unpauses the paused VM Instance automatically as defined in the rule
 * Admin user checks that VM instance is running using Horizon UI

Redeployment Scenario
This scenario shows predeployment enforcement employment in case of redeployment of already deployed environment. Scenario Steps:
 * The cloud admin(s) sets policy rule (predeployment enforcement):
 * Tenant Demo can only use MySQL as a DB vendor
 * End user creates environment with PostgreSQL DB in "Demo" tenant
 * End user deploys environment, but it fails due to predeployment enforcement rule above
 * Environment is populated to Congress using datasource driver
 * End user edits environment
 * remove PostgreSQL DB
 * adds MySQL DB
 * End User redeploys environment
 * predeployment enforcement rules allows redeployment

Prerequisities

 * OpenStack
 * enabled services Murano, Mistral, and Congress
 * enabled policy enforcement in Murano documentation http://murano.readthedocs.org/en/latest/articles/policy_enf_setup.html
 * create qa tenant
 * assign demo user to roles Member, anotherrole in the tenant (same roles as demo user has in demo tenant)
 * assign admin user to role admin in the tenant (same roles as admin user has in demo tenant)
 * VM images
 * Ubuntu based image
 * Centos (RedHat) based image
 * Images have to have installed Murano Agent
 * if you are behind proxy, then images have to have configured proxy
 * Networking
 * Access to internet is required
 * if you have proxy access to internet, you have to configured it in your lab, and in VM images
 * It is preferred to have configured float IPs in order to access deployed application from other nodes that DevStack server

Application
Application is based on packages used in murano-apps, murano-app-incubator, and.

Application packages are available here: https://github.com/filip-blaha/policy-guided-fulfillment-demo :


 * io.murano.apps.java.PetClinic
 * main application package:
 * installs pet clinic web application to Tomcat
 * configures database
 * deploys Mistral workflow in order to populate DB with data, check that Tomcat is running and sends email
 * has dependency on SqlDatabase, so one of MySql, PostgreSql can be choosen
 * has dependency on Tomcat
 * io.murano.apps.apache.Tomcat
 * Installs Tomcat
 * io.murano.databases.SqlDatabase
 * Base package for databases. It defines base actions for database configuration.
 * io.murano.databases.MySql
 * MySql database package
 * io.murano.databases.PostgreSql
 * Postgres database package

Create packages and upload them into OpenStack:

git clone https://github.com/filip-blaha/policy-guided-fulfillment-demo cd policy-guided-fulfillment-demo . ~/devstack/openrc admin admin # if you are using devstack, otherwise you have to setup env manually ./make-package.sh io.murano.apps.apache.Tomcat murano package-import --is-public io.murano.apps.apache.Tomcat.zip ./make-package.sh io.murano.apps.java.PetClinic murano package-import --is-public io.murano.apps.java.PetClinic.zip ./make-package.sh io.murano.databases.MySql murano package-import --is-public io.murano.databases.MySql.zip ./make-package.sh io.murano.databases.PostgreSql murano package-import --is-public io.murano.databases.PostgreSql.zip ./make-package.sh io.murano.databases.SqlDatabase murano package-import --is-public io.murano.databases.SqlDatabase.zip

Policy Rules
Following policy rules defines predeploy enforcement of the demo. Please setup Congress Murano driver as mentioned in http://murano.readthedocs.org/en/latest/articles/policy_enf_setup.html (configure driver for demo tenant).

Execute following commands to create the policies and policy rules:

. ~/devstack/openrc admin admin # if you are using devstack, otherwise you have to setup env manually

openstack congress policy rule create murano_system 'predeploy_errors(eid,oid,msg) :- murano:objects(oid,eid,type), murano:objects(eid,tid, "io.murano.Environment"), murano:parent_types(oid,"io.murano.Application"), tenantName(tid,tname), not allowedApp(type, tname),concat("Unsupported application detected: ", type, tmsg1),concat(tmsg1, ", ", tmsg2),objName(oid, oname), concat(tmsg2, oname, msg)' openstack congress policy rule create murano_system 'objName(oid,oname) :- murano:properties(oid, "name", oname)' openstack congress policy rule create murano_system 'tenantName(tid, tname) :- keystone:tenants(en,desc,tname,tid)' openstack congress policy rule create murano_system 'allowedApp("io.murano.databases.MySql", "demo")' openstack congress policy rule create murano_system 'allowedApp("io.murano.apps.java.PetClinic", "demo")' openstack congress policy rule create murano_system 'allowedApp("io.murano.apps.apache.Tomcat", "demo")' openstack congress policy rule create murano_system 'allowedApp("io.murano.databases.PostgreSql", "qa")' openstack congress policy rule create murano_system 'allowedApp("io.murano.apps.java.PetClinic", "qa")' openstack congress policy rule create murano_system 'allowedApp("io.murano.apps.apache.Tomcat", "qa")'
 * 1) Predeploy policy rules
 * 1) allowed app main rules
 * 1) demo tenant
 * 1) qa tenant

openstack congress policy rule create murano_system 'vm_of_murano_instance(oid, server_id) :- murano:parent_types(oid, "io.murano.resources.Instance"), murano:properties(oid, "ipAddresses", ip_addr), neutronv2:fixed_ips(port_id=port_id, ip_address=ip_addr),neutronv2:ports(id=port_id, device_id=server_id ), nova:servers(id=server_id)' openstack congress policy rule create murano_system 'not_running_vm_of_murano_instance(oid, server_id) :- vm_of_murano_instance(oid, server_id), nova:servers(id=server_id,status=status), not equal(status,"ACTIVE")' openstack congress policy rule create murano_system 'vm_of_murano_instance2(oid) :- vm_of_murano_instance(oid,sid)' openstack congress policy rule create murano_system 'not_running_vm_of_murano_instance2(oid) :- not_running_vm_of_murano_instance(oid,sid)' openstack congress policy rule create murano_system 'murano_instance_vm_status(oid, vm_status) :- vm_of_murano_instance(oid, vm_id), nova:servers(id=vm_id,status=vm_status)' openstack congress policy rule create murano_system 'murano_instance_vm_status(oid, " ") :- murano:parent_types(oid,"io.murano.resources.Instance"), not vm_of_murano_instance2(oid)' openstack congress policy rule create murano_system 'murano_env_of_object(oid,eid) :- murano:connected(eid,oid), murano:objects(eid,tid,"io.murano.Environment")' openstack congress policy rule create murano_system 'error(oid,eid,vmstatus) :- murano:parent_types(oid,"io.murano.resources.Instance"), murano_env_of_object(oid,eid), murano_instance_vm_status(oid,vmstatus), not equal(vmstatus,"ACTIVE")' openstack congress policy rule create murano_system 'error_msg(oid,eid,msg) :- error(oid,eid,vmstatus), murano:properties(eid,"name",ename), concat("Environment ",ename,msg1), murano:properties(oid,"name",iname), concat(msg1,", Instance ",msg2), concat(msg2, iname, msg3), concat(msg3, " has not running VM - VM status: ", msg4), concat(msg4,vmstatus,msg)'
 * 1) Post-deployment rules
 * 1) returns list of Instance-VM
 * 1) return list of Instance-VM where VM is not running (is not ACTIVE)
 * 1) find murano instances with vm (for not)
 * 1) find murano instances with not running vm (for not)
 * 1) get server status
 * 1) get environment of oid
 * 1) error - murano instance w/o vm, or instance in not ACTIVE status
 * 1) error_msg - like error, but it provides text message with the error

Application Predeployment Scenario
* The cloud admin(s) sets 2 policy rules: ** Tenant Demo can only use MySQL as a DB vendor ** Tenant QA can only use PostgreSQL as a DB vendor Done by creating policy rules in setup guide.

* The admin wishes to model a Tomcat application that depends on a DB ** Tenant Demo admin models it with a MySQL DB ** Tenant QA admin models it with a PostgreSQL DB Done by uploading application packages

* The admin deploys the application models to Murano. The package will also include the Mistral workflow (so we will not depend on it being pre-deployed to Mistral). Application developer wants to add a workflow to the application deployment that will verify the Tomcat is up and running (by sending a GET request) and will send an email notifying whether the deployment has completed successfully or not. Then it will insert some initial data to the application (4 new owners) by calling REST API. The uploading of a flow is done as part of the application PetClinic.yaml. After the upload, the flow is called - it checks the status of Tomcat, sends email and inserts data. The email address that is used is: policyguideddemo@gmail.com

* End user of tenant Demo wishes to deploy the Tomcat application that relies on PostgreSQL ** Deployment fails since the policy rules restrict this DB for tenant Demo Environment is created as follow by user demo:
 * Add Application Pet Clinic into environment
 * Add New Database
 * Choose PostgreSql database
 * On Instance panel use Ubuntu VM image
 * End-of-Database
 * Add New Container
 * On Instance panel use Centos VM image
 * End-of-Container
 * On database panel enter password
 * End-of-Application

* End user of tenant Demo asks Murano to deploy tomcat with MySQL DB ** Application is deployed *** At the end of deployment phase, the workflow is deployed to Mistral (programmatically, by this specific application deployment logic) *** The workflow is triggered by the application class **** data are populated into DB **** mail will be sent to the end user indicating whether Tomcat has been deployed and whether it is responsive, if yes then init data will be inserted by the flow. Environment is created as follow by user demo:
 * Add Application Pet Clinic into environment
 * Add New Database
 * Choose MySQL database
 * On Instance panel use Ubuntu VM image
 * End-of-Database
 * Add New Container
 * On Instance panel use Centos VM image
 * End-of-Container
 * On database panel enter password
 * End-of-Application

After deployment you can access PetClinic application in browser (if you have configured float IPs, and you checked floating IP assignement on Tomcat component). URL is in the log of PetClinic component - in environment page, click on PetClinic in component list, then click on Tab Logs

Application Postdeployment Scenario
* When application is deployed, admin user creates error rules to check if deployed application's infrastructure is correct ** rule will check if VMs created by Murano instances exist or are running * Admin user pauses VM instance of deployed application using Horizon UI * Admin user then will execute the policy rule to see the invalid application * Admin user resumes VM instance of deployed application using Horizon UI * Admin user then will execute the policy rule to check that instances are runningSteps $ . ~/devstack/openrc demo demo $ openstack congress policy simulate murano_system 'error_msg(oid,eid,msg)' ' ' murano_system $ . ~/devstack/openrc demo demo $ openstack congress policy simulate murano_system 'error_msg(oid,eid,msg)' ' ' murano_system error_msg("3e3e0465-d565-431e-bb0d-e505476993c8", "9c4adde90fb94bc3ba2720e638027bb6", "Environment petclinic, Instance epabxi9llzvf91 has not running VM - VM status: PAUSED") $ . ~/devstack/openrc demo demo $ openstack congress policy simulate murano_system 'error_msg(oid,eid,msg)' ' ' murano_system
 * Policy rules are created policy rules in setup guide.
 * Admin user or demo user executes following Congress query, which shall returns empty result (as all VMs of Murano instances are running)
 * Admin user or demo user pauses VM instances using Horizon UI/Project/Compute/Instance
 * Admin user or demo user  executes following Congress query, which shall returns ID of Murano object representing instance and ID of corresponding Murano environment
 * Admin user or demo user resumes VM instances using Horizon UI/Project/Compute/Instance
 * Admin user or demo user  executes following Congress query, which shall returns empty result (as all VMs of Murano instances are running)

Automatic Remediation Scenario
$ . ~/devstack/openrc demo demo $ openstack congress policy rule create murano_system 'execute[ nova:servers.unpause(server_id) ] :- murano_instance_vm_status(oid, vm_status), equal(vm_status,"PAUSED"), not_running_vm_of_murano_instance(oid, server_id)'
 * When application is deployed, admin user creates rule which unpauses VM instances which is paused


 * Admin user pauses VM instance of deployed application using Horizon UI
 * Congress will unpauses the paused VM Instance automatically as defined in the rule
 * Admin user checks that VM instance is running using Horizon UI

Remove the rule using $ . ~/devstack/openrc demo demo $ openstack congress policy rule delete murano_system 

Redeployment Scenario
* The cloud admin(s) sets policy rule (predeployment enforcement): ** Tenant Demo can only use MySQL as a DB vendor Done by creating policy rules in setup guide.

* End user creates environment with PostgreSQL DB in "Demo" tenant Environment is created as follow by user demo:
 * Add Database
 * Choose PostgreSql database
 * On Instance panel use Ubuntu VM image
 * End-of-Database

* End user deploys environment, but it fails due to predeployment enforcement rule above User "demo" clicks on "deploy"

* Environment is populated to Congress using datasource driver It can be checked by following CLI

$ openstack congress datasource row list murano objects

* End user edits environment ** remove PostgreSQL DB ** adds MySQL DB Done by Murano UI

* End User redeploys environment User "demo" clicks on "deploy"

** predeployment enforcement rules allows redeployment Environment is deployed.