Difference between revisions of "Congress"

Revision as of 23:36, 28 April 2014

Policy as a service ("Congress")

• Source code
• Overall design doc
• Data integration design doc

Mission

Congress is an OpenStack project to provide policy as a service across any collection of cloud services in order to offer governance and compliance for dynamic infrastructures.

Why Congress

IT services will always be governed and brought into compliance with business-level policies.

In the past, policy was enforced manually, e.g. by someone sending an email asking for an application to be added to the network, secured by specific firewall entries, connected to an agreed-on storage, and so on. In the cloud era, IT has become more agile: users expect immediate delivery of services, a level of responsiveness that is unattainable by the team responsible for governance. Hence, manual enforcement is no longer feasible.

Both enterprises and vendors have fielded engines for enforcing policy (semi)-automatically, creating a fragmented market where enterprises reinvent the wheel while maintaining their own code, and vendors fail to meet enterprise needs, either for technical reasons or because their solutions require vertical integration and lock-in.

The Congress policy service enables IT services to extend their OpenStack footprint by onboarding new applications while keeping the strong compliance and governance dictated by their own business policies. All of that leveraging a community-driven implementation in which vendors can plug into a common interface.

What is Congress

Congress aims to provide an extensible open-source framework for governance and regulatory compliance across any cloud services (e.g. application, network, compute and storage) within a dynamic infrastructure. It is a cloud service whose sole responsibility is policy enforcement.

Congress aims to include the following functionality:

• Allow cloud administrators and tenants to use a high-level, general purpose, declarative language to describe business logic. The policy language does not include a fixed collection of policy types or built-in enforcement mechanisms; rather, a policy simply defines which states of the cloud are in compliance and which are not, where the state of the cloud is the collection of data provided by the cloud services available to Congress. Some examples:
  • Application A is only allowed to communicate with application B.
  • Virtual machine owned by tenant A should always have a public network connection if tenant A is part of the group B.
  • Virtual machine A should never be provisioned in a different geographic region than storage B.
• Offer a pluggable architecture that connects to any collection of cloud services
• Enforce policy
  • Proactively: preventing violations before they occur
  • Reactively: correcting violations after they occur
  • Interactively: give administrators insight into policy and its violations, e.g. identifying violations, explaining their causes, computing potential remediations, simulating a sequence of changes.

Policy Language

The policy language for Congress is Datalog, which is basically SQL but with a syntax that is closer to traditional programming languages. This declarative language was chosen because its semantics are well-known to a broad range of DevOps, yet its syntax is more terse making it better suited for expressing real-world policies. The grammar is given below.

<policy> ::= <rule>*
<rule> ::= <atom> COLONMINUS <literal> (COMMA <literal>)*
<literal> ::= <atom>
<literal> ::= NOT <atom>
<atom> ::= TABLENAME LPAREN <term> (COMMA <term>)* RPAREN
<term> ::= INTEGER | FLOAT | STRING | VARIABLE

Use Cases

Below we list use cases and the Congress release at which we hope to support that use case. Please feel free to add additional use cases that you would like Congress to support.

Roadmap

Here is a list of features that would be useful in future releases. We have not prioritized or assigned any of these, so if any of you developers think they sound fun, let us know and we can explain more.

• Basic Policy language implementation
  • Multi-threaded Datalog implementation
  • Bottom-up datalog evaluation
  • Query optimization
  • Materialized implementation: automated view selection/inlining

• Enhanced Policy language
  • ActiveDirectory facade
  • Syntax improvements (modals like insert/delete)
  • Support for compound objects (safe, stratified recursion)
  • Richer support for describing actions in Enforcement policy
  • Modules

• Policy structure
  • Multi-tenant
  • Multi-stakeholder (tenants, finance, operations, etc.)

• Enforcement
  • Execution of (rich) actions
  • Carve out subpolicies and push to other components, e.g. Neutron
  • Add consultation with Congress to other OS components, e.g. Nova/Neutron
  • Proper remediation enumeration with Classification+Action policies
  • Find ways of automatically choosing the proper remediation strategy (e.g. priorities/monotonicity)
  • Give cloud owner way of configuring how proactive/reactive/etc. based on information from separate policies.

• Libraries
  • Data source drivers for common OS and non-OS components
  • HIPAA, etc. encoding
  • Ontologies for different sectors, e.g. finance

• Policy Analysis
  • Look for infinite loops through Enforcement policy (using Action policy)
  • Compare Access control policy and Classification policy for redundancy
  • Change impact analysis

• Dashboard
  • IDE for policy (different levels: raw-Datalog, AD, checkbox-based)
  • List violations
  • Explain violations (step-by-step tracing thru policy)
  • Simulate state change and action execution
  • Enumerate remediations for a given violation

• Architecture and API
  • Formalize and implement full introspection and query APIs
  • Distribute across multiple nodes
  • Ensure Congress can use another Congress instance as data

• Authentication and Access Control
  • Add authentication and access controls on API/dashboard
  • When remediating, which user(s) are executing actions? Does Congress need admin credentials for all its cloud services or are user credentials part of actions? Need proper storage for those credentials. Etc.

Relationship to Other Projects

Related OpenStack components

• Keystone: Keystone is an identity service providing authentication and high-level authorization for OpenStack. Congress can leverage Keystone as an input for policies. For example, an auditor might want to ensure that the running system is consistent with current Keystone authorization decisions.

• Heat: Heat is an orchestration service for application provisioning and lifecycle management. Congress can ensure that applications managed by Heat are consistent with business policy.

• Mistral: Keystone is a task management service, or in other words Workflow as a service. Its primary use cases include Cron-for-the-cloud, execution of long-running tasks, and big data analysis. Congress could potentially utilize Mistral to execute actions that bring the cloud back into policy compliance.

Policy Initiatives

• SolverScheduler (Nova blueprint): The SolverScheduler provides an interface for using different constraint solvers to solve placement problems for Nova. Depending on how it is applied, could be used for initial provisioning, re-balancing loads, or both.

• Runtime Policies (Nova blueprint): The Runtime Policies proposal aims to enforce several specific types of policies (Load Balancing, High Availability, Energy consumption) at run-time, dynamically rebalancing VM deployment so as to stay in compliance with the stated policy. Some proof of concepts have been implemented and are looking for a home project, as of early March 2014.

• Gantt: A scheduler framework for use by different OpenStack components. Used to be a subgroup of Nova and focused on scheduling VMs based on resource utilization. Includes plugin framework for making arbitrary metrics available to the scheduler.

• Neat: The Neat project aims to rebalance VMs based on load. Conceptually, this appears to be subsumed by the Runtime Policies effort.

• Neutron policy group: This group aims to add a policy API to Neutron, where tenants express policy between groups of networks, ports, etc., and that policy is enforced. Policy statements are of the form "for every network flow between groups A and B that satisfies these conditions, apply a constraint on that flow". The constraints that can be enforced on a flow will grow as the enforcement engine matures; currently, the constraints are Allow and Deny, but there are plans for quality-of-service constraints as well.

• Open Attestation: This project provides an SDK for verifying host integrity. It provides some policy-based management capabilities, though documentation is limited.

Congress is complementary to all of these efforts. It is intended to be a general-purpose policy component and hence could (potentially) express any of the policies described above. However, to enforce those policies Congress would carve off subpolicies and send them to the appropriate enforcement point in the cloud. For example, Congress might carve off the compute Load Balancing policy and send it to the Runtime Policy engine and/or carve off the networking policy and sending it to the Neutron policy engine. Part of the goal of Congress is to give administrators a single place to write and inspect the policy being enforced throughout the datacenter or cloud, distribute the relevant portions of that policy to all of the available enforcement points, and monitor the state of the cloud to let administrators know if the cloud is in compliance. (Congress also attempts to correct policy violations when they occur, but optimization policies such as many of those addressed above will not be enforceable by Congress directly.)

[Thanks to Jay Lau, Gokul Kandiraju, and others on the mailing list for help compiling this list.]