Difference between revisions of "Congress"

Revision as of 15:00, 31 March 2014

Policy as a service ("Congress")

• Source code
• Overall design doc
• Data integration design doc

Mission

Congress is an OpenStack project to provide policy as a service across any collection of cloud services in order to offer governance and compliance for dynamic infrastructures.

Why Congress

IT services will always be governed and brought into compliance with business-level policies.

In the past, policy was enforced manually, e.g. by someone sending an email asking for an application to be added to the network, secured by specific firewall entries, connected to an agreed-on storage, and so on. In the cloud era, IT has become more agile: users expect immediate delivery of services, a level of responsiveness that is unattainable by the team responsible for governance. Hence, manual enforcement is no longer feasible.

Both enterprises and vendors have fielded engines for enforcing policy (semi)-automatically, creating a fragmented market where enterprises reinvent the wheel while maintaining their own code, and vendors fail to meet enterprise needs, either for technical reasons or because their solutions require vertical integration and lock-in.

The Congress policy service enables IT services to extend their OpenStack footprint by onboarding new applications while keeping the strong compliance and governance dictated by their own business policies. All of that leveraging a community-driven implementation in which vendors can plug into a common interface.

What is Congress

Congress aims to provide an extensible open-source framework for governance and regulatory compliance across any cloud services (e.g. application, network, compute and storage) within a dynamic infrastructure. It is a cloud service whose sole responsibility is policy enforcement.

Congress aims to include the following functionality:

• Allow cloud administrators and tenants to use a high-level, general purpose, declarative language to describe business logic. The policy language does not include a fixed collection of policy types or built-in enforcement mechanisms; rather, a policy simply defines which states of the cloud are in compliance and which are not, where the state of the cloud is the collection of data provided by the cloud services available to Congress. Some examples:
  • Application A is only allowed to communicate with application B.
  • Virtual machine owned by tenant A should always have a public network connection if tenant A is part of the group B.
  • Virtual machine A should never be provisioned in a different geographic region than storage B.
• Offer a pluggable architecture that connects to any collection of cloud services
• Enforce policy
  • Proactively: preventing violations before they occur
  • Reactively: correcting violations after they occur
  • Interactively: give administrators insight into policy and its violations, e.g. identifying violations, explaining their causes, computing potential remediations, simulating a sequence of changes.

Policy Language

The policy language for Congress is Datalog, which is basically SQL but with a syntax that is closer to traditional programming languages. This declarative language was chosen because its semantics are well-known to a broad range of DevOps, yet its syntax is more terse making it better suited for expressing real-world policies. The grammar is given below.

<policy> ::= <rule>*
<rule> ::= <atom> COLONMINUS <literal> (COMMA <literal>)*
<literal> ::= <atom>
<literal> ::= NOT <atom>
<atom> ::= TABLENAME LPAREN <term> (COMMA <term>)* RPAREN
<term> ::= INTEGER | FLOAT | STRING | VARIABLE

Use Cases

Below we list use cases and the Congress release at which we hope to support that use case. Please feel free to add additional use cases that you would like Congress to support.

Roadmap

• Formalize and implement full introspection and query APIs
• Integrate with OpenStack services
  • Reactive enforcement: listen to RabbitMQ bus so that we can respond to violations that have already occurred.
  • Proactive enforcement: enable Keystone-like interception of API calls to stop violations before they occur.
• Investigate and improve performance and scalability
• Develop dashboard for cloud administrators

Relationship to Other Projects

Related OpenStack components

• Keystone: Keystone is an identity service providing authentication and high-level authorization for OpenStack. Congress can leverage Keystone as an input for policies. For example, an auditor might want to ensure that the running system is consistent with current Keystone authorization decisions.

• Heat: Heat is an orchestration service for application provisioning and lifecycle management. Congress can ensure that applications managed by Heat are consistent with business policy.

Policy Initiatives

• SolverScheduler (Nova blueprint): The SolverScheduler provides an interface for using different constraint solvers to solve placement problems for Nova. Depending on how it is applied, could be used for initial provisioning, re-balancing loads, or both.

• Runtime Policies (Nova blueprint): The Runtime Policies proposal aims to enforce several specific types of policies (Load Balancing, High Availability, Energy consumption) at run-time, dynamically rebalancing VM deployment so as to stay in compliance with the stated policy. Some proof of concepts have been implemented and are looking for a home project, as of early March 2014.

• Gantt: A scheduler framework for use by different OpenStack components. Used to be a subgroup of Nova and focused on scheduling VMs based on resource utilization. Includes plugin framework for making arbitrary metrics available to the scheduler.

• Neat: The Neat project aims to rebalance VMs based on load. Conceptually, this appears to be subsumed by the Runtime Policies effort.

• Neutron policy group: This group aims to add a policy API to Neutron, where tenants express policy between groups of networks, ports, etc., and that policy is enforced. Policy statements are of the form "for every network flow between groups A and B that satisfies these conditions, apply a constraint on that flow". The constraints that can be enforced on a flow will grow as the enforcement engine matures; currently, the constraints are Allow and Deny, but there are plans for quality-of-service constraints as well.

• Open Attestation: This project provides an SDK for verifying host integrity. It provides some policy-based management capabilities, though documentation is limited.

Congress is complementary to all of these efforts. It is intended to be a general-purpose policy component and hence could (potentially) express any of the policies described above. However, to enforce those policies Congress would carve off subpolicies and send them to the appropriate enforcement point in the cloud. For example, Congress might carve off the compute Load Balancing policy and send it to the Runtime Policy engine and/or carve off the networking policy and sending it to the Neutron policy engine. Part of the goal of Congress is to give administrators a single place to write and inspect the policy being enforced throughout the datacenter or cloud, distribute the relevant portions of that policy to all of the available enforcement points, and monitor the state of the cloud to let administrators know if the cloud is in compliance. (Congress also attempts to correct policy violations when they occur, but optimization policies such as many of those addressed above will not be enforceable by Congress directly.)

[Thanks to Jay Lau, Gokul Kandiraju, and others on the mailing list for help compiling this list.]