Difference between revisions of "Xenapi-security-groups"
| Line 1: | Line 1: | ||
__NOTOC__ | __NOTOC__ | ||
| − | + | = XenAPI support for security groups = | |
| − | |||
* '''Launchpad Entry''': [[NovaSpec]]:xenapi-security-groups | * '''Launchpad Entry''': [[NovaSpec]]:xenapi-security-groups | ||
* '''Created''': [https://launchpad.net/~salvatore-orlando Salvatore Orlando] | * '''Created''': [https://launchpad.net/~salvatore-orlando Salvatore Orlando] | ||
* '''Contributors''': [https://launchpad.net/~salvatore-orlando Salvatore Orlando] & everybody else who wants to join! | * '''Contributors''': [https://launchpad.net/~salvatore-orlando Salvatore Orlando] & everybody else who wants to join! | ||
| + | |||
== Summary == | == Summary == | ||
| − | + | This blueprint concerns the implementation of the security groups functionality in Openstack's compute driver for XenAPI. This will fill one of the existing gaps between XenAPI and libvirt. | |
| − | This blueprint concerns the implementation of the security groups functionality in Openstack's compute driver for XenAPI. | ||
| − | This will fill one of the existing gaps between XenAPI and libvirt. | ||
== Background == | == Background == | ||
| − | + | For an explanation of the concept of security group, please have a look at the [http://nova.openstack.org/nova.concepts.html nova concepts page] Related launchpad blueprint: nova:xenapi-security-groups | |
| − | For an explanation of the concept of security group, please have a look at the [http://nova.openstack.org/nova.concepts.html nova concepts page] | ||
| − | Related launchpad blueprint: nova:xenapi-security-groups | ||
This feature has been implemented for the libvirt backend since the early days of nova. Two distinct drivers are provided (called Firewall Drivers): one based on iptables and one on nwfilter.In particular the iptables driver uses the nwfilter driver for enforcing basic filtering rules, which are the equivalent of XenServer's [https://github.com/openstack/nova/blob/master/plugins/xenserver/doc/networking.rst VIF isolation rules]. | This feature has been implemented for the libvirt backend since the early days of nova. Two distinct drivers are provided (called Firewall Drivers): one based on iptables and one on nwfilter.In particular the iptables driver uses the nwfilter driver for enforcing basic filtering rules, which are the equivalent of XenServer's [https://github.com/openstack/nova/blob/master/plugins/xenserver/doc/networking.rst VIF isolation rules]. | ||
| Line 22: | Line 18: | ||
== Use cases == | == Use cases == | ||
| − | |||
# A user starts an instance which is associated with one or more security groups (association via project). nova-compute enforces the security group(s)' rules on the host were the instance is spawned; | # A user starts an instance which is associated with one or more security groups (association via project). nova-compute enforces the security group(s)' rules on the host were the instance is spawned; | ||
# A user creates/updates/delete a rule for a security group. nova-compute ensure the rules enforced for this security groups for all interested instances are in sync with the ones specified by the user. | # A user creates/updates/delete a rule for a security group. nova-compute ensure the rules enforced for this security groups for all interested instances are in sync with the ones specified by the user. | ||
| Line 28: | Line 23: | ||
== Requirements == | == Requirements == | ||
| + | R1) XS/XCP uses Linux Bridge networking stack, security groups should be configured through iptables rules. R2A) XS/XCP uses Open vSwitch networking stack, security groups still configured through iptables R2B) XS/XCP uses Open vSwitch networking stack, security groups are configured through flow tables in Open vSwitch R3) Security groups must be consistent across instance reboots R4) Security groups must be re-enforced when a host become active after having been down for any reason (re-initializiation) R5) iptables rules or OVS flow entries for a security group associated with an instance must be cleaned up when the instance is terminated or suspended. R6) CRUD operations on a security group must re-enforce security groups on instances to ensure that iptables rules or OVS flow entries are in sync with the database | ||
| − | + | R3-R6 can be satisfied by ensuring the security groups are correctly enforced when relevant events in the Compute or API service occur (e.g.: instance reboot, or security group deletion). R1-R2 can be satisfied by providing two distinct drivers for implementing security groups: one based on iptables, and one based on Open vSwitch. | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | R3-R6 can be satisfied by ensuring the security groups are correctly enforced when relevant events in the Compute or API service occur (e.g.: instance reboot, or security group deletion). | ||
| − | R1-R2 can be satisfied by providing two distinct drivers for implementing security groups: one based on iptables, and one based on Open vSwitch. | ||
== Implementation == | == Implementation == | ||
| − | |||
Implementation is currently progress. Before pushing for review on gerrit, we could share code on launchpad or github. | Implementation is currently progress. Before pushing for review on gerrit, we could share code on launchpad or github. | ||
| Line 48: | Line 34: | ||
Most of the code already implemented for libvirt can be very useful for [[XenServer]] as well. We need however to keep in mind that, unlike libvirt, in XenAPI we need to enforce the rules in dom0, whereas nova-compute is running in a VM. To this aim, the developers of the libvirt feature have been very helpful, as they provided a mechanism to provide the iptables manager with an arbitrary command execution mechanism. Basically this would allow us to replace local subprocess.Popen calls with calls to an appropriate XenAPI plugin. | Most of the code already implemented for libvirt can be very useful for [[XenServer]] as well. We need however to keep in mind that, unlike libvirt, in XenAPI we need to enforce the rules in dom0, whereas nova-compute is running in a VM. To this aim, the developers of the libvirt feature have been very helpful, as they provided a mechanism to provide the iptables manager with an arbitrary command execution mechanism. Basically this would allow us to replace local subprocess.Popen calls with calls to an appropriate XenAPI plugin. | ||
| − | = Changes to Firewall Driver Structure = | + | === Changes to Firewall Driver Structure === |
| − | + | * Move the abstract class FirewallDriver from nova.virt.libvirt into nova.virt | |
| − | * Move the abstract class | ||
* Remove KVM items (nwfilter) for IPTablesDriver and 'promote' it into nova.virt | * Remove KVM items (nwfilter) for IPTablesDriver and 'promote' it into nova.virt | ||
* Make sure the IPTablesDriver in nova.virt.libvirt extends from the one in nova.virt and add the KVM items (nwfilter) | * Make sure the IPTablesDriver in nova.virt.libvirt extends from the one in nova.virt and add the KVM items (nwfilter) | ||
| Line 57: | Line 42: | ||
Tasks not yet started: | Tasks not yet started: | ||
| − | * Develop a new Firewall Driver in nova.virt based on Open vSwitch. | + | * Develop a new Firewall Driver in nova.virt based on Open vSwitch. ''We should not put it in nova.virt.xenapi'', we want to keep it generic enough so that it could work on libvirt as well with minor change. |
| − | |||
| − | |||
| + | === Command execution on dom0 through XenAPI plugin === | ||
We actually need to run the following commands: | We actually need to run the following commands: | ||
| Line 70: | Line 54: | ||
We can operate on the xenhost plugin, which already contains a _runcommand function. This function executes a command in dom0. | We can operate on the xenhost plugin, which already contains a _runcommand function. This function executes a command in dom0. | ||
''Note #1: _runcommand is not exposed directly by the plugin (that would have been rather surprising).'' | ''Note #1: _runcommand is not exposed directly by the plugin (that would have been rather surprising).'' | ||
| − | ''Note #2: _runcommand command uses subprocess.Popen with shell=True. This is normally unsafe. It is fine in this context so far as the 'args' parameter to Popen'' was always a single-item list. For our job we might need multiple arguments list (also for alignment with nova's utils.execute). shell parameter should be set to False. | + | ''Note #2: _runcommand command uses subprocess.Popen with shell=True. This is normally unsafe. It is fine in this context so far as the 'args' parameter to Popen'' was always a single-item list. ''For our job we might need multiple arguments list (also for alignment with nova's utils.execute). The 'shell' parameter should be set to False. '' |
| − | * Modifiy _runcommand for improving security and reading process input from STDIN | + | * <s>Modifiy _runcommand for improving security and reading process input from STDIN</s> |
| − | * Expose the commands listed above through xenapi 'xenhost' plugin | + | * <s>Expose the commands listed above through xenapi 'xenhost' plugin</s> |
| − | * iptables-save and iptables-restore | + | ** <s>iptables-save and iptables-restore</s> |
| − | * ovs-ofctl and ovs-vsctl | + | ** ovs-ofctl and ovs-vsctl |
| − | Tasks not yet started: | + | Tasks not yet started: |
* Create an execute function to be passed to the [[IpTables]]/OVS manager which calls the xenapi plugin | * Create an execute function to be passed to the [[IpTables]]/OVS manager which calls the xenapi plugin | ||
| − | = Changes to XenAPI virt driver = | + | === Changes to XenAPI virt driver === |
| − | |||
We need to add calls for enforcing/destroying security groups on: | We need to add calls for enforcing/destroying security groups on: | ||
| Line 94: | Line 77: | ||
Note: we also need to react when security groups are edited using the EC2 API/ | Note: we also need to react when security groups are edited using the EC2 API/ | ||
| − | = Testing = | + | === Testing === |
| − | |||
* Unit tests throughout. Empirically verify code coverage as close as possible to 100%, stub-out only things that necessarily have to be stubbed out (plugin calls). | * Unit tests throughout. Empirically verify code coverage as close as possible to 100%, stub-out only things that necessarily have to be stubbed out (plugin calls). | ||
* Functional tests covering the use cases discussed in this page. | * Functional tests covering the use cases discussed in this page. | ||
Revision as of 08:45, 14 October 2011
XenAPI support for security groups
- Launchpad Entry: NovaSpec:xenapi-security-groups
- Created: Salvatore Orlando
- Contributors: Salvatore Orlando & everybody else who wants to join!
Summary
This blueprint concerns the implementation of the security groups functionality in Openstack's compute driver for XenAPI. This will fill one of the existing gaps between XenAPI and libvirt.
Background
For an explanation of the concept of security group, please have a look at the nova concepts page Related launchpad blueprint: nova:xenapi-security-groups
This feature has been implemented for the libvirt backend since the early days of nova. Two distinct drivers are provided (called Firewall Drivers): one based on iptables and one on nwfilter.In particular the iptables driver uses the nwfilter driver for enforcing basic filtering rules, which are the equivalent of XenServer's VIF isolation rules.
Also note that both drivers also provide another feature, the Provider Firewall Rules; this feature is currently not supported in XenAPI, and will be implemented on top of the functionality offered by the Security groups (blueprint nova:xenapi-provider-firewall)
As of the diablo release Security Groups are currently managed through the EC2 API only, and cannot be maanged through the dashboard.
Use cases
- A user starts an instance which is associated with one or more security groups (association via project). nova-compute enforces the security group(s)' rules on the host were the instance is spawned;
- A user creates/updates/delete a rule for a security group. nova-compute ensure the rules enforced for this security groups for all interested instances are in sync with the ones specified by the user.
- A user creates a new security groups and associates it with one or more project. The rules for this security groups are then enforced on all the hosts where interested instances are running.
Requirements
R1) XS/XCP uses Linux Bridge networking stack, security groups should be configured through iptables rules. R2A) XS/XCP uses Open vSwitch networking stack, security groups still configured through iptables R2B) XS/XCP uses Open vSwitch networking stack, security groups are configured through flow tables in Open vSwitch R3) Security groups must be consistent across instance reboots R4) Security groups must be re-enforced when a host become active after having been down for any reason (re-initializiation) R5) iptables rules or OVS flow entries for a security group associated with an instance must be cleaned up when the instance is terminated or suspended. R6) CRUD operations on a security group must re-enforce security groups on instances to ensure that iptables rules or OVS flow entries are in sync with the database
R3-R6 can be satisfied by ensuring the security groups are correctly enforced when relevant events in the Compute or API service occur (e.g.: instance reboot, or security group deletion). R1-R2 can be satisfied by providing two distinct drivers for implementing security groups: one based on iptables, and one based on Open vSwitch.
Implementation
Implementation is currently progress. Before pushing for review on gerrit, we could share code on launchpad or github.
Current focus is on IPTables driver, before addressing OVS driver we need to clarify whether port ranges can be handled as set of flow entries or explicit OVS support is required.
Most of the code already implemented for libvirt can be very useful for XenServer as well. We need however to keep in mind that, unlike libvirt, in XenAPI we need to enforce the rules in dom0, whereas nova-compute is running in a VM. To this aim, the developers of the libvirt feature have been very helpful, as they provided a mechanism to provide the iptables manager with an arbitrary command execution mechanism. Basically this would allow us to replace local subprocess.Popen calls with calls to an appropriate XenAPI plugin.
Changes to Firewall Driver Structure
- Move the abstract class FirewallDriver from nova.virt.libvirt into nova.virt
- Remove KVM items (nwfilter) for IPTablesDriver and 'promote' it into nova.virt
- Make sure the IPTablesDriver in nova.virt.libvirt extends from the one in nova.virt and add the KVM items (nwfilter)
- Make sure that the unit tests still pass
Tasks not yet started:
- Develop a new Firewall Driver in nova.virt based on Open vSwitch. We should not put it in nova.virt.xenapi, we want to keep it generic enough so that it could work on libvirt as well with minor change.
Command execution on dom0 through XenAPI plugin
We actually need to run the following commands:
- iptables-restore for enforcing security groups (configuration read from stdin)
- iptables-save for persisting security groups
- ovs-ofctl for manipulating flow entries
- ovs-vsctl for finding bridges and interfaces
We can operate on the xenhost plugin, which already contains a _runcommand function. This function executes a command in dom0. Note #1: _runcommand is not exposed directly by the plugin (that would have been rather surprising). Note #2: _runcommand command uses subprocess.Popen with shell=True. This is normally unsafe. It is fine in this context so far as the 'args' parameter to Popen was always a single-item list. For our job we might need multiple arguments list (also for alignment with nova's utils.execute). The 'shell' parameter should be set to False.
-
Modifiy _runcommand for improving security and reading process input from STDIN -
Expose the commands listed above through xenapi 'xenhost' plugin-
iptables-save and iptables-restore - ovs-ofctl and ovs-vsctl
-
Tasks not yet started:
- Create an execute function to be passed to the IpTables/OVS manager which calls the xenapi plugin
Changes to XenAPI virt driver
We need to add calls for enforcing/destroying security groups on:
- Instance creation
- Instance reboot
- Instance termination
- Instance suspension
- Host initialisation (enfore security groups on all instances running on host)
The above list has been redacted started from security groups hooks in libvirt drivers.
Note: we also need to react when security groups are edited using the EC2 API/
Testing
- Unit tests throughout. Empirically verify code coverage as close as possible to 100%, stub-out only things that necessarily have to be stubbed out (plugin calls).
- Functional tests covering the use cases discussed in this page.
