MoveIpAllocation

The current architecture of networking in nova and why the allocation should be moved was explained in an email that has been included below.

Current

Currently, there are three strategies for networking, implemented by different managers: FlatManager -- ip addresses are grabbed from a network and injected into the image on launch. All instances are attached to the same manually configured bridge. FlatDHCPManager -- ip addresses are grabbed from a network, and a single bridge is created for all instances. A dhcp server is started to pass out addresses VlanManager -- each project gets its own vlan, bridge and network. A dhcpserver is started for each vlan, and all instances are bridged into that vlan.

The implementation of creating bridges, vlans, dhcpservers, and firewall rules is done by the driver linux_net. This layer of abstraction is so that we can at some point support configuring hardware switches etc. using the same managers.

My goal with the Manager refactor was to move all of the code relating to each component into the Manager classes. For example, all code relating to networks/ips would be done by NetworkManager. That said, code relating to businesss objects needs to be run by three separate workers, and not all of it has been encapsulated in the manager. For example, the initial creation of the record for instances/volumes is done by nova-api by hitting the db directly instead of using a method on the manager.

Here is some horrible ascii art of what each component does during run instance. The manager's methods are called directly for local setup, and they are called via rpc to nova-network which runs the manager wrapped by a service which exposes public methods to rpc. The *d items aren't specifically related to networking but help make it clear when things occur +----------------+ +----------------+ +----------------+ | nova-api | | nova-network | | nova-compute |

+ +----------------+ +----------------+

(direct manager)            (service)             (direct manager)

create instance in db

find correct network host (get_network) call (if no host) -----> sets up network

                        (set_network_host)

<----------------------- returns host allocates fixed ip (allocate_fixed_ip) cast-------------------> sets up fixed ip V (setup_fixed_ip) cast---------~~goes through scheduler~~-------> (ComputeManager.run_instance) | sets up compute network | (setup_compute_network) | * launches instance | sets up security groups V (done by compute driver)

return instance data

This all works and allows us to return an ip very quickly from an api call, but it does have some issues. Most importantly, in some network models, the api server may not have enough information to know how to allocate an ip. For example, in the flat mode, we may want to have a cluster of compute nodes associated with a particular network host, and get_network returns a network based on the compute host instead of the project as we do in vlan mode.

Future (IMO)

I think we should move the allocating of the ip further down the stack like so:

+ +----------------+ +----------------+

(direct manager)            (service)             (direct manager)

create instance in db

check for available network resources (check_free_ips?) cast---------~~goes through scheduler~~-------> (ComputeManager.run_instance) | find correct network host | (get_network) | sets up network <------- call (if no host) | (set_network_host) | -----------------------> returns host | sets up compute network | (setup_compute_network) | allocates fixed ip <---- call (because we're not in a rush) | (allocate_fixed_ip) | (setup_fixed_ip) | -----------------------> returns fixed_ip | * launches instance | sets up security groups V (done by compute driver)

return instance data (with no ip address)

It is also possible to do some of this in scheduler instead of compute if we want to keep compute really dumb in terms of communicating over the queue.

Floating Ips

The floating ip allocation is simpler and rarer, so we actually just to a simple call through the queue and all of the setup is done by nova-network. It is important to note that floating ips are associated with fixed ips, not instances directly, and more than one floating ip can be associated with each fixed ip. They are implemented with iptables forwarding rules.

Security Groups

Soren did the security group code, so I'll let him do an overview. They are implemented on the host level. I have a couple patches I need to submit to speed them up and to allow intra-project communication in vlan mode.