VlanNetworkSetup
Setting up your cloud to work in a VLAN tagged environment.
In some cases, you may have a large IP space which is cut up into smaller subnets. The smaller subnets are then trunked together at the switch level (dividing layer 3 by layer 2) so that all machines in the larger IP space can communicate. The purpose of this is generally to control the size of broadcast domains.
Using projects as a way to logically separate each VLAN, we can setup our cloud in this environment. Please note that you must have IP forwarding enabled for this network mode to work.
- Obtain the parameters for each network
- This includes netmask, broadcast, gateway, ethernet device and VLAN ID
- Please note that currently eth0 is hardcoded as the vlan_interface. If you need to attach your bridges to a device other than eth0, you will need to add following flag to /etc/nova/nova.conf
--vlan_interface=eth1
- For the purposes of this document, we will use the following (intentionally complex in an attempt to cover most situations):
- VLANs: 171, 172, 173 and 174
- IP Blocks: 10.1.171.0/24, 10.1.172.0/24, 10.1.173.0/24 and 10.1.174.0/24
- Each VLAN maps to it's corresponding /24 (171 = 10.1.171.0/24, etc)
- Each VLAN will get it's own bridge device, which is in the format br_$VLANID
- Each /24 has an upstream default gateway on .1
- The first 6 IPs in each /24 are reserved
- Create the networks for nova to pull from1
nova-manage --config-file=/etc/nova/nova.conf network create public 10.1.171.0/24 1 256 nova-manage --config-file=/etc/nova/nova.conf network create public 10.1.172.0/24 1 256 nova-manage --config-file=/etc/nova/nova.conf network create public 10.1.173.0/24 1 256 nova-manage --config-file=/etc/nova/nova.conf network create public 10.1.174.0/24 1 256
- Login to your DB and determine the network id assigned to each VLAN
select id,cidr from networks;
- Update the DB to match your network settings. The following script will generate SQL based on the predetermined settings for this example. You'll need to modify this to fit your environment
#!/bin/sh
if [ -z $1 ]; then
echo "You need to specify the vlan to modify"
fi
if [ -z $2 ]; then
echo "You need to specify a network id number (check the DB for the network you want to update)"
fi
VLAN=$1
ID=$2
cat > vlan.sql << __EOF_
update networks set vlan = '$VLAN' where id = $ID;
update networks set bridge = 'br_$VLAN' where id = $ID;
update networks set gateway = '10.1.$VLAN.7' where id = $ID;
update networks set dhcp_start = '10.1.$VLAN.8' where id = $ID;
update fixed_ips set reserved = 1 where address in ('10.1.$VLAN.1','10.1.$VLAN.2','10.1.$VLAN.3','10.1.$VLAN.4','10.1.$VLAN.5','10.1.$VLAN.6','10.1.$VLAN.7');
__EOF_
- Once you have verified the SQL is correct, run it on your DB
- You'll need to do this for every VLAN you have
- Next we create a project manager
nova-manage --flagfile=/etc/nova/nova.conf user admin $username
- Then we create a project and assign that user as the admin user
nova-manage --flagfile=/etc/nova/nova.conf project create $projectname $username
- Finally, we get the credentials for the user we just created (this will also assign one of the networks to this project)
nova-manage --flagfile=/etc/nova/nova.conf project zipfile $projectname $username
When you start nova-network, the bridge devices and associated VLAN tags will be created. When you create a new VM you must determine (either manually or programatically) which VLAN it should be a part of, and start the VM in the corresponding project.
In certain cases, the network manager may not properly tear down bridges and VLANs when it is stopped. If you attempt to restart the network manager and it does not start, check the logs for errors indicating that a bridge device already exists. If this is the case, you will likely need to tear down the bridge and VLAN devices manually.
vconfig rem vlanXXX ifconfig br_XXX down brctl delbr br_XXX
