Deployment Tool for OpenStack(Nova, Glance and Swift) using Puppet
Deployment Tool for Diablo and Essex (Dodai-deploy)
Please refer to https://github.com/nii-cloud/dodai-deploy/wiki/User-guide.
Deployment Tool for Cactus
Current Release Status
Deployment Tool for Cactus can be downloaded from http://launchpad.net/nova-deployment-tool/cactus/cactus1.3/+download/nova-deployment-tool-cactus.tgz
The release can be used on AWS and other similar clouds like Eucalyptus clouds as well as on bare metal environments by using LXC or qemu.
Release notes
Nova deployment tool is an OpenStack deployment tool now.
- You can install swift and glance, as well.
Overview
These instructions are on how to install Nova on multiple machines (or a single machine).
- You can improve performance and availability by installing Nova, Glance and Swift on multiple servers in various configurations.
- You can install Nova, Glace and Swift automatically by simply editing the configuration file (deploy.conf) and running the Deployment Tool (deploy.py in Nova Deployment Tool in Launchpad).
The set-up is based on Ubuntu 10.04, Ubuntu 10.10 and Ubuntu 11.04 with a Puppet server available. Make sure you are running one of these Ubuntu versions. (CentOS support may be available, later.)
You can download the latest version of the Deployment tool at https://launchpad.net/nova-deployment-tool/.
You can view a screencast recording showing how to install Nova using the Deployment tool.
-- Introduction Video --
The following are the basic steps to install Nova without the Deployment tool.
RabbitMQ Installation
Mysql Installation
User name: root, Password: nova
Create DB
DB name: nova
Nova Components Installation
Euca2ools Installation
Nova Network Range Registration
Network range CIDR: 10.0.0.0/, Number of subnetworks: 3 Number of IP in each subnetwork: 16
Create a Nova user
User name: anne Role: admin
Create a Nova project
Project name: IRT
Export project certificate into zip file. And then, unzip the file.
Directory where zip file is to be extracted: /tmp/env/
Register environment variables for running Euca2ools
You can install/test/uninstall Nova, Glance and Swift with the deployment tool as follows.
Deploy.py takes care of the details by using Puppet. Puppet is an automation tool with standardized scripts that manage a machine's configuration. Read the introduction at Puppet's website an Introduction to Puppet on the PuppetLabs.
Install by typing the following command.
python deploy.py install
Confirm that the installation succeeded by typing the following command.
python deploy.py test
Uninstall Nova components by typing the following command.
python deploy.py uninstall
python deploy.py all = python deploy.py uninstall; python deploy.py install; python deploy.py test
Prerequisites
- The servers must be connected to a subnet.
- You must have root user permission for installation and service provision.
- You must configure the installation server to access the Puppet server by name.
- Ensure that the Puppet server can access nova, glance and swift component servers by name.
The command examples in this document identify the user as “nii”. You should change the name but you need to create the same users on all Nova, Glance and Swift component servers in ~/DeploymentTool/conf/deploy.conf (ssh_user=’user’).
If you do not change the default setting of the VolumeManager in the nova-volume, LVM must be used.
- The tool does not support system configurations other than those listed above. If you want to use other configurations, you have to change the configuration after runing the deployment tool or modify the deployment tool.
This deployment tool has been tested under the following configurations.
- Nova-compute components are installed on multiple servers.
- OS: Ubuntu10.04, Ubuntu10.10 or Ubuntu11.04
- Multiple nova network mode (Vlan Mode, Flat Mode)
- Multiple virtual software (KVM, UML, LXC)
Because of the current nova implementation constraints, Nova-api,nova-objectstore and euca2ools must be installed on one server.
We have not tested Flat DHCP Mode or Xen setting, yet. However, they should work fine if you follow the instructions in the Notes, below.
Although we conducted extensive tests, we were unable to test every configuration. Please let us know of any problems that occur in your environment by contacting us at https://answers.launchpad.net/nova-deployment-tool.
We will try to resolve any problem you send us and make the tool better for stackers.
Note: The configurations, which are not described on this document, are Nova default settings. Note also that, although we have not done so ourselves, you should be able to change the network mode to flat DHCP mode and hypervisor to Xen if you follow the instructions in the Notes section below.
Step 1 Requirements for Installation
1.1. Set up Nova, Glance and Swift PPA
Type or copy/paste the following command to use Nova PPA on all nova component servers.
sudo apt-get install python-software-properties -y sudo add-apt-repository ppa:openstack-release/2011.2 sudo apt-get update
1.2. Edit sudoers file
Type or copy/paste the visudo command to set ‘user’ (= nii in this document) as a sudouer on all nova component servers.
sudo visudo
Append the following lines.
nii ALL=(ALL) NOPASSWD:ALL nova ALL=(ALL) NOPASSWD:ALL
1.3. Configuration for SSH without password
1.3.1. Generate public and private key pairs
The Deployment tool needs to connect to all nova component servers without having the operator enter a password for any of the servers.
Type or copy/paste the following command to generate public and private key pairs on the server running the Deployment tool.
ssh-keygen -t rsa -N '' -f ~/.ssh/id_rsa
1.3.2. Copy public key to nova, swift and glance component servers
Copy the generated public key to all component servers.
Next, type or copy/paste the following commands to register the public keys on all component servers.
ssh-copy-id nii@<each component server name>
1.3.3. Create Swift -storage folder and mount device
First, create a Swift-storage folder and mount device on each swift-storage server.
The commands vary depending on which destination (Partition or Lookback device) is to be used.
The steps are detailed in the sections that follow. “$storage_path” and “$storage_dev” are defined in “deploy.conf”.
1.3.3.1 Partition
sudo apt-get install xfsprogs -y sudo mkdir -p $storage_path/$storage_dev sudo sh -c "echo '/dev/$storage_dev $storage_path/$storage_dev xfs noatime,nodiratime,nobarrier,logbufs=8 0 0' >> /etc/fstab" sudo mount $storage_path/$storage_dev
1.3.3.2 Loopback device
sudo apt-get install xfsprogs -y sudo mkdir -p $storage_path/$storage_dev sudo dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000 sudo mkfs.xfs -i size=1024 /srv/swift-disk sudo sh -c "echo '/srv/swift-disk $storage_path/$storage_dev xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0' >> /etc/fstab" sudo mount $storage_path/$storage_dev
Step 2 Installation
2.1. Download Deployment Tool
Download the DeploymetTool and extract the contents.
wget http://launchpad.net/nova-deployment-tool/cactus/cactus1.3/+download/nova-deployment-tool-cactus.tgz tar xzvf nova-deployment-tool-cactus.tgz
2.2. The Deployment tool Directory
The Deployment tool directory contains the following:
DeploymentTool
|-- Readme
|-- conf
| |-- deploy.conf
| |-- etc-default
| | `-- puppetmaster
| |-- logging.conf
| |-- network.conf
| `-- puppet-server
| |-- autosign.conf
| |-- fileserver.conf
| `-- puppet.conf
|-- deploy.py
|-- files
| |-- external_nodes.py
| |-- nova
| | |-- mysql-init.sh
| | `-- mysql-preseed.sh
| `-- swift
| |-- cert.crt
| |-- cert.key
| |-- proxy-init.sh
| |-- rsync-init.sh
| `-- storage-init.sh
|-- log
|-- manifests
| |-- packages
| | |-- glance.pp
| | |-- nova.pp
| | `-- swift.pp
| `-- site.pp
|-- templates
| |-- glance
| | `-- glance.conf.erb
| |-- nova
| | `-- nova.conf.erb
| `-- swift
| |-- account-server.conf.erb
| |-- container-server.conf.erb
| |-- object-server.conf.erb
| |-- proxy-server.conf.erb
| |-- rsyncd.conf.erb
| |-- storage-servers.erb
| `-- swift.conf.erb
|-- test
| |-- nova
| | |-- images
| | `-- test.sh
| `-- swift
| `-- test.sh
`-- uninstall
|-- default
| └-- uninstall.sh
|-- glance
| `-- uninstall.sh
|-- nova
| `-- uninstall.sh
`-- swift
`-- uninstall.sh2.3. Change Configuration file
You must change the configuration file in order to execute the Deployment tool according to your environment and configuration design.
2.3.1 Edit “deploy.conf"
Configuration values are set with the following policy.
- [default] section:
- Common configuration values
- [nova] section:
- Nova configuration values which are defined in nova.conf and are usually changed.
- [glance] section:
- Glance configuration values which are defined in glance.conf and are usually changed.
- [swift] section:
- Swift configuration values which are defined in swift.conf and are usually changed.
If you want to change the configurations not defined with deploy.conf, you have to directly change the content of the .erb file in the templates folder.
If you want to install multiple nova-computes, configure "nova-compute=ubuntu3,ubuntu8” in the deploy.conf.
If you want to install swift storage, configure “swift-storage=ubuntu3,ubuntu8”.
Type or copy/paste the following configuration information in deploy.conf.
<begin ~/DeploymentTool/conf/deploy.conf> [default] puppet_server=ubuntu7 ssh_user=nii [nova] nova_api=ubuntu7 nova_objectstore=ubuntu7 nova_compute=ubuntu7 nova_scheduler=ubuntu7 nova_network=ubuntu7 nova_volume=ubuntu7 euca2ools=ubuntu7 mysql=ubuntu7 glance_host=ubuntu7 libvirt_type=kvm network_manager=nova.network.manager.VlanManager image_service=nova.image.glance.GlanceImageService network_interface=eth0 network_ip_range=10.0.0.0/24 volume_group=ubuntu7 fixed_range=10.0.0.0/8 network_size=5000 [glance] glance=ubuntu7 default_store=swift [swift] swift_proxy=ubuntu7 swift_storage=ubuntu7 account=system username=root password=testpass storage_path=/srv/node storage_dev=sdb1 ring_builder_replicas=1 super_admin_key=swauth <end ~/DeploymentTool/conf/deploy.conf>
Here are the definitions of the values which are used in deploy.conf.
1. default section
- puppet_server … Name of server in which the puppet server is installed
- ssh_user … User name that is used to SSH into a nova component
2. nova section
- nova_api … Name of server in which the nava-api component is installed
- nova_objectstore … Name of server in which the nova-objectstore component is installed*
- nova_compute … Name of server in which the nova-compute component is installed
- nova_scheduler … Name of server in which the nova-scheduler component is installed
- nova_network … Name of server in which the nova-network component is installed
- nova_volume … Name of server in which the nova-volume component is installed
- euca2ools … Name of server that runs the test sequence
- mysql … Name of server in which mysql is installed
- glance_host … Glance server name
- libvirt_type … Virtualization type
- network_manager … Network management class name
- image_service … Image management class name
- network_interface … Network interface that is used in the nova-compute component
- network_ip_range … IP address range used by guest VMS. This value should be included in the values of fixed_range.
- volume_group … LVM volume group name that is used in the nova volume component
- fixed_range … Range of IP addresses used in all projects. If you want to change the value, please also change the IP addresses X.X.X.X of the command "nova-manage network create X.X.X.X ..." in file setup-network.sh, and the IP addresses should include the new value.
- network_size … Number of IP addresses used by Guest VM in all projects
If you want to know the “volume group name”, type “vgs” command.
sudo vgs
3. glance section
- glance … Name of server in which the glance is installed
- default_store … Image store that is used in glance. Available value: file, swift, s3
Because of the current nova implementation constraints, Nova-api,nova-objectstore and euca2ools must have been loaded in one server.
4. swift section
- swift_proxy … Name of server in which the glance is installed
- swift_storage … Name of server in which the swift storage is installed
- account … swift account name
- username … swift user name
- password … swift password
- storage_path … Folder for saving account, container and object information in swift storage server
- storage_dev … Device holding account, container and object information
- ring_builder_replicas … Number of account, container, and object copies. The value has to be equal or less than the number of swift-storage servers.
- super_admin_key … A key for creating swift users
If you install swift on Ubuntu 11.04, due to the bug https://bugs.launchpad.net/swift/+bug/796404 swift_proxy should be installed on the different machine from the machine where swift_storage will be installed.
If you want to know the server's name, type "hostname" command.
hostname --fqdn
2.3.2 Download machine image file
Download the machine image file, and put it in the“DeploymentTool/image” directory.
image="cirros-0.3.0-x86_64-uec.tar.gz" wget https://launchpad.net/cirros/trunk/0.3.0/+download/$image mv $image ~/DeploymentTool/test/nova/images/image_kvm.tgz
2.4. Start Installation
Type the following command to uninstall/install/test Nova, Swift and Glance.
python deploy.py all
Uninstall/install/test only Nova
python deploy.py all nova
Uninstall/install/test only Swift
python deploy.py all swift
Uninstall/install/test only Glance
python deploy.py all glance
Step 3 Confirm installation
3.1. Nova
Type or copy/paste the following command to check that the installation succeeded in ecua2ools installed server. *
* Let’s assume that the instance’s IP address from the euca-describe-instances response is 10.0.0.1.
sudo -i source /tmp/nova/env/novarc euca-describe-instances ssh -i /tmp/nova/mykey.priv root@10.0.0.1
3.2. Glance
Type or copy/paste the following command to check that the glance service is running in the glance installed server.
sudo service glance-api status sudo service glance-registry status
3.3. Swift
Type or copy/paste the following command on swift-proxy server to check if the "testfile" was uploaded.
st -A https://127.0.0.1:8080/auth/v1.0 -U $account:$username -K $password list myfiles
Notes
4.1. How to change the network mode
4.1.1. Using Flat mode
4.1.1.1 Configure “deploy.conf"
If Flat mode is used instead of VLAN DHCP mode, change the “network manager” in deploy.conf as follows, and start the installation.
network_manage=nova.network.manager.FlatManager
4.1.1.2 Configure “network.conf"
The following is an example of the network bridge config information in network.conf.
<begin ~/DeploymentTool/conf/network.conf >
bridge_ports eth0
bridge_stp off
bridge_maxwait 0
bridge_fd 0
<end ~/DeploymentTool/conf/ network.conf>4.1.2 Using Flat DHCP mode
If Flat DHCP mode is used instead of VLAN DHCP mode, change the “network manager” in deploy.conf as follows, and start the installation.
network_manage=nova.network.manager.FlatDHCPManager
4.2. How to change the virtualization type
4.2.1 Using Xen
If Xen is used instead of KVM, start the installation after you do the following:
(1) Install the necessary package for using Xen
(2) Change the libvirt.type in deploy.conf as follows, and start installation
libvirt_type=xen
(3) Download the machine image file of the Xen version for testing and copy it to ~/DeploymentTool/images/image_xen.tgz.
4.2.2 Use UML（User Mode Linux）
If UML is used instead of KVM, start the installation after you change the following:
(1) Change the libvirt.type in deploy.conf as follows, and then start the installation.
libvirt_type=uml
(2) Download the machine image file of the UML version for testing and copy it to ~/DeploymentTool/images/image_uml.gz.
4.2.3 Use LXC（Linux Containers）
If LXC is used instead of KVM, start the installation after you change the following: You should use the latest Ubuntu Server release, Natty Narwhal.
(1) Setup cgroups.
sudo mkdir /cgroup sudo sh -c "echo 'none /cgroup cgroup cpuacct,memory,devices,cpu,freezer,blkio 0 0' >> /etc/fstab" sudo reboot
(2) Change the libvirt.type in deploy.conf as follows, and then start the installation.
libvirt_type=lxc
(3) Download the machine image file of the LXC version for testing and copy it to ~/DeploymentTool/images/image_lxc.tgz.
4.3. Installation without an Internet connection
Create a repository in the same subnet and set the config for it to be used.
4.4. Uninstallation
Type or copy/paste the following command to uninstall the components.
python deploy.py uninstall
4.5. Use own server certificates in swift proxy servers
There is a server certificate for Swift Proxy in the "~/Deployment/files/swift/” directory.
If you want to use an original server certificate, type or copy/past the following command to create the key.
openssl req new x509 nodes out cert.crt keyout cert.key
Copy the server certificate and the key to the "~/Deployment/files/swift/ " directory, and overwrite existing ones.
-- Instruction Videos --
To a Single Server
To Multiple Servers
Deployment Tool Architecture
1. Set physical machines environments
2. Install the deployment tool
3. Set Deployment Tool configuration file according to each configuration
4. Run Deployment Tool
- ① Install puppet master and puppet agent
- ② Run puppet agent (nova deployment happens, here)
- ③ Run test script
- ④ Get test results
Deployment Tool on AWS
--Launch an AWS instance for deployment --
(Probably, we can use multiple AWS instances for deployment as well, however, I have not tried, yet.)
-- Default user(ubuntu) login to the instance --
1. Ssh password login setting
sudo sed -i 's/PasswordAuthentication no/PasswordAuthentication yes/' /etc/ssh/sshd_config sudo service ssh restart
2. User add (nii)
sudo useradd -d /home/nii -m nii sudo passwd nii
3. Repository updates
sudo apt-get install python-software-properties -y sudo add-apt-repository ppa:nova-core/release sudo apt-get update
4. Sudo users (nii, nova)
sudo visudo
5. Append the following lines.
nii ALL=(ALL) NOPASSWD:ALL nova ALL=(ALL) NOPASSWD:ALL
6. Create cgroup
sudo mkdir /cgroup sudo sh -c "echo 'none /cgroup cgroup cpuacct,memory,devices,cpu,freezer,blkio 0 0'>> /etc/fstab" sudo reboot
-- Deployment user(nii) login to the instance --
7. Create volume group
privdns=`hostname --fqdn` sudo apt-get install lvm2 -y dd if=/dev/zero of=volume.data bs=1M count=512 sudo losetup /dev/loop7 volume.data sudo vgcreate $privdns /dev/loop7
8. Copy public key to component servers
privdns=`hostname --fqdn` ssh-keygen -t rsa -N '' -f ~/.ssh/id_rsa ssh-copy-id nii@"$privdns"
9. Edit configuration files
privdns=`hostname --fqdn` wget http://launchpad.net/nova-deployment-tool/cactus/cactus1.3/+download/nova-deployment-tool-cactus.tgz tar xvfz nova-deployment-tool-cactus.tgz cd DeploymentTool sed -i "s/ubuntu7/$privdns/" ./conf/deploy.conf sed -i 's/libvirt_type=kvm/libvirt_type=lxc/' ./conf/deploy.conf sed -i 's/image_service/#image_service/' ./conf/deploy.conf
10. Download a machine image for LXC
cd ~/DeploymentTool/test/nova/images wget https://s3.amazonaws.com/nova-deployment-tool/image_lxc.gz
11. Deploy
cd ~/DeploymentTool/ python deploy.py all nova
12. You can use nova on AWS at least in a single machine configuration.