Swift Installation Notes on SuSE SLE 11 64bit
Bellow I will provide some steps into deploying openstack-swift on a SuSE enterprise system. Basically, this is a setup that will include all openstack-swift functionality into a single box so that one can get acquainted with the openstack-swift installation/setup procedures.
Dependencies
- python 2.6
rsync (>3.0), curl, memcached, sqlite3 (>= 3.6.22), xfsprogs
python-support (>=0.90.0), python-openssl, python-setuptools, python-pastedeploy
python-simplejson, python-netifaces, python-webob (>=0.9.7 ), python-xattr
python-greenlet (>=0.3.1), python-eventlet (>=0.9.8)
Installing non-python packages
suse11:~ # zypper install curl memcached xfsprogs
The sqlite3 that comes with SuSE11-SP1 will not work, therefore a newer version needs to the installed. I used the version found in the opensuse build project for server:database. First add the repo and then install the new packages.
suse11:~ # zypper addrepo -n "sqlite3" http://download.opensuse.org/repositories/server:/database/SLE_11_SP1 "db:sqlite3" suse11:~ # zypper install -r sqlite3 sqlite3-3.7.5-38.2.x86_64
Installing python packages
Most python packages that are required are not present on the SuSE LE 11 default repos. But no worries, we will add a repo later from the opensuse build project that will provide us with the missing ones.
suse11:~ # zypper install python-openssl python-setuptools
Installing openstack-swift packages
The following details how to add the openstack opensuse repository and installing the openstack-swift packages. When installing the openstack-swift packages all dependencies will be automatically installed for you.
suse11:~ # zypper addrepo -name="isv:B1-Systems:OpenStack" http://download.opensuse.org/repositories/isv:/B1-Systems:/OpenStack/SLE_11_SP1 "isv:B1-Systems:OpenStack" suse11:~ # zypper refresh suse11:~ # zypper install openstack-swift Refreshing service 'susecloud'. Loading repository data... Reading installed packages... Resolving package dependencies... The following NEW packages are going to be installed: openstack-swift python-configobj python-curl python-eventlet python-flup python-greenlet python-netifaces python-nose python-openid python-paste python-pastedeploy python-simplejson python-tempita python-urljr python-webob python-webtest python-wsgiproxy python-yadis pyxattr The following packages are not supported by their vendor: openstack-swift python-configobj python-curl python-eventlet python-flup python-greenlet python-netifaces python-nose python-openid python-paste python-pastedeploy python-simplejson python-tempita python-urljr python-webob python-webtest python-wsgiproxy python-yadis pyxattr 19 new packages to install. ... suse11:~ # zypper install openstack-swift-auth openstack-swift-account openstack-swift-container openstack-swift-object openstack-swift-proxy Refreshing service 'susecloud'. Loading repository data... Reading installed packages... Resolving package dependencies... The following NEW packages are going to be installed: openstack-swift-account openstack-swift-auth openstack-swift-container openstack-swift-object openstack-swift-proxy The following packages are not supported by their vendor: openstack-swift-account openstack-swift-auth openstack-swift-container openstack-swift-object openstack-swift-proxy 5 new packages to install. ...
Setting up storage location using a block device
suse11~ # parted -s /dev/sdf mklabel gpt
suse11~ # parted -l /dev/sdf
Model: /vbd-212 8 (scsi)
Disk /dev/sdf: 8590MB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
suse11~ # parted -s /dev/sdf mkpart primary xfs 0 2G
suse11~ # parted -s /dev/sdf mkpart primary xfs 2G 4G
suse11~ # parted -s /dev/sdf mkpart primary xfs 4G 6G
suse11~ # parted -s /dev/sdf mkpart primary xfs 6G 8G
suse11:~ # for i in `seq 1 4`; do mkfs.xfs -i size=1024 -L sdf$i /dev/sdf$i ; done
suse11:~ # mkdir -p /srv/node/sdf{1..4}
suse11:~ # cat /etc/fstab
/dev/sda1 / ext3 acl,user_xattr 1 1
proc /proc proc defaults 0 0
sysfs /sys sysfs noauto 0 0
debugfs /sys/kernel/debug debugfs noauto 0 0
devpts /dev/pts devpts mode=0620,gid=5 0 0
/dev/sdf1 /srv/node/sdf1 xfs defaults,noatime,nodiratime,nobarrier,logbufs=8 0 0
/dev/sdf2 /srv/node/sdf2 xfs defaults,noatime,nodiratime,nobarrier,logbufs=8 0 0
/dev/sdf3 /srv/node/sdf3 xfs defaults,noatime,nodiratime,nobarrier,logbufs=8 0 0
/dev/sdf4 /srv/node/sdf4 xfs defaults,noatime,nodiratime,nobarrier,logbufs=8 0 0
suse11:~ # mount -a
suse11:~ # chown swift.swift /srv/node/*
Setting up rsync configuration
Configure /etc/rsyncd.conf with the information below and then restart the service
uid = swift gid = swift log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid address = 127.0.0.1 [account6012] max connections = 25 path = /srv/node/sdf1 read only = false lock file = /var/lock/account6012.lock [account6022] max connections = 25 path = /srv/node/sdf2 read only = false lock file = /var/lock/account6022.lock [account6032] max connections = 25 path = /srv/node/sdf3 read only = false lock file = /var/lock/account6032.lock [account6042] max connections = 25 path = /srv/node/sdf4 read only = false lock file = /var/lock/account6042.lock [container6011] max connections = 25 path = /srv/node/sdf1 read only = false lock file = /var/lock/container6011.lock [container6021] max connections = 25 path = /srv/node/sdf2 read only = false lock file = /var/lock/container6021.lock [container6031] max connections = 25 path = /srv/node/sdf3 read only = false lock file = /var/lock/container6031.lock [container6041] max connections = 25 path = /srv/node/sdf4 read only = false lock file = /var/lock/container6041.lock [object6010] max connections = 25 path = /srv/node/sdf1 read only = false lock file = /var/lock/object6010.lock [object6020] max connections = 25 path = /srv/node/sdf2 read only = false lock file = /var/lock/object6020.lock [object6030] max connections = 25 path = /srv/node/sdf3 read only = false lock file = /var/lock/object6030.lock [object6040] max connections = 25 path = /srv/node/sdf4 read only = false lock file = /var/lock/object6040.lock
Setting up openstack-swift configuration files
Please note that you can find sample config files under /usr/share/doc/packages/openstack-swift-auth/ with more detailed information. You could use those and then just change the parameters that are needed with those provided below.
/etc/swift/swift.conf
[swift-hash] # random unique string that can never change (DO NOT LOSE) swift_hash_path_suffix = changeme
Configuring DevAuth: /etc/swift/auth-server.conf
[DEFAULT] bind_ip = 0.0.0.0 bind_port = 11000 workers = 1 user = swift swift_dir = /etc/swift [pipeline:main] pipeline = auth-server [app:auth-server] use = egg:swift#auth super_admin_key = devauth reseller_prefix = AUTH default_cluster_url = http://127.0.0.1:8080/v1 token_life = 86400 node_timeout = 10
Configuring proxy-server: /etc/swift/proxy-server/proxy-server.conf
[DEFAULT] bind_ip = 0.0.0.0 bind_port = 8080 swift_dir = /etc/swift user = swift log_facility = LOG_LOCAL1 log_level = INFO [pipeline:main] pipeline = catch_errors healthcheck cache ratelimit auth proxy-server [app:proxy-server] use = egg:swift#proxy log_name = proxy-server log_facility = LOG_LOCAL1 log_level = INFO allow_account_management = true [filter:auth] use = egg:swift#auth log_name = auth-server log_facility = LOG_LOCAL1 log_level = INFO reseller_prefix = AUTH ip = 127.0.0.1 port = 11000 ssl = false prefix = / node_timeout = 10 super_admin_key = swauthkey [filter:healthcheck] use = egg:swift#healthcheck log_facility = LOG_LOCAL1 log_level = INFO [filter:cache] use = egg:swift#memcache log_facility = LOG_LOCAL1 log_level = INFO memcache_servers = 127.0.0.1:11211 [filter:ratelimit] use = egg:swift#ratelimit log_facility = LOG_LOCAL1 log_level = INFO [filter:domain_remap] use = egg:swift#domain_remap log_facility = LOG_LOCAL1 log_level = INFO [filter:domain_remap] use = egg:swift#domain_remap log_facility = LOG_LOCAL1 log_level = INFO [filter:catch_errors] use = egg:swift#catch_errors log_facility = LOG_LOCAL1 log_level = INFO log_headers = False [filter:cname_lookup] use = egg:swift#cname_lookup log_facility = LOG_LOCAL1 log_level = INFO
Configuring account-server: /etc/swift/account-server/{1..4}.account-server.conf
Each config should have unique bind_port (6012, 6022, 6032, 6042) and devices (sdf1, sdf2, sdf3, sdf4).
Example for the first 1.account-server.conf file shown below with port 6012 and devide sdf1[DEFAULT] bind_ip = 0.0.0.0 bind_port = 6012 user = swift swift_dir = /etc/swift devices = /srv/node/sdf1 mount_check = false log_facility = LOG_LOCAL2 [pipeline:main] pipeline = account-server [app:account-server] use = egg:swift#account set log_name = account-server set log_facility = LOG_LOCAL2 set log_level = INFO [account-replicator] log_name = account-replicator log_facility = LOG_LOCAL2 log_level = INFO log_facility = LOG_LOCAL2 log_level = INFO [account-stats] log_name = account-stats log_facility = LOG_LOCAL2 log_level = INFO proxy_server_conf = /etc/swift/proxy-server/proxy-server.conf [account-auditor] log_name = account-auditor log_facility = LOG_LOCAL2 log_level = INFO [account-reaper] log_name = account-reaper log_facility = LOG_LOCAL2 log_level = INFO
Configuring container-server: /etc/swift/container-server/1.container-server.conf
Each config should have unique bind_port (6011, 6021, 6031, 6041) and devices (sdf1, sdf2, sdf3, sdf4)
Example for the first 1.container-server.conf file shown below with port 6011 and devide sdf1[DEFAULT] bind_ip = 0.0.0.0 bind_port = 6011 swift_dir = /etc/swift devices = /srv/node/sdf1 log_name = swift log_facility = LOG_LOCAL3 log_level = INFO [pipeline:main] pipeline = container-server [app:container-server] use = egg:swift#container log_name = container-server log_facility = LOG_LOCAL3 log_level = INFO [container-replicator] log_name = container-replicator log_facility = LOG_LOCAL3 log_level = INFO vm_test_mode = yes [container-updater] log_name = container-updater log_facility = LOG_LOCAL3 log_level = INFO [container-auditor] log_name = container-auditor log_facility = LOG_LOCAL3 log_level = INFO
Configuring object-server: /etc/swift/object-server/1.object-server.conf
Each config should have unique bind_port (6010, 6020, 6030, 6040) and devices (sdf1, sdf2, sdf3, sdf4)
Example for the first 1.object-server.conf file shown below with port 6010 and devide sdf1[DEFAULT] bind_ip = 0.0.0.0 bind_port = 6010 user = swift swift_dir = /etc/swift devices = /srv/node/sdf1 mount_check = false log_name = swift log_facility = LOG_LOCAL4 log_level = INFO [pipeline:main] pipeline = object-server [app:object-server] use = egg:swift#object log_name = object-server log_facility = LOG_LOCAL4 log_level = INFO [object-replicator] log_name = object-replicator log_facility = LOG_LOCAL4 log_level = INFO vm_test_mode = yes [object-updater] log_name = object-updater log_facility = LOG_LOCAL4 log_level = INFO [object-auditor] log_name = object-auditor log_facility = LOG_LOCAL4 log_level = INFO
Setting up ring script & create ring
/usr/local/bin/remakerings.sh (chmod +x)
cd /etc/swift rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz swift-ring-builder object.builder create 18 3 1 swift-ring-builder object.builder add z1-127.0.0.1:6010/sdf1 1 swift-ring-builder object.builder add z2-127.0.0.1:6020/sdf2 1 swift-ring-builder object.builder add z3-127.0.0.1:6030/sdf3 1 swift-ring-builder object.builder add z4-127.0.0.1:6040/sdf4 1 swift-ring-builder object.builder rebalance swift-ring-builder container.builder create 18 3 1 swift-ring-builder container.builder add z1-127.0.0.1:6011/sdf1 1 swift-ring-builder container.builder add z2-127.0.0.1:6021/sdf2 1 swift-ring-builder container.builder add z3-127.0.0.1:6031/sdf3 1 swift-ring-builder container.builder add z4-127.0.0.1:6041/sdf4 1 swift-ring-builder container.builder rebalance swift-ring-builder account.builder create 18 3 1 swift-ring-builder account.builder add z1-127.0.0.1:6012/sdf1 1 swift-ring-builder account.builder add z2-127.0.0.1:6022/sdf2 1 swift-ring-builder account.builder add z3-127.0.0.1:6032/sdf3 1 swift-ring-builder account.builder add z4-127.0.0.1:6042/sdf4 1 swift-ring-builder account.builder rebalance
Create rings
suse11:/etc/swift # /usr/local/bin/remakerings.sh Device z1-127.0.0.1:6010/sdf1_"" with 1.0 weight got id 0 Device z2-127.0.0.1:6020/sdf2_"" with 1.0 weight got id 1 Device z3-127.0.0.1:6030/sdf3_"" with 1.0 weight got id 2 Device z4-127.0.0.1:6040/sdf4_"" with 1.0 weight got id 3 Reassigned 262144 (100.00%) partitions. Balance is now 0.00. Device z1-127.0.0.1:6011/sdf1_"" with 1.0 weight got id 0 Device z2-127.0.0.1:6021/sdf2_"" with 1.0 weight got id 1 Device z3-127.0.0.1:6031/sdf3_"" with 1.0 weight got id 2 Device z4-127.0.0.1:6041/sdf4_"" with 1.0 weight got id 3 Reassigned 262144 (100.00%) partitions. Balance is now 0.00. Device z1-127.0.0.1:6012/sdf1_"" with 1.0 weight got id 0 Device z2-127.0.0.1:6022/sdf2_"" with 1.0 weight got id 1 Device z3-127.0.0.1:6032/sdf3_"" with 1.0 weight got id 2 Device z4-127.0.0.1:6042/sdf4_"" with 1.0 weight got id 3 Reassigned 262144 (100.00%) partitions. Balance is now 0.00.
Setting up the rsyslogd configuration
suse11:~ # cat /etc/rsyslog.d/openstack-swift.conf local0.* /var/log/swift/auth.log local1.* /var/log/swift/proxy.log local2.* /var/log/swift/account.log local3.* /var/log/swift/container.log local4.* /var/log/swift/object.log
SuSE init scripts for openstack
Download them from SuSE init scripts github
Start DevAuth and Main services
suse11:/etc/swift # /etc/init.d/openstack-swift-auth start Starting auth-server suse11:/etc/swift # /etc/init.d/openstack-swift-proxy start Starting proxy-server suse11:/etc/swift # for i in account container object ; do /etc/init.d/openstack-swift-$i start ; done Starting account-server Starting account-server Starting account-server Starting account-server Starting container-server Starting container-server Starting container-server Starting container-server Starting object-server Starting object-server Starting object-server Starting object-server
Start secondary services
suse11:/etc/swift # for i in account-auditor container-auditor object-auditor ; do /etc/init.d/openstack-swift-$i start ; done Starting account-auditor Starting account-auditor Starting account-auditor Starting account-auditor Starting container-auditor Starting container-auditor Starting container-auditor Starting container-auditor Starting object-auditor Starting object-auditor Starting object-auditor Starting object-auditor suse11:/etc/swift # for i in account-replicator container-replicator object-replicator ; do /etc/init.d/openstack-swift-$i start ; done Starting account-replicator Starting account-replicator Starting account-replicator Starting account-replicator Starting container-replicator Starting container-replicator Starting container-replicator Starting container-replicator Starting object-replicator Starting object-replicator Starting object-replicator Starting object-replicator suse11:/etc/swift # for i in container-updater object-updater ; do /etc/init.d/openstack-swift-$i start ; done Starting container-updater Starting container-updater Starting container-updater Starting container-updater Starting object-updater Starting object-updater Starting object-updater Starting object-updater suse11:/etc/swift # /etc/init.d/openstack-swift-account-reaper start Starting account-reaper Starting account-reaper Starting account-reaper Starting account-reaper
Create account & get Auth info
suse11:~ # swift-auth-add-user -K devauth -a openstack tester testerpw http://127.0.0.1:8080/v1/AUTH_a85ea676b110482c95198aa027380f1a suse11: ~ # curl -i -H "X-Storage-User: openstack:tester" -H "X-Storage-Pass: testerpw" http://127.0.0.1:11000/v1.0 HTTP/1.1 204 No Content X-Storage-Url: http://127.0.0.1:8080/v1/AUTH_a85ea676b110482c95198aa027380f1a X-Storage-Token: AUTH_tkf76e235f26fe4a10871404da83833c75 X-Auth-Token: AUTH_tkf76e235f26fe4a10871404da83833c75 Content-Length: 0 Date: Fri, 18 Feb 2011 17:22:33 GMT
Testing things out
suse11: ~ # export XT="X-Auth-Token: AUTH_tkf76e235f26fe4a10871404da83833c75" suse11: ~ # export XU="http://127.0.0.1:8080/v1/AUTH_a85ea676b110482c95198aa027380f1a" suse11:~ # curl -i -H "$XT" -X PUT $XU/test-container HTTP/1.1 201 Created Content-Length: 18 Content-Type: text/plain; charset=UTF-8 Date: Fri, 18 Feb 2011 20:32:27 GMT suse11:~ # curl -i -H "$XT" -X HEAD $XU HTTP/1.1 204 No Content X-Account-Object-Count: 0 X-Account-Bytes-Used: 0 X-Account-Container-Count: 1 Content-Length: 0 Date: Fri, 18 Feb 2011 20:47:24 GMT suse11:~ # curl -i -H "$XT" -X GET $XU/ HTTP/1.1 200 OK X-Account-Object-Count: 0 X-Account-Bytes-Used: 0 X-Account-Container-Count: 1 Content-Length: 15 Content-Type: text/plain; charset=utf8 Date: Fri, 18 Feb 2011 22:07:50 GMT test-container suse11:~ # curl -i -H "$XT" -T /root/sqlite3-3.6.16-2.3.x86_64.rpm -X PUT $XU/test-container/sqlite3-3.6.16-2.3.x86_64.rpm HTTP/1.1 100 Continue HTTP/1.1 201 Created Content-Length: 118 Content-Type: text/plain; charset=UTF-8 Etag: 58843e7b8f4f30a6a3963155b3385dc2 Last-Modified: Mon, 21 Feb 2011 17:45:03 GMT Date: Mon, 21 Feb 2011 17:45:03 GMT suse11:~ # swift-get-nodes /etc/swift/account.ring.gz AUTH_a85ea676b110482c95198aa027380f1a Account AUTH_a85ea676b110482c95198aa027380f1a Container None Object None Partition 138668 Hash 876b0099bcf2258917d806e6ba82f87e Server:Port Device 127.0.0.1:6012 sdf1 Server:Port Device 127.0.0.1:6032 sdf3 Server:Port Device 127.0.0.1:6022 sdf2 Server:Port Device 127.0.0.1:6042 sdf4 [Handoff] curl -I -XHEAD "http://127.0.0.1:6012/sdf1/138668/AUTH_a85ea676b110482c95198aa027380f1a" curl -I -XHEAD "http://127.0.0.1:6032/sdf3/138668/AUTH_a85ea676b110482c95198aa027380f1a" curl -I -XHEAD "http://127.0.0.1:6022/sdf2/138668/AUTH_a85ea676b110482c95198aa027380f1a" curl -I -XHEAD "http://127.0.0.1:6042/sdf4/138668/AUTH_a85ea676b110482c95198aa027380f1a" # [Handoff] ssh 127.0.0.1 "ls -lah /srv/node/sdf1/accounts/138668/87e/876b0099bcf2258917d806e6ba82f87e/" ssh 127.0.0.1 "ls -lah /srv/node/sdf3/accounts/138668/87e/876b0099bcf2258917d806e6ba82f87e/" ssh 127.0.0.1 "ls -lah /srv/node/sdf2/accounts/138668/87e/876b0099bcf2258917d806e6ba82f87e/" ssh 127.0.0.1 "ls -lah /srv/node/sdf4/accounts/138668/87e/876b0099bcf2258917d806e6ba82f87e/" # [Handoff]