Difference between revisions of "Swift/DevstackSetupForKeystoneV3"

Revision as of 17:31, 12 August 2014

Setting up devstack to run swift with keystone v3 API

Install devstack

Instructions for devstack all-in-one installation are here: http://devstack.org/guides/single-machine.html

Edit devstack/local.conf as described on that page.

Before running stack.sh there's a few more config changes to make.

1. If you have proxy settings in your environment be sure to set no_proxy for local address ranges e.g.

 export no_proxy=192.168.58.0/8,localhost,127.0.0.0/8,.localdomain

(replace 192.168.58.0 with your VM subnet)

2. Enable swift to run in devstack

By default devstack does not run swift, so add a line to devstack/local.conf in the Swift section:

 enable_service s-proxy s-object s-container s-account

(see http://devstack.org/configuration.html)

3. Enable keystone v3 API to be used by swift's auth middleware

Add line to devstack/lib/swift in in function configure_swift(), section starting with comment "# Configure Keystone":

 iniset ${SWIFT_CONFIG_PROXY_SERVER} filter:authtoken auth_version v3.0

(at time of writing this was at line 369 in devstack/lib/swift )

4. Configure keystone to use UUID tokens rather than PKI - this may not be necessary but avoids any issues arising from large PKI tokens exceeding Swift's default max header size.

Edit existing variable in devstack/lib/keystone:

 KEYSTONE_TOKEN_FORMAT=${KEYSTONE_TOKEN_FORMAT:-UUID}

Now run devstack:

$ cd devstack
$ ./stack.sh

Enter SERVICE_TOKEN password when prompted (examples below assume this is 'ADMIN').

With devstack running, check the [filter:authtoken] section in /etc/swift/proxy-server.conf to verify that auth_version = v3.0 is set.

Install openstackclient

openstackclient will be used to set up users and projects in non-default keystone domains - keystoneclient CLI does not support non-default domains.

On a client machine:

$ git clone https://github.com/openstack/python-openstackclient.git openstackclient
$ cd openstackclient
[ may be needed: $ sudo pip install -r requirements.txt ]
$ sudo python setup.py install
$ cd ..

The openstack client command line help is here: https://wiki.openstack.org/wiki/OpenStackClient/Commands

Use openstackclient to create a domain, and a user and project in that domain

keystone-v3-setup.sh script will perform these steps: https://gist.github.com/alistairncoles/ae9d5f92063b58afeb88#file-keystone-v3-setup-sh

NOTE: in following examples replace 'u132.localdomain' with your devstack hostname/IP address and replace 'ADMIN' with whatever password you entered for SERVICE_TOKEN when running stack.sh

Create a domain named d1 (note use of --os-url and --os-token to manage keystone)

$ openstack --os-url http://u132.localdomain:5000/v3 --os-identity-api-version 3 --os-token=ADMIN domain create d1
+---------+----------------------------------------------------------------------------------------+
| Field   | Value                                                                                  | 
+---------+----------------------------------------------------------------------------------------+
| enabled | True                                                                                   |
| id      | b91b1a2be2784448a44f82ed1feafef8                                                       |
| links   | {u'self': u'http://u132.localdomain:5000/v3/domains/b91b1a2be2784448a44f82ed1feafef8'} |
| name    | d1                                                                                     |
+---------+----------------------------------------------------------------------------------------+

Create a project named p1 in domain d1

$ openstack --os-url http://u132.localdomain:5000/v3 --os-identity-api-version 3 --os-token=ADMIN project create p1 --domain d1
+-------------+-----------------------------------------------------------------------------------------+
| Field       | Value                                                                                   |
+-------------+-----------------------------------------------------------------------------------------+
| description |                                                                                         |
| domain_id   | b91b1a2be2784448a44f82ed1feafef8                                                        |
| enabled     | True                                                                                    |
| id          | 3a64e71a64a84c4796b93b109cd2b5ba                                                        |
| links       | {u'self': u'http://u132.localdomain:5000/v3/projects/3a64e71a64a84c4796b93b109cd2b5ba'} |
| name        | p1                                                                                      |
+-------------+-----------------------------------------------------------------------------------------+

Create a user named u1 in domain d1

$ openstack --os-url http://u132.localdomain:5000/v3 --os-identity-api-version 3 --os-token=ADMIN user create u1 --domain d1 --password testing
+-----------+--------------------------------------------------------------------------------------+
| Field     | Value                                                                                |
+-----------+--------------------------------------------------------------------------------------+
| domain_id | b91b1a2be2784448a44f82ed1feafef8                                                     |
| enabled   | True                                                                                 |
| id        | f227284da36849a39b29db3798d00979                                                     |
| links     | {u'self': u'http://u132.localdomain:5000/v3/users/f227284da36849a39b29db3798d00979'} |
| name      | u1                                                                                   |
+-----------+--------------------------------------------------------------------------------------+

Create a role named admin (note, use 'admin' role because devstack does not configure the role 'swiftoperator' in swift-proxy-server.conf)

$ openstack --os-url http://u132.localdomain:5000/v3 --os-identity-api-version 3 --os-token=ADMIN role create admin
+-------+--------------------------------------------------------------------------------------+
| Field | Value                                                                                |
+-------+--------------------------------------------------------------------------------------+
| id    | 587f0885f6174436bd1cd5b0862324b2                                                     |
| links | {u'self': u'http://u132.localdomain:5000/v3/roles/587f0885f6174436bd1cd5b0862324b2'} |
| name  | admin                                                                                |
+-------+--------------------------------------------------------------------------------------+

Assign user u1 the role admin on project p1

$ openstack --os-url http://u132.localdomain:5000/v3 --os-identity-api-version 3 --os-token=ADMIN role add --user u1 --project p1 admin

This may return an error because 'admin' role already exists - ignore:

ERROR: cliff.app Conflict occurred attempting to store role. (IntegrityError) (1062, "Duplicate entry 'admin' for key 'ixu_role_name'") 'INSERT INTO role (id, name, extra) VALUES (%s, %s, %s)' ('342b7b9103ff4c1284b1926c6e2bcc2c', 'admin', '{}') (HTTP 409)

Use swiftclient to create an account

Patch swiftclient from from https://review.openstack.org/#/c/91788/ to get keystone v3 support.

stat the account (note -V 3 option and /v3 at end of auth url)

$ swift --os-auth-url http://u132.localdomain:5000/v3 --os-username u1 --os-user-domain-name d1 --os-project-name p1 --os-project-domain-name d1 --os-password testing -V 3 stat
No handlers could be found for logger "keystoneclient.httpclient"
      Account: AUTH_3a64e71a64a84c4796b93b109cd2b5ba
   Containers: 0
      Objects: 0
        Bytes: 0
 Content-Type: text/plain; charset=utf-8
  X-Timestamp: 1406021227.61343
   X-Trans-Id: tx405d56d8da1a454492a58-0053ce2e6b 
X-Put-Timestamp: 1406021227.61343

Create a container

$ swift --os-auth-url http://u132.localdomain:5000/v3 --os-username u1 --os-user-domain-name d1 --os-project-name p1 --os-project-domain-name d1 --os-password testing -V 3 post c1
No handlers could be found for logger "keystoneclient.httpclient"

List the account containers using swiftclient

$ swift --os-auth-url http://u132.localdomain:5000/v3 --os-username u1 --os-user-domain-name d1 --os-project-name p1 --os-project-domain-name d1 --os-password testing -V 3 list
No handlers could be found for logger "keystoneclient.httpclient"
c1

You can also list the account containers using openstackclient (note: now using --os-auth-url and user credentials)

$ openstack --os-auth-url http://u132.localdomain:5000/v3 --os-identity-api-version 3 --os-username u1 --os-user-domain-name d1 --os-project-name p1 --os-project-domain-name d1 --os-password testing container list
WARNING: keystoneclient.httpclient Failed to retrieve management_url from token
+------+
| Name |
+------+
| c1   |
+------+ list

Creating endpoint for SAIO VM in keystone

Once you have devstack (or any instance of keystone) running, you may want to use it as an id service for an SAIO instance. To do that you need to create an endpoint in keystone's service catalog that points to your SAIO (by default devstack installs endpoints for the devstack swift instance - we're going to add another endpoint for an SAIO).

Note: make sure your SAIO and devstack machines are time-sync'd: the tokens generated by keystone will have an expiry time that is checked on the SAIO by swift authtoken middleware.

First, you can list the existing endpoints:

Note: in following examples replace 'u133.localdomain' with your devstack hostname/IP address and replace 'ADMIN' with whatever password you entered for SERVICE_TOKEN when running stack.sh

$ openstack --os-url http://u133.localdomain:5000/v3 --os-identity-api-version 3 --os-token=ADMIN endpoint list 
+----------------------------------+-----------+--------------+----------------+---------+-----------+-------------------------------------------------------+
| ID                               | Region    | Service Name | Service Type   | Enabled | Interface | URL                                                   | 
+----------------------------------+-----------+--------------+----------------+---------+-----------+-------------------------------------------------------+
<other services snipped>
| a859cc9ea3fa44d99e7dfedfa0bd835d | RegionOne | swift        | object-store   | True    | admin     | http://192.168.58.133:8080                            |
| d7aa041b33f845ca9916a26243f394a2 | RegionOne | swift        | object-store   | True    | internal  | http://192.168.58.133:8080/v1/AUTH_$(tenant_id)s      |
| de71171eaf80404da55239f28c765222 | RegionOne | swift        | object-store   | True    | public    | http://192.168.58.133:8080/v1/AUTH_$(tenant_id)s      |
+----------------------------------+-----------+--------------+----------------+---------+-----------+-------------------------------------------------------+

All of the existing endpoint are in RegionOne - we will create a new endpoint in another region ('MyRegion') so that swiftclient can select the endpoint based on region. The endpoint url uses special syntax to have keystone automatically insert the tenant id after the AUTH_ part.

Note: replace saio-1.localdomain with the hostname/IP of your SAIO machine.

$ openstack --os-url http://u133.localdomain:5000/v3 --os-token ADMIN endpoint create --region MyRegion object-store public 'http://saio-1.localdomain:8080/v1/AUTH_$(tenant_id)s' 
+--------------+----------------------------------------------------------------------------------------+
| Field        | Value                                                                                  | 
+--------------+----------------------------------------------------------------------------------------+
| enabled      | True                                                                                   |
| id           | a303fdceb8e145869dad2caef0cb7d6b                                                       |
| interface    | public                                                                                 |
| links        | {u'self': u'http://192.168.58.133:5000/v3/endpoints/a303fdceb8e145869dad2caef0cb7d6b'} |
| region       | MyRegion                                                                               |
| service_id   | a510395df29949f8b1531eb79e367924                                                       |
| service_name | swift                                                                                  |
| service_type | object-store                                                                           |
| url          | http://saio-1.localdomain:8080/v1/AUTH_$(tenant_id)s                                   | 
+--------------+----------------------------------------------------------------------------------------+

Next we need to configure swift's auth_token middleware config to use devstack keystone to validate tokens. In proxy-server.conf you will need some changes w.r.t. proxy-server.conf-sample, to enable and configure using keystone auth:

[pipeline:main]
pipeline = catch_errors gatekeeper healthcheck proxy-logging cache container_sync bulk tempurl ratelimit authtoken keystoneauth container-quotas account-quotas slo dlo proxy-logging proxy-server

[filter:authtoken]
log_level = DEBUG 
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
delay_auth_decision = true
auth_version = v3.0
auth_port = 35357
auth_host = u133.localdomain
auth_protocol = http
auth_uri = http://u133.localdomain:5000/v3
admin_tenant_name = service
admin_user = swift
admin_password = ADMIN
#admin_token = ADMIN                                                                                                                           
cache = swift.cache
include_service_catalog = False

Restart the swift proxy-server, and try to stat an account:

$ swift --os-auth-url http://u133.localdomain:5000/v3 --os-username u1 --os-user-domain-name d1 --os-project-name p1 --os-project-domain-name d1 --os-password testing -V 3 --os-region MyRegion stat
      Account: AUTH_05257da6eba143cd8af9d40bf5fcddc5
   Containers: 0
      Objects: 0
        Bytes: 0
 Content-Type: text/plain; charset=utf-8
  X-Timestamp: 1407862944.15104
   X-Trans-Id: txf62ec2664851468abbe61-0053ea489f
X-Put-Timestamp: 1407862944.15104

(You may find that you do not need to specify --os-region in the swift command line - swift parses the catalog returned by keystone and selects the first object-store endpoint it finds, which may happen to be the one you created for MyRegion).