Difference between revisions of "Manila/docs/HOWTO use tempest with manila"
Vponomaryov (talk | contribs) (→Configuring Tempest before running tests) |
Vponomaryov (talk | contribs) (→Where do Tempest tests for Manila live?) |
||
Line 16: | Line 16: | ||
=Where do Tempest tests for Manila live?= | =Where do Tempest tests for Manila live?= | ||
Manila stores its tests for Tempest in its repo here: | Manila stores its tests for Tempest in its repo here: | ||
− | https://github.com/ | + | https://github.com/openstack/manila/tree/master/contrib/tempest |
These are pluggable-like files which do not require changing of core Tempest files. | These are pluggable-like files which do not require changing of core Tempest files. |
Revision as of 19:28, 17 February 2015
Contents
- 1 Foreword
- 2 Where do Tempest tests for Manila live?
- 3 Configuring Tempest before running tests
- 3.1 Cluster related options in general
- 3.1.1 Section "cli"
- 3.1.2 Section "identity"
- 3.1.2.1 Option "uri"
- 3.1.2.2 Option "uri_v3"
- 3.1.2.3 Option "username"
- 3.1.2.4 Option "password"
- 3.1.2.5 Option "tenant_name"
- 3.1.2.6 Option "alt_username"
- 3.1.2.7 Option "alt_password"
- 3.1.2.8 Option "alt_tenant_name"
- 3.1.2.9 Option "admin_username"
- 3.1.2.10 Option "admin_password"
- 3.1.2.11 Option "admin_tenant_name"
- 3.1.3 Section "network"
- 3.2 Manila related options
- 3.2.1 Section "service_available"
- 3.2.2 Section "share"
- 3.2.2.1 Option "catalog_type"
- 3.2.2.2 Option "endpoint_type"
- 3.2.2.3 Option "multitenancy_enabled"
- 3.2.2.4 Option "enable_protocols"
- 3.2.2.5 Option "enable_ip_rules_for_protocols"
- 3.2.2.6 Option "enable_sid_rules_for_protocols"
- 3.2.2.7 Option "username_for_sid_rules"
- 3.2.2.8 Option "storage_protocol"
- 3.2.2.9 Option "share_network_id"
- 3.2.2.10 Option "alt_share_network_id"
- 3.2.2.11 Option "admin_share_network_id"
- 3.2.2.12 Option "multi_backend"
- 3.2.2.13 Option "backend_names"
- 3.2.2.14 Option "build_interval"
- 3.2.2.15 Option "build_timeout"
- 3.1 Cluster related options in general
- 4 How to run tests
- 5 How to write own tests
Foreword
This page describes how Tempest project for Manila project testing can be used.
It is assumed that user who reads this is acquainted with, at least, common information about Tempest:
http://docs.openstack.org/developer/tempest/ https://github.com/openstack/tempest
Also, it is assumed, that the following projects has been installed in test environmentː
- cinder - glance - keystone - manila - neutron - nova
Where do Tempest tests for Manila live?
Manila stores its tests for Tempest in its repo here:
https://github.com/openstack/manila/tree/master/contrib/tempest
These are pluggable-like files which do not require changing of core Tempest files. But the way of its pluggability is implemented as work-around for Tempest, which does not have pluggable functionality for 'exceptions', 'config' and 'clients' modules. It is compatible with master branch of Tempest.
Just copy these files to Tempest dirː
$ TEMPEST_DIR=/path/to/tempest $ MANILA_DIR=/path/to/manila $ cp -rv ${MANILA_DIR}/contrib/tempest/tempest/* ${TEMPEST_DIR}/tempest
Configuring Tempest before running tests
Any test running with Tempest requires update of Tempest's config file located in
etc/tempest.conf
If it does not exist, just copy template "tempest.conf.sample"
cp tempest.conf.sample tempest.conf
Manila's config extension is defined in
https://github.com/openstack/manila/blob/master/contrib/tempest/tempest/config_share.py
Subsections below describe sections and options that can be updated with Tempest's config file.
Required options for Manila tests are described below.
Section "cli"
Option "enabled"
Set it to
True
value, to be able to run Manila cli tests.
Section "identity"
Option "uri"
If using localhost it should look likeː
http://127.0.0.1:5000/v2.0/
Used for all tests, if v2 auth enabled.
Option "uri_v3"
If using localhost it should look likeː
http://127.0.0.1:5000/v3/
Option "username"
If using devstack installation it should have the following valueː
demo
Option "password"
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'demo' user.
Option "tenant_name"
If using devstack installation it should have the following valueː
demo
Option "alt_username"
If using devstack installation it should have the following valueː
alt_demo
Option "alt_password"
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'alt_demo' user.
Option "alt_tenant_name"
If using devstack installation it should have the following valueː
alt_demo
Option "admin_username"
If using devstack installation it should have the following valueː
admin
Option "admin_password"
If using devstack installation it should have the following valueː
secrete
Password can be redefined, just put here password from 'admin' user.
Option "admin_tenant_name"
If using devstack installation it should have the following valueː
admin
Section "network"
Option "public_network_id"
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron net-list
Option "public_router_id"
This option is used for creation of isolated tenants. Its value can be retrieved from:
$ neutron router-list
Section "service_available"
This section is defined within vanila Tempest. But "manila" option is defined within Manila's extension.
Option "manila"
typeː boolean default valueː True descriptionː Whether or not manila is expected to be available
This section is defined with Manila's extension.
Option "catalog_type"
typeː str default valueː share descriptionː Catalog type of the Share service. Should be changed only if it is changed in Manila itself.
Option "endpoint_type"
typeː str default valueː publicURL descriptionː The endpoint type to use for the share service. Expected values are 'public', 'admin', 'internal', 'publicURL', 'adminURL' and 'internalURL'.
Option "multitenancy_enabled"
typeː boolean default valueː True descriptionː This option is used to determine backend driver type, multitenant driver uses share-networks, but single-tenant doesn't.
Current singletenant drivers: LVM, GlusterFS, NetApp 7-mode Current multitenant drivers: Generic, NetApp Cluster-mode (cmode) Default driver in manila is Generic driver, so "multitenancy_enabled" option is enabled by default.
Option "enable_protocols"
typeː list default valueː nfs,cifs descriptionː The first value of the list is protocol by default, items of the list show enabled protocols.
Expected values are: nfs - will be tested only NFS type of shares and common actions cifs - will be tested only CIFS type of shares and common actions nfs,cifs - will be tested both share types and common actions, for common actions will be used nfs protocol cifs,nfs - will be tested both share types and common actions, for common actions will be used cifs protocol
Option "enable_ip_rules_for_protocols"
typeː list default valueː nfs,cifs descriptionː Whether allow running of ip rules tests or not, and which share protocols should be used.
Expected values are: nfs - sid rules will be tested only for nfs protocol cifs - sid rules will be tested only for cifs protocol nfs,cifs - sid rules will be tested with both share protocols cifs,nfs - sid rules will be tested with both share protocols
Option "enable_sid_rules_for_protocols"
typeː list default valueː (empty) descriptionː Whether allow running of sid (username/usergroup) rules tests or not, and which share protocols should be used.
Expected values are: nfs - sid rules will be tested only for nfs protocol cifs - sid rules will be tested only for cifs protocol nfs,cifs - sid rules will be tested with both share protocols cifs,nfs - sid rules will be tested with both share protocols
Option "username_for_sid_rules"
typeː str default valueː Administrator descriptionː What name should be used with sid rule tests? Specify it if 'sid rule' tests are enabled.
Option "storage_protocol"
typeː str default valueː "" (empty) descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.
typeː str default valueː "" (empty) descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for member creds.
typeː str default valueː "" (empty) descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for alternative member creds. At the moment there are no tests, that use share_network with creds for alternative user.
typeː str default valueː descriptionː If you want Tempest to use some specific share-network instead of autocreated, specify it. This value is used for admin creds.
Option "multi_backend"
typeː boolean default valueː False descriptionː Does your environment has more than 1 backend or not? If yes, than you can set it to True to enable multibackend tests in tempest.
Option "backend_names"
typeː list default valueː (empty) descriptionː names of backends in Manila, that Tempest should use. Tempest will use first two values from list and will use it only if option "multi_backend" is Enabled.
Option "build_interval"
typeː int default valueː 3 descriptionː time to wait between resend of requests.
Option "build_timeout"
typeː int default valueː 500 descriptionː time to wait until raise of Exception with timeout.
How to run tests
Run only cli tests
tox -evenv bash tools/pretty_tox.sh \"tempest.cli.*manila*\"
Run only api tests
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share*\"
Run only api and cli tests
tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila*\"
Run only api and cli tests with specified amount of threads
export TEMPEST_CONCURRENCY=4 tox -evenv bash tools/pretty_tox.sh \"tempest.api.share* tempest.cli.*manila* -- --concurrency=$TEMPEST_CONCURRENCY\"
Above action will try to run Tempest in 4 threads, if your host does have at least 4 cores. By default it uses amount of threads equal to amount of available cores.
Manila tests are OK using concurrent threads.
If something went wrong with test launch
Exception may be raised on step where tests are gathered and parsed and as a result we would see something similar to the followingː
running testr Non-zero exit code (2) from test listing. stdout='\xb3)\x01@d@Ytempest.api.share.admin.test_admin_actions.AdminActionsTest.test_force_delete_share[gate]<Z\x90"\xb3)\x01@g@\\ ... tempest.scenario.test_volume_boot_pattern6B\x1e\xcc', stderr=None error: testr failed (3) running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \ OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \ OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \ OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \ ${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list ====== Totals ====== Run: 0 in 0.0 sec. - Passed: 0 - Skipped: 0 - Failed: 0 ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh tempest/api/share/test_extensions.py' __________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________ ERROR: venv: commands failed
There could be different causes for that, in most cases - dependency issues, when some dependency is not installed and can not be imported. Or python syntax error, if something were changed in code, etc...
As we can see, it does not show us real problem and we need dig root cause up. It can be done by a lot of means, for example, try nextː
nosetests -sv path/to/tests
or
.tox/venv/bin/python -m testtools.run discover -t ./ path/to/tests
How to write own tests
Where to put new tests
All tests, that cover admin API are stored in
tempest/api/share/admin/*
All tests, that cover non-admin API are stored in
tempest/api/share/*
Add method as API caller
All such methods are stored in
tempest/services/share/json/shares_client.py
XML is considered as deprecated and tests for it were removed, so, only json "client" is expected.
Positive/Negative
Positive and negative tests are separated into different modules, module with "negative" tests should have "_negative" suffix. Example of module with positive testsː
tempest/api/share/test_shares.py
Example of module with negative testsː
tempest/api/share/test_shares_negative.py
Resolving concurrency issue
If some test is sensible to changes that can be done by other tests, this test should use isolated tenant, example of it can be found in quota tests, located in
tempest/api/share/admin/test_quotas.py
Reusing resources
If several tests have same dependencies, they should use same resources if possible, it can be created in "setUpClass" method. Example can be found hereː
tempest/api/share/test_shares_actions.py
Utilization of resources
Utilization stuff is defined in the following moduleː
tempest/api/share/base.py
Basic actionsː
1) All setUpClass methods should be decorated with "@test.safe_setup" decorator. Reasonː if we get Exception on some step in setUpClass, tearDownClass won't start. So, this decorator runs 'tearDownClass' in any case.
2) Create resources that should be deleted with methods from base class. correctː self.create_share() - will add resources to deletion queue incorrectː self.shares_client.create_share() - won't add resource to deletion queue
Common infoː
All resources in deletion queue will be deleted in LIFO sequence - "last in first out" or "last created first deleted".
There are two deletion queuesː - runs after each test - runs after each test-suite (class with tests) Be attentive setting up it.