Jump to: navigation, search

Difference between revisions of "Murano/TestsDocumentation"

(Murano Continious Integration Service)
(How To Run)
 
(19 intermediate revisions by 2 users not shown)
Line 8: Line 8:
  
 
Murano CI url: [https://murano-ci.mirantis.com/ murano-ci.mirantis.com]
 
Murano CI url: [https://murano-ci.mirantis.com/ murano-ci.mirantis.com]
 +
 +
There are two jobs for each repository: one of them run on Ubuntu, another one on CentOS.
  
 
Here you can see several Jenkins jobs with different targets:
 
Here you can see several Jenkins jobs with different targets:
* Jobs with postfix 'on-commit' allow to verify each commit.
+
* Jobs 'murano-dashboard-integration-tests-*' allow to verify each commit to murano-dashboard repository on different distributive.
* Jobs with prefix 'prepare-devbox' allow to prepare VMs in OpenStack testing environment for fast Murano installation before the 'on-commit' jobs.
+
* Jobs 'murano-engine-app-deployment-tests-*' allow to verify each commit to murano repository on different distributive
* Jobs with postfix 'night-job-0.4' or 'full_run' allow to run long tests per night and validate that all integration tests are PASSED.
 
  
  
Line 18: Line 19:
  
 
=== Murano Automated Tests: UI Tests ===
 
=== Murano Automated Tests: UI Tests ===
Murano project has a Web User Interface and we have the test suite for Murano Web UI. All UI tests are located in [https://github.com/stackforge/murano-tests/tree/master/webUI murano-tests repository]. Here we have several tests suites for Murano WebUI testing.
+
Murano project has a Web User Interface and we have the test suite for Murano Web UI. All UI tests are located in [https://github.com/stackforge/murano-dashboard/functionaltests murano-dashboard repository].  
 +
 
 +
All automated tests for Murano Web UI writen on python using advanced Selenium library, which allow to find Web elements using captions for fields and other information to find elements without/with xpath.
 +
 
 +
====How To Run====
 +
'''Prerequisites: '''
 +
 
 +
Install nose using:
 +
 
 +
:''easy_install nose''
 +
:or
 +
:''pip install nose''
 +
This will install the nose libraries, as well as the nosetests script, which you can use to automatically discover and run tests.
 +
 
 +
Install external python libraries are required for Murano Web UI tests:
 +
* testtools
 +
* [https://pypi.python.org/pypi/selenium selenium].
 +
cloud computing openstack
 +
 
 +
'''Download and run'''
 +
 
 +
Make sure that additional components are installed.
 +
 
 +
*Clone murano-dashboard git repository:
 +
 
 +
:''git clone https://github.com/openstack/murano-dashboard''
  
All automated tests for Murano Web UI writed with Robot Framework and advanced Selenium library, which allow to find Web elements using captions for fields and other information to find elements without xpath.
+
*Firstly, need to change default settings in functionaltests/config/config_file.conf, set appropriate urls and credentials.
  
 +
    [common]
 +
    horizon_url = http://127.0.0.1/horizon
 +
    murano_url = http://127.0.0.1:8082
 +
    user = WebTestUser
 +
    password = swordfish
 +
    tenant = WebTestProject
 +
    keystone_url = http://127.0.0.1:5000/v2.0/
  
 +
 +
*Go to the "functionaltests" directory where tests are stored
 +
*A part of tests work with application catalog and upload set of application packages for that. To upload a set of standard packages from special Murano repository need to create and execute following script which clone repository with packages, archive and store them in 'functionaltests' folder:
 +
 +
    git_url="https://github.com/murano-project/murano-app-incubator"
 +
    clone_dir="murano-app-incubator"
 +
    git clone $git_url $clone_dir
 +
    cd $clone_dir
 +
    for package_dir in io.murano.*
 +
    do
 +
        if [ -d "$package_dir" ]; then
 +
            if [ -f "${package_dir}/manifest.yaml" ]; then
 +
                sudo bash make-package.sh $package_dir
 +
            fi
 +
        fi
 +
    done
 +
 +
*All preparation are done.
 +
*All tests are grouped for a few suites, to specify which tests/suite to run, pass test/suite names on the command line:
 +
**to run all tests: ''nosetests sanity_check.py''
 +
**to run one suite: ''nosetests sanity_check.py:<test suite name>''
 +
**to run one test: ''nosetests sanity_check.py:<test suite name>.<test name>''
 +
 +
 +
In case of SUCCESS you should see something like this:
 +
 +
..................................
 +
 +
Ran 34 tests in 1.440s
 +
 +
OK
 +
 +
There are also a number of command line options that can be used to control the test execution and generated outputs. For help with nosetests’ many command-line options, try:
 +
 +
:''nosetests -h''
  
 
=== Murano Automated Tests: Tempest Tests ===
 
=== Murano Automated Tests: Tempest Tests ===
 
All Murano services have tempest-based automated tests, which allow to verify API interfaces and deployment scenarious.
 
All Murano services have tempest-based automated tests, which allow to verify API interfaces and deployment scenarious.
  
Tempest tests for Murano are located in two different repositories:
+
Tempest tests for Murano are located in : [https://github.com/stackforge/murano/tree/master/functionaltests Basic Repository]
# [https://github.com/Mirantis/tempest/tree/platform/stable/havana/tempest/api/murano Basic Repository] with full test suites for different Murano services
 
# [https://github.com/stackforge/murano-tests/tree/master/tempest Custom Tests] with advanced tests for specific cases.
 
  
  
The following Python files are contain basic tests suites for different Murano components:
+
 
 +
The following Python files are contain basic tests suites for different Murano components.
 +
 
 +
Tests on API which running on devstack gate [https://github.com/stackforge/murano/tree/master/functionaltests/api API Tests]:
 
* test_murano_envs.py contains test suite with actions on murano's environments(create, delete, get and etc.)
 
* test_murano_envs.py contains test suite with actions on murano's environments(create, delete, get and etc.)
 
* test_murano_sessions.py contains test suite with actions on murano's sessions(create, delete, get and etc.)
 
* test_murano_sessions.py contains test suite with actions on murano's sessions(create, delete, get and etc.)
 
* test_murano_services.py contains test suite with actions on murano's services(create, delete, get and etc.)
 
* test_murano_services.py contains test suite with actions on murano's services(create, delete, get and etc.)
* test_murano_deploy.py contains test suite with actions on deploy some basic services
+
* test_murano_repository.py contains test suite with actions on murano's package repository
* test_murano_metadata.py contains test suite with actions on murano's metadata repository
+
 
 +
Tests on engine which running on murano-ci [https://github.com/stackforge/murano/tree/master/functionaltests/engine Engine Tests]:
 +
* base.py containts base test class and tests with actions on deploy murano's services
  
  
  
If you want to know, what steps does this test, in code in comments for this test you can see test's scenario.
+
If you want to know, what steps this test does, for this test you can see test's scenario in code in comments.
 
For example:
 
For example:
  
Line 60: Line 131:
 
         self.environments.append(env)
 
         self.environments.append(env)
 
         resp, infa = self.get_environment_by_id(env['id'])
 
         resp, infa = self.get_environment_by_id(env['id'])
         assert resp['status'] == '200'
+
         self.assertEqual(200, resp.status)
        assert infa['name'] == 'test'
+
        self.assertEqual('test', infa['name'])
 
         self.delete_environment(env['id'])
 
         self.delete_environment(env['id'])
 
         self.environments.pop(self.environments.index(env))
 
         self.environments.pop(self.environments.index(env))

Latest revision as of 04:15, 27 September 2015

Murano Automated Tests Description

This page describes automated tests for OpenStack Murano project: how you can download and run tests, how understand the root of problems with FAILed tests and detailed description about each test, which executes for per commit.

Murano Continious Integration Service

Murano project has the CI server, which tests all commits for Murano components and allow to verify that new code does not break nothing.

Murano CI uses OpenStack QA cloud for testing infrastructure.

Murano CI url: murano-ci.mirantis.com

There are two jobs for each repository: one of them run on Ubuntu, another one on CentOS.

Here you can see several Jenkins jobs with different targets:

  • Jobs 'murano-dashboard-integration-tests-*' allow to verify each commit to murano-dashboard repository on different distributive.
  • Jobs 'murano-engine-app-deployment-tests-*' allow to verify each commit to murano repository on different distributive


Other jobs allow to build and test Murano documentation and perform another usefull work to support Murano CI infrastructure.

Murano Automated Tests: UI Tests

Murano project has a Web User Interface and we have the test suite for Murano Web UI. All UI tests are located in murano-dashboard repository.

All automated tests for Murano Web UI writen on python using advanced Selenium library, which allow to find Web elements using captions for fields and other information to find elements without/with xpath.

How To Run

Prerequisites:

Install nose using:

easy_install nose
or
pip install nose

This will install the nose libraries, as well as the nosetests script, which you can use to automatically discover and run tests.

Install external python libraries are required for Murano Web UI tests:

cloud computing openstack

Download and run

Make sure that additional components are installed.

  • Clone murano-dashboard git repository:
git clone https://github.com/openstack/murano-dashboard
  • Firstly, need to change default settings in functionaltests/config/config_file.conf, set appropriate urls and credentials.
   [common]
   horizon_url = http://127.0.0.1/horizon
   murano_url = http://127.0.0.1:8082
   user = WebTestUser
   password = swordfish
   tenant = WebTestProject
   keystone_url = http://127.0.0.1:5000/v2.0/


  • Go to the "functionaltests" directory where tests are stored
  • A part of tests work with application catalog and upload set of application packages for that. To upload a set of standard packages from special Murano repository need to create and execute following script which clone repository with packages, archive and store them in 'functionaltests' folder:
   git_url="https://github.com/murano-project/murano-app-incubator"
   clone_dir="murano-app-incubator"
   git clone $git_url $clone_dir
   cd $clone_dir
   for package_dir in io.murano.*
   do
       if [ -d "$package_dir" ]; then
           if [ -f "${package_dir}/manifest.yaml" ]; then
               sudo bash make-package.sh $package_dir
           fi
       fi
   done
  • All preparation are done.
  • All tests are grouped for a few suites, to specify which tests/suite to run, pass test/suite names on the command line:
    • to run all tests: nosetests sanity_check.py
    • to run one suite: nosetests sanity_check.py:<test suite name>
    • to run one test: nosetests sanity_check.py:<test suite name>.<test name>


In case of SUCCESS you should see something like this:

..................................

Ran 34 tests in 1.440s

OK

There are also a number of command line options that can be used to control the test execution and generated outputs. For help with nosetests’ many command-line options, try:

nosetests -h

Murano Automated Tests: Tempest Tests

All Murano services have tempest-based automated tests, which allow to verify API interfaces and deployment scenarious.

Tempest tests for Murano are located in : Basic Repository


The following Python files are contain basic tests suites for different Murano components.

Tests on API which running on devstack gate API Tests:

  • test_murano_envs.py contains test suite with actions on murano's environments(create, delete, get and etc.)
  • test_murano_sessions.py contains test suite with actions on murano's sessions(create, delete, get and etc.)
  • test_murano_services.py contains test suite with actions on murano's services(create, delete, get and etc.)
  • test_murano_repository.py contains test suite with actions on murano's package repository

Tests on engine which running on murano-ci Engine Tests:

  • base.py containts base test class and tests with actions on deploy murano's services


If you want to know, what steps this test does, for this test you can see test's scenario in code in comments. For example:

   @attr(type='smoke')
   def test_get_environment(self):
       """
       Get environment by id
       Test create environment, afterthat test try to get
       environment's info, using environment's id,
       and finally delete this environment
       Target component: Murano
       Scenario:
           1. Send request to create environment.
           2. Send request to get environment
           3. Send request to delete environment
       """
       resp, env = self.create_environment('test')
       self.environments.append(env)
       resp, infa = self.get_environment_by_id(env['id'])
       self.assertEqual(200, resp.status)
       self.assertEqual('test', infa['name'])
       self.delete_environment(env['id'])
       self.environments.pop(self.environments.index(env))