Jump to: navigation, search

Difference between revisions of "Murano/TestsDocumentation"

(Murano Continious Integration Service)
(Murano Automated Tests: UI Tests)
Line 22: Line 22:
 
All automated tests for Murano Web UI writed with Robot Framework and advanced Selenium library, which allow to find Web elements using captions for fields and other information to find elements without xpath.
 
All automated tests for Murano Web UI writed with Robot Framework and advanced Selenium library, which allow to find Web elements using captions for fields and other information to find elements without xpath.
  
 +
===How To Run===
 +
'''Prerequisites: '''
 +
These components are required for Murano Web UI tests:
 +
* Murano
 +
* Python
 +
* BeatufulSoap
 +
:''apt-get install python-bs4''
 +
::Beautiful Soup 4 is published through PyPi, so if you can’t install it with the system packager, you can install it with easy_install orpip. The package name is beautifulsoup4, and the same package works on Python 2 and Python 3.
 +
:''easy_install beautifulsoup4''
 +
:''pip install beautifulsoup4''
 +
*Selenium2Library
 +
:''pip install robotframework-selenium2library''
 +
*Boffin.WebUIlib
  
 +
'''Download and run'''
 +
Make sure that additional components are installed.
 +
Clone murano-tests git repository:
 +
''git clone https://github.com/stackforge/murano-tests''
 +
 +
Go to the webUI directory where tests for dashboard are stored
 +
All tests are grouped for a few suites, select the suite that you want to run and execute:
 +
:''pybot <full_name_of_the_suite>''
 +
The tests in this file will execute and generate the following reports:
 +
report.html - the test results summary
 +
log.html - the test results details
 +
output.xml - the test results in a portable XML format
 +
There are also a number of command line options that can be used to control the test execution and generated outputs. Complete list can be viewed by issuing pybot --help.
 +
 +
==Tests Structure==
 +
Functional tests for Web UI  are based on Robot Framework. The web UI tests allow to perform complex integration testing with REST API service, REST API client, orchestrator component and Murano dashboard component.
 +
Main requirements for design are describe here [[https://code.google.com/p/robotframework/wiki/HowToWriteGoodTestCases]]
 +
All tests and test suites for Web UI comply with the requirements described above.
 +
Tests are divided into groups called suites according to the functionality that they check. Suite has a understandable name that consists of two parts:
 +
''<main header><Murano release version> (e.g.  Sanity_tests_0.4.txt).''
 +
Test suites:
 +
:Sanity_tests_0.X.txt contains tests that make a primary check (can we log in in murano-dashboard, can we create environment, can we mark image)
 +
:Creation_and_deletion_0.X.txt contains test with actions on murano's services(can we create and delete murano services)
 +
:Deploy_0.X.txt contains tests with actions on deploy some basic services
 +
:Coverage_of_bugs_0.X.txt contains tests that cover scenarios where regression bugs are possible
 +
:Murano_Repository_0.X.txt contains tests with actions on murano's metadata repository (compose new service, modify existing service, download/upload service)
 +
:Fields_validation.txt contains tests that check validation of the fields (min/max value, regex validation)
  
 
=== Murano Automated Tests: Tempest Tests ===
 
=== Murano Automated Tests: Tempest Tests ===

Revision as of 15:56, 22 January 2014

Murano Automated Tests Description

This page describes automated tests for OpenStack Murano project: how you can download and run tests, how understand the root of problems with FAILed tests and detailed description about each test, which executes for per commit.

Murano Continious Integration Service

Murano project has the CI server, which tests all commits for Murano components and allow to verify that new code does not break nothing.

Murano CI uses OpenStack QA cloud for testing infrastructure.

Murano CI url: murano-ci.mirantis.com

Here you can see several Jenkins jobs with different targets:

  • Jobs with postfix 'on-commit' allow to verify each commit.
  • Jobs with prefix 'prepare-devbox' allow to prepare VMs in OpenStack testing environment for fast Murano installation before the 'on-commit' jobs.
  • Jobs with postfix 'night-job-0.4' or 'full_run' allow to run long tests per night and validate that all integration tests are PASSED.


Other jobs allow to build and test Murano documentation and perform another usefull work to support Murano CI infrastructure.

Murano Automated Tests: UI Tests

Murano project has a Web User Interface and we have the test suite for Murano Web UI. All UI tests are located in murano-tests repository. Here we have several tests suites for Murano WebUI testing.

All automated tests for Murano Web UI writed with Robot Framework and advanced Selenium library, which allow to find Web elements using captions for fields and other information to find elements without xpath.

How To Run

Prerequisites: These components are required for Murano Web UI tests:

  • Murano
  • Python
  • BeatufulSoap
apt-get install python-bs4
Beautiful Soup 4 is published through PyPi, so if you can’t install it with the system packager, you can install it with easy_install orpip. The package name is beautifulsoup4, and the same package works on Python 2 and Python 3.
easy_install beautifulsoup4
pip install beautifulsoup4
  • Selenium2Library
pip install robotframework-selenium2library
  • Boffin.WebUIlib

Download and run Make sure that additional components are installed. Clone murano-tests git repository: git clone https://github.com/stackforge/murano-tests

Go to the webUI directory where tests for dashboard are stored All tests are grouped for a few suites, select the suite that you want to run and execute:

pybot <full_name_of_the_suite>

The tests in this file will execute and generate the following reports: report.html - the test results summary log.html - the test results details output.xml - the test results in a portable XML format There are also a number of command line options that can be used to control the test execution and generated outputs. Complete list can be viewed by issuing pybot --help.

Tests Structure

Functional tests for Web UI are based on Robot Framework. The web UI tests allow to perform complex integration testing with REST API service, REST API client, orchestrator component and Murano dashboard component. Main requirements for design are describe here [[1]] All tests and test suites for Web UI comply with the requirements described above. Tests are divided into groups called suites according to the functionality that they check. Suite has a understandable name that consists of two parts: <main header><Murano release version> (e.g. Sanity_tests_0.4.txt). Test suites:

Sanity_tests_0.X.txt contains tests that make a primary check (can we log in in murano-dashboard, can we create environment, can we mark image)
Creation_and_deletion_0.X.txt contains test with actions on murano's services(can we create and delete murano services)
Deploy_0.X.txt contains tests with actions on deploy some basic services
Coverage_of_bugs_0.X.txt contains tests that cover scenarios where regression bugs are possible
Murano_Repository_0.X.txt contains tests with actions on murano's metadata repository (compose new service, modify existing service, download/upload service)
Fields_validation.txt contains tests that check validation of the fields (min/max value, regex validation)

Murano Automated Tests: Tempest Tests

All Murano services have tempest-based automated tests, which allow to verify API interfaces and deployment scenarious.

Tempest tests for Murano are located in two different repositories:

  1. Basic Repository with full test suites for different Murano services
  2. Custom Tests with advanced tests for specific cases.


The following Python files are contain basic tests suites for different Murano components:

  • test_murano_envs.py contains test suite with actions on murano's environments(create, delete, get and etc.)
  • test_murano_sessions.py contains test suite with actions on murano's sessions(create, delete, get and etc.)
  • test_murano_services.py contains test suite with actions on murano's services(create, delete, get and etc.)
  • test_murano_deploy.py contains test suite with actions on deploy some basic services
  • test_murano_metadata.py contains test suite with actions on murano's metadata repository


If you want to know, what steps does this test, in code in comments for this test you can see test's scenario. For example:

   @attr(type='smoke')
   def test_get_environment(self):
       """
       Get environment by id
       Test create environment, afterthat test try to get
       environment's info, using environment's id,
       and finally delete this environment
       Target component: Murano
       Scenario:
           1. Send request to create environment.
           2. Send request to get environment
           3. Send request to delete environment
       """
       resp, env = self.create_environment('test')
       self.environments.append(env)
       resp, infa = self.get_environment_by_id(env['id'])
       assert resp['status'] == '200'
       assert infa['name'] == 'test'
       self.delete_environment(env['id'])
       self.environments.pop(self.environments.index(env))