Difference between revisions of "Blueprint-speed-up-tempest"
| Line 5: | Line 5: | ||
== Summary == | == Summary == | ||
| − | + | Scale test cases horizontally and vertically. | |
== Release Note == | == Release Note == | ||
| Line 11: | Line 11: | ||
== Rationale == | == Rationale == | ||
| + | The gate time has impact to the development speed. | ||
Parallel test case execution on multiple machines can dramatically speed up Unit Test execution. | Parallel test case execution on multiple machines can dramatically speed up Unit Test execution. | ||
== User stories == | == User stories == | ||
| − | 'Gate is tooo slow' | + | * 'Gate is tooo slow' |
| + | * 'I would like use the same test runner with all OpenStack component and handling the test runs in similar way' | ||
| + | * 'I would like to follow the xUnit recommendations ' | ||
== Assumptions == | == Assumptions == | ||
| − | + | Happier developers. | |
== Design == | == Design == | ||
| − | Replacing the notesttests test runner to testtools/testresources/testrepository combination. | + | * Replacing the notesttests test runner to testtools/testresources/testrepository combination. |
| + | * Mitigate the common fixture usage | ||
== Implementation == | == Implementation == | ||
| + | |||
| + | 1. Increase the single thread risks in order to be less confuse-able with parallel issues. | ||
| + | The tempest code uses waits for delete to mitigate the chance of running out from resources, this type of issue will happen in parallel. | ||
| + | 2. fix numbered, order dependent issues | ||
| + | 3. Add an option to old and new code path | ||
| + | 4. Replace the cheaper setUpClasses | ||
| + | 5. Replace the expensive expensive ones (server) | ||
| + | 6. traceable logging ?? | ||
| + | |||
=== UI Changes === | === UI Changes === | ||
| − | + | output will change. | |
=== Code Changes === | === Code Changes === | ||
| − | + | Remove complex skip decisions, test selection should be done by attributes based on the configuration file and skip exceptions. | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | Remove complex skip decisions, test selection should be done by attributes. | ||
=== Additional Notes === | === Additional Notes === | ||
| + | The test cases rarely needs to writes to the volumes. | ||
10GB volume storage will not be enough, but the Thin provisioning can help. | 10GB volume storage will not be enough, but the Thin provisioning can help. | ||
| − | |||
More then 64 MB memory flavor type might be necessary for testing heat properly, but it will be rare case. | More then 64 MB memory flavor type might be necessary for testing heat properly, but it will be rare case. | ||
| − | |||
=== Migration === | === Migration === | ||
| − | + | Change the tox settings. | |
| + | Add post processing | ||
== Test/Demo Plan == | == Test/Demo Plan == | ||
| − | + | ||
== Unresolved issues == | == Unresolved issues == | ||
| − | https://bugs.launchpad.net/nova/+bug/1016633 | + | * https://bugs.launchpad.net/nova/+bug/1016633 |
| − | |||
Revision as of 21:28, 19 May 2013
- Launchpad Entry: TempestSpec:speed-up-tempest
- Created: 18 May 2013
- Contributors:
Contents
Summary
Scale test cases horizontally and vertically.
Release Note
Rationale
The gate time has impact to the development speed. Parallel test case execution on multiple machines can dramatically speed up Unit Test execution.
User stories
- 'Gate is tooo slow'
- 'I would like use the same test runner with all OpenStack component and handling the test runs in similar way'
- 'I would like to follow the xUnit recommendations '
Assumptions
Happier developers.
Design
- Replacing the notesttests test runner to testtools/testresources/testrepository combination.
- Mitigate the common fixture usage
Implementation
1. Increase the single thread risks in order to be less confuse-able with parallel issues. The tempest code uses waits for delete to mitigate the chance of running out from resources, this type of issue will happen in parallel. 2. fix numbered, order dependent issues 3. Add an option to old and new code path 4. Replace the cheaper setUpClasses 5. Replace the expensive expensive ones (server) 6. traceable logging ??
UI Changes
output will change.
Code Changes
Remove complex skip decisions, test selection should be done by attributes based on the configuration file and skip exceptions.
Additional Notes
The test cases rarely needs to writes to the volumes. 10GB volume storage will not be enough, but the Thin provisioning can help.
More then 64 MB memory flavor type might be necessary for testing heat properly, but it will be rare case.
Migration
Change the tox settings. Add post processing
Test/Demo Plan
Unresolved issues
