Jump to: navigation, search

Difference between revisions of "StarlingX/Test"

(Team Information)
 
(65 intermediate revisions by 6 users not shown)
Line 2: Line 2:
  
 
=== Team Information ===
 
=== Team Information ===
* Project Lead: '''Ada Cabrales''' <ada.cabrales@intel.com>
+
* Project Lead: TBD
* Technical Lead: '''Jose Perez''' <jose.perez.carranza@intel.com>
+
* Technical Lead: TBD
* Quality Assurance Engineer: '''Elio Martinez''' <elio.martinez.monroy@intel.com>
+
* PyTest Developer Lead: '''Yang Liu''' <yang.liu@windriver.com>
* Contributors:
+
* Team members:
  
 
=== Team Objective / Priorities ===
 
=== Team Objective / Priorities ===
* Responsible for verification of StarlingX, including automated and manual testing
+
* Verification and validation of StarlingX system - take a look at the [[StarlingX/TestStrategy|test strategy]]
 +
* Consolidate an automated test suite using an unified Framework
 +
* Reduce to 0 the amount of manual testing
 +
 
 +
=== Documentation ===
 +
* [[StarlingX/TestStrategy|Test strategy]]
 +
* [[StarlingX/TestCaseDocument|Documenting a test]]
 +
 
 +
=== Project calls ===
 +
Bi-weekly meetings on Tuesdays at 9:00am PDT / 1600 UTC
 +
* Zoom link: https://zoom.us/j/342730236
 +
** Dialing in from phone:
 +
** Dial(for higher quality, dial a number based on your current location): US: +1 669 900 6833 or +1 646 876 9923
 +
** Meeting ID: 342 730 236
 +
** International numbers available: https://zoom.us/u/ed95sU7aQ
 +
Agenda and meeting minutes [https://etherpad.openstack.org/p/stx-test are in this etherpad]
  
 
=== Story Board Tags ===
 
=== Story Board Tags ===
Line 30: Line 45:
 
*** [https://bugs.launchpad.net/starlingx/+bugs?field.tag=stx.test+stx.2018.10&field.tags_combinator=ALL stx.2018.10 Open Bugs]
 
*** [https://bugs.launchpad.net/starlingx/+bugs?field.tag=stx.test+stx.2018.10&field.tags_combinator=ALL stx.2018.10 Open Bugs]
  
=== Status ===
+
=== Sanity Information ===
* Capture status - what's the cadence? weekly?
+
'''Overview'''
* 2018/08/06:
+
*4 configurations are run:
* 2018/08/13:
+
**AIO-SX (Simplex)
 
+
**AIO-DX (Duplex)
===Latest ISO image Sanity Summary===
+
**Standard Local Storage (2+2)
*ISO: stx-2018-08-30-97.iso
+
**Standard External Storage (2+2+2)
*ISO: stx-2018-08-31-98.iso
+
* 2 environments are run:
*ISO: stx-2018-09-01-99.iso
+
**Baremetal
*ISO: stx-2018-09-02-100.iso
+
**Virtual
*ISO: stx-2018-09-03-101.iso
+
*Execution is run with no proxy settings, with proxy settings and with local registry.
*ISO: stx-2018-09-04-102.iso
+
*Execution is run in a variety of hardware
**All these iso images are having the same behavior
+
*Execution with Robot Framework developed by Intel and PyTest Framework developed by WR.
*Installation: [PASS] , 5 test cases success
+
*Execution split in Sanity Platform and Sanity OpenStack
*Provisioning: [FAIL] error during 2nd controller provisioning, last test fail from 4
+
------------------------------------------------
*Tests after Provisioning: [FAIL] as consequence from provisioning failure, 10 pass from 28 tests
+
'''Tests Cycle'''
**More details regarding results, please contact elio.martinez.monroy@intel.com
+
* Each site, Intel and WR, runs two configuration for two weeks, then switch.
**Most of the failures are related with the compute installation.
+
*Baremetal Environment will be executed by Intel and WR
**Full logs and reports will be uploaded soon
+
*Virtual Environment will be executed by Intel only
*Related bugs:
+
*Execution of Sanity will be run with PyTest Framework that is already in the public repo.
** https://bugs.launchpad.net/starlingx/+bug/1790213
+
*Execution during Week 1 & 2:
** https://bugs.launchpad.net/starlingx/+bug/1790514
+
**Intel:
** https://bugs.launchpad.net/starlingx/+bug/1790716
+
***AIO-SX
** https://bugs.launchpad.net/starlingx/+bug/1790718
+
***Standard External Storage (2+2+2)
 +
**WR:
 +
***AIO-DX
 +
***Standard Local Storage (2+2)
 +
*Execution during Week 3 & 4:
 +
**Intel:
 +
***AIO-DX
 +
***Standard Local Storage (2+2)
 +
**WR:
 +
***AIO-SX
 +
***Standard External Storage (2+2+2)
 +
------------------------------------------------
 +
'''Frameworks'''
 +
*Robot Framework: 61 TCs
 +
*PyTest Framework: 70 TCs
 +
* [[StarlingX/Test/SanityTests|Sanity Tests]]
 +
* [https://docs.google.com/spreadsheets/d/1Xr5-XjZlQ3wxukwH2nlu5HzdHJ0hDXvtmdcDbxiY6FY/edit#gid=0 Unified Sanity Record]
 +
------------------------------------------------
 +
'''Launchpads'''
 +
* N/A
 +
------------------------------------------------
 +
'''Notes'''
 +
* Pings from external is not enabled yet, we are able only to ping from the node where VM lives to the VM, no matter what configuration is.
 +
* Ping between VMs is also possible, but the logic to automate this is not ready.

Latest revision as of 23:28, 9 January 2020

StarlingX Test Sub-project

Team Information

  • Project Lead: TBD
  • Technical Lead: TBD
  • PyTest Developer Lead: Yang Liu <yang.liu@windriver.com>
  • Team members:

Team Objective / Priorities

  • Verification and validation of StarlingX system - take a look at the test strategy
  • Consolidate an automated test suite using an unified Framework
  • Reduce to 0 the amount of manual testing

Documentation

Project calls

Bi-weekly meetings on Tuesdays at 9:00am PDT / 1600 UTC

Agenda and meeting minutes are in this etherpad

Story Board Tags

All story board stories created for this team should use the tag "stx.test" and the prefix [Test]

Team Work Items

Sanity Information

Overview

  • 4 configurations are run:
    • AIO-SX (Simplex)
    • AIO-DX (Duplex)
    • Standard Local Storage (2+2)
    • Standard External Storage (2+2+2)
  • 2 environments are run:
    • Baremetal
    • Virtual
  • Execution is run with no proxy settings, with proxy settings and with local registry.
  • Execution is run in a variety of hardware
  • Execution with Robot Framework developed by Intel and PyTest Framework developed by WR.
  • Execution split in Sanity Platform and Sanity OpenStack

Tests Cycle

  • Each site, Intel and WR, runs two configuration for two weeks, then switch.
  • Baremetal Environment will be executed by Intel and WR
  • Virtual Environment will be executed by Intel only
  • Execution of Sanity will be run with PyTest Framework that is already in the public repo.
  • Execution during Week 1 & 2:
    • Intel:
      • AIO-SX
      • Standard External Storage (2+2+2)
    • WR:
      • AIO-DX
      • Standard Local Storage (2+2)
  • Execution during Week 3 & 4:
    • Intel:
      • AIO-DX
      • Standard Local Storage (2+2)
    • WR:
      • AIO-SX
      • Standard External Storage (2+2+2)

Frameworks


Launchpads

  • N/A

Notes

  • Pings from external is not enabled yet, we are able only to ping from the node where VM lives to the VM, no matter what configuration is.
  • Ping between VMs is also possible, but the logic to automate this is not ready.