Jump to: navigation, search

Difference between revisions of "StarlingX/Test"

(→‎Team Information)
(47 intermediate revisions by 6 users not shown)
Line 4: Line 4:
 
* Project Lead: '''Ada Cabrales''' <ada.cabrales@intel.com>
 
* Project Lead: '''Ada Cabrales''' <ada.cabrales@intel.com>
 
* Technical Lead: '''Jose Perez''' <jose.perez.carranza@intel.com>
 
* Technical Lead: '''Jose Perez''' <jose.perez.carranza@intel.com>
−
* Quality Assurance Engineer:  '''Elio Martinez''' <elio.martinez.monroy@intel.com>
+
* PyTest Developer Lead: '''Yang Liu''' <yang.liu@windriver.com>
−
* Contributors:
+
* Team members:
 +
**  '''Cristopher Lemus''' <cristopher.j.lemus.contreras@intel.com>
 +
** '''Elio Martinez''' <elio.martinez.monroy@intel.com>
 +
** '''Fernando Hernandez''' <fernando.hernandez.gonzalez@intel.com>
 +
** '''Juan Carlos Alonso''' <juan.carlos.alonso@intel.com>
 +
** '''Juan Pablo Gomez''' <juan.p.gomez@intel.com>
 +
**  '''Maria Perez''' <maria.g.perez.ibarra@intel.com>
 +
**  '''Numan Waheed''' <Numan.Waheed@windriver.com>
 +
**  '''Ricardo Perez''' <ricardo.o.perez@intel.com>
  
 
=== Team Objective / Priorities ===
 
=== Team Objective / Priorities ===
−
* Responsible for verification of StarlingX, including automated and manual testing
+
* Verification and validation of StarlingX system - take a look at the [[StarlingX/TestStrategy|test strategy]]
 +
* Consolidate an automated test suite using an unified Framework
 +
* Reduce to 0 the amount of manual testing
 +
 
 +
=== Documentation ===
 +
* [[StarlingX/TestStrategy|Test strategy]]
 +
* [[StarlingX/TestCaseDocument|Documenting a test]]
 +
* [[StarlingX/Test/SanityTests|Sanity Tests]]
  
 
=== Project calls ===
 
=== Project calls ===
Line 39: Line 54:
 
*** [https://bugs.launchpad.net/starlingx/+bugs?field.tag=stx.test+stx.2018.10&field.tags_combinator=ALL stx.2018.10 Open Bugs]
 
*** [https://bugs.launchpad.net/starlingx/+bugs?field.tag=stx.test+stx.2018.10&field.tags_combinator=ALL stx.2018.10 Open Bugs]
  
−
===Latest ISO image Sanity Summary===
+
=== Sanity Information ===
−
*ISO: stx-2018-09-02-100.iso
+
Please refer to this [[StarlingX/Test_Sanity_info|link]] for information regarding the sanity test.
−
*ISO: stx-2018-09-03-101.iso
+
 
−
*ISO: stx-2018-09-04-102.iso
+
===Latest ISO images for Sanity Test===
−
*ISO: stx-2018-09-05-103.iso
+
The ISOs are generated and published by CENGN ([[http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/]])
−
*ISO: stx-2018-09-06-104.iso
+
*ISO: 20181203
−
*ISO: stx-2018-09-07-106.iso
+
*ISO: 20181206
−
*ISO: stx-2018-09-08-107.iso
+
 
−
*ISO: stx-2018-09-09-108.iso
+
===Results of ISO: 20181206===
−
*ISO: stx-2018-09-10-110.iso
+
Sanity Test is executed on Virtual Environment
−
*ISO: stx-2018-09-11-111.iso
 
−
*ISO: stx-2018-09-12-112.iso
 
−
*ISO: stx-2018-09-13-113.iso
 
−
*ISO: stx-2018-09-13-114.iso
 
−
*ISO: stx-2018-09-15-115.iso
 
−
*ISO: stx-2018-09-16-116.iso
 
−
*ISO: stx-2018-09-17-117.iso
 
−
*ISO: stx-2018-09-18-120.iso
 
−
*ISO: stx-2018-09-19-121.iso
 
  
−
*Installation [PASS]: 4 TCs Passed, 0 TCs Failed.
+
'''Simplex'''
−
*Provisioning [FAIL]: 2 TCs Passed, 4 TCs Failed.
+
*Setup - 04 TCs [PASS]
−
** After configured storage and added LVM storage backend to Controller-0, the Controller-0 stays in "Config out-of-date" state and storage backend stays in "configuring" state.
+
*Provisioning - 01 TCs [PASS]
−
** Controller-0 cannot be unlocked since: "Lvm is configuring. All storage backends must be configured before operation is allowed."
+
*Sanity - 18 TCs [PASS]
−
** Based on Wind River feedback, test framework was modified to wait for 30 min and for 60 min since state will change to "configured" eventually,  but controller and storage states never change and it cause tests fails.
+
Total: 23 TCs [PASS]
−
** Issue in Launchpad will be open.
+
------------------------------------------------
−
*Sanity [FAIL]: 9 TCs Paseed, 11 TCs Failed.
+
'''Duplex'''
−
*Total: 29 TCs Executed, 15 TCs Passed, 15 TCs Failed.
+
*Setup - 04 TCs [PASS]
−
*More details regarding results, please contact elio.martinez.monroy@intel.com and juan.carlos.alonso@intel.com
+
*Provisioning - 01 TCs [PASS]
 +
*Sanity - 19 TCs [PASS]
 +
Total: 24 TCs [PASS]
 +
------------------------------------------------
 +
'''Multinode Controller Storage'''
 +
*Setup - 04 TCs [PASS]
 +
*Provisioning - 01 TCs [PASS]
 +
*Sanity - 19 TCs [PASS]
 +
Total: 24 TCs [PASS]
 +
------------------------------------------------
 +
'''Multinode External Storage'''
 +
*Setup - 04 TCs [PASS]
 +
*Provisioning - 01 TCs [PASS]
 +
*Sanity - 19 TCs [PASS]
 +
Total: 24 TCs [PASS]
 +
------------------------------------------------
 +
*More details regarding results, please contact juan.carlos.alonso@intel.com
  
 
**Full logs and reports will be uploaded soon
 
**Full logs and reports will be uploaded soon
−
*Related bugs:
+
 
−
** https://bugs.launchpad.net/starlingx/+bug/1790213 (Fixed)
+
=== stx.2018.10 Test Plan / Status ===
−
** https://bugs.launchpad.net/starlingx/+bug/1790514
+
* [[StarlingX/stx.2018.10_Testplan|stx.2018.10 Testplan]]
−
** https://bugs.launchpad.net/starlingx/+bug/1790716 (Fixed)
+
* Intel Test Status: Ada to fill in
−
** https://bugs.launchpad.net/starlingx/+bug/1790718 (Fixed)
+
* [[StarlingX/stx.2018.10_WR_TestStatus|WR Test Status]]

Revision as of 15:26, 2 July 2019

StarlingX Test Sub-project

Team Information

  • Project Lead: Ada Cabrales <ada.cabrales@intel.com>
  • Technical Lead: Jose Perez <jose.perez.carranza@intel.com>
  • PyTest Developer Lead: Yang Liu <yang.liu@windriver.com>
  • Team members:
    • Cristopher Lemus <cristopher.j.lemus.contreras@intel.com>
    • Elio Martinez <elio.martinez.monroy@intel.com>
    • Fernando Hernandez <fernando.hernandez.gonzalez@intel.com>
    • Juan Carlos Alonso <juan.carlos.alonso@intel.com>
    • Juan Pablo Gomez <juan.p.gomez@intel.com>
    • Maria Perez <maria.g.perez.ibarra@intel.com>
    • Numan Waheed <Numan.Waheed@windriver.com>
    • Ricardo Perez <ricardo.o.perez@intel.com>

Team Objective / Priorities

  • Verification and validation of StarlingX system - take a look at the test strategy
  • Consolidate an automated test suite using an unified Framework
  • Reduce to 0 the amount of manual testing

Documentation

Project calls

Bi-weekly meetings on Tuesdays at 9:00am PDT / 1600 UTC

Agenda and meeting minutes are in this etherpad

Story Board Tags

All story board stories created for this team should use the tag "stx.test" and the prefix [Test]

Team Work Items

Sanity Information

Please refer to this link for information regarding the sanity test.

Latest ISO images for Sanity Test

The ISOs are generated and published by CENGN ([[1]])

  • ISO: 20181203
  • ISO: 20181206

Results of ISO: 20181206

Sanity Test is executed on Virtual Environment

Simplex

  • Setup - 04 TCs [PASS]
  • Provisioning - 01 TCs [PASS]
  • Sanity - 18 TCs [PASS]

Total: 23 TCs [PASS]

Duplex

  • Setup - 04 TCs [PASS]
  • Provisioning - 01 TCs [PASS]
  • Sanity - 19 TCs [PASS]

Total: 24 TCs [PASS]

Multinode Controller Storage

  • Setup - 04 TCs [PASS]
  • Provisioning - 01 TCs [PASS]
  • Sanity - 19 TCs [PASS]

Total: 24 TCs [PASS]

Multinode External Storage

  • Setup - 04 TCs [PASS]
  • Provisioning - 01 TCs [PASS]
  • Sanity - 19 TCs [PASS]

Total: 24 TCs [PASS]

  • More details regarding results, please contact juan.carlos.alonso@intel.com
    • Full logs and reports will be uploaded soon

stx.2018.10 Test Plan / Status

Retrieved from "https://wiki.openstack.org/w/index.php?title=StarlingX/Test&oldid=170897"