Jump to: navigation, search

Difference between revisions of "StarlingX/Test"

m (Documentation)
(Team Information)
 
(6 intermediate revisions by 2 users not shown)
Line 2: Line 2:
  
 
=== Team Information ===
 
=== Team Information ===
* Project Lead: '''Ada Cabrales''' <ada.cabrales@intel.com>
+
* Project Lead: TBD
* Technical Lead: '''Jose Perez''' <jose.perez.carranza@intel.com>
+
* Technical Lead: TBD
* Team members:  
+
* PyTest Developer Lead: '''Yang Liu''' <yang.liu@windriver.com>
**  '''Cristopher Lemus''' <cristopher.j.lemus.contreras@intel.com>
+
* Team members:
**  '''Elio Martinez''' <elio.martinez.monroy@intel.com>
 
** '''Fernando Hernandez''' <fernando.hernandez.gonzalez@intel.com>
 
** '''Juan Carlos Alonso''' <juan.carlos.alonso@intel.com>
 
** '''Juan Pablo Gomez''' <juan.p.gomez@intel.com>
 
**  '''Maria Perez''' <maria.g.perez.ibarra@intel.com>
 
**  '''Numan Waheed''' <Numan.Waheed@windriver.com>
 
**  '''Ricardo Perez''' <ricardo.o.perez@intel.com>
 
  
 
=== Team Objective / Priorities ===
 
=== Team Objective / Priorities ===
Line 22: Line 15:
 
* [[StarlingX/TestStrategy|Test strategy]]
 
* [[StarlingX/TestStrategy|Test strategy]]
 
* [[StarlingX/TestCaseDocument|Documenting a test]]
 
* [[StarlingX/TestCaseDocument|Documenting a test]]
* [[StarlingX/Test/SanityTests|Sanity Tests]]
 
  
 
=== Project calls ===
 
=== Project calls ===
Line 54: Line 46:
  
 
=== Sanity Information ===
 
=== Sanity Information ===
Please refer to this [[StarlingX/Test_Sanity_info|link]] for information regarding the sanity test.
+
'''Overview'''
 
+
*4 configurations are run:
===Latest ISO images for Sanity Test===
+
**AIO-SX (Simplex)
The ISOs are generated and published by CENGN ([[http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/]])
+
**AIO-DX (Duplex)
*ISO: 20181203
+
**Standard Local Storage (2+2)
*ISO: 20181206
+
**Standard External Storage (2+2+2)
 
+
* 2 environments are run:
===Results of ISO: 20181206===
+
**Baremetal
Sanity Test is executed on Virtual Environment
+
**Virtual
 
+
*Execution is run with no proxy settings, with proxy settings and with local registry.
'''Simplex'''
+
*Execution is run in a variety of hardware
*Setup - 04 TCs [PASS]
+
*Execution with Robot Framework developed by Intel and PyTest Framework developed by WR.
*Provisioning - 01 TCs [PASS]
+
*Execution split in Sanity Platform and Sanity OpenStack
*Sanity - 18 TCs [PASS]
 
Total: 23 TCs [PASS]
 
 
------------------------------------------------
 
------------------------------------------------
'''Duplex'''
+
'''Tests Cycle'''
*Setup - 04 TCs [PASS]
+
* Each site, Intel and WR, runs two configuration for two weeks, then switch.
*Provisioning - 01 TCs [PASS]
+
*Baremetal Environment will be executed by Intel and WR
*Sanity - 19 TCs [PASS]
+
*Virtual Environment will be executed by Intel only
Total: 24 TCs [PASS]
+
*Execution of Sanity will be run with PyTest Framework that is already in the public repo.
 +
*Execution during Week 1 & 2:
 +
**Intel:
 +
***AIO-SX
 +
***Standard External Storage (2+2+2)
 +
**WR:
 +
***AIO-DX
 +
***Standard Local Storage (2+2)
 +
*Execution during Week 3 & 4:
 +
**Intel:
 +
***AIO-DX
 +
***Standard Local Storage (2+2)
 +
**WR:
 +
***AIO-SX
 +
***Standard External Storage (2+2+2)
 
------------------------------------------------
 
------------------------------------------------
'''Multinode Controller Storage'''
+
'''Frameworks'''
*Setup - 04 TCs [PASS]
+
*Robot Framework: 61 TCs
*Provisioning - 01 TCs [PASS]
+
*PyTest Framework: 70 TCs
*Sanity - 19 TCs [PASS]
+
* [[StarlingX/Test/SanityTests|Sanity Tests]]
Total: 24 TCs [PASS]
+
* [https://docs.google.com/spreadsheets/d/1Xr5-XjZlQ3wxukwH2nlu5HzdHJ0hDXvtmdcDbxiY6FY/edit#gid=0 Unified Sanity Record]
 
------------------------------------------------
 
------------------------------------------------
'''Multinode External Storage'''
+
'''Launchpads'''
*Setup - 04 TCs [PASS]
+
* N/A
*Provisioning - 01 TCs [PASS]
 
*Sanity - 19 TCs [PASS]
 
Total: 24 TCs [PASS]
 
 
------------------------------------------------
 
------------------------------------------------
*More details regarding results, please contact juan.carlos.alonso@intel.com
+
'''Notes'''
 
+
* Pings from external is not enabled yet, we are able only to ping from the node where VM lives to the VM, no matter what configuration is.
**Full logs and reports will be uploaded soon
+
* Ping between VMs is also possible, but the logic to automate this is not ready.
 
 
=== stx.2018.10 Test Plan / Status ===
 
* [[StarlingX/stx.2018.10_Testplan|stx.2018.10 Testplan]]
 
* Intel Test Status: Ada to fill in
 
* [[StarlingX/stx.2018.10_WR_TestStatus|WR Test Status]]
 

Latest revision as of 23:28, 9 January 2020

StarlingX Test Sub-project

Team Information

  • Project Lead: TBD
  • Technical Lead: TBD
  • PyTest Developer Lead: Yang Liu <yang.liu@windriver.com>
  • Team members:

Team Objective / Priorities

  • Verification and validation of StarlingX system - take a look at the test strategy
  • Consolidate an automated test suite using an unified Framework
  • Reduce to 0 the amount of manual testing

Documentation

Project calls

Bi-weekly meetings on Tuesdays at 9:00am PDT / 1600 UTC

Agenda and meeting minutes are in this etherpad

Story Board Tags

All story board stories created for this team should use the tag "stx.test" and the prefix [Test]

Team Work Items

Sanity Information

Overview

  • 4 configurations are run:
    • AIO-SX (Simplex)
    • AIO-DX (Duplex)
    • Standard Local Storage (2+2)
    • Standard External Storage (2+2+2)
  • 2 environments are run:
    • Baremetal
    • Virtual
  • Execution is run with no proxy settings, with proxy settings and with local registry.
  • Execution is run in a variety of hardware
  • Execution with Robot Framework developed by Intel and PyTest Framework developed by WR.
  • Execution split in Sanity Platform and Sanity OpenStack

Tests Cycle

  • Each site, Intel and WR, runs two configuration for two weeks, then switch.
  • Baremetal Environment will be executed by Intel and WR
  • Virtual Environment will be executed by Intel only
  • Execution of Sanity will be run with PyTest Framework that is already in the public repo.
  • Execution during Week 1 & 2:
    • Intel:
      • AIO-SX
      • Standard External Storage (2+2+2)
    • WR:
      • AIO-DX
      • Standard Local Storage (2+2)
  • Execution during Week 3 & 4:
    • Intel:
      • AIO-DX
      • Standard Local Storage (2+2)
    • WR:
      • AIO-SX
      • Standard External Storage (2+2+2)

Frameworks


Launchpads

  • N/A

Notes

  • Pings from external is not enabled yet, we are able only to ping from the node where VM lives to the VM, no matter what configuration is.
  • Ping between VMs is also possible, but the logic to automate this is not ready.