Revision as of 14:39, 11 September 2023

StarlingX Test Sub-project

Team Information

Project Lead: Rob Cooke
Technical Lead: Rob Cooke

Team Objective / Priorities

Verification and validation of StarlingX system - take a look at the test strategy
Consolidate an automated test suite using an unified Framework
Reduce to 0 the amount of manual testing

Documentation

Project calls

Bi-weekly meetings on Tuesdays at 9:00am PDT / 1600 UTC

Zoom link: https://zoom.us/j/342730236
- Dialing in from phone:
- Dial(for higher quality, dial a number based on your current location): US: +1 669 900 6833 or +1 646 876 9923
- Meeting ID: 342 730 236
- International numbers available: https://zoom.us/u/ed95sU7aQ

Agenda and meeting minutes are in this etherpad

Story Board Tags

All story board stories created for this team should use the tag "stx.test" and the prefix [Test]

Team Work Items

Validation - high level tasks: https://ethercalc.openstack.org/50vke3z9qta3
Story Board
- All
  - Active Stories
  - Merged Stories
- stx.2018.10
  - stx.2018.10 Active Stories
  - stx.2018.10 Merged Stories
Launchpad
- Launchpad queries have been added below. It's still TBD whether the test team will use launchpad for automation bugs(?)
- All
  - Open Bugs
  - Fixed Bugs
- stx.2018.10
  - stx.2018.10 Open Bugs

Sanity Information

Overview

2 configurations are run on Baremetal:
- AIO-SX (Simplex)
- AIO-DX (Duplex)

Execution is run once per week on Wind River provided labs and hardware
Execution with Pytest Framework developed by Wind River.
Execution split in Sanity Platform and Sanity OpenStack

Tests Cycle

Sanity for both Platform and Openstack are run once per week
Sanities are executed on Baremetal h/w provided by Wind River
Executed for AIO-SX and AIO-DX configurations
Regression is executed near the end of the release test cycle
Regression is run within the same framework as the weekly Sanity for Platform
Executed for AIO-SX and AIO-DX

Frameworks

Robot Framework: 61 TCs - dormant was maintained by Intel who have ceased providing this support
PyTest Framework: 70 TCs
Sanity Tests

Launchpads

N/A

Notes

@@ Line 45: / Line 45: @@
 === Sanity Information ===
 '''Overview'''
-*4 configurations are run:
+*2 configurations are run on Baremetal:
 **AIO-SX (Simplex)
 **AIO-DX (Duplex)
-**Standard Local Storage (2+2)
-**Standard External Storage (2+2+2)
+*Execution is run once per week on Wind River provided labs and hardware
-* 2 environments are run:
+*Execution with Pytest Framework developed by Wind River.
-**Baremetal
-**Virtual
-*Execution is run with no proxy settings, with proxy settings and with local registry.
-*Execution is run in a variety of hardware
-*Execution with Robot Framework developed by Intel and PyTest Framework developed by WR.
 *Execution split in Sanity Platform and Sanity OpenStack
 ------------------------------------------------
 '''Tests Cycle'''
-* Each site, Intel and WR, runs two configuration for two weeks, then switch.
+* Sanity for both Platform and Openstack are run once per week
-*Baremetal Environment will be executed by Intel and WR
+* Sanities are executed on Baremetal h/w provided by Wind River
-*Virtual Environment will be executed by Intel only
+* Executed for AIO-SX and AIO-DX configurations
-*Execution of Sanity will be run with PyTest Framework that is already in the public repo.
+*
-*Execution during Week 1 & 2:
+* Regression is executed near the end of the release test cycle
-**Intel:
+* Regression is run within the same framework as the weekly Sanity for Platform
-***AIO-SX
+* Executed for AIO-SX and AIO-DX
-***Standard External Storage (2+2+2)
-**WR:
-***AIO-DX
-***Standard Local Storage (2+2)
-*Execution during Week 3 & 4:
-**Intel:
-***AIO-DX
-***Standard Local Storage (2+2)
-**WR:
-***AIO-SX
-***Standard External Storage (2+2+2)
 ------------------------------------------------
 '''Frameworks'''
-*Robot Framework: 61 TCs
+*Robot Framework: 61 TCs - dormant was maintained by Intel who have ceased providing this support
 *PyTest Framework: 70 TCs
 * [[StarlingX/Test/SanityTests|Sanity Tests]]
-* [https://docs.google.com/spreadsheets/d/1Xr5-XjZlQ3wxukwH2nlu5HzdHJ0hDXvtmdcDbxiY6FY/edit#gid=0 Unified Sanity Record]
 ------------------------------------------------
 '''Launchpads'''
@@ Line 88: / Line 71: @@
 ------------------------------------------------
 '''Notes'''
-* Pings from external is not enabled yet, we are able only to ping from the node where VM lives to the VM, no matter what configuration is.
+*
-* Ping between VMs is also possible, but the logic to automate this is not ready.

Difference between revisions of "StarlingX/Test"