Jump to: navigation, search

Difference between revisions of "StarlingX/Test"

(→‎Sanity Information)
 
(81 intermediate revisions by 8 users not shown)
Line 2: Line 2:
  
 
=== Team Information ===
 
=== Team Information ===
−
* Project Lead: '''Ada Cabrales''' <ada.cabrales@intel.com>
+
* Project Lead: Rob Cooke
−
* Technical Lead: '''Jose Perez''' <jose.perez.carranza@intel.com>
+
* Technical Lead: Rob Cooke
−
* Quality Assurance Engineer:  '''Elio Martinez''' <elio.martinez.monroy@intel.com>
 
−
* Contributors:
 
  
 
=== Team Objective / Priorities ===
 
=== Team Objective / Priorities ===
−
* Responsible for verification of StarlingX, including automated and manual testing
+
* Verification and validation of StarlingX system - take a look at the [[StarlingX/TestStrategy|test strategy]]
 +
* Consolidate an automated test suite using an unified Framework
 +
* Reduce to 0 the amount of manual testing
 +
 
 +
=== Documentation ===
 +
* [[StarlingX/TestStrategy|Test strategy]]
 +
* [[StarlingX/TestCaseDocument|Documenting a test]]
 +
 
 +
=== Project calls ===
 +
Bi-weekly meetings on Tuesdays at 9:00am PDT / 1600 UTC
 +
* Zoom link: https://zoom.us/j/342730236
 +
** Dialing in from phone:
 +
** Dial(for higher quality, dial a number based on your current location): US: +1 669 900 6833 or +1 646 876 9923
 +
** Meeting ID: 342 730 236
 +
** International numbers available: https://zoom.us/u/ed95sU7aQ
 +
Agenda and meeting minutes [https://etherpad.openstack.org/p/stx-test are in this etherpad]
  
 
=== Story Board Tags ===
 
=== Story Board Tags ===
Line 15: Line 28:
 
=== Team Work Items ===
 
=== Team Work Items ===
 
* Validation - high level tasks: https://ethercalc.openstack.org/50vke3z9qta3
 
* Validation - high level tasks: https://ethercalc.openstack.org/50vke3z9qta3
−
* All Stories
+
* Story Board
−
** [https://storyboard.openstack.org/#!/story/list?status=active&tags=stx.test&project_group_id=86 Active]  
+
** All
−
** [https://storyboard.openstack.org/#!/story/list?status=merged&tags=stx.test&project_group_id=86 Merged]
+
*** [https://storyboard.openstack.org/#!/story/list?status=active&tags=stx.test&project_group_id=86 Active Stories]  
−
* stx.2018.10 Stories
+
*** [https://storyboard.openstack.org/#!/story/list?status=merged&tags=stx.test&project_group_id=86 Merged Stories]
−
** [https://storyboard.openstack.org/#!/story/list?status=active&tags=stx.test&tags=stx.2018.10&project_group_id=86 stx.2018.10 stx.2018.10 Active Items]
+
** stx.2018.10  
−
** [https://storyboard.openstack.org/#!/story/list?status=merged&tags=stx.test&tags=stx.2018.10&project_group_id=86 stx.2018.10 stx.2018.10 Merged Items]
+
*** [https://storyboard.openstack.org/#!/story/list?status=active&tags=stx.test&tags=stx.2018.10&project_group_id=86 stx.2018.10 Active Stories]
 +
*** [https://storyboard.openstack.org/#!/story/list?status=merged&tags=stx.test&tags=stx.2018.10&project_group_id=86 stx.2018.10 Merged Stories]
 +
* Launchpad
 +
** Launchpad queries have been added below. It's still TBD whether the test team will use launchpad for automation bugs(?)
 +
** All
 +
*** [https://bugs.launchpad.net/starlingx/+bugs?field.tag=stx.test Open Bugs]
 +
*** [https://bugs.launchpad.net/starlingx/+bugs?field.searchtext=&orderby=-importance&field.status%3Alist=FIXRELEASED&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_commenter=&field.subscriber=&field.structural_subscriber=&field.tag=stx.test&field.tags_combinator=ANY&field.has_cve.used=&field.omit_dupes.used=&field.omit_dupes=on&field.affects_me.used=&field.has_patch.used=&field.has_branches.used=&field.has_branches=on&field.has_no_branches.used=&field.has_no_branches=on&field.has_blueprints.used=&field.has_blueprints=on&field.has_no_blueprints.used=&field.has_no_blueprints=on&search=Search Fixed Bugs]
 +
** stx.2018.10
 +
*** [https://bugs.launchpad.net/starlingx/+bugs?field.tag=stx.test+stx.2018.10&field.tags_combinator=ALL stx.2018.10 Open Bugs]
 +
 
 +
=== Sanity Information ===
 +
'''Overview'''
 +
*2 configurations are run on Baremetal:
 +
**AIO-SX (Simplex)
 +
**AIO-DX (Duplex)
 +
 
 +
*Execution is run on Wind River provided labs and hardware
 +
*Execution with Pytest Framework developed by Wind River.
 +
*Execution split in Sanity Platform and Sanity OpenStack
 +
------------------------------------------------
 +
'''Tests Cycle'''
  
−
=== Status ===
+
* Sanity for both Platform and Openstack are run once per week
−
* Capture status - what's the cadence? weekly?
+
* Sanities are executed on Baremetal h/w provided by Wind River
−
* 2018/08/06:
+
* Executed for AIO-SX and AIO-DX configurations
−
* 2018/08/13:
+
* Results are shared by email to the community distribution list
 +
*
 +
* Regression is executed near the end of the release test cycle once all content is in the release
 +
* Automated regression typically includes two executions
 +
* Automated Regression is run within the same framework as the weekly Sanity for Platform
 +
* Executed for AIO-SX and AIO-DX
 +
* Report is generated and shared with the community via email
 +
------------------------------------------------
 +
'''Frameworks'''
 +
*Robot Framework: 61 TCs - dormant was maintained by Intel who have ceased providing this support
 +
*PyTest Framework: 70 TCs
 +
* [[StarlingX/Test/SanityTests|Sanity Tests]]
 +
------------------------------------------------
 +
'''Launchpads'''
 +
* N/A
 +
------------------------------------------------
 +
'''Notes'''
 +
* No participation outside of Wind River on test activities
 +
* Intel used to assist with Sanity and Regression - but this is no longer the case
 +
* Intel used to maintain the test repo - again no longer the case

Latest revision as of 20:49, 12 September 2023

StarlingX Test Sub-project

Team Information

  • Project Lead: Rob Cooke
  • Technical Lead: Rob Cooke

Team Objective / Priorities

  • Verification and validation of StarlingX system - take a look at the test strategy
  • Consolidate an automated test suite using an unified Framework
  • Reduce to 0 the amount of manual testing

Documentation

Project calls

Bi-weekly meetings on Tuesdays at 9:00am PDT / 1600 UTC

Agenda and meeting minutes are in this etherpad

Story Board Tags

All story board stories created for this team should use the tag "stx.test" and the prefix [Test]

Team Work Items

Sanity Information

Overview

  • 2 configurations are run on Baremetal:
    • AIO-SX (Simplex)
    • AIO-DX (Duplex)
  • Execution is run on Wind River provided labs and hardware
  • Execution with Pytest Framework developed by Wind River.
  • Execution split in Sanity Platform and Sanity OpenStack

Tests Cycle

  • Sanity for both Platform and Openstack are run once per week
  • Sanities are executed on Baremetal h/w provided by Wind River
  • Executed for AIO-SX and AIO-DX configurations
  • Results are shared by email to the community distribution list
  • Regression is executed near the end of the release test cycle once all content is in the release
  • Automated regression typically includes two executions
  • Automated Regression is run within the same framework as the weekly Sanity for Platform
  • Executed for AIO-SX and AIO-DX
  • Report is generated and shared with the community via email

Frameworks

  • Robot Framework: 61 TCs - dormant was maintained by Intel who have ceased providing this support
  • PyTest Framework: 70 TCs
  • Sanity Tests

Launchpads

  • N/A

Notes

  • No participation outside of Wind River on test activities
  • Intel used to assist with Sanity and Regression - but this is no longer the case
  • Intel used to maintain the test repo - again no longer the case
Retrieved from "https://wiki.openstack.org/w/index.php?title=StarlingX/Test&oldid=183664"