StarlingX/Test

= StarlingX Test Sub-project =

Team Information

 * Project Lead: TBD
 * Technical Lead: TBD
 * PyTest Developer Lead: Yang Liu 
 * Team members:

Team Objective / Priorities

 * Verification and validation of StarlingX system - take a look at the test strategy
 * Consolidate an automated test suite using an unified Framework
 * Reduce to 0 the amount of manual testing

Documentation

 * Test strategy
 * Documenting a test

Project calls
Bi-weekly meetings on Tuesdays at 9:00am PDT / 1600 UTC Agenda and meeting minutes are in this etherpad
 * Zoom link: https://zoom.us/j/342730236
 * Dialing in from phone:
 * Dial(for higher quality, dial a number based on your current location): US: +1 669 900 6833 or +1 646 876 9923
 * Meeting ID: 342 730 236
 * International numbers available: https://zoom.us/u/ed95sU7aQ

Story Board Tags
All story board stories created for this team should use the tag "stx.test" and the prefix [Test]

Team Work Items

 * Validation - high level tasks: https://ethercalc.openstack.org/50vke3z9qta3
 * Story Board
 * All
 * Active Stories
 * Merged Stories
 * stx.2018.10
 * stx.2018.10 Active Stories
 * stx.2018.10 Merged Stories
 * Launchpad
 * Launchpad queries have been added below. It's still TBD whether the test team will use launchpad for automation bugs(?)
 * All
 * Open Bugs
 * Fixed Bugs
 * stx.2018.10
 * stx.2018.10 Open Bugs

Sanity Information
Overview
 * 4 configurations are run:
 * AIO-SX (Simplex)
 * AIO-DX (Duplex)
 * Standard Local Storage (2+2)
 * Standard External Storage (2+2+2)
 * 2 environments are run:
 * Baremetal
 * Virtual
 * Execution is run with no proxy settings, with proxy settings and with local registry.
 * Execution is run in a variety of hardware
 * Execution with Robot Framework developed by Intel and PyTest Framework developed by WR.
 * Execution split in Sanity Platform and Sanity OpenStack

Tests Cycle
 * Each site, Intel and WR, runs two configuration for two weeks, then switch.
 * Baremetal Environment will be executed by Intel and WR
 * Virtual Environment will be executed by Intel only
 * Execution of Sanity will be run with PyTest Framework that is already in the public repo.
 * Execution during Week 1 & 2:
 * Intel:
 * AIO-SX
 * Standard External Storage (2+2+2)
 * WR:
 * AIO-DX
 * Standard Local Storage (2+2)
 * Execution during Week 3 & 4:
 * Intel:
 * AIO-DX
 * Standard Local Storage (2+2)
 * WR:
 * AIO-SX
 * Standard External Storage (2+2+2)

Frameworks
 * Robot Framework: 61 TCs
 * PyTest Framework: 70 TCs
 * Sanity Tests
 * Unified Sanity Record

Launchpads
 * N/A

Notes
 * Pings from external is not enabled yet, we are able only to ping from the node where VM lives to the VM, no matter what configuration is.
 * Ping between VMs is also possible, but the logic to automate this is not ready.