Jump to: navigation, search

Difference between revisions of "RefStack/UseCases"

(Requirements Map)
m
 
(11 intermediate revisions by 5 users not shown)
Line 1: Line 1:
Use Cases are a powerful way of identifying requirements for a software project.  In the initial phase of RefStack definition, these cases have been used to drive design and architecture.  They are also currently used to drive development milestones.
+
Use Cases are a powerful way of identifying requirements for a software project.  In the initial phase of Refstack definition, these cases have been used to drive design and architecture.  They are also currently used to drive development milestones.
indicates first pass feature..  
 
  
== ReStack Use Cases ==
+
== RefStack Use Cases ==
 +
Here is the link to the original RefStack use cases with first pass are marked with **.<br />
 +
https://etherpad.openstack.org/p/DefCoreRefStackUseCases2015<br />
  
 +
In the following sections, the first pass Use Cases are highlighted in '''Bold'''.
  
 
=== Ops: A Private Cloud Operator ===
 
=== Ops: A Private Cloud Operator ===
I want to:
+
I want to:
* preview results before they are public   
+
* '''preview results before they are public'''  
 
* compare my test runs over time   
 
* compare my test runs over time   
 
* compare my test runs against clouds with similar characteristics (tagging)   
 
* compare my test runs against clouds with similar characteristics (tagging)   
* be able to ignore tests that are not important                       
+
* '''be able to ignore tests that are not important'''                      
* have tests grouped into capabilities so I am not overwhelmed   
+
* '''have tests grouped into capabilities so I am not overwhelmed'''  
* highlight tests that are critical for operations   
+
* '''highlight tests that are critical for operations'''  
* be able to upload my tempest results to refstack for analysis   
+
* '''be able to upload my tempest results to refstack for analysis'''  
* be able to remove results  
+
* '''be able to remove results'''
  
 
=== User: An OpenStack User ===
 
=== User: An OpenStack User ===
Line 20: Line 22:
 
* make sure that the capabilities I require are available
 
* make sure that the capabilities I require are available
 
* op 3, 4, 5 and 6
 
* op 3, 4, 5 and 6
* be able to post results from my own tests
+
* '''be able to post results from my own tests'''
 
* (foundation 7)
 
* (foundation 7)
  
Line 33: Line 35:
 
* control tags that can be applied to test run against my cloud  
 
* control tags that can be applied to test run against my cloud  
 
* contest results that I disagree with  
 
* contest results that I disagree with  
* be able to mark a result as official  
+
* '''be able to mark a result as official'''
* know that test submission are coming from validated users  
+
* '''know that test submission are coming from validated users'''
 
* be able to excludes tests for capabilities we have not implemented (ops 4)  
 
* be able to excludes tests for capabilities we have not implemented (ops 4)  
 
* be able to give customers information about my compatibility (ops 3)
 
* be able to give customers information about my compatibility (ops 3)
Line 42: Line 44:
 
=== Foundation: the OpenStack Foundation ===
 
=== Foundation: the OpenStack Foundation ===
 
I need/want:  
 
I need/want:  
* way to determine which tests are passing in my install base  
+
* '''way to determine which tests are passing in my install base'''
* way for community to indicate important capabilities  
+
* '''way for community to indicate important capabilities'''
 
* way to communicate which tests are “must pass” on a version by version basis  
 
* way to communicate which tests are “must pass” on a version by version basis  
 
* way to certify that a result includes all the “must pass” tests  
 
* way to certify that a result includes all the “must pass” tests  
Line 55: Line 57:
 
I need to:
 
I need to:
 
* be able to validate which clouds my product will work with (user 1)  
 
* be able to validate which clouds my product will work with (user 1)  
* be able to upload test results that are outside the upstream
+
* be able to upload test results that are outside the upstream
  
== Requirements Map ==
+
[[category: InteropWG]]
 
 
Below are derived requirements from the Use Cases and implied requirements based on the purposes and goals of the RefStack project.  This in no way should be construed as complete, but does provide guidance in the design and implementation of RefStack.<br/>
 
 
 
{| class="wikitable"
 
|-
 
! scope="col"|Requirement!! scope="col"|OpenStack Foundation!! scope="col"|Cloud Vendor!! scope="col"|subsystem Vendor!! scope="col"|Cloud Operator!! scope="col"|User!! scope="col"|Prospective User
 
|-
 
!scope="row"|Manage Tests
 
|-
 
|Identify test sets completeness|| X || X || X || X ||  X || X
 
|-
 
| Identify test sets by capability || X || X || X || X || X || X
 
|-
 
| Identify test sets by OpenStack version || X || X || X || X || ||
 
|-
 
| Identify "required pass" tests by version || X || X || X || X || ||
 
|-
 
| Identify tests that are specific to hardware/software architectures || X || X || X || X || ||
 
|-
 
!scope="row"|Manage Test Runs
 
|-
 
| Select test sets by capability ||  || X || X || X || X || X
 
|-
 
| Drill down to individual tests/test results ||  || X || X || X || X || ?
 
|-
 
| Select/Deselect individual tests (based on relevance) ||  || X || X || X || X || X
 
|-
 
| Highlight/select Critical Operational capabilities test sets ||  || X || X || X || X || X
 
|- style="background-color:white"
 
| Run tests against cloud connected to internet ||  || X || X || X || X  ||
 
|-
 
| Run tests against cloud isolated from internet ||  || X || X || X ||  ||
 
|-
 
| Run selected subset of tests ||  || X || X || X || X ||
 
|-
 
| View set of previously run tests when selecting for new run ||  || X || X || X || X ||
 
|-
 
!scope="row"|Manage Test Results
 
|-
 
| Preview test results (before publishing) ||  || X || X || X ||  ||
 
|-
 
| Compare unpublished test results over time  ||  || X || X || X || X ||
 
|-
 
| Compare published test results over time  || X || X || X || X || X || X
 
|-
 
| Compare local test results to RefStack test results ||  || X || X || X || X ||
 
|-
 
| Upload test results to RefStack for analysis ||  || X || X || X || X ||
 
|-
 
| Publish test results to RefStack ||  || X || X || X || X ||
 
|-
 
| Validate Publishers || X || X || X || X || X || X
 
|-
 
| Validate published test runs || X || X || X || X || X || X
 
|-
 
| Identify results by publisher and/or class of publisher || X || X || X || X || X || X
 
|-
 
| Identify results by certification (is this redundant to above?) || X || X || X || X || X || X
 
|-
 
| Identify official vendor results || X || X || X || X || X || X
 
|-
 
| Identify results by version || X || X || X || X || X || X
 
|-
 
| Remove test results from RefStack  || X || X || X || X || X ||
 
|-
 
| View test results by cloud  || X || X || X || X || X || X
 
|-
 
| Identify capabilities not tested  || X || X || X || X || X || X
 
|-
 
| Identify Capabilities where test run contains non-exercised tests || X || X || X || X || X || X
 
|-
 
| Compare unpublished results to clouds with similar characteristics ||  || X || X || X || X ||
 
|-
 
| Compare published results to clouds with similar characteristics || X || X || X || X || X || X
 
|-
 
!scope="row"|Manage RefStack Business Processes
 
|-
 
| Provide Vote mechanism for Capabilities  ||  || X || X || X || X || X
 
|-
 
| Display OpenStack.org certification for approved clouds, HW, SW, etc || X || X || X || X || X || X
 
|-
 
| Display OpenStack.org compliance for approved companies, organizations, schools, etc || X || X || X || X || X || X
 
|-
 
| Maintain connections to certification and compliance documentation || X || X || X || X || X || X
 
|-
 
| Ensure changes to certifications, compliance, test results, and all other critical data is restricted to authorized individuals/organizations || X || X || X || X || X || X
 
|-
 
| Submit protest to contest published test results  || X || X || X || X || ? ||
 
|-
 
| Track/correct/remove protests and results protested || X || X || X || X || X || X
 
|-
 
| Submit request for Dispute Resolution || X || X || X || X ||  ||
 
|-
 
| Publish results of Dispute Resolution || X || X || X || X || X || X
 
|-
 
| Data mine Results database for aggregates info, trends, etc  || X || X || X || X || ? ||
 
|-
 
| Provide independent verification of xx% of official (and nonofficial?) test results  || X ||  ||  ||  || ? || X
 
|-
 
| Business rules of what constitutes "approved" test results  || X || X || X || X || X || X
 
|-
 
| Privacy and Use Policies || X || X || X || X || X || X
 
|}
 
=== Reference Materials ===
 
[https://docs.google.com/document/d/1O1KH3RcPR2ZshlV3LS1qPBaywBCyXlzuFdinmYZNAM8/edit?pli=1 Original DefCore RefStack UseCases]
 

Latest revision as of 00:20, 10 December 2016

Use Cases are a powerful way of identifying requirements for a software project. In the initial phase of Refstack definition, these cases have been used to drive design and architecture. They are also currently used to drive development milestones.

RefStack Use Cases

Here is the link to the original RefStack use cases with first pass are marked with **.
https://etherpad.openstack.org/p/DefCoreRefStackUseCases2015

In the following sections, the first pass Use Cases are highlighted in Bold.

Ops: A Private Cloud Operator

I want to:

  • preview results before they are public
  • compare my test runs over time
  • compare my test runs against clouds with similar characteristics (tagging)
  • be able to ignore tests that are not important
  • have tests grouped into capabilities so I am not overwhelmed
  • highlight tests that are critical for operations
  • be able to upload my tempest results to refstack for analysis
  • be able to remove results

User: An OpenStack User

I want to:

  • make sure that the capabilities I require are available
  • op 3, 4, 5 and 6
  • be able to post results from my own tests
  • (foundation 7)

Prospect: a Prospective OpenStack User

I want to:

  • compare results without uploading tests
  • not be forced to create a lp account to get data
  • not see results that are not accurate (approved results only)

Vendor / Organization: an OpenStack Vendor

I want to:

  • control tags that can be applied to test run against my cloud
  • contest results that I disagree with
  • be able to mark a result as official
  • know that test submission are coming from validated users
  • be able to excludes tests for capabilities we have not implemented (ops 4)
  • be able to give customers information about my compatibility (ops 3)
  • results that I don’t control to be statistically relevant
  • expose aggregate results to prospects (if they are favorable)

Foundation: the OpenStack Foundation

I need/want:

  • way to determine which tests are passing in my install base
  • way for community to indicate important capabilities
  • way to communicate which tests are “must pass” on a version by version basis
  • way to certify that a result includes all the “must pass” tests
  • to indicate organizations that are in compliance
  • way to settle disputes (ops 3)
  • to highlight gaps in capability coverage
  • to ensure that results posted are accurate
  • link between RefStack result and documentation at the appropriate version

Vendor: Relies on OpenStack API’s

I need to:

  • be able to validate which clouds my product will work with (user 1)
  • be able to upload test results that are outside the upstream