Jump to: navigation, search

Difference between revisions of "StarlingX/Test/SanityTests"

(→‎Sanity-Platform)
Line 1: Line 1:
 
= Sanity Test=
 
= Sanity Test=
−
== ''Sanity-OpenStack'' ==
+
== ''Robot Framework'' ==
−
 
 
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
−
! iso  || DATE
+
! SANITY-OPENSTACK
 
|-
 
|-
−
|http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/20190609T233000Z/outputs/iso/|| 10 - Jun- 2019
+
! 01-Instance-From-Image
 
|-
 
|-
−
| . || .
+
|Create Flavors For Instances
 
|-
 
|-
−
| ||  
+
|Create Images For Instances
−
|}
+
|-
−
 
+
|Create Networks For Instances
−
{| class="wikitable"
+
|-
 +
|Launch Instances
 +
|-
 +
|Suspend Resume Instances
 +
|-
 +
|Set Error Active Flags Instances
 +
|-
 +
|Pause Unpause Instances
 +
|-
 +
|Stop Start Instances
 +
|-
 +
|Lock Unlock Instances
 +
|-
 +
|Reboot Instances
 +
|-
 +
|Rebuild Instances
 +
|-
 +
|Resize Instances
 +
|-
 +
|Set Unset Properties Instances
 +
|-
 +
|Evacuate Instances From Hosts
 +
|-
 +
! 02-Instance-From-Volume
 +
|-
 +
|Create Flavors For Instances
 +
|-
 +
|Create Images For Instances
 +
|-
 +
|Create Networks For Instances
 
|-
 
|-
−
! Provision-Containers || AIO - Simplex Virtual  || AIO - Baremetal || bug
+
|Create Volume For Instances
 
|-
 
|-
−
| 1. Provisioning Simplex System || PASS || PASS ||
+
|Launch Instances
 
|-
 
|-
−
| 2. Provisioning Duplex System || N/A || FAIL || https://bugs.launchpad.net/starlingx/+bug/1832269
+
|Suspend Resume Instances
 
|-
 
|-
−
| 3. Provisioning Standard Non-Storage System || N/A|| FAIL|| https://bugs.launchpad.net/starlingx/+bug/1832269
+
|Set Error Active Flags Instances
 
|-
 
|-
−
| 4. Provisioning Standard Storage System || N/A|| PASS ||
+
|Pause Unpause Instances
−
|}
 
−
 
 
−
{| class="wikitable"
 
−
! 01-Instance-From-Image || AIO - Simplex Virtual  || AIO - Duplex Virtual || Standard - Local Storage (2+2) Virtual || Standard - Dedicated Storage (2+2+2) Virtual || AIO - Simplex BM || AIO - Duplex BM  || Standard - Local Storage (2+2) BM || Standard - Dedicated Storage (2+2+2) BM || bug
 
 
|-
 
|-
−
| 1. Create Flavors for Instances || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|Stop Start Instances
 
|-
 
|-
−
| 2. Create Images for Instances || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|Lock Unlock Instances
 
|-
 
|-
−
| 3. Create Networks for Instances ||PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|Reboot Instances
 
|-
 
|-
−
| 4. Launch Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Rebuild Instances
 
|-
 
|-
−
| 5. Suspend Resume Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Resize Instances
 
|-
 
|-
−
| 6. Set Error Active Flags Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Set Unset Properties Instances
 
|-
 
|-
−
| 7. Pause Unpause Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Evacuate Instances From Hosts
 
|-
 
|-
−
| 8. Stop Start Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
! 03-Instance-From-Snapshot
 
|-
 
|-
−
| 9. Lock Unlock Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832237
+
|Create Flavors For Instances
 
|-
 
|-
−
| 10. Reboot Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Create Images For Instances
 
|-
 
|-
−
| 11. Rebuild Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Create Networks For Instances
 
|-
 
|-
−
| 12. Resize Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Create Volume For Instances
 
|-
 
|-
−
| 13. Set Unset Properties Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Create Snapshot For Instance
 
|-
 
|-
−
| 14. Evacuate Instances From Hosts || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Launch Instances
−
|}
 
−
 
 
−
{| class="wikitable"
 
 
|-
 
|-
−
! 02-Instance-From-Volume || AIO - Simplex Virtual  || AIO - Duplex Virtual || Standard - Local Storage (2+2) Virtual || Standard - Dedicated Storage (2+2+2) Virtual || AIO - Simplex BM || AIO - Duplex BM  || Standard - Local Storage (2+2) BM || Standard - Dedicated Storage (2+2+2) BM || bug
+
|Suspend Resume Instances
 
|-
 
|-
−
| 1. Create Flavors for Instances || PASS ||PASS || PASS||PASS||PASS||PASS||PASS||PASS||
+
|Set Error Active Flags Instances
 
|-
 
|-
−
| 2. Create Images for Instances || PASS ||PASS || PASS||PASS||PASS||PASS||PASS||PASS||
+
|Pause Unpause Instances
 
|-
 
|-
−
| 3. Create Networks for Instance || PASS ||PASS || PASS||PASS||PASS||PASS||PASS||PASS||
+
|Stop Start Instances
 
|-
 
|-
−
| 4. Create Volume for Instances ||  PASS ||PASS || PASS||PASS||PASS||PASS||PASS||PASS||
+
|Lock Unlock Instances
 
|-
 
|-
−
| 5. Launch Instances ||FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Reboot Instances
 
|-
 
|-
−
| 6. Suspend Resume Instance || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Rebuild Instances
 
|-
 
|-
−
| 7. Set Error Active Flags Instance || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Resize Instances
 
|-
 
|-
−
| 8. Pause Unpause Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Set Unset Properties Instances
 
|-
 
|-
−
| 9. Stop Start Instances ||FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Evacuate Instances From Hosts
 
|-
 
|-
−
| 10. Lock Unlock Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
! 04-Instance-From-Heat-Template
 
|-
 
|-
−
| 11. Reboot Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Create Flavors for Instance
 
|-
 
|-
−
|12. Rebuild Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Create Images For Instances
 
|-
 
|-
−
| 13. Resize Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Create Networks For Instances
 
|-
 
|-
−
| 14. Set Unset Properties Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Create Instance Trough Stack
 
|-
 
|-
−
| 15. Evacuate Instances From Hosts || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
! 05-Measurements-For-Metric
−
|}
 
−
 
 
−
{| class="wikitable"
 
 
|-
 
|-
−
! 03-Instance-From-Snapshot  || AIO - Simplex Virtual  || AIO - Duplex Virtual || Standard - Local Storage (2+2) Virtual || Standard - Dedicated Storage (2+2+2) Virtual || AIO - Simplex BM || AIO - Duplex BM  || Standard - Local Storage (2+2) BM || Standard - Dedicated Storage (2+2+2) BM || bug
+
|Create Image For Metrics
 
|-
 
|-
−
| 1. Create Flavors for Instances || PASS ||PASS || PASS||PASS||PASS||PASS||PASS||PASS||
+
|Update Image Name
 
|-
 
|-
−
| 2. Create Images for Instances || PASS ||PASS || PASS||PASS||PASS||PASS||PASS||PASS||
+
|Update Image Disk Ram Size
 
|-
 
|-
−
| 3. Create Networks for Instance || PASS ||PASS || PASS||PASS||PASS||PASS||PASS||PASS||
+
! SANITY-PLATFORM
 
|-
 
|-
−
| 4. Create Volume for Instances || PASS ||PASS || PASS||PASS||PASS||PASS||PASS||PASS||
+
! 01-OpenStack-Pod-Healthy
 
|-
 
|-
−
| 5. Create Snapshot for Instance || PASS ||PASS || PASS||PASS||PASS||PASS||PASS||PASS||
+
|OpenStack PODs Healthy
 
|-
 
|-
−
| 6. Launch Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Reapply STX OpenStack
 
|-
 
|-
−
| 7. Suspend Resume Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|STX OpenStack Override Update Reset
 
|-
 
|-
−
| 8. Set Error Active Flags Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Kube System Services
 
|-
 
|-
−
| 9. Pause Unpause Instances ||FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Create Check Delete POD
 
|-
 
|-
−
| 10. Stop Start Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
! 02-Host-Management
 
|-
 
|-
−
| 11. Lock Unlock Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A||  https://bugs.launchpad.net/starlingx/+bug/1832237
+
|Add Controller Host Simplex
 
|-
 
|-
−
| 12. Reboot Instances ||FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Swact Controller Host Simplex
 
|-
 
|-
−
| 13. Rebuild Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Lock Active Controller
 
|-
 
|-
−
| 14. Resize Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Lock Unlock Standby Controller
 
|-
 
|-
−
| 15. Set Unset Properties Instances || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Lock Unlock Compute Hosts
 
|-
 
|-
−
| 16. Evacuate Instances From Hosts || FAIL ||FAIL|| FAIL||N/A||FAIL||N/A||N/A||N/A|| https://bugs.launchpad.net/starlingx/+bug/1832279
+
|Lock Unlock Storage Hosts
−
 
 
 
|-
 
|-
−
 
 
|}
 
|}
−
 
+
== ''PyTest Framework'' ==
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
−
! 04-Instance-From-Heat-Template || AIO - Simplex Virtual  || AIO - Duplex Virtual || Standard - Local Storage (2+2) Virtual || Standard - Dedicated Storage (2+2+2) Virtual || AIO - Simplex BM || AIO - Duplex BM  || Standard - Local Storage (2+2) BM || Standard - Dedicated Storage (2+2+2) BM || bug
+
! SANITY
 +
|-
 +
|test_measurements_for_metric[image.size]
 +
|-
 +
|test_ssh_to_hosts
 +
|-
 +
|test_system_alarms_and_events_on_lock_unlock_compute
 +
|-
 +
|test_horizon_create_delete_instance
 +
|-
 +
|test_lock_unlock_host[controller]
 
|-
 
|-
−
| 1. Create Flavors for Instances || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|test_system_persist_over_host_reboot[controller]
 
|-
 
|-
−
| 2. Create Images for Instances ||PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|test_swact_controllers
 
|-
 
|-
−
| 3. Create Networks for Instance || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|test_ping_between_two_vms[default-virtio-virtio]
 
|-
 
|-
−
| 4. Create Instance Trough Stack || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|test_vm_meta_data_retrieval
−
|}
 
−
 
 
−
{| class="wikitable"
 
 
|-
 
|-
−
! 05-Measurements-For-Metric || AIO - Simplex Virtual  || AIO - Duplex Virtual || Standard - Local Storage (2+2) Virtual || Standard - Dedicated Storage (2+2+2) Virtual || AIO - Simplex BM || AIO - Duplex BM  || Standard - Local Storage (2+2) BM || Standard - Dedicated Storage (2+2+2) BM || bug
+
|test_migrate_vm[ubuntu_14-cold-dedicated]
 
|-
 
|-
−
| 1. Create Image For Metrics || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|test_migrate_vm[tis-centos-guest-live-None]
 
|-
 
|-
−
| 2. Update Image Name || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|test_nova_actions[tis-centos-guest-dedicated-pause-unpause]
 
|-
 
|-
−
| 3. Update Image Disk Ram Size || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|test_nova_actions[ubuntu_14-shared-stop-start]
−
|}
 
−
 
 
−
{| class="wikitable"
 
 
|-
 
|-
−
! Networking-Tests || AIO - Simplex Virtual  || AIO - Duplex Virtual || Standard - Local Storage (2+2) Virtual || Standard - Dedicated Storage (2+2+2) Virtual || AIO - Simplex BM || AIO - Duplex BM  || Standard - Local Storage (2+2) BM || Standard - Dedicated Storage (2+2+2) BM || bug
+
|test_nova_actions[ubuntu_14-dedicated-auto_recover]
 
|-
 
|-
−
| 1. Ping from Controller to Instances ||N/A|| N/A|| N/A|| N/A|| N/A|| N/A||N/A||N/A||
+
|test_nova_actions[tis-centos-guest-dedicated-suspend-resume]
 
|-
 
|-
−
| 2. Ping Between Instances || N/A|| N/A|| N/A|| N/A|| N/A|| N/A||N/A||N/A||
+
|test_openstack_services_healthy
−
|}
 
−
 
 
−
== ''Sanity-Platform'' ==
 
−
 
 
−
{| class="wikitable"
 
 
|-
 
|-
−
! 01-Host-Management || AIO - Simplex Virtual  ||AIO - Duplex Virtual|| Standard - Local Storage (2+2) Virtua || Standard - Dedicated Storage (2+2+2) Virtua|| AIO - Simplex BM || AIO - Duplex BM|| Standard - Local Storage (2+2) BM || Standard - Dedicated Storage (2+2+2) BM || Bug
+
|test_evacuate_vms
 
|-
 
|-
−
| 1. Lock Active Controller || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|test_reapply_stx_openstack_no_change[controller-0]
 
|-
 
|-
−
| 2. Lock Unlock Standby Controller || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|test_reapply_stx_openstack_no_change[controller-1]
 
|-
 
|-
−
| 3.  Lock Unlock Compute Hosts || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|test_stx_openstack_helm_override_update_and_reset
−
|}
 
−
 
 
−
{| class="wikitable"
 
 
|-
 
|-
−
! 02-sanity-platform || AIO - Simplex Virtual  ||AIO - Duplex Virtual|| Standard - Local Storage (2+2) Virtua || Standard - Dedicated Storage (2+2+2) Virtua|| AIO - Simplex BM || AIO - Duplex BM|| Standard - Local Storage (2+2) BM || Standard - Dedicated Storage (2+2+2) BM || Bug
+
|test_system_coredumps_and_crashes[core_dumps]
 
|-
 
|-
−
| 1. OpenStack PODs Healthy || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|test_system_coredumps_and_crashes[crash_reports]
 
|-
 
|-
−
| 2. Reapply STX OpenStack || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
|test_system_alarms
 
|-
 
|-
−
| 3. STX OpenStack Override Update Reset || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
! PLATFORM SANITY
 
|-
 
|-
−
| 4. Kube System Services || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
! SX SANITY
 
|-
 
|-
−
| 5. Create Check Delete POD || PASS ||FAIL|| FAIL||N/A||PASS||N/A||N/A||N/A||
+
! CPE SANITY
 
|}
 
|}

Revision as of 21:23, 22 October 2019

Sanity Test

Robot Framework

SANITY-OPENSTACK
01-Instance-From-Image
Create Flavors For Instances
Create Images For Instances
Create Networks For Instances
Launch Instances
Suspend Resume Instances
Set Error Active Flags Instances
Pause Unpause Instances
Stop Start Instances
Lock Unlock Instances
Reboot Instances
Rebuild Instances
Resize Instances
Set Unset Properties Instances
Evacuate Instances From Hosts
02-Instance-From-Volume
Create Flavors For Instances
Create Images For Instances
Create Networks For Instances
Create Volume For Instances
Launch Instances
Suspend Resume Instances
Set Error Active Flags Instances
Pause Unpause Instances
Stop Start Instances
Lock Unlock Instances
Reboot Instances
Rebuild Instances
Resize Instances
Set Unset Properties Instances
Evacuate Instances From Hosts
03-Instance-From-Snapshot
Create Flavors For Instances
Create Images For Instances
Create Networks For Instances
Create Volume For Instances
Create Snapshot For Instance
Launch Instances
Suspend Resume Instances
Set Error Active Flags Instances
Pause Unpause Instances
Stop Start Instances
Lock Unlock Instances
Reboot Instances
Rebuild Instances
Resize Instances
Set Unset Properties Instances
Evacuate Instances From Hosts
04-Instance-From-Heat-Template
Create Flavors for Instance
Create Images For Instances
Create Networks For Instances
Create Instance Trough Stack
05-Measurements-For-Metric
Create Image For Metrics
Update Image Name
Update Image Disk Ram Size
SANITY-PLATFORM
01-OpenStack-Pod-Healthy
OpenStack PODs Healthy
Reapply STX OpenStack
STX OpenStack Override Update Reset
Kube System Services
Create Check Delete POD
02-Host-Management
Add Controller Host Simplex
Swact Controller Host Simplex
Lock Active Controller
Lock Unlock Standby Controller
Lock Unlock Compute Hosts
Lock Unlock Storage Hosts

PyTest Framework

SANITY
test_measurements_for_metric[image.size]
test_ssh_to_hosts
test_system_alarms_and_events_on_lock_unlock_compute
test_horizon_create_delete_instance
test_lock_unlock_host[controller]
test_system_persist_over_host_reboot[controller]
test_swact_controllers
test_ping_between_two_vms[default-virtio-virtio]
test_vm_meta_data_retrieval
test_migrate_vm[ubuntu_14-cold-dedicated]
test_migrate_vm[tis-centos-guest-live-None]
test_nova_actions[tis-centos-guest-dedicated-pause-unpause]
test_nova_actions[ubuntu_14-shared-stop-start]
test_nova_actions[ubuntu_14-dedicated-auto_recover]
test_nova_actions[tis-centos-guest-dedicated-suspend-resume]
test_openstack_services_healthy
test_evacuate_vms
test_reapply_stx_openstack_no_change[controller-0]
test_reapply_stx_openstack_no_change[controller-1]
test_stx_openstack_helm_override_update_and_reset
test_system_coredumps_and_crashes[core_dumps]
test_system_coredumps_and_crashes[crash_reports]
test_system_alarms
PLATFORM SANITY
SX SANITY
CPE SANITY
Retrieved from "https://wiki.openstack.org/w/index.php?title=StarlingX/Test/SanityTests&oldid=172841"