StarlingX/stx.2018.10 Testplan
< StarlingX
Revision as of 23:37, 21 September 2018 by Ada.cabrales (talk | contribs) (Created page with "=== Test plan for stx.2018.10 === {| class="wikitable" |- ! Test Number !! Domain !! Test description |- | 1 || Simplex || Admin Password change on Simplex |-...")
Test plan for stx.2018.10
| Test Number | Domain | Test description |
|---|---|---|
| 1 | Simplex | Admin Password change on Simplex |
| 2 | Simplex | Power down/up recovery on Simplex |
| 3 | Simplex | Lock unlock Simplex |
| 4 | Simplex | Pmon monitored process |
| 5 | Simplex | Unsupported sysInv Commands |
| 6 | Simplex | Verify branding on Simplex |
| 7 | Simplex | Verify memory assignment on Simplex |
| 8 | Simplex | Verify migration is rejected on Simplex |
| 9 | Simplex | Verify resize and rebuild on Simplex |
| 10 | Simplex | Lock/unlock 10 times |
| 11 | Simplex | Reboot system 10 times |
| 12 | Simplex | Verify add/Delete controller and compute hosts |
| 13 | Simplex | Verify host lock/unlock soak - 10 iterations of all host types |
| 14 | Simplex | Verify several controller swacts |
| 15 | Simplex | Horizon login screen displays StarlingX |
| 16 | Simplex | Horizon system type shows All-in-one |
| 17 | Fault Management | Alarms: Verify order of columns are correct |
| 18 | Fault Management | Verify order of alarms reverse chronologically |
| 19 | Fault Management | Hierarchical Suppression of Alarms for Locked Compute Node |
| 20 | Fault Management | Hierarchical Suppression of Alarms for Locked Controller Node |
| 21 | Fault Management | Suppress alarm and verify alarms on CLI |
| 22 | Fault Management | Suppress alarm and verify alarms on GUI |
| 23 | Fault Management | Verify new Alarms are not listed when they are suppressed |
| 24 | Fault Management | SNMP CLI - Communities: Comunity can be deleted |
| 25 | Fault Management | SNMP CLI - Comunitites: Create new comunity. |
| 26 | Fault Management | SNMP CLI- Communities: Check comunity details |
| 27 | High Availability | Controlled Swact Timing |
| 28 | High Availability | Uncontrolled Swact Timing |
| 29 | High Availability | Controller HA: Swact the controllers 20 times. Verify VM launch and deletions after every swact |
| 30 | High Availability | Halt standby controller from the Linux command line (halt -f) |
| 31 | High Availability | Standby controller lock (GUI) |
| 32 | High Availability | Validate that you can use the https address to access Horizon |
| 33 | High Availability | Kill all services (one by one) managed by SM (on active controller) |
| 34 | High Availability | Kill Critical Process on the active Controller node |
| 35 | High Availability | Kill Critical Process on the standby Controller node |
| 36 | High Availability | Kill Major Process on the standby Controller node |
| 37 | High Availability | Controller: Service Group Redundancy Alarm |
| 38 | High Availability | Service Group State alarm |
| 39 | High Availability | system CLI - display HA system servicegroup list and state |
| 40 | High Availability | System CLI - display HA system service list and state |
| 41 | High Availability | With both controllers running, confirm crm_mon command and validate all services running on at least one controller |
| 42 | Horizon | Edit Image for volume in Horizon and add Instance Auto Recovery, verify metadata updated |
| 43 | Horizon | Edit Image of snapshot in Horizon and add Instance Auto Recovery, verify metadata updated |
| 44 | Horizon | Verify that after installation of the second controller the GUI does not stop working |
| 45 | Horizon | Horizon login time using All-in-one |
| 46 | Horizon | Horizon login time using Regular |
| 47 | Horizon | Horizon login time using Storage |
| 48 | Installation and Config | During config controller enter invalid values and verify that they are rejected - DB Size below and above |
| 49 | Installation and Config | During config controller enter invalid values and verify that they are rejected - OAM addresses are not in the oam pool selected |
| 50 | Installation and Config | During config controller enter invalid values and verify that they are rejected - Valid format for IP |
| 51 | Installation and Config | During config controller enter invalid values and verify that they are rejected -Image Volume Storage |
| 52 | Installation and Config | Import profile and apply to multiple nodes |
| 53 | Installation and Config | Swact controllers during configure compute |
| 54 | Installation and Config | SysInv: iprofile create, show attr, delete |
| 55 | Installation and Config | Ceph back-end using static IP addressing - using non-interactive config_controller |
| 56 | Installation and Config | Validate ability to edit Host Personality, Hostname in Horizon prior node provisioning |
| 57 | Installation and Config | Verify that logical volumes can be resized up on the controller (CLI/GUI) |
| 58 | Installation and Config | Attempt to decrease partition size |
| 59 | Installation and Config | Attempt to increase partition size |
| 60 | Installation and Config | Attempt unlock host where nova-local lvg exists but does not have physical volume |
| 61 | Installation and Config | Controller node basic provisioning check |
| 62 | Installation and Config | Validate modified system host-disk-list CLI output |
| 63 | Installation and Config | Validate Partition Modify operations |
| 64 | Networking | Off-line static configuration |
| 65 | Networking | Off-line static configuration for External OAM Interface |
| 66 | Networking | On-line Static configuration validation: compatible NTP configuration should be applied via CLI |
| 67 | Networking | On-line Static configuration validation: Not compatible configuration for External OAM Interface should not be applied |
| 68 | Networking | On-line Static configuration validation : compatible configuration for External OAM Interface should be applied via CLI |
| 69 | Networking | On-line Static configuration validation: swact controllers should be rejected until controller-0 config is not up-to-date |
| 70 | Networking | Provider network Down Alarm |
| 71 | Networking | Static configuration validation: compatible NTP configuration should be applied via GUI |
| 72 | Networking | Verify alarm generation for neutron DHCP agent scheduling states |
| 73 | Networking | Verify alarm generation for neutron L3 agent scheduling states |
| 74 | Networking | Verify alarm generation for neutron provider network state |
| 75 | Networking | On-line Static configuration validation: swact controllers should be rejected until controller-0 config for External OAM Interface is done |
| 76 | Networking | Verify appropriate values should be used for modifying of interfaces (CLI, GUI) |
| 77 | Networking | Verify ethernet management interface is updated successfully on controller |
| 78 | Networking | Verify ethernet OAM interface is updated successfully on controller. |
| 79 | Networking | Verify LAG OAM interface is updated successfully on controller |
| 80 | Networking | Verify that internal customer managed tenant network works |
| 81 | Networking | Verify that System name can be modified via CLI and GUI |
| 82 | Networking | Verify that unlocked powered off host can not be deleted (CLI, GUI) |
| 83 | Networking | Verify that Vswitch CPU/Numa isolation parameters can be applied via iprofile |
| 84 | Networking | interfaces cannot be deleted on unlocked host |
| 85 | Networking | Appropriate values should be used to modify if ports |
| 86 | Networking | Appropriate values should be used to add new interfaces |
| 87 | Networking | Verify GUI support for crypto device config |
| 88 | Networking | Verify Traffic Control Class setup |
| 89 | Networking | Boot Instance with Virtio interfaces using new NIC |
| 90 | Networking | Audit Provider Network Connectivity Test after all slave computes reboot |
| 91 | Networking | Audit Provider Network Connectivity Test after deleting vxlan segment range |
| 92 | Networking | Audit Provider Network Connectivity Test of type flat |
| 93 | Networking | Providernet Connectivity Test List using CLI |
| 94 | Networking | Network Topology page in Horizon after clean install but prior to providernet and compute config |
| 95 | Networking | Enable/Disable internal dns resolution 2+2 standard system |
| 96 | Networking | Enable/Disable internal dns resolution All-In-One system |
| 97 | Networking | Power down/up recovery test with direct connect CPE mode, followed by swact operations |
| 98 | Networking | New cli commands for system modify system_mode=duplex-direct |
| 99 | Networking | New cli output for system_mode in 'system show' |
| 100 | Nova | Check Nova Diagnostics |
| 101 | Nova | Flavor - Adding access to the flavor |
| 102 | Nova | Flavor - Removing access to the flavor |
| 103 | Nova | Flavor - Flavor in use can not be modified (or deleted) |
| 104 | Nova | ImageSnapshot - snapshot a running server, show snapshot and delete image |
| 105 | Nova | Aggregates - Update aggregates and host(s) |
| 106 | Nova | CPUPolicy - CPU Policy set in Horizon extra specs |
| 107 | Nova | CPUPolicy - From the cli, set CPU Policy in the extra spec (valid value) |
| 108 | Nova | CPUPolicy - From both Horizon and cli, test validation of hw:cpu_policy - invalid or empty values |
| 109 | Nova | CPUPolicy - Test case sensitive entries for the value of the hw:cpu_policy |
| 110 | Nova | CPUScaling - Using cli/Horizon,verify error returned if the extra spec setting for minimum CPU exceeds flavor vCPUs set |
| 111 | Nova | CPUScaling - Verify error when attempt to set the minimum CPU to value below minimum valid range- zero or negative |
| 112 | Nova | Evacuations of instances with cpu thread policies using cli command |
| 113 | Nova | MemPageSize -Verify invalid or empty values for hw:mem_page_size |
| 114 | Nova | MigrationTime - Test Live migrate operation is not allowed (from the GUI) if the other compute is locked |
| 115 | Nova | MigrationTime - Create Flavor extra spec live migration max downtime within valid range for live migration max downtime |
| 116 | Nova | Server Actions -Rebuild interaction |
| 117 | Nova | Server Actions - Nova delete instance in error state |
| 118 | Nova | SharedCPU - Using horizon/cli, create flavor with hw:cpu_policy “dedicated” and valid hw:wrs:shared_vcpu values |
| 119 | Nova | SharedCPU - Instance with hw:wrs:shared_vcpu respected on resize |
| 120 | Nova | Local Storage - Neutron port status after successful migration ACTIVE (cold, live, block) |
| 121 | Nova | Local Storage - Local LVM backed instance - cold, live, block migration combinations |
| 122 | Nova | Local Storage - Launch instance from volume snapshot (created using Horizon) with Local_CoW Image Backed storage |
| 123 | Nova | Local Storage - Launch instance from volume snapshot (created using cli) with Local_CoW Image Backed storage |
| 124 | Nova | Local Storage - Delete volume snapshot - instance has been launched from it |
| 125 | Nova | Local Storage - Delete volume snapshot - instance has never been launched from it |
| 126 | Nova | Local Storage - Update metadata in volume and snapshot |
| 127 | Nova | RecoveryHeartBeat - Using cli/horizon set flavor extra sw:wrs:guest:heartbeat to valid values |
| 128 | Security | SSH - SSH root access sshd config file changed, Connection rejected. |
| 129 | Security | Passwd - wrsroot Password expiration. |
| 130 | Security | Keystone - Nova user Passwords are protected. |
| 131 | Security | Keystone - Change admin password. |
| 132 | Security | Keystone - Adding users to keystone user list via horizon. |
| 133 | Security | Passwd - system admin user is capable of changing password quality. |
| 134 | Security | Passwd - wrsroot changed password and propagated. |
| 135 | Security | Horizon - relogin after timed out horizon session. |
| 136 | SNMP | SNMP V2C Agent init and launching on HA Controller(s) (w/default config) |
| 137 | SNMP | SNMP CLI - System Group: Verify SNMP get/getNext for system group |
| 138 | SNMP | SNMP CLI - Trap dest: Verify trap receiver |
| 139 | SNMP | snmpd process recovers properly |
| 140 | SNMP | Verification of SNMP get/getnext/set with and without configured community |
| 141 | SNMP | Verify that the SNMP warmStart trap is properly generated when controllers switchover |
| 142 | SNMP | SNMP - audit logging configuration |
| 143 | SNMP | SNMP - successful GETBULK request logging |
| 144 | SNMP | SNMP - Verify log permissions and ownership |
| 145 | Storage | Verify snapshot via cinder snapshot-create |
| 146 | Storage | 20 10gb volumes - controlled/uncontrolled swact |
| 147 | Storage | controlled/uncontrolled swact with no volumes |
| 148 | Storage | Volume clone time |
| 149 | Storage | Disable Local Storage |
| 150 | Storage | Launch an instance using localstorage flavour boot from image |
| 151 | Storage | kill VM and restart VM and validate no issues with re-attaching to Cinder Volume with external storage |
| 152 | Storage | Verify logs of activity |
| 153 | System Inventory | change the dns server ip addresses using cli |
| 154 | System Inventory | change the dns server ip addresses using gui |
| 155 | System Inventory | change the mtu value of the data interface using api |
| 156 | System Inventory | change the mtu value of the data interface using cli |
| 157 | System Inventory | change the mtu value of the data interface using gui |
| 158 | System Inventory | change the mtu value of the oam interface using gui |
| 159 | System Inventory | change the size of the image storage pool on a ceph-based system using gui |
| 160 | System Inventory | Check Resource Usage panel working properly |
| 161 | System Inventory | export hosts information host-bulk-export api |
| 162 | System Inventory | export hosts information host-bulk-export cli |
| 163 | System Inventory | invalid inputs for number of hugepages will be rejected gui |
| 164 | System Inventory | modify number of hugepages using cli |
| 165 | System Inventory | modify number of hugepages using gui |
| 166 | System Inventory | query the product type on cpe system using cli |
| 167 | System Inventory | query the product type on cpe system using gui |
| 168 | System Inventory | query the product type on std system using cli |
| 169 | System Inventory | query the product type on std system using gui |
| 170 | System Inventory | Test creating and applying interface profile |
| 171 | System Inventory | verify that alarm can be deleted using cli |
| 172 | System Inventory | verify that the cpu data can be seen via cli |
| 173 | System Inventory | verify the system type is read-only and cannot be changed via cli |
| 174 | System Inventory | verify the system type is read-only and cannot be changed via gui |
| 175 | System Inventory | verify wrong interface profiles will be rejected |
| 176 | System Inventory | Verify CLI system infra-modify rejects action=any_value_not_apply |
| 177 | System Inventory | Verify CLI system ntp-modify rejects action=any_value_not_apply |
| 178 | System Inventory | Verify CLI system pm-modify does not require option action=apply |
| 179 | System | Power down/up recovery recovery of system with storage |
| 180 | System | Evacuation of VM via rebooting a compute node |
| 181 | System | Guest failures: kill VM |
| 182 | System | Guest failures: rebootVM |
| 183 | System | Halt -f on active controller and confirm other controller takes activity; also then confirm can launch new VMs |
| 184 | System | kill Compute node and re-schedule VM elsewhere and validate no issues with re-attaching to Cinder Volume |
| 185 | System | Kill kvm process multiple times-10 verify VM instance recovery |
| 186 | System | Lock/unlcok standby controller and swact 10 times |
| 187 | System | Lock/unlock a compute |
| 188 | System | Recovery after DOS |
| 189 | System | Run 10 Controller swacts triggered by reboot -f command |
| 190 | System | Run 10 evacuate tests |
| 191 | System | Run 10 VM reboots and ensure automatic recovery |
| 192 | System | Run 15 Controller swacts triggered by reboot command |
| 193 | System | Run 15 controller swacts via cli followed by VM launch/delete |
| 194 | System | Run cold/live migrations |
| 195 | Maintenance | Neutron - Verify that the same actions that were allowed on horizon can still take place with authentication |
| 196 | Maintenance | Verify host power-down/power-up feature for locked compute node via CLI |
| 197 | Maintenance | Verify host power-down/power-up feature for locked compute node via GUI |
| 198 | Maintenance | Verify host power-down/power-up feature for locked controller node via CLI |
| 199 | Maintenance | Verify host power-down/power-up feature for unlocked compute node via CLI |
| 200 | Maintenance | Verify host power-down/power-up feature for unlocked compute node via GUI |
| 201 | Maintenance | Verify host power-down/power-up feature for unlocked controller node via CLI |
| 202 | Maintenance | Verify host power-down/power-up feature for unlocked controller node via GUI |
| 203 | Maintenance | Verify host power-down/power-up feature for unlocked storage node via CLI |
| 204 | Maintenance | Verify host power-down/power-up feature for unlocked storage node via GUI |
| 205 | Maintenance | Verify audit interval is 120 seconds for all groups after initial provisioning |
| 206 | Maintenance | Verify BMC auto monitoring |
| 207 | Maintenance | Verify BMC deprovisioning behavior (no board management controller) |
| 208 | Maintenance | Verify BMC provisioning with bad IP leads to BMC access alarm |
| 209 | Maintenance | Verify changing BMC MAC or IP address to invalid values (sensors go offline) |
| 210 | Maintenance | double fault evacuation of vm with controller and compute nodes |
| 211 | Maintenance | Controller Node HeartBeat Failure Handling |
| 212 | Maintenance | Verify mtce heartbeat parameters can be modified by user and result are accurate |
| 213 | Maintenance | Lock force a compute with instances that won’t or can’t move (compute should be rebooted). |
| 214 | Maintenance | Pull Management Cable on Active Controller |
| 215 | Maintenance | Verify that compute host will reboot if management network is admin down |
| 216 | Maintenance | Verify that in-active controller host will reboot if management network is admin down |
| 217 | Maintenance | Compute Resource Maintenance and Local Alarm |
| 218 | Maintenance | Auto-Provision and Auto-Enable of controller-0 |
| 219 | Maintenance | Delete Compute Host |
| 220 | Maintenance | Verify Host 'Delete' Feature |
| 221 | Maintenance | Verify if an alarm is present on node it is cleared after it is deleted |
| 222 | Maintenance | verify storage system install success in progress bar preinstall, installing, post install, installed |
| 223 | Maintenance | Lock Compute Host With Live Instance Migration - Failure path |
| 224 | Maintenance | Lock Compute Host With Live Instance Migration - Success path |
| 225 | Maintenance | swact rejected inactive controller |
| 226 | Maintenance | reject swact to failed controller |
| 227 | Maintenance | verify that personality cant be changed for host |
| 228 | Maintenance | verify compute cant be swacted |
| 229 | Maintenance | name cannot be changed on unlocked host |
| 230 | Maintenance | Verfiy SysInv DB rejects hosts with same hostname or mgmt_mac or mgmt_ip |
| 231 | Maintenance | Time stamp correct in Alarms after time zone has been updated |
| 232 | Maintenance | Time stamp correct in Logs after time zone has been updated |
| 233 | Maintenance | Time zone can revert to default setting |
| 234 | Maintenance | Time zone persistent after controller lock and unlock |
| 235 | Maintenance | Time zone persistent after controller reboot |
| 236 | Maintenance | Time zone persistent after controller swact |
| 237 | Maintenance | Verify that the CPU data is seen on the Host Details Page in Horizon |
| 238 | Maintenance | Verify that the Memory data is seen on the Host Details Page in Horizon |
