StarlingX/stx.2018.10 Testplan
< StarlingX
Revision as of 23:38, 2 October 2018 by Ada.cabrales (talk | contribs)
Test plan for stx.2018.10
Test Number | Domain | Test description |
---|---|---|
1 | Simplex | Admin Password change on Simplex |
2 | Simplex | Power down/up recovery on Simplex |
3 | Simplex | Lock unlock Simplex |
4 | Simplex | Pmon monitored process |
5 | Simplex | Unsupported sysInv Commands |
6 | Simplex | Verify branding on Simplex |
7 | Simplex | Verify memory assignment on Simplex |
8 | Simplex | Verify migration is rejected on Simplex |
9 | Simplex | Verify resize and rebuild on Simplex |
10 | Simplex | Lock/unlock 10 times |
11 | Simplex | Reboot system 10 times |
12 | Simplex | Verify add/Delete controller and compute hosts |
13 | Simplex | Verify host lock/unlock soak - 10 iterations of all host types |
14 | Simplex | Verify several controller swacts |
15 | Simplex | Horizon login screen displays StarlingX |
16 | Simplex | Horizon system type shows All-in-one |
17 | Fault Management | Alarms: Verify order of columns are correct |
18 | Fault Management | Verify order of alarms reverse chronologically |
19 | Fault Management | Hierarchical Suppression of Alarms for Locked Compute Node |
20 | Fault Management | Hierarchical Suppression of Alarms for Locked Controller Node |
21 | Fault Management | Suppress alarm and verify alarms on CLI |
22 | Fault Management | Suppress alarm and verify alarms on GUI |
23 | Fault Management | Verify new Alarms are not listed when they are suppressed |
24 | Fault Management | SNMP CLI - Communities: Comunity can be deleted |
25 | Fault Management | SNMP CLI - Comunitites: Create new comunity. |
26 | Fault Management | SNMP CLI- Communities: Check comunity details |
27 | High Availability | Controlled Swact Timing |
28 | High Availability | Uncontrolled Swact Timing |
29 | High Availability | Controller HA: Swact the controllers 20 times. Verify VM launch and deletions after every swact |
30 | High Availability | Halt standby controller from the Linux command line (halt -f) |
31 | High Availability | Standby controller lock (GUI) |
32 | High Availability | Validate that you can use the https address to access Horizon |
33 | High Availability | Kill all services (one by one) managed by SM (on active controller) |
34 | High Availability | Kill Critical Process on the active Controller node |
35 | High Availability | Kill Critical Process on the standby Controller node |
36 | High Availability | Kill Major Process on the standby Controller node |
37 | High Availability | Controller: Service Group Redundancy Alarm |
38 | High Availability | Service Group State alarm |
39 | High Availability | system CLI - display HA system servicegroup list and state |
40 | High Availability | System CLI - display HA system service list and state |
41 | High Availability | With both controllers running, confirm crm_mon command and validate all services running on at least one controller |
42 | Horizon | Edit Image for volume in Horizon and add Instance Auto Recovery, verify metadata updated |
43 | Horizon | Edit Image of snapshot in Horizon and add Instance Auto Recovery, verify metadata updated |
44 | Horizon | Verify that after installation of the second controller the GUI does not stop working |
45 | Horizon | Horizon login time using All-in-one |
46 | Horizon | Horizon login time using Regular |
47 | Horizon | Horizon login time using Storage |
48 | Installation and Config | During config controller enter invalid values and verify that they are rejected - DB Size below and above |
49 | Installation and Config | During config controller enter invalid values and verify that they are rejected - OAM addresses are not in the oam pool selected |
50 | Installation and Config | During config controller enter invalid values and verify that they are rejected - Valid format for IP |
51 | Installation and Config | During config controller enter invalid values and verify that they are rejected -Image Volume Storage |
52 | Installation and Config | Import profile and apply to multiple nodes |
53 | Installation and Config | Swact controllers during configure compute |
54 | Installation and Config | SysInv: iprofile create, show attr, delete |
55 | Installation and Config | Ceph back-end using static IP addressing - using non-interactive config_controller |
56 | Installation and Config | Validate ability to edit Host Personality, Hostname in Horizon prior node provisioning |
57 | Installation and Config | Verify that logical volumes can be resized up on the controller (CLI/GUI) |
58 | Installation and Config | Attempt to decrease partition size |
59 | Installation and Config | Attempt to increase partition size |
60 | Installation and Config | Attempt unlock host where nova-local lvg exists but does not have physical volume |
61 | Installation and Config | Controller node basic provisioning check |
62 | Installation and Config | Validate modified system host-disk-list CLI output |
63 | Installation and Config | Validate Partition Modify operations |
64 | Networking | Off-line static configuration |
65 | Networking | Off-line static configuration for External OAM Interface |
66 | Networking | On-line Static configuration validation: compatible NTP configuration should be applied via CLI |
67 | Networking | On-line Static configuration validation: Not compatible configuration for External OAM Interface should not be applied |
68 | Networking | On-line Static configuration validation : compatible configuration for External OAM Interface should be applied via CLI |
69 | Networking | On-line Static configuration validation: swact controllers should be rejected until controller-0 config is not up-to-date |
70 | Networking | Provider network Down Alarm |
71 | Networking | Static configuration validation: compatible NTP configuration should be applied via GUI |
72 | Networking | Verify alarm generation for neutron DHCP agent scheduling states |
73 | Networking | Verify alarm generation for neutron L3 agent scheduling states |
74 | Networking | Verify alarm generation for neutron provider network state |
75 | Networking | On-line Static configuration validation: swact controllers should be rejected until controller-0 config for External OAM Interface is done |
76 | Networking | Verify appropriate values should be used for modifying of interfaces (CLI, GUI) |
77 | Networking | Verify ethernet management interface is updated successfully on controller |
78 | Networking | Verify ethernet OAM interface is updated successfully on controller. |
79 | Networking | Verify LAG OAM interface is updated successfully on controller |
80 | Networking | Verify that internal customer managed tenant network works |
81 | Networking | Verify that System name can be modified via CLI and GUI |
82 | Networking | Verify that unlocked powered off host can not be deleted (CLI, GUI) |
83 | Networking | Verify that Vswitch CPU/Numa isolation parameters can be applied via iprofile |
84 | Networking | interfaces cannot be deleted on unlocked host |
85 | Networking | Appropriate values should be used to modify if ports |
86 | Networking | Appropriate values should be used to add new interfaces |
87 | Networking | Verify GUI support for crypto device config |
88 | Networking | Verify Traffic Control Class setup |
89 | Networking | Boot Instance with Virtio interfaces using new NIC |
90 | Networking | Audit Provider Network Connectivity Test after all slave computes reboot |
91 | Networking | Audit Provider Network Connectivity Test after deleting vxlan segment range |
92 | Networking | Audit Provider Network Connectivity Test of type flat |
93 | Networking | Providernet Connectivity Test List using CLI |
94 | Networking | Network Topology page in Horizon after clean install but prior to providernet and compute config |
95 | Networking | Enable/Disable internal dns resolution 2+2 standard system |
96 | Networking | Enable/Disable internal dns resolution All-In-One system |
97 | Networking | Power down/up recovery test with direct connect CPE mode, followed by swact operations |
98 | Networking | New cli commands for system modify system_mode=duplex-direct |
99 | Networking | New cli output for system_mode in 'system show' |
100 | Nova | Check Nova Diagnostics |
101 | Nova | Flavor - Adding access to the flavor |
102 | Nova | Flavor - Removing access to the flavor |
103 | Nova | Flavor - Flavor in use can not be modified (or deleted) |
104 | Nova | ImageSnapshot - snapshot a running server, show snapshot and delete image |
105 | Nova | Aggregates - Update aggregates and host(s) |
106 | Nova | CPUPolicy - CPU Policy set in Horizon extra specs |
107 | Nova | CPUPolicy - From the cli, set CPU Policy in the extra spec (valid value) |
108 | Nova | CPUPolicy - From both Horizon and cli, test validation of hw:cpu_policy - invalid or empty values |
109 | Nova | CPUPolicy - Test case sensitive entries for the value of the hw:cpu_policy |
110 | Nova | CPUScaling - Using cli/Horizon,verify error returned if the extra spec setting for minimum CPU exceeds flavor vCPUs set |
111 | Nova | CPUScaling - Verify error when attempt to set the minimum CPU to value below minimum valid range- zero or negative |
112 | Nova | Evacuations of instances with cpu thread policies using cli command |
113 | Nova | MemPageSize -Verify invalid or empty values for hw:mem_page_size |
114 | Nova | MigrationTime - Test Live migrate operation is not allowed (from the GUI) if the other compute is locked |
115 | Nova | MigrationTime - Create Flavor extra spec live migration max downtime within valid range for live migration max downtime |
116 | Nova | Server Actions -Rebuild interaction |
117 | Nova | Server Actions - Nova delete instance in error state |
118 | Nova | SharedCPU - Using horizon/cli, create flavor with hw:cpu_policy “dedicated” and valid hw:wrs:shared_vcpu values |
119 | Nova | SharedCPU - Instance with hw:wrs:shared_vcpu respected on resize |
120 | Nova | Local Storage - Neutron port status after successful migration ACTIVE (cold, live, block) |
121 | Nova | Local Storage - Local LVM backed instance - cold, live, block migration combinations |
122 | Nova | Local Storage - Launch instance from volume snapshot (created using Horizon) with Local_CoW Image Backed storage |
123 | Nova | Local Storage - Launch instance from volume snapshot (created using cli) with Local_CoW Image Backed storage |
124 | Nova | Local Storage - Delete volume snapshot - instance has been launched from it |
125 | Nova | Local Storage - Delete volume snapshot - instance has never been launched from it |
126 | Nova | Local Storage - Update metadata in volume and snapshot |
127 | Nova | RecoveryHeartBeat - Using cli/horizon set flavor extra sw:wrs:guest:heartbeat to valid values |
128 | Security | SSH - SSH root access sshd config file changed, Connection rejected. |
129 | Security | Passwd - wrsroot Password expiration. |
130 | Security | Keystone - Nova user Passwords are protected. |
131 | Security | Keystone - Change admin password. |
132 | Security | Keystone - Adding users to keystone user list via horizon. |
133 | Security | Passwd - system admin user is capable of changing password quality. |
134 | Security | Passwd - wrsroot changed password and propagated. |
135 | Security | Horizon - relogin after timed out horizon session. |
136 | SNMP | SNMP V2C Agent init and launching on HA Controller(s) (w/default config) |
137 | SNMP | SNMP CLI - System Group: Verify SNMP get/getNext for system group |
138 | SNMP | SNMP CLI - Trap dest: Verify trap receiver |
139 | SNMP | snmpd process recovers properly |
140 | SNMP | Verification of SNMP get/getnext/set with and without configured community |
141 | SNMP | Verify that the SNMP warmStart trap is properly generated when controllers switchover |
142 | SNMP | SNMP - audit logging configuration |
143 | SNMP | SNMP - successful GETBULK request logging |
144 | SNMP | SNMP - Verify log permissions and ownership |
145 | Storage | Verify snapshot via cinder snapshot-create |
146 | Storage | 20 10gb volumes - controlled/uncontrolled swact |
147 | Storage | controlled/uncontrolled swact with no volumes |
148 | Storage | Volume clone time |
149 | Storage | Disable Local Storage |
150 | Storage | Launch an instance using localstorage flavour boot from image |
151 | Storage | kill VM and restart VM and validate no issues with re-attaching to Cinder Volume with external storage |
152 | Storage | Verify logs of activity |
153 | System Inventory | change the dns server ip addresses using cli |
154 | System Inventory | change the dns server ip addresses using gui |
155 | System Inventory | change the mtu value of the data interface using api |
156 | System Inventory | change the mtu value of the data interface using cli |
157 | System Inventory | change the mtu value of the data interface using gui |
158 | System Inventory | change the mtu value of the oam interface using gui |
159 | System Inventory | change the size of the image storage pool on a ceph-based system using gui |
160 | System Inventory | Check Resource Usage panel working properly |
161 | System Inventory | export hosts information host-bulk-export api |
162 | System Inventory | export hosts information host-bulk-export cli |
163 | System Inventory | invalid inputs for number of hugepages will be rejected gui |
164 | System Inventory | modify number of hugepages using cli |
165 | System Inventory | modify number of hugepages using gui |
166 | System Inventory | query the product type on cpe system using cli |
167 | System Inventory | query the product type on cpe system using gui |
168 | System Inventory | query the product type on std system using cli |
169 | System Inventory | query the product type on std system using gui |
170 | System Inventory | Test creating and applying interface profile |
171 | System Inventory | verify that alarm can be deleted using cli |
172 | System Inventory | verify that the cpu data can be seen via cli |
173 | System Inventory | verify the system type is read-only and cannot be changed via cli |
174 | System Inventory | verify the system type is read-only and cannot be changed via gui |
175 | System Inventory | verify wrong interface profiles will be rejected |
176 | System Inventory | Verify CLI system infra-modify rejects action=any_value_not_apply |
177 | System Inventory | Verify CLI system ntp-modify rejects action=any_value_not_apply |
178 | System Inventory | Verify CLI system pm-modify does not require option action=apply |
179 | System | Power down/up recovery recovery of system with storage |
180 | System | Evacuation of VM via rebooting a compute node |
181 | System | Guest failures: kill VM |
182 | System | Guest failures: rebootVM |
183 | System | Halt -f on active controller and confirm other controller takes activity; also then confirm can launch new VMs |
184 | System | kill Compute node and re-schedule VM elsewhere and validate no issues with re-attaching to Cinder Volume |
185 | System | Kill kvm process multiple times-10 verify VM instance recovery |
186 | System | Lock/unlcok standby controller and swact 10 times |
187 | System | Lock/unlock a compute |
188 | System | Recovery after DOS |
189 | System | Run 10 Controller swacts triggered by reboot -f command |
190 | System | Run 10 evacuate tests |
191 | System | Run 10 VM reboots and ensure automatic recovery |
192 | System | Run 15 Controller swacts triggered by reboot command |
193 | System | Run 15 controller swacts via cli followed by VM launch/delete |
194 | System | Run cold/live migrations |
195 | Maintenance | Neutron - Verify that the same actions that were allowed on horizon can still take place with authentication |
196 | Maintenance | Verify host power-down/power-up feature for locked compute node via CLI |
197 | Maintenance | Verify host power-down/power-up feature for locked compute node via GUI |
198 | Maintenance | Verify host power-down/power-up feature for locked controller node via CLI |
199 | Maintenance | Verify host power-down/power-up feature for unlocked compute node via CLI |
200 | Maintenance | Verify host power-down/power-up feature for unlocked compute node via GUI |
201 | Maintenance | Verify host power-down/power-up feature for unlocked controller node via CLI |
202 | Maintenance | Verify host power-down/power-up feature for unlocked controller node via GUI |
203 | Maintenance | Verify host power-down/power-up feature for unlocked storage node via CLI |
204 | Maintenance | Verify host power-down/power-up feature for unlocked storage node via GUI |
205 | Maintenance | Verify audit interval is 120 seconds for all groups after initial provisioning |
206 | Maintenance | Verify BMC auto monitoring |
207 | Maintenance | Verify BMC deprovisioning behavior (no board management controller) |
208 | Maintenance | Verify BMC provisioning with bad IP leads to BMC access alarm |
209 | Maintenance | Verify changing BMC MAC or IP address to invalid values (sensors go offline) |
210 | Maintenance | double fault evacuation of vm with controller and compute nodes |
211 | Maintenance | Controller Node HeartBeat Failure Handling |
212 | Maintenance | Verify mtce heartbeat parameters can be modified by user and result are accurate |
213 | Maintenance | Lock force a compute with instances that won’t or can’t move (compute should be rebooted). |
214 | Maintenance | Pull Management Cable on Active Controller |
215 | Maintenance | Verify that compute host will reboot if management network is admin down |
216 | Maintenance | Verify that in-active controller host will reboot if management network is admin down |
217 | Maintenance | Compute Resource Maintenance and Local Alarm |
218 | Maintenance | Auto-Provision and Auto-Enable of controller-0 |
219 | Maintenance | Delete Compute Host |
220 | Maintenance | Verify Host 'Delete' Feature |
221 | Maintenance | Verify if an alarm is present on node it is cleared after it is deleted |
222 | Maintenance | verify storage system install success in progress bar preinstall, installing, post install, installed |
223 | Maintenance | Lock Compute Host With Live Instance Migration - Failure path |
224 | Maintenance | Lock Compute Host With Live Instance Migration - Success path |
225 | Maintenance | swact rejected inactive controller |
226 | Maintenance | reject swact to failed controller |
227 | Maintenance | verify that personality cant be changed for host |
228 | Maintenance | verify compute cant be swacted |
229 | Maintenance | name cannot be changed on unlocked host |
230 | Maintenance | Verfiy SysInv DB rejects hosts with same hostname or mgmt_mac or mgmt_ip |
231 | Maintenance | Time stamp correct in Alarms after time zone has been updated |
232 | Maintenance | Time stamp correct in Logs after time zone has been updated |
233 | Maintenance | Time zone can revert to default setting |
234 | Maintenance | Time zone persistent after controller lock and unlock |
235 | Maintenance | Time zone persistent after controller reboot |
236 | Maintenance | Time zone persistent after controller swact |
237 | Maintenance | Verify that the CPU data is seen on the Host Details Page in Horizon |
238 | Maintenance | Verify that the Memory data is seen on the Host Details Page in Horizon |
Please take a look here for the test with instructions included.