Oath FFU Juno To Ocata

Preparing to upgrade

1. Create CMR.
2. Finalize CMR steps
3. Testing chef bootstrap & converge for api, mq, db, HVs
4. Verify runbooks for SRE/PE
5. Add new VIPs to LB
6. Validate control plane pipeline
7. HV preparation
   1. Upgrade all compute nodes to RHEL 7
8. DB preparation
   1. archive deleted rows
   2. validate backups
9. Verify network ACLs are correct
10. Update the Horizon Banner with CMR information
11. Add VIP settings in Chef recipe
12. Internal announcements on intranet, email, etc
13. Build new jumphosts that use openstack-client, rather than the novaclient

Upgrade

1. Take cluster snapshot of VMs status
2. Start CMR
3. Snooze monitoring alerts and stop functional testing jobs
4. Stop api and mq services
5. Stop nova compute on all HVs & confirm
6. Backup db
7. Run DB migration script
8. Re-image API and MQ nodes to RHEL-7
9. Chef bootstrap API, MQ, and DB nodes
10. Env and runlist association
11. Start deployment for API, MQ, DB nodes.
12. Deploy to two Compute nodes online as canaries
13. Bring cluster online with the two canary compute nodes
14. Validate VM creation, deletion work as expected. Ensure VMs don't disappear from canary hypervisors.
15. Upgrade remaining compute nodes
16. Post deployment checks for Control plane and HVs
17. Clean up any additional db backup did during migration