Difference between revisions of "NovaVMware/Minesweeper/Status"

Revision as of 23:50, 20 February 2014

Current Service Status

No known issues at this time CI is offline or running with limited capacity

Blocking Bugs

These bugs may be causing Minesweeper to fail in the form of failing tests. If there are fixes available, they need to be merged soon for Minesweeper to run correctly.

Severity	Bug	Fix (needs review attention)
Medium	VMware: Unable to launch instance because vmware_base already exists	VMware: fix race for datastore directory existence
Low	VMware: attaching a volume to the VM failed	VMware: Support hot-adding for VMs with IDE controllers

Service Disruption History

All times are in PST.

Event Time	Description
Feb 20, 2:46 PM	A commit that added a test for cinder backup caused some tests to fail. These tests are not supported and has been added to the exclude list.
Feb 18, 11:10 AM	A commit that added 2 new tests in Tempest caused build failures yesterday. These tests are not supported by the VMware Cinder driver and have been added to the exclude list.
Feb 14, 10:38 AM	Last night, Minesweeper ran out of floating IPs causing builds to fail. Back online but running with only 2 executors.
Feb 10, 8:48 PM	Rescue bug has been merged. Infrastructure issues are still being addressed. CI is currently offline.
Feb 10, 2:58 PM	One infra issue resolved having to do with proxy failure. Minesweeper is still running on 1 executor while other infrastructure issues are being addressed.
Feb 10, 2:33 PM	Rescue patch is in gate now, however, infra issues are still being worked on.
Feb 9, 10:47 AM	Infra issues are being worked on. Minesweeper is also hitting a Rescue bug causing some tests to fail.
Feb 7, 2:46 PM	Keystone requirement problem has been resolved as per this bug. CI remains offline due to infrastructure issues.
Feb 7, 1:30 PM	Infrastructure issues are still affecting builds. Builds have also started to become affected by keystone requirements bug. Minesweeper will be taken offline until these issues are resolved.
Feb 6, 10:53 AM	Hypervisor crash contributed to build failures, currently fixing. Quota database problem still affecting builds, looking to manually fix entries in the database.
Feb 5, 4:07 PM	Identified Concurrent requests to quotas reserve sometimes lock db table as a possible cause for infra issues. Still investigating.
Feb 5, 1:47 AM	Small percentage of builds failing due to infra issues. Investigating.
Feb 4, 5:00 PM	Infrastructure issue have been resolved.
Feb 3, 8:29 AM	Infrastructure issue involving testbed provision. Small percentage of builds may be failing.

@@ Line 17: / Line 17: @@
 | [https://bugs.launchpad.net/nova/+bug/1266579 VMware: Unable to launch instance because vmware_base already exists]
 | [https://review.openstack.org/#/c/65306 VMware: fix race for datastore directory existence]
+|-
+| Low
+| [https://bugs.launchpad.net/nova/+bug/1226543 VMware: attaching a volume to the VM failed]
+| [https://review.openstack.org/#/c/65799/ VMware: Support hot-adding for VMs with IDE controllers]
 |}