Difference between revisions of "NovaVMware/Minesweeper/Status"

Revision as of 18:55, 6 February 2014

Current Service Status

No known issues at this time There are known issues currently affecting CI

Blocking Bugs

From time to time, there may be critical bugs that may be preventing Minesweeper from running correctly. This section lists these open bugs and any related fixes that need to be merged in order for successful test results.

Bug	Fix (needs review attention)
VMware: exception handling throws error - AttributeError: 'NoneType' object has no attribute 'getChildren'	VMware: raise more specific exception
VMware: Unable to launch instance because vmware_base already exists	VMware: fix race for datastore directory existence
VMware: Unable to delete image cache vmdk while launching instance	VMware: prevent race for vmdk deletion
Concurrent requests to quotas reserve sometimes lock db table

Service Disruption History

All times are in PST.

Event Time	Description
Feb 6, 10:53 AM	Hypervisor crash contributed to build failures, currently fixing. Quota database problem still affecting builds, looking to manually fix entries in the database.
Feb 5, 4:07 PM	Identified Concurrent requests to quotas reserve sometimes lock db table as a possible cause for infra issues. Still investigating.
Feb 5, 1:47 AM	Small percentage of builds failing due to infra issues. Investigating.
Feb 4, 5:00 PM	Infrastructure issue have been resolved.
Feb 3, 8:29 AM	Infrastructure issue involving testbed provision. Small percentage of builds may be failing.

@@ Line 33: / Line 33: @@
 ! Event Time
 ! Description
+|-
+| Feb 6, 10:53 AM
+| Hypervisor crash contributed to build failures, currently fixing. Quota database problem still affecting builds, looking to manually fix entries in the database.
 |-
 | Feb 5, 4:07 PM