Jump to: navigation, search

Difference between revisions of "PowerKVM"

(→‎Current Status)
(→‎Contact)
 
(48 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
= Overview =
 
= Overview =
−
This is the IBM KVM on Power CI status page. For questions and more information, please contact kvmpower@linux.vnet.ibm.com. You can also ping rfolco or mmedvede on OpenStack IRC channels @Freenode.
+
This is the IBM KVM on Power CI status page.
  
−
The KVM on Power CI cloud currently runs Openstack Mitaka release. Devstack-gate jobs run on Ubuntu Vivid single use slaves. Publicly available ppc64le version of CirrOS is used for second level guests. Previously, custom CirrOS image had to be built with mainline kernel for ppc64 as described in [1].
+
The KVM on Power CI performs tests using Ubuntu Xenial single use VMs. The VMs are spun up on OpenStack cloud, currently Mitaka release. CirrOS image with kernel >= 4.8.0-22 had to be built to support second level KVM acceleration on ppc64le (see [https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1634129 bug]).
  
−
Logs are uploaded to the Softlayer's Swift service[2] on PROJECT/PATCH_NUMBER[-2:]/PATCH_NUMBER/PATCH_SET directory structure. The test jobs are using devstack-gate to setup devstack and run the Tempest testsuite. Logs stored on Softlayer Swift expire in 1 month.
+
Logs are uploaded to IBM Softlayer's Swift [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/index.html container] and expire in 1 month. The test jobs are using [http://github.com/openstack-infra/devstack-gate devstack-gate] to setup devstack and run the Tempest test suite.
  
−
Checks for ironic project utilize real hardware.
+
Checks for Ironic project utilize bare metal hardware (ppc64le architecture).
  
−
* [1] https://www.ibm.com/developerworks/community/blogs/fe313521-2e95-46f2-817d-44a4f27eba32/entry/building_a_tiny_ppc_cloud_image_for_ci_testing?lang=en
+
== Contact ==
−
* [2] http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci
+
IRC: mjturek on OpenStack Freenode channels
  
 
= Current Status =
 
= Current Status =
−
* Reporting '''DISABLED'''
+
* nova, cinder, devstack, os-brick,  diskimage-builder reporting '''ENABLED''', voting only on success due to instability
 +
* ironic reporting '''ENABLED'''
  
 
{| class="wikitable"
 
{| class="wikitable"
Line 19: Line 20:
 
! Start !! End !! Description
 
! Start !! End !! Description
 
|-
 
|-
−
| 2016-11-02 13:00 UTC  || 2016-11-02 21:00 UTC || Network outage for maintenance  
+
| 2018-11-06 20:00 UTC  || 2018-11-07 16:00 UTC || Cloud provider down
 +
|-
 +
| 2018-09-28 18:00 UTC  || 2018-10-01 15:00 UTC || Scheduled lab maintenance
 +
|-
 +
| 2018-04-04 03:00 UTC  || 2018-04-04 18:00 UTC || Cloud provider down
 +
|-
 +
| 2018-03-21 22:00 UTC  || 2018-03-22 17:00 UTC || Network down after an upgrade
 +
|-
 +
| 2018-02-28 14:00 UTC  || 2018-02-28 21:00 UTC || Network outage
 +
|-
 +
| 2017-06-21 19:02 UTC  || 2017-06-21 19:58 UTC || Network
 +
|-
 +
| 2017-06-17 06:00 UTC  || 2017-06-19 12:00 UTC || Zuul failed
 +
|-
 +
| 2017-05-23 12:00 UTC  || 2017-05-24 20:00 UTC || Too many tempest failures due to breaking changes in glance, os-brick, nova
 +
|-
 +
| 2017-04-15 04:00 UTC  || 2017-04-17 16:00 UTC || Zuul failure due to dependency update (voluptuous)
 +
|-
 +
| 2017-04-11 11:30 UTC  || 2017-04-11 14:00 UTC || Network
 +
|-
 +
| 2017-04-07 19:30 UTC  || 2017-04-08 03:00 UTC || Network
 +
|-
 +
| 2017-03-16 15:00 UTC  || 2017-03-16 18:00 UTC || Zuul fail
 +
|-
 +
| 2017-03-07 15:00 UTC  || 2017-03-07 17:00 UTC || apt cache server is down
 +
|-
 +
| 2017-02-13 18:50 UTC  || 2017-02-14 01:00 UTC || Cloud down after a power outage
 +
|-
 +
| 2017-02-07 16:30 UTC  || 2017-02-07 18:00 UTC || Network outage
 +
|-
 +
| 2016-12-08 12:00 UTC  || 2016-12-08 17:00 UTC || pypi mirror is down
 +
|-
 +
| 2016-11-15 14:00 UTC  || 2016-11-17 15:00 UTC || Packet loss on route to git.o.o
 +
|-
 +
| 2016-11-02 13:00 UTC  || 2016-11-02 18:00 UTC || Network outage for maintenance
 
|-
 
|-
 
| 2016-10-04 09:00 UTC  || 2016-10-05 13:00 UTC || devstack-vivid failure due pyeclib requirement updated to 1.3.1
 
| 2016-10-04 09:00 UTC  || 2016-10-05 13:00 UTC || devstack-vivid failure due pyeclib requirement updated to 1.3.1
Line 94: Line 129:
  
 
= Projects Tested =
 
= Projects Tested =
−
Current test status for all tested projects can be seen at  
+
Current test status for all tested projects can be seen at
−
[http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/ Zuul Testing Report].
+
[http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/index.html Zuul Testing Report].
  
 
The report is compiled form zuul logs and is updated periodically. It might include aborted or otherwise terminated
 
The report is compiled form zuul logs and is updated periodically. It might include aborted or otherwise terminated
Line 102: Line 137:
 
== Reporting ==
 
== Reporting ==
  
−
* [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/devstack/ Devstack]
+
* Devstack
−
* [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/ironic/ Ironic]
+
* Ironic
−
* [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/nova/ Nova]
+
* Nova
 +
* Cinder
 +
* os-brick
 +
* diskimage-builder
 +
 
  
 
== Non-reporting ==
 
== Non-reporting ==
  
−
* [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/cinder/ Cinder]
+
* Devstack-gate
−
* [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/devstack-gate/ Devstack-gate]
+
* Neutron
−
* [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/glance/ Glance]
+
* Glance
−
* [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/keystone/ Keystone]
+
* Keystone
−
* [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/nova/periodic/periodic-gate-ibm-nova-python27/ Nova Python27]
+
* Nova Unit tests
−
* [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/requirements/ Requirements]
+
* Requirements
−
* [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/swift/ Swift]
+
* Swift
−
* [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/tempest/ Tempest]
+
* Tempest
  
 
= Skipped Tests =
 
= Skipped Tests =
−
Nova:
+
 
−
3 tests are skipped due to missing PCI hotplug support in CirrOS ppc64le version 2015-09-23:
+
There are currently a few skipped tests, see test run logs for details.
−
* tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario
 
−
* tempest.scenario.test_encrypted_cinder_volumes.TestEncryptedCinderVolumes.test_encrypted_cinder_volumes_luks
 
−
* tempest.scenario.test_encrypted_cinder_volumes.TestEncryptedCinderVolumes.test_encrypted_cinder_volumes_cryptsetup
 
−
See test run logs for details.
 
  
 
== Known Issues ==
 
== Known Issues ==
 
* Missing PCI hotplug support in CirrOS ppc64le
 
* Missing PCI hotplug support in CirrOS ppc64le
 
   * Neutron (3) failures to missing interface attach support (PCI hotplug) in CirrOS
 
   * Neutron (3) failures to missing interface attach support (PCI hotplug) in CirrOS
−
* Xenial stability issues for experimental jobs (Swift), possibly related to MySQL
 
  
 
== PowerKVM CI related blogs ==
 
== PowerKVM CI related blogs ==

Latest revision as of 02:59, 17 February 2019

Overview

This is the IBM KVM on Power CI status page.

The KVM on Power CI performs tests using Ubuntu Xenial single use VMs. The VMs are spun up on OpenStack cloud, currently Mitaka release. CirrOS image with kernel >= 4.8.0-22 had to be built to support second level KVM acceleration on ppc64le (see bug).

Logs are uploaded to IBM Softlayer's Swift container and expire in 1 month. The test jobs are using devstack-gate to setup devstack and run the Tempest test suite.

Checks for Ironic project utilize bare metal hardware (ppc64le architecture).

Contact

IRC: mjturek on OpenStack Freenode channels

Current Status

  • nova, cinder, devstack, os-brick, diskimage-builder reporting ENABLED, voting only on success due to instability
  • ironic reporting ENABLED
Outage history
Start End Description
2018-11-06 20:00 UTC 2018-11-07 16:00 UTC Cloud provider down
2018-09-28 18:00 UTC 2018-10-01 15:00 UTC Scheduled lab maintenance
2018-04-04 03:00 UTC 2018-04-04 18:00 UTC Cloud provider down
2018-03-21 22:00 UTC 2018-03-22 17:00 UTC Network down after an upgrade
2018-02-28 14:00 UTC 2018-02-28 21:00 UTC Network outage
2017-06-21 19:02 UTC 2017-06-21 19:58 UTC Network
2017-06-17 06:00 UTC 2017-06-19 12:00 UTC Zuul failed
2017-05-23 12:00 UTC 2017-05-24 20:00 UTC Too many tempest failures due to breaking changes in glance, os-brick, nova
2017-04-15 04:00 UTC 2017-04-17 16:00 UTC Zuul failure due to dependency update (voluptuous)
2017-04-11 11:30 UTC 2017-04-11 14:00 UTC Network
2017-04-07 19:30 UTC 2017-04-08 03:00 UTC Network
2017-03-16 15:00 UTC 2017-03-16 18:00 UTC Zuul fail
2017-03-07 15:00 UTC 2017-03-07 17:00 UTC apt cache server is down
2017-02-13 18:50 UTC 2017-02-14 01:00 UTC Cloud down after a power outage
2017-02-07 16:30 UTC 2017-02-07 18:00 UTC Network outage
2016-12-08 12:00 UTC 2016-12-08 17:00 UTC pypi mirror is down
2016-11-15 14:00 UTC 2016-11-17 15:00 UTC Packet loss on route to git.o.o
2016-11-02 13:00 UTC 2016-11-02 18:00 UTC Network outage for maintenance
2016-10-04 09:00 UTC 2016-10-05 13:00 UTC devstack-vivid failure due pyeclib requirement updated to 1.3.1
2016-07-13 20:00 UTC 2016-07-14 18:00 UTC Internal pypi mirror is down
2016-06-26 05:00 UTC 2016-06-27 22:00 UTC Timeout while installing python XStatic-Angular-Bootstrap
2016-06-24 02:20 UTC 2016-06-24 16:00 UTC Pypi mirror server down, and pypi.python.org is too weak to handle the load
2016-06-21 21:00 UTC 2016-06-23 02:00 UTC Underlying cloud is down: no patches were missed, long delay in posting results
2016-05-25 00:00 UTC 2016-05-26 16:00 UTC devstack setup failure due to our Ubuntu images using upstart, infrastracture problems
2016-05-23 08:00 UTC 2016-05-23 11:00 UTC Zuul failed, zuul-merger hanging
2016-05-12 12:30 UTC 2016-05-12 14:30 UTC Zuul failed
2016-05-08 08:00 UTC 2016-05-09 11:00 UTC Zuul lost connection to gerrit.o.o
2016-04-29 05:30 UTC 2016-04-29 16:30 UTC Jobs failing due to missing dev dependency
2016-04-19 12:30 UTC 2016-04-19 19:00 UTC Network outage
2016-04-10 11:00 UTC 2016-04-11 19:00 UTC
2016-03-15 17:30 UTC 2016-03-16 12:05 UTC Zuul down
2016-03-03 14:00 UTC 2016-03-03 18:00 UTC Zuul fails to listen gerrit.o.o events
2016-02-13 13:00 UTC 2016-02-15 14:00 UTC Zuul reconnect failure after gerrit.o.o restart
2016-02-12 10:00 UTC 2016-02-12 19:00 UTC Network loss
2016-01-26 19:00 UTC 2016-01-27 15:00 UTC Network loss
2016-01-19 10:00 UTC 2016-01-19 18:00 UTC Loss of Jenkins master
2015-12-21 12:15 UTC 2015-12-22 12:45 UTC Jobs failing due to launchpad bug #1447704
2015-12-04 20:00 UTC 2015-12-07 18:00 UTC Jobs are unstable
2015-11-30 04:00 UTC 2015-11-30 13:00 UTC nodepool failing
2015-10-19 09:00 UTC 2015-10-19 17:00 UTC reporting failures due to outdated pypi mirror
2015-08-24 00:00 UTC 2015-08-24 13:00 UTC bad zuul configuration
2015-08-08 06:00 UTC 2015-08-10 12:50 UTC zuul failed to update configuration
2015-07-30 13:00 UTC 2015-07-31 21:00 UTC Numerous failures due to upper_constraints, disabled constraints for the gate
2015-07-27 12:00 UTC 2015-07-27 16:00 UTC zuul/jenkins failure
2015-07-03 13:00 UTC 2015-07-03 19:00 UTC Gate was failing due to devstack-gate patch that enabled pypi package upper constraints
2015-07-01 00:00 UTC 2015-07-01 14:00 UTC Jenkins crashed (leap second suspected as the cause)
2015-06-30 01:00 UTC 2015-06-30 13:00 UTC Zuul did not reconnect to gerrit after connectivity loss
2015-06-16 16:00 UTC 2015-06-16 19:00 UTC Could not install python-libguestfs on Fedora 21 guests, yum cache problems
2015-05-16 17:30 UTC 2015-05-18 02:00 UTC Zuul service down
2015-05-04 16:00 UTC 2015-05-04 17:00 UTC Maintenance
2015-04-25 19:00 UTC 2015-04-26 16:00 UTC Zuul VM down
2015-04-11 08:00 UTC 2015-04-11 17:00 UTC Internal network down
2015-04-10 14:00 UTC 2015-04-10 19:00 UTC Internal network down

Projects Tested

Current test status for all tested projects can be seen at Zuul Testing Report.

The report is compiled form zuul logs and is updated periodically. It might include aborted or otherwise terminated builds.

Reporting

  • Devstack
  • Ironic
  • Nova
  • Cinder
  • os-brick
  • diskimage-builder


Non-reporting

  • Devstack-gate
  • Neutron
  • Glance
  • Keystone
  • Nova Unit tests
  • Requirements
  • Swift
  • Tempest

Skipped Tests

There are currently a few skipped tests, see test run logs for details.

Known Issues

  • Missing PCI hotplug support in CirrOS ppc64le
 * Neutron (3) failures to missing interface attach support (PCI hotplug) in CirrOS

PowerKVM CI related blogs

Retrieved from "https://wiki.openstack.org/w/index.php?title=PowerKVM&oldid=168409"