Difference between revisions of "PowerKVM"
(→Current Status) |
(→Contact) |
||
(133 intermediate revisions by 6 users not shown) | |||
Line 1: | Line 1: | ||
− | This is the IBM | + | = Overview = |
+ | This is the IBM KVM on Power CI status page. | ||
+ | The KVM on Power CI performs tests using Ubuntu Xenial single use VMs. The VMs are spun up on OpenStack cloud, currently Mitaka release. CirrOS image with kernel >= 4.8.0-22 had to be built to support second level KVM acceleration on ppc64le (see [https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1634129 bug]). | ||
− | + | Logs are uploaded to IBM Softlayer's Swift [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/index.html container] and expire in 1 month. The test jobs are using [http://github.com/openstack-infra/devstack-gate devstack-gate] to setup devstack and run the Tempest test suite. | |
− | + | ||
+ | Checks for Ironic project utilize bare metal hardware (ppc64le architecture). | ||
+ | == Contact == | ||
+ | IRC: mjturek on OpenStack Freenode channels | ||
− | + | = Current Status = | |
+ | * nova, cinder, devstack, os-brick, diskimage-builder reporting '''ENABLED''', voting only on success due to instability | ||
+ | * ironic reporting '''ENABLED''' | ||
{| class="wikitable" | {| class="wikitable" | ||
Line 13: | Line 20: | ||
! Start !! End !! Description | ! Start !! End !! Description | ||
|- | |- | ||
− | | 2015-04-11 08:00 UTC || || Internal network down | + | | 2018-11-06 20:00 UTC || 2018-11-07 16:00 UTC || Cloud provider down |
+ | |- | ||
+ | | 2018-09-28 18:00 UTC || 2018-10-01 15:00 UTC || Scheduled lab maintenance | ||
+ | |- | ||
+ | | 2018-04-04 03:00 UTC || 2018-04-04 18:00 UTC || Cloud provider down | ||
+ | |- | ||
+ | | 2018-03-21 22:00 UTC || 2018-03-22 17:00 UTC || Network down after an upgrade | ||
+ | |- | ||
+ | | 2018-02-28 14:00 UTC || 2018-02-28 21:00 UTC || Network outage | ||
+ | |- | ||
+ | | 2017-06-21 19:02 UTC || 2017-06-21 19:58 UTC || Network | ||
+ | |- | ||
+ | | 2017-06-17 06:00 UTC || 2017-06-19 12:00 UTC || Zuul failed | ||
+ | |- | ||
+ | | 2017-05-23 12:00 UTC || 2017-05-24 20:00 UTC || Too many tempest failures due to breaking changes in glance, os-brick, nova | ||
+ | |- | ||
+ | | 2017-04-15 04:00 UTC || 2017-04-17 16:00 UTC || Zuul failure due to dependency update (voluptuous) | ||
+ | |- | ||
+ | | 2017-04-11 11:30 UTC || 2017-04-11 14:00 UTC || Network | ||
+ | |- | ||
+ | | 2017-04-07 19:30 UTC || 2017-04-08 03:00 UTC || Network | ||
+ | |- | ||
+ | | 2017-03-16 15:00 UTC || 2017-03-16 18:00 UTC || Zuul fail | ||
+ | |- | ||
+ | | 2017-03-07 15:00 UTC || 2017-03-07 17:00 UTC || apt cache server is down | ||
+ | |- | ||
+ | | 2017-02-13 18:50 UTC || 2017-02-14 01:00 UTC || Cloud down after a power outage | ||
+ | |- | ||
+ | | 2017-02-07 16:30 UTC || 2017-02-07 18:00 UTC || Network outage | ||
+ | |- | ||
+ | | 2016-12-08 12:00 UTC || 2016-12-08 17:00 UTC || pypi mirror is down | ||
+ | |- | ||
+ | | 2016-11-15 14:00 UTC || 2016-11-17 15:00 UTC || Packet loss on route to git.o.o | ||
+ | |- | ||
+ | | 2016-11-02 13:00 UTC || 2016-11-02 18:00 UTC || Network outage for maintenance | ||
+ | |- | ||
+ | | 2016-10-04 09:00 UTC || 2016-10-05 13:00 UTC || devstack-vivid failure due pyeclib requirement updated to 1.3.1 | ||
+ | |- | ||
+ | | 2016-07-13 20:00 UTC || 2016-07-14 18:00 UTC || Internal pypi mirror is down | ||
+ | |- | ||
+ | | 2016-06-26 05:00 UTC || 2016-06-27 22:00 UTC || Timeout while installing python XStatic-Angular-Bootstrap | ||
+ | |- | ||
+ | | 2016-06-24 02:20 UTC || 2016-06-24 16:00 UTC || Pypi mirror server down, and pypi.python.org is too weak to handle the load | ||
+ | |- | ||
+ | | 2016-06-21 21:00 UTC || 2016-06-23 02:00 UTC || Underlying cloud is down: no patches were missed, long delay in posting results | ||
+ | |- | ||
+ | | 2016-05-25 00:00 UTC || 2016-05-26 16:00 UTC || devstack setup failure due to our Ubuntu images using upstart, infrastracture problems | ||
+ | |- | ||
+ | | 2016-05-23 08:00 UTC || 2016-05-23 11:00 UTC || Zuul failed, zuul-merger hanging | ||
+ | |- | ||
+ | | 2016-05-12 12:30 UTC || 2016-05-12 14:30 UTC || Zuul failed | ||
+ | |- | ||
+ | | 2016-05-08 08:00 UTC || 2016-05-09 11:00 UTC || Zuul lost connection to gerrit.o.o | ||
+ | |- | ||
+ | | 2016-04-29 05:30 UTC || 2016-04-29 16:30 UTC || Jobs failing due to missing dev dependency | ||
+ | |- | ||
+ | | 2016-04-19 12:30 UTC || 2016-04-19 19:00 UTC || Network outage | ||
+ | |- | ||
+ | | 2016-04-10 11:00 UTC || 2016-04-11 19:00 UTC || | ||
+ | |- | ||
+ | | 2016-03-15 17:30 UTC || 2016-03-16 12:05 UTC || Zuul down | ||
+ | |- | ||
+ | | 2016-03-03 14:00 UTC || 2016-03-03 18:00 UTC || Zuul fails to listen gerrit.o.o events | ||
+ | |- | ||
+ | | 2016-02-13 13:00 UTC || 2016-02-15 14:00 UTC || Zuul reconnect failure after gerrit.o.o restart | ||
+ | |- | ||
+ | | 2016-02-12 10:00 UTC || 2016-02-12 19:00 UTC || Network loss | ||
+ | |- | ||
+ | | 2016-01-26 19:00 UTC || 2016-01-27 15:00 UTC || Network loss | ||
+ | |- | ||
+ | | 2016-01-19 10:00 UTC || 2016-01-19 18:00 UTC || Loss of Jenkins master | ||
+ | |- | ||
+ | | 2015-12-21 12:15 UTC || 2015-12-22 12:45 UTC || Jobs failing due to launchpad bug #1447704 | ||
+ | |- | ||
+ | | 2015-12-04 20:00 UTC || 2015-12-07 18:00 UTC || Jobs are unstable | ||
+ | |- | ||
+ | | 2015-11-30 04:00 UTC || 2015-11-30 13:00 UTC || nodepool failing | ||
+ | |- | ||
+ | | 2015-10-19 09:00 UTC || 2015-10-19 17:00 UTC || reporting failures due to outdated pypi mirror | ||
+ | |- | ||
+ | | 2015-08-24 00:00 UTC || 2015-08-24 13:00 UTC || bad zuul configuration | ||
+ | |- | ||
+ | | 2015-08-08 06:00 UTC || 2015-08-10 12:50 UTC || zuul failed to update configuration | ||
+ | |- | ||
+ | | 2015-07-30 13:00 UTC || 2015-07-31 21:00 UTC || Numerous failures due to upper_constraints, disabled constraints for the gate | ||
+ | |- | ||
+ | | 2015-07-27 12:00 UTC || 2015-07-27 16:00 UTC || zuul/jenkins failure | ||
+ | |- | ||
+ | | 2015-07-03 13:00 UTC || 2015-07-03 19:00 UTC || Gate was failing due to devstack-gate patch that enabled pypi package upper constraints | ||
+ | |- | ||
+ | | 2015-07-01 00:00 UTC || 2015-07-01 14:00 UTC || Jenkins crashed (leap second suspected as the cause) | ||
+ | |- | ||
+ | | 2015-06-30 01:00 UTC || 2015-06-30 13:00 UTC || Zuul did not reconnect to gerrit after connectivity loss | ||
+ | |- | ||
+ | | 2015-06-16 16:00 UTC || 2015-06-16 19:00 UTC || Could not install python-libguestfs on Fedora 21 guests, yum cache problems | ||
+ | |- | ||
+ | | 2015-05-16 17:30 UTC || 2015-05-18 02:00 UTC || Zuul service down | ||
+ | |- | ||
+ | | 2015-05-04 16:00 UTC || 2015-05-04 17:00 UTC || Maintenance | ||
+ | |- | ||
+ | | 2015-04-25 19:00 UTC || 2015-04-26 16:00 UTC || Zuul VM down | ||
+ | |- | ||
+ | | 2015-04-11 08:00 UTC || 2015-04-11 17:00 UTC || Internal network down | ||
|- | |- | ||
| 2015-04-10 14:00 UTC || 2015-04-10 19:00 UTC || Internal network down | | 2015-04-10 14:00 UTC || 2015-04-10 19:00 UTC || Internal network down | ||
Line 19: | Line 128: | ||
|} | |} | ||
− | = Projects = | + | = Projects Tested = |
− | == | + | Current test status for all tested projects can be seen at |
− | + | [http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/index.html Zuul Testing Report]. | |
+ | |||
+ | The report is compiled form zuul logs and is updated periodically. It might include aborted or otherwise terminated | ||
+ | builds. | ||
+ | |||
+ | == Reporting == | ||
+ | |||
+ | * Devstack | ||
+ | * Ironic | ||
* Nova | * Nova | ||
− | * | + | * Cinder |
+ | * os-brick | ||
+ | * diskimage-builder | ||
− | == | + | |
− | + | == Non-reporting == | |
− | * | + | |
− | * | + | * Devstack-gate |
+ | * Neutron | ||
+ | * Glance | ||
+ | * Keystone | ||
+ | * Nova Unit tests | ||
+ | * Requirements | ||
+ | * Swift | ||
* Tempest | * Tempest | ||
= Skipped Tests = | = Skipped Tests = | ||
− | |||
− | + | There are currently a few skipped tests, see test run logs for details. | |
− | + | ||
− | |||
− | |||
== Known Issues == | == Known Issues == | ||
− | + | * Missing PCI hotplug support in CirrOS ppc64le | |
− | * | + | * Neutron (3) failures to missing interface attach support (PCI hotplug) in CirrOS |
− | = | + | == PowerKVM CI related blogs == |
− | * | + | * [https://developer.ibm.com/opentech/2016/04/21/kvm-on-power-ci-increasing-the-test-coverage/ KVM on Power CI: Increasing the test coverage] |
− | * | + | * [https://developer.ibm.com/opentech/2016/04/20/ironic-continuous-integration-testing-with-hardware-pool-management/ Ironic CI with physical Power target nodes] |
− | * | + | * [https://www.ibm.com/developerworks/community/blogs/fe313521-2e95-46f2-817d-44a4f27eba32/entry/building_a_tiny_ppc_cloud_image_for_ci_testing?lang=en CirrOS] |
− | * | + | * [https://www.ibm.com/developerworks/community/blogs/fe313521-2e95-46f2-817d-44a4f27eba32/entry/using_ibmveth_and_ibmvscsi_on_openstack?lang=en Using ibmveth and ibmvscsi] |
− | * | + | * [https://www.ibm.com/developerworks/community/blogs/fe313521-2e95-46f2-817d-44a4f27eba32/entry/spawning_a_little_endian_instance_in_openstack?lang=en LE guest] |
+ | * [https://www.ibm.com/developerworks/community/blogs/fe313521-2e95-46f2-817d-44a4f27eba32/entry/powerkvm_openstack_ci_a_custom_swift_upload_script?lang=en Swift script] | ||
+ | * [https://www.ibm.com/developerworks/community/blogs/fe313521-2e95-46f2-817d-44a4f27eba32/entry/building_your_openstack_3rd_party_ci_system1?lang=en Continuous Integration] | ||
+ | * [https://www.ibm.com/developerworks/community/blogs/fe313521-2e95-46f2-817d-44a4f27eba32/entry/openstack_nova_scheduler_disable_ram_weigher?lang=en RAM Weigher] | ||
+ | * [https://www.ibm.com/developerworks/community/blogs/fe313521-2e95-46f2-817d-44a4f27eba32/entry/setup_ibm_storwize_for_openstack_cinder_and_glance?lang=en Storwize] | ||
+ | * [http://www.kurttaylor.com/blog/2014/11/third-party-ci-work-group-at-kilo-summit/ Third Party CI Working Group] | ||
+ | * [http://www.kurttaylor.com/blog/2015/03/clean-swift-object-store/ Cleaning up Swift] | ||
+ | * [http://www.kurttaylor.com/blog/2015/05/ci-tools-repo/ Third Party CI Working Group Tools Repository] |
Latest revision as of 02:59, 17 February 2019
Contents
Overview
This is the IBM KVM on Power CI status page.
The KVM on Power CI performs tests using Ubuntu Xenial single use VMs. The VMs are spun up on OpenStack cloud, currently Mitaka release. CirrOS image with kernel >= 4.8.0-22 had to be built to support second level KVM acceleration on ppc64le (see bug).
Logs are uploaded to IBM Softlayer's Swift container and expire in 1 month. The test jobs are using devstack-gate to setup devstack and run the Tempest test suite.
Checks for Ironic project utilize bare metal hardware (ppc64le architecture).
Contact
IRC: mjturek on OpenStack Freenode channels
Current Status
- nova, cinder, devstack, os-brick, diskimage-builder reporting ENABLED, voting only on success due to instability
- ironic reporting ENABLED
Start | End | Description |
---|---|---|
2018-11-06 20:00 UTC | 2018-11-07 16:00 UTC | Cloud provider down |
2018-09-28 18:00 UTC | 2018-10-01 15:00 UTC | Scheduled lab maintenance |
2018-04-04 03:00 UTC | 2018-04-04 18:00 UTC | Cloud provider down |
2018-03-21 22:00 UTC | 2018-03-22 17:00 UTC | Network down after an upgrade |
2018-02-28 14:00 UTC | 2018-02-28 21:00 UTC | Network outage |
2017-06-21 19:02 UTC | 2017-06-21 19:58 UTC | Network |
2017-06-17 06:00 UTC | 2017-06-19 12:00 UTC | Zuul failed |
2017-05-23 12:00 UTC | 2017-05-24 20:00 UTC | Too many tempest failures due to breaking changes in glance, os-brick, nova |
2017-04-15 04:00 UTC | 2017-04-17 16:00 UTC | Zuul failure due to dependency update (voluptuous) |
2017-04-11 11:30 UTC | 2017-04-11 14:00 UTC | Network |
2017-04-07 19:30 UTC | 2017-04-08 03:00 UTC | Network |
2017-03-16 15:00 UTC | 2017-03-16 18:00 UTC | Zuul fail |
2017-03-07 15:00 UTC | 2017-03-07 17:00 UTC | apt cache server is down |
2017-02-13 18:50 UTC | 2017-02-14 01:00 UTC | Cloud down after a power outage |
2017-02-07 16:30 UTC | 2017-02-07 18:00 UTC | Network outage |
2016-12-08 12:00 UTC | 2016-12-08 17:00 UTC | pypi mirror is down |
2016-11-15 14:00 UTC | 2016-11-17 15:00 UTC | Packet loss on route to git.o.o |
2016-11-02 13:00 UTC | 2016-11-02 18:00 UTC | Network outage for maintenance |
2016-10-04 09:00 UTC | 2016-10-05 13:00 UTC | devstack-vivid failure due pyeclib requirement updated to 1.3.1 |
2016-07-13 20:00 UTC | 2016-07-14 18:00 UTC | Internal pypi mirror is down |
2016-06-26 05:00 UTC | 2016-06-27 22:00 UTC | Timeout while installing python XStatic-Angular-Bootstrap |
2016-06-24 02:20 UTC | 2016-06-24 16:00 UTC | Pypi mirror server down, and pypi.python.org is too weak to handle the load |
2016-06-21 21:00 UTC | 2016-06-23 02:00 UTC | Underlying cloud is down: no patches were missed, long delay in posting results |
2016-05-25 00:00 UTC | 2016-05-26 16:00 UTC | devstack setup failure due to our Ubuntu images using upstart, infrastracture problems |
2016-05-23 08:00 UTC | 2016-05-23 11:00 UTC | Zuul failed, zuul-merger hanging |
2016-05-12 12:30 UTC | 2016-05-12 14:30 UTC | Zuul failed |
2016-05-08 08:00 UTC | 2016-05-09 11:00 UTC | Zuul lost connection to gerrit.o.o |
2016-04-29 05:30 UTC | 2016-04-29 16:30 UTC | Jobs failing due to missing dev dependency |
2016-04-19 12:30 UTC | 2016-04-19 19:00 UTC | Network outage |
2016-04-10 11:00 UTC | 2016-04-11 19:00 UTC | |
2016-03-15 17:30 UTC | 2016-03-16 12:05 UTC | Zuul down |
2016-03-03 14:00 UTC | 2016-03-03 18:00 UTC | Zuul fails to listen gerrit.o.o events |
2016-02-13 13:00 UTC | 2016-02-15 14:00 UTC | Zuul reconnect failure after gerrit.o.o restart |
2016-02-12 10:00 UTC | 2016-02-12 19:00 UTC | Network loss |
2016-01-26 19:00 UTC | 2016-01-27 15:00 UTC | Network loss |
2016-01-19 10:00 UTC | 2016-01-19 18:00 UTC | Loss of Jenkins master |
2015-12-21 12:15 UTC | 2015-12-22 12:45 UTC | Jobs failing due to launchpad bug #1447704 |
2015-12-04 20:00 UTC | 2015-12-07 18:00 UTC | Jobs are unstable |
2015-11-30 04:00 UTC | 2015-11-30 13:00 UTC | nodepool failing |
2015-10-19 09:00 UTC | 2015-10-19 17:00 UTC | reporting failures due to outdated pypi mirror |
2015-08-24 00:00 UTC | 2015-08-24 13:00 UTC | bad zuul configuration |
2015-08-08 06:00 UTC | 2015-08-10 12:50 UTC | zuul failed to update configuration |
2015-07-30 13:00 UTC | 2015-07-31 21:00 UTC | Numerous failures due to upper_constraints, disabled constraints for the gate |
2015-07-27 12:00 UTC | 2015-07-27 16:00 UTC | zuul/jenkins failure |
2015-07-03 13:00 UTC | 2015-07-03 19:00 UTC | Gate was failing due to devstack-gate patch that enabled pypi package upper constraints |
2015-07-01 00:00 UTC | 2015-07-01 14:00 UTC | Jenkins crashed (leap second suspected as the cause) |
2015-06-30 01:00 UTC | 2015-06-30 13:00 UTC | Zuul did not reconnect to gerrit after connectivity loss |
2015-06-16 16:00 UTC | 2015-06-16 19:00 UTC | Could not install python-libguestfs on Fedora 21 guests, yum cache problems |
2015-05-16 17:30 UTC | 2015-05-18 02:00 UTC | Zuul service down |
2015-05-04 16:00 UTC | 2015-05-04 17:00 UTC | Maintenance |
2015-04-25 19:00 UTC | 2015-04-26 16:00 UTC | Zuul VM down |
2015-04-11 08:00 UTC | 2015-04-11 17:00 UTC | Internal network down |
2015-04-10 14:00 UTC | 2015-04-10 19:00 UTC | Internal network down |
Projects Tested
Current test status for all tested projects can be seen at Zuul Testing Report.
The report is compiled form zuul logs and is updated periodically. It might include aborted or otherwise terminated builds.
Reporting
- Devstack
- Ironic
- Nova
- Cinder
- os-brick
- diskimage-builder
Non-reporting
- Devstack-gate
- Neutron
- Glance
- Keystone
- Nova Unit tests
- Requirements
- Swift
- Tempest
Skipped Tests
There are currently a few skipped tests, see test run logs for details.
Known Issues
- Missing PCI hotplug support in CirrOS ppc64le
* Neutron (3) failures to missing interface attach support (PCI hotplug) in CirrOS