Jump to: navigation, search

Difference between revisions of "Fuel/CI"

(Add information about Puppet OpenStack CI)
(Packaging CI Overview)
Line 89: Line 89:
 
== Packaging CI Overview ==
 
== Packaging CI Overview ==
  
Packages are built and published using [https://github.com/openstack/fuel-mirror/tree/master/perestroika Perestroika].
+
Packaging CI for Fuel builds RPM-packages for all Fuel repositories. Packages are built and published using [https://github.com/openstack/fuel-mirror/tree/master/perestroika Perestroika].
  
Packaging CI consists of one Jenkins master node, which is connected to Zuul instance via Gearman plugin. Zuul is connected to Openstack Gerrit using service account '''Fuel Packaging CI'''. There is also several Jenkins slaves running Ubuntu 14.04 (LTS):
+
Packaging CI consists of one Jenkins master node [https://packaging-ci.fuel-infra.org packaging-ci.fuel-infra.org], which is connected to Zuul instance [http://gate.fuel-infra.org gate.fuel-infra.org] via Gearman plugin. Zuul is attached to Openstack Gerrit using service account '''Fuel Packaging CI'''.
 +
 
 +
There are also several Jenkins slaves running Ubuntu 14.04 (LTS):
 
* one publisher and mirror host (VM)
 
* one publisher and mirror host (VM)
 
* package building slave(s) (HW)
 
* package building slave(s) (HW)
Line 97: Line 99:
 
* slave(s) for deployment (system) tests (HW)
 
* slave(s) for deployment (system) tests (HW)
  
=== Links ===
+
Built packages are available at [http://packages.fuel-infra.org packages.fuel-infra.org] site.
 +
 
 +
=== Packaging CI Links ===
 +
* [https://review.openstack.org/#/c/266911/ Packaging CI Spec]
 
* [https://github.com/fuel-infra/jenkins-jobs/tree/master/servers/pkgs-ci-pub Jenkins Job Builder configuration]
 
* [https://github.com/fuel-infra/jenkins-jobs/tree/master/servers/pkgs-ci-pub Jenkins Job Builder configuration]
 
* [https://infra-ci.fuel-infra.org/job/zuul-maintainer/lastSuccessfulBuild/artifact/build/gate01-scc.fuel-infra.org/ Zuul layout]
 
* [https://infra-ci.fuel-infra.org/job/zuul-maintainer/lastSuccessfulBuild/artifact/build/gate01-scc.fuel-infra.org/ Zuul layout]
 
* [https://github.com/fuel-infra/puppet-manifests/tree/master/modules/zuul/manifests Puppet manifests]
 
* [https://github.com/fuel-infra/puppet-manifests/tree/master/modules/zuul/manifests Puppet manifests]
 
* [http://mirror.fuel-infra.org/devops/ubuntu/all/ Package repository for Ubuntu containing Zuul package and its dependencies]
 
* [http://mirror.fuel-infra.org/devops/ubuntu/all/ Package repository for Ubuntu containing Zuul package and its dependencies]
* [https://packaging-ci.fuel-infra.org Jenkins instance]
+
* [https://packaging-ci.fuel-infra.org packaging-ci.fuel-infra.org] - Jenkins
* [http://gate.fuel-infra.org Zuul instance]
+
* [http://gate.fuel-infra.org gate.fuel-infra.org] - Zuul
* [http://packages.fuel-infra.org Package repositories]
+
* [http://packages.fuel-infra.org packages.fuel-infra.org] - packages repository

Revision as of 15:55, 2 March 2016

Fuel CI Overview

Fuel project uses the OpenStack Gerrit infrastructure and follows its Development workflow.

Additionally to that, there is a Fuel CI -- third-party CI service which runs additional checks and tests which are not yet supported in OpenStack Infra gate.

Fuel CI consists of one Jenkins master node, which is connected to Openstack Gerrit via Gerrit Trigger plugin using service account Fuel CI. There is also about 30 bare-metal Jenkins slaves. All Jenkins slaves are running Ubuntu 14.04 and perform unit tests and deployment tests via fuel-devops/fuel-qa framework.

Links

Deployment tests

Generic deployment test

To run deployment for Fuel you need:

  • ISO image for Fuel node [1]
  • fuel-devops [2] - the cli tool which manages virtual machines and stores state (vm names, network interfaces..) in a PostgreSQL database
  • fuel-qa [3] - test framework based on proboscis

Basic setup is described in [4].

And the test flow is as follows:

  • with fuel-devops tool:
    • create several vm's connected via internal network - a so-called devops environment
  • with fuel-qa framework:
Step 1. install Fuel node on first vm using the ISO image provided by the local path on the host server
Step 2. bootstrap other vms with basic OS image provided on Fuel ISO
Step 3. configure Fuel environment via API according to certain scenario
Step 4. run deployment

Test scenarios are described in fuel-qa documentation, see for example [5].

Deployment test on CI

fuel-library code is essentially a set of puppet manifests which are used to deploy the enviroment configuration defined via Fuel interface. These manifests are delivered to Fuel node as RPM package [6].

To save time and resources on CI we don't recreate environment from scratch for every tests but regularly take a "stable enough" ISO, upload it to Jenkins slaves, create base environment (steps 1. and 2.) and snapshot all its vms.

Then, on every commit we

  1. rebuild a fuel-library package in a CentOS-based docker container
  2. revert devops environment from snapshot
  3. upload and install package on Fuel node
  4. run the deployment test scenario (steps 3. and 4.)

You can refer to detailed logs in [7]

CI for Puppet OpenStack

Fuel CI acts as a third-party CI for a set of Puppet OpenStack project modules used in fuel-library:

  • puppet-aodh
  • puppet-ceilometer
  • puppet-cinder
  • puppet-glance
  • puppet-ironic
  • puppet-heat
  • puppet-horizon
  • puppet-keystone
  • puppet-murano
  • puppet-neutron
  • puppet-nova
  • puppet-openstacklib
  • puppet-sahara
  • puppet-swift

For every commit to these projects we start a set of builds which currently includes fuel-library noop tests and 2 deployment tests.

Deployment tests share the same workflow and test scenarios with the deployment tests we're running for fuel-library.

For any questions regarding Fuel CI for Puppet OpenStack please contact #fuel-dev IRC channel.

Packaging CI Overview

Packaging CI for Fuel builds RPM-packages for all Fuel repositories. Packages are built and published using Perestroika.

Packaging CI consists of one Jenkins master node packaging-ci.fuel-infra.org, which is connected to Zuul instance gate.fuel-infra.org via Gearman plugin. Zuul is attached to Openstack Gerrit using service account Fuel Packaging CI.

There are also several Jenkins slaves running Ubuntu 14.04 (LTS):

  • one publisher and mirror host (VM)
  • package building slave(s) (HW)
  • slave(s) for install tests (HW)
  • slave(s) for deployment (system) tests (HW)

Built packages are available at packages.fuel-infra.org site.

Packaging CI Links