Revision as of 17:27, 29 May 2015

Driver Testing

Description

The Cinder community (and other OpenStack projects) have agreed that if a vendor wishes to submit a driver for their particular storage device that said vendor should also be required to set up a third party CI system in their lab which runs Tempest volume tests against their storage device for every Cinder commit, and provides feedback in to Gerrit.

Third Party CI Requirements

• See the official Third Party Testing wiki.
• Test all volume drivers your company has integrated in Cinder.
• Test all fabrics your solution uses.

For example, if your company has two volume drivers in Cinder and they both use ISCSI and FibreChannel, you would need to have a CI that tests against four backends and reports the results for each backend, for every Cinder upstream patch

Existing CI Solutions

• Puppet modules for deploying OpenStack CI
  • Git repo
  • Sample Jenkins Job Builder Job Template for Cinder drivers
  • Fork of Jay Pipe's external test repo and more up-to-date. Uses nodepool.

• Jay Pipe's external testing series
  • Note: Jay's repo is outdated, but the articles are useful to read. A more updated fork exists.
  • Git Repo
  • Understanding the OpenStack CI
  • Setting up CI Part 1
  • Setting up a CI Part 2

Current Reporting Cinder CI's

See the list.

Questions

• Join Third Party Meeting
• Reach out to IRC nicks DuncanT or asselin on Freenode #openstack-cinder.

FAQ

What tests do I use?

Use the OpenStack integration test suite Tempest. The volume related tests can be started with the following command from a Tempest repo:

tox -e all -- volume | tee -a console.log.out

For those using devstack-gate export this variable before running the job:

export DEVSTACK_GATE_TEMPEST_REGEX="volume"

How do I configure DevStack so my Driver Passes Tempest?

Sample local.conf for devstack setup.

[[local|localrc]]
ADMIN_PASSWORD=password
MYSQL_PASSWORD=password
RABBIT_PASSWORD=password
SERVICE_PASSWORD=password
SERVICE_TOKEN=password

# These options define  expected driver capabilities
TEMPEST_VOLUME_DRIVER=foo
TEMPEST_VOLUME_VENDOR="Foo Inc"
TEMPEST_STORAGE_PROTOCOL=iSCSI

# These options allow you to specify a branch other than "master" be used
CINDER_REPO=https://review.openstack.org/openstack/cinder
CINDER_BRANCH=refs/changes/83/72183/4

# Disable security groups entirely
Q_USE_SECGROUP=False
LIBVIRT_FIREWALL_DRIVER=nova.virt.firewall.NoopFirewallDriver
CINDER_SECURE_DELETE=False

[[post-config|$CINDER_CONF]]
volume_driver=cinder.volume.drivers.foo.FooDriver

How do I run my CI to test all cinder patches with my driver not yet merged?

If using devstack-gate use the pre-test-hook to cherry-pick your driver on top of the cinder patch under review

function pre_test_hook {
        cd $BASE/new/cinder
        git fetch https://review.openstack.org/openstack/cinder $PATCH && git cherry-pick FETCH_HEAD
}

Note you may wish to substitute the review.openstack.org repo with your own github location before submitting to gerrit.

Otherwise you can make the changes prior to calling stack.sh or via a custom devstack plugin.

When thirdparty CI voting will be required?

Once third party CI's become more common and stable, we'll revisit the subject. This was agreed in a discussion on October 15th 2014. In a discussion on April 8th 2015, it was agreed that open source solutions that have Infra hosted CI's could have the chance to vote.

How do I test FC drivers?

Use PCI Passthrough to give the VM access to the FC PCI Device Scripts available here: https://git.openstack.org/cgit/stackforge/third-party-ci-tools/tree/provisioning_scripts/fibre_channel