Cinder/tested-3rdParty-drivers
Contents
- 1 Driver Testing
- 1.1 Description
- 1.2 Third Party CI Requirements
- 1.3 Existing CI Solutions
- 1.4 Current Reporting Cinder CI's
- 1.5 Questions
- 1.6 FAQ
- 1.6.1 What tests do I use?
- 1.6.2 How do I configure DevStack so my Driver Passes Tempest?
- 1.6.3 How do I run my CI to test all cinder patches with my driver not yet merged?
- 1.6.4 When thirdparty CI voting will be required?
- 1.6.5 How do I test FC drivers?
- 1.6.6 How do I trigger my CI to rerun on gerrit comments?
Driver Testing
Description
The Cinder community (and other OpenStack projects) have agreed that if a vendor wishes to submit a driver for their particular storage device that said vendor should also be required to set up a third party CI system in their lab which runs Tempest volume tests against their storage device for every Cinder commit, and provides feedback in to Gerrit.
Third Party CI Requirements
- See the official Third Party Testing wiki.
- Test all volume drivers your company has integrated in Cinder.
- Post all test results. Even if you have failure results, those need to be posted.
- Test all fabrics your solution uses.
For example, if your company has two volume drivers in Cinder and they both use ISCSI and FibreChannel, you would need to have a CI that tests against four backends and reports the results for each backend, for every Cinder upstream patch
Existing CI Solutions
- Puppet modules for deploying OpenStack CI
- Git repo
- Mailing list information about CI
- Questions can be answered on the #openstack-infra IRC channel
- For historical background you can review Jay Pipe's external testing series
- Note: Jay's repo is outdated, but the articles are useful to read. A more updated fork exists.
- Git Repo
- Understanding the OpenStack CI
- Setting up CI Part 1
- Setting up a CI Part 2
Current Reporting Cinder CI's
See the list.
Questions
- Join Third Party Meeting
- Reach out to IRC nicks DuncanT or asselin on Freenode #openstack-cinder.
FAQ
What tests do I use?
Use the OpenStack integration test suite Tempest. The volume related tests can be started with the following command from a Tempest repo:
tox -e all -- volume | tee -a console.log.out
For those using devstack-gate export this variable before running the job:
export DEVSTACK_GATE_TEMPEST_REGEX="volume"
How do I configure DevStack so my Driver Passes Tempest?
Sample local.conf for devstack setup.
[[local|localrc]] ADMIN_PASSWORD=password MYSQL_PASSWORD=password RABBIT_PASSWORD=password SERVICE_PASSWORD=password SERVICE_TOKEN=password # These options define expected driver capabilities TEMPEST_VOLUME_DRIVER=foo TEMPEST_VOLUME_VENDOR="Foo Inc" TEMPEST_STORAGE_PROTOCOL=iSCSI # These options allow you to specify a branch other than "master" be used CINDER_REPO=https://review.openstack.org/openstack/cinder CINDER_BRANCH=refs/changes/83/72183/4 # Disable security groups entirely Q_USE_SECGROUP=False LIBVIRT_FIREWALL_DRIVER=nova.virt.firewall.NoopFirewallDriver CINDER_SECURE_DELETE=False [[post-config|$CINDER_CONF]] volume_driver=cinder.volume.drivers.foo.FooDriver
How do I run my CI to test all cinder patches with my driver not yet merged?
If using devstack-gate use the pre-test-hook to cherry-pick your driver on top of the cinder patch under review. If you've submitted your change to gerrit, you can use something like the following:
function pre_test_hook { echo "Cherry-picking latest driver from gerrit" cd $BASE/new/cinder # fetch latest patchset from gerrit UPSTREAM_REMOTE=https://review.openstack.org/openstack/cinder # set this to your gerrit change number CHANGE_NUM=999999 PATCHSET_BASE=refs/changes/${CHANGE_NUM:(-2)}/$CHANGE_NUM LATEST_PATCHSET=$(git ls-remote $UPSTREAM_REMOTE $PATCHSET_BASE/\* | sort -t/ -k 5 -n | tail -n1 | cut -d$'\t' -f2) if [ -z "$LATEST_PATCHSET" ]; then echo "Failed to determine latest patchset of $PATCHSET_BASE from $UPSTREAM_REMOTE" exit 1 fi echo "Latest patchset ref is $LATEST_PATCHSET" git fetch $UPSTREAM_REMOTE $LATEST_PATCHSET && git cherry-pick FETCH_HEAD }
If you haven't submitted your change to gerrit yet, you'll have to fetch the change from your own repository (github or otherwise). For example:
function pre_test_hook { echo "Cherry-picking latest driver from private repository" cd $BASE/new/cinder VENDOR_REMOTE=https://www.github.com/vendor/cinder-driver REFSPEC=bp/new-cinder-driver git fetch $VENDOR_REMOTE $REFSPEC && git cherry-pick FETCH_HEAD }
Otherwise you can make the changes prior to calling stack.sh or via a custom devstack plugin.
When thirdparty CI voting will be required?
Once third party CI's become more common and stable, we'll revisit the subject. This was agreed in a discussion on October 15th 2014. In a discussion on April 8th 2015, it was agreed that open source solutions that have Infra hosted CI's could have the chance to vote.
How do I test FC drivers?
Use PCI Passthrough to give the VM access to the FC PCI Device Scripts available here: http://git.openstack.org/cgit/openstack/third-party-ci-tools/tree/provisioning_scripts/fibre_channel
How do I trigger my CI to rerun on gerrit comments?
It is required for your CI to rerun jobs if the following comment is posted in a gerrit review:
"run-<CI NAME>"
So if your CI account name is "Super Cool Storage CI", the CI should be able to be retriggered by leaving the comment:
run-Super Cool Storage CI
CI's do not need to kick off tests until Jenkin's has given a +1, though it is up to the maintainer if you want to trigger immediately on all patchsets.
For the openstack-ci configuration, change the following to switch to only trigger on a Jenkin's +1:
In layout.yaml:
Delete the line:
- event: patchset-created
And then add:
- event: comment-added require-approval: - verified: 1 username: Jenkins