Difference between revisions of "Neutron/FunctionalGateSetup"
Paul Michali (talk | contribs) m (→Move To Check Queue) |
Paul Michali (talk | contribs) (→Improving Performance) |
||
(19 intermediate revisions by 2 users not shown) | |||
Line 65: | Line 65: | ||
fi | fi | ||
− | function gate_hook | + | function gate_hook { |
bash -xe $BASE/new/neutron-vpnaas/neutron_vpnaas/tests/contrib/gate_hook.sh dsvm-functional | bash -xe $BASE/new/neutron-vpnaas/neutron_vpnaas/tests/contrib/gate_hook.sh dsvm-functional | ||
− | + | } | |
export -f gate_hook | export -f gate_hook | ||
− | function post_test_hook | + | function post_test_hook { |
bash -xe $BASE/new/neutron-vpnaas/neutron_vpnaas/tests/contrib/post_test_hook.sh dsvm-functional | bash -xe $BASE/new/neutron-vpnaas/neutron_vpnaas/tests/contrib/post_test_hook.sh dsvm-functional | ||
− | + | } | |
export -f post_test_hook | export -f post_test_hook | ||
Line 85: | Line 85: | ||
A quick check was made in openstack-infra/devstack-gate/features.yaml to make sure that the q-vpn service was started up for neutron. It was, no nothing was needed there. | A quick check was made in openstack-infra/devstack-gate/features.yaml to make sure that the q-vpn service was started up for neutron. It was, no nothing was needed there. | ||
− | Pushed up for review. Did get failure of gate-project-config-layout test. Log was truncated, so didn’t know what the error was and had to ask infra folks. They pointed me to console log to see the error (can go to https://jenkins06.openstack.org/job/gate-project-config-laylout/. There is not a check-neutron-vpnaas-dsvm-functional defined. Needed to add to project.yaml in jenkins/jobs/. Under the neutron-vpnaas project, I added job definitions: | + | To check the job definitions, you can run '''tox -e zuul''', and it will show the jobs run, and any conditions (branches, exclusions, etc). |
+ | |||
+ | Pushed up for review (it might be preferable to set the title of the commit message to "Create experimental job for <your-project-here>" such that reviewers understand the experimental nature). Did get failure of gate-project-config-layout test. Log was truncated, so didn’t know what the error was and had to ask infra folks. They pointed me to console log to see the error (can go to https://jenkins06.openstack.org/job/gate-project-config-laylout/. There is not a check-neutron-vpnaas-dsvm-functional defined. Needed to add to project.yaml in jenkins/jobs/. Under the neutron-vpnaas project, I added job definitions: | ||
- project: | - project: | ||
Line 107: | Line 109: | ||
branch-override: default | branch-override: default | ||
− | Made sure that Jenkins passes for the review and get approval from Infra folks. This goes under openstack-infra review obviously. | + | Made sure that Jenkins passes for the review and get approval from Infra folks and the Neutron liaisons for infra (https://wiki.openstack.org/wiki/CrossProjectLiaisons#Infra). This goes under openstack-infra review obviously. |
+ | |||
+ | Ref: [https://review.openstack.org/#/c/148616/ Review 148616] | ||
+ | |||
+ | === UPDATE === | ||
+ | If you have multiple similar jobs (like was eventually added for neutron-vpnaas), you can create one template that is shared. In this example, we have neutron-vpnaas-dsvm-functional and neutron-vpnaas-dsvm-functioanl-sswan jobs defined. The template was modified to look like this: | ||
+ | |||
+ | - job-template: | ||
+ | name: '{pipeline}-neutron-vpnaas-dsvm-{vpnaastest}{job-suffix}' | ||
+ | node: '{node}' | ||
+ | |||
+ | wrappers: | ||
+ | - build-timeout: | ||
+ | timeout: 125 | ||
+ | - timestamps | ||
+ | |||
+ | builders: | ||
+ | - print-template-name: | ||
+ | template-name: "{template-name}" | ||
+ | - link-logs | ||
+ | - net-info | ||
+ | - devstack-checkout | ||
+ | - shell: | | ||
+ | #!/bin/bash -xe | ||
+ | export PYTHONUNBUFFERED=true | ||
+ | export DEVSTACK_GATE_UNSTACK=1 | ||
+ | export DEVSTACK_GATE_TIMEOUT=120 | ||
+ | export DEVSTACK_GATE_TEMPEST=0 | ||
+ | export DEVSTACK_GATE_EXERCISES=0 | ||
+ | export DEVSTACK_GATE_NEUTRON=1 | ||
+ | export DEVSTACK_GATE_INSTALL_TESTONLY=1 | ||
+ | export BRANCH_OVERRIDE={branch-override} | ||
+ | if [ "$BRANCH_OVERRIDE" != "default" ] ; then | ||
+ | export OVERRIDE_ZUUL_BRANCH=$BRANCH_OVERRIDE | ||
+ | fi | ||
+ | |||
+ | function gate_hook { | ||
+ | bash -xe $BASE/new/neutron-vpnaas/neutron_vpnaas/tests/contrib/gate_hook.sh dsvm-{vpnaastest} | ||
+ | } | ||
+ | export -f gate_hook | ||
+ | |||
+ | function post_test_hook { | ||
+ | bash -xe $BASE/new/neutron-vpnaas/neutron_vpnaas/tests/contrib/post_test_hook.sh dsvm-{vpnaastest} | ||
+ | } | ||
+ | export -f post_test_hook | ||
+ | |||
+ | cp devstack-gate/devstack-vm-gate-wrap.sh ./safe-devstack-vm-gate-wrap.sh | ||
+ | ./safe-devstack-vm-gate-wrap.sh | ||
+ | publishers: | ||
+ | - test-results | ||
+ | - devstack-logs | ||
+ | - console-log | ||
+ | |||
+ | As you can see, '''vpnaastest''' is added in the template name, and is passed into the hook scripts, as an argument. The hook scripts will get either dsvm-functional, or dsvm-functional-sswan, and can perform actions for the two jobs based on that argument. | ||
+ | |||
+ | In addition to these changes (and adding the jobs to the experimental queue), the following changes are needed to jenkins/jobs/projects.yaml: | ||
+ | |||
+ | - project: | ||
+ | name: neutron-vpnaas | ||
+ | node: bare-trusty | ||
+ | tarball-site: tarballs.openstack.org | ||
+ | doc-publisher-site: docs.openstack.org | ||
+ | |||
+ | jobs: | ||
+ | - python-jobs | ||
+ | - translation-jobs | ||
+ | - python-constraints-jobs | ||
+ | - python-kilo-bitrot-jobs | ||
+ | - python-liberty-bitrot-jobs | ||
+ | - openstack-publish-jobs | ||
+ | - openstack-releasenotes-jobs | ||
+ | - '{pipeline}-neutron-vpnaas-dsvm-{vpnaastest}{job-suffix}': | ||
+ | pipeline: gate | ||
+ | node: devstack-trusty | ||
+ | vpnaastest: functional | ||
+ | job-suffix: '' | ||
+ | branch-override: default | ||
+ | - '{pipeline}-neutron-vpnaas-dsvm-{vpnaastest}{job-suffix}': | ||
+ | pipeline: gate | ||
+ | node: devstack-trusty | ||
+ | vpnaastest: functional-sswan | ||
+ | job-suffix: '' | ||
+ | branch-override: default | ||
+ | |||
+ | This defines the vpnaastest argument used in the template, for each of the jobs. | ||
== Quick Test On Local Host (optional) == | == Quick Test On Local Host (optional) == | ||
Line 137: | Line 223: | ||
== Add Hooks and Test == | == Add Hooks and Test == | ||
− | The easiest way here, is to copy the files from neutron/neutron/tests/contrib, into the repo (neutron-vpnaas/neutron_vpnaas/tests/contrib), and tweak them (yeah, cut and paste :). The devstack-vm-gate.sh and functions.sh from Neutron should not be used. The gate_hook.sh should run devstack-vm- | + | The easiest way here, is to copy the files from neutron/neutron/tests/contrib, into the repo (neutron-vpnaas/neutron_vpnaas/tests/contrib), and tweak them (yeah, cut and paste :). The devstack-vm-gate.sh and functions.sh from Neutron should not be used. The gate_hook.sh should run devstack-vm-gate.sh in openstack-infra/devstack-gate/ project area. |
− | For the neutron-vpnaas repo, the following changes were made to the files | + | For the neutron-vpnaas repo, the following changes were made to the files (note: both Neutron and VPN repos have been simplified and the updated VPN code is shown here): |
* Indicated neutron-vpnaas repo in README | * Indicated neutron-vpnaas repo in README | ||
− | * | + | * Set project name to neutron-vpnaas in gate-hook.sh |
− | * | + | * Handle both the dsvm-functional and dsvm-functional-sswan jobs (the same way) in gate_hook.sh and post_test_hook.sh. |
+ | * Added NEUTRON_VPN_PATH in gate_hook.sh so that can point to both the Neutron and VPN repo areas. | ||
+ | * Source configure_for_vpn_func_testing.sh script, which in turn uses the configure_for_func_testing.sh script from Neutron. | ||
+ | * Use NEUTRON_VPN_PATH in post_test_hook.sh to set file ownership. | ||
+ | |||
Here is gate_hook.sh (see repo for latest): | Here is gate_hook.sh (see repo for latest): | ||
− | #!/bin/bash | + | #!/usr/bin/env bash |
set -ex | set -ex | ||
− | + | VENV=${1:-"dsvm-functional"} | |
− | $BASE/new/devstack-gate/devstack-vm-gate.sh | + | case $VENV in |
+ | dsvm-functional | dsvm-functional-sswan) | ||
+ | # The following need to be set before sourcing | ||
+ | # configure_for_func_testing. | ||
+ | GATE_DEST=$BASE/new | ||
+ | GATE_STACK_USER=stack | ||
+ | NEUTRON_PATH=$GATE_DEST/neutron | ||
+ | PROJECT_NAME=neutron-vpnaas | ||
+ | NEUTRON_VPN_PATH=$GATE_DEST/$PROJECT_NAME | ||
+ | DEVSTACK_PATH=$GATE_DEST/devstack | ||
+ | IS_GATE=True | ||
+ | |||
+ | source $NEUTRON_VPN_PATH/tools/configure_for_vpn_func_testing.sh | ||
+ | |||
+ | # Make the workspace owned by the stack user | ||
+ | sudo chown -R $STACK_USER:$STACK_USER $BASE | ||
+ | |||
+ | configure_host_for_vpn_func_testing | ||
+ | ;; | ||
+ | api) $BASE/new/devstack-gate/devstack-vm-gate.sh ;; | ||
+ | esac | ||
+ | |||
+ | |||
+ | Here is the tools/configure_for_vpn_func_testing.sh for VPN: | ||
+ | |||
+ | #!/usr/bin/env bash | ||
− | # | + | # Licensed under the Apache License, Version 2.0 (the "License"); |
− | # | + | # you may not use this file except in compliance with the License. |
− | # | + | # You may obtain a copy of the License at |
− | + | # | |
− | + | # http://www.apache.org/licenses/LICENSE-2.0 | |
− | + | # | |
+ | # Unless required by applicable law or agreed to in writing, software | ||
+ | # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT | ||
+ | # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the | ||
+ | # License for the specific language governing permissions and limitations | ||
+ | # under the License. | ||
− | + | set -e | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | IS_GATE=${IS_GATE:-False} | |
− | + | PROJECT_NAME=${PROJECT_NAME:-neutron-vpnaas} | |
− | + | REPO_BASE=${GATE_DEST:-$(cd $(dirname "$BASH_SOURCE")/../.. && pwd)} | |
− | |||
− | + | source $REPO_BASE/neutron/tools/configure_for_func_testing.sh | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | function _install_vpn_package { | |
− | + | if [ "$VENV" == "dsvm-functional-sswan" ] | |
− | + | then | |
− | + | IPSEC_PACKAGE=strongswan | |
− | + | else | |
− | + | IPSEC_PACKAGE=openswan | |
− | + | fi | |
− | |||
− | |||
− | + | echo_summary "Installing $IPSEC_PACKAGE" | |
− | + | neutron_vpn_install_agent_packages | |
− | + | } | |
− | + | ||
− | + | ||
− | + | function _configure_vpn_ini_file { | |
− | + | echo_summary "Configuring VPN ini file" | |
+ | |||
+ | local temp_ini=$(mktemp) | ||
+ | cp $REPO_BASE/$PROJECT_NAME/etc/vpn_agent.ini $temp_ini | ||
+ | if [ "$IPSEC_PACKAGE" == "strongswan" ]; then | ||
+ | iniset_multiline $temp_ini vpnagent vpn_device_driver neutron_vpnaas.services.vpn.device_drivers.strongswan_ipsec.StrongSwanDriver | ||
+ | if is_fedora; then | ||
+ | iniset $temp_ini strongswan default_config_area /usr/share/strongswan/templates/config/strongswan.d | ||
+ | fi | ||
+ | else | ||
+ | iniset_multiline $temp_ini vpnagent vpn_device_driver neutron_vpnaas.services.vpn.device_drivers.ipsec.OpenSwanDriver | ||
+ | fi | ||
+ | sudo install -d -o $STACK_USER /etc/neutron/ | ||
+ | sudo mv $temp_ini $Q_VPN_CONF_FILE | ||
+ | } | ||
+ | |||
+ | |||
+ | function configure_host_for_vpn_func_testing { | ||
+ | echo_summary "Configuring for VPN functional testing" | ||
+ | if [ "$IS_GATE" == "True" ]; then | ||
+ | configure_host_for_func_testing | ||
+ | fi | ||
+ | _install_vpn_package | ||
+ | _configure_vpn_ini_file | ||
+ | } | ||
+ | |||
+ | |||
+ | if [ "$IS_GATE" != "True" ]; then | ||
+ | configure_host_for_vpn_func_testing | ||
+ | fi | ||
− | + | It essentially does additional setup for VPN and then performs the Neutron configure_host_for_func_testing() method. | |
− | #!/bin/bash | + | For the post_test_hook.sh script VPN changed the Neutron script to look like this: |
+ | |||
+ | #!/usr/bin/env bash | ||
set -xe | set -xe | ||
− | + | NEUTRON_VPNAAS_DIR="$BASE/new/neutron-vpnaas" | |
TEMPEST_DIR="$BASE/new/tempest" | TEMPEST_DIR="$BASE/new/tempest" | ||
SCRIPTS_DIR="/usr/local/jenkins/slave_scripts" | SCRIPTS_DIR="/usr/local/jenkins/slave_scripts" | ||
+ | |||
+ | VENV=${1:-"dsvm-functional"} | ||
function generate_testr_results { | function generate_testr_results { | ||
Line 215: | Line 354: | ||
sudo -H -u $owner chmod o+rw -R .testrepository | sudo -H -u $owner chmod o+rw -R .testrepository | ||
if [ -f ".testrepository/0" ] ; then | if [ -f ".testrepository/0" ] ; then | ||
− | .tox/$ | + | .tox/$VENV/bin/subunit-1to2 < .testrepository/0 > ./testrepository.subunit |
− | .tox/$ | + | .tox/$VENV/bin/python $SCRIPTS_DIR/subunit2html.py ./testrepository.subunit testr_results.html |
gzip -9 ./testrepository.subunit | gzip -9 ./testrepository.subunit | ||
gzip -9 ./testr_results.html | gzip -9 ./testr_results.html | ||
Line 223: | Line 362: | ||
} | } | ||
− | + | case $VENV in | |
− | + | dsvm-functional | dsvm-functional-sswan) | |
− | + | owner=stack | |
− | + | sudo_env= | |
− | + | ;; | |
− | + | api) | |
+ | owner=tempest | ||
+ | # Configure the api tests to use the tempest.conf set by devstack. | ||
+ | sudo_env="TEMPEST_CONFIG_DIR=$TEMPEST_DIR/etc" | ||
+ | ;; | ||
+ | esac | ||
# Set owner permissions according to job's requirements. | # Set owner permissions according to job's requirements. | ||
− | cd $ | + | cd $NEUTRON_VPNAAS_DIR |
− | sudo chown -R $owner:stack $ | + | sudo chown -R $owner:stack $NEUTRON_VPNAAS_DIR |
− | |||
− | |||
# Run tests | # Run tests | ||
− | echo "Running neutron $ | + | echo "Running neutron $VENV test suite" |
set +e | set +e | ||
− | sudo -H -u $owner tox -e $ | + | sudo -H -u $owner $sudo_env tox -e $VENV |
testr_exit_code=$? | testr_exit_code=$? | ||
set -e | set -e | ||
Line 248: | Line 390: | ||
In addition to the hook support, create a dummy functional test in tests/functional. Push this for review, and from Gerrit, enter in a comment with ‘check experimental’ to run the functional test from the experimental pipeline. | In addition to the hook support, create a dummy functional test in tests/functional. Push this for review, and from Gerrit, enter in a comment with ‘check experimental’ to run the functional test from the experimental pipeline. | ||
+ | |||
+ | Ref: [https://review.openstack.org/#/c/150990 Review 150990] | ||
== Move To Check Queue == | == Move To Check Queue == | ||
Line 268: | Line 412: | ||
- name: openstack/nova | - name: openstack/nova | ||
− | Now, the test will run for every patch reviewed for the repo, but the testing will not be voting. | + | Now, the test will run for every patch reviewed for the repo, but the testing will not be voting. Again, ensure that neutron liaisons for infra review the commit (Infra will withhold +A, until a liaison reviews). |
Ref: [https://review.openstack.org/#/c/152602/ Review 152602] | Ref: [https://review.openstack.org/#/c/152602/ Review 152602] | ||
Line 277: | Line 421: | ||
For this, zulu/layout.yaml is again modified to remove the lines that make the check non-voting, and add a gate entry for the project. | For this, zulu/layout.yaml is again modified to remove the lines that make the check non-voting, and add a gate entry for the project. | ||
− | + | diff --git a/zuul/layout.yaml b/zuul/layout.yaml | |
+ | index d8ca382..578dd32 100755 | ||
+ | --- a/zuul/layout.yaml | ||
+ | +++ b/zuul/layout.yaml | ||
+ | @@ -666,8 +666,6 @@ jobs: | ||
+ | branch: ^(?!stable/icehouse).*$ | ||
+ | - name: gate-designate-dsvm-bind9 | ||
+ | branch: ^(?!stable/(?:icehouse|juno)).*$ | ||
+ | - - name: check-neutron-vpnaas-dsvm-functional | ||
+ | - voting: false | ||
+ | - name: ^(gate|check)-tempest-dsvm-networking-odl | ||
+ | branch: ^(?!stable) | ||
+ | voting: false | ||
+ | @@ -1431,6 +1429,7 @@ projects: | ||
+ | - check-neutron-vpnaas-dsvm-functional | ||
+ | gate: | ||
+ | - gate-neutron-vpnaas-pylint | ||
+ | + - gate-neutron-vpnaas-dsvm-functional | ||
+ | |||
+ | - name: openstack/nova | ||
+ | template: | ||
+ | |||
+ | As before, get a neutron liaison to review, so that infra can +A the review. | ||
+ | |||
+ | Ref: [https://review.openstack.org/156281 Review 156281] | ||
+ | |||
+ | == Odds & Ends == | ||
+ | There are a few other things to consider, when creating jobs (both functional and non-functional). This section will highlight them. | ||
+ | |||
+ | === Improving Performance === | ||
+ | On thing that can be done to improve the performance, is to skip running jobs based on the change set. A common one is to skip running some jobs when the changes were only for documentation files. For example (zuul/layout.yaml): | ||
+ | |||
+ | # Skip non-doc jobs for doc-only neutron-vpnaas changes. The dsvm | ||
+ | # tests area already skipped by another check. | ||
+ | - name: ^.*neutron-vpnaas-python(27|34)(-constraints)?$ | ||
+ | skip-if: | ||
+ | - project: ^openstack/neutron-vpnaas$ | ||
+ | all-files-match-any: | ||
+ | - ^.*\.rst$ | ||
+ | - ^doc/.*$ | ||
+ | - ^releasenotes/.*$ | ||
+ | |||
+ | This will skip the python27 and python34 constraints and non-constraints jobs, if the changes are only to .rst files or files in the doc or releasenotes areas. You can confirm this filter, by running '''tox -e zuul''' and examining the output. | ||
+ | |||
+ | === Older Releases === | ||
+ | When adding in new jobs, you may want to exclude from running them for older releases, as the jobs may not be supported, may not have hook support, and/or may not have targets implemented in tox.ini. To exclude a job from a release, you can add the following (zuul/layout.yaml): | ||
+ | |||
+ | # Don't run python34 on Kilo | ||
+ | - name: ^gate-neutron-vpnaas-python34$ | ||
+ | branch: ^(?!stable/kilo).*$ | ||
+ | |||
+ | LIkewise, if you are replacing an existing job with another job, you may not want to run the older job with current releases, so you can specify that it runs with only specific releases, like this (zuul/layout.yaml): | ||
+ | # Run the non-constraints coverage on Kilo | ||
+ | - name: neutron-vpnaas-coverage | ||
+ | branch: ^(?:stable/kilo).*$ | ||
+ | |||
+ | You can specify multiple releases. For example, to only run a job if it is NOT a Kilo and NOT a Liberty release, you'd specify: | ||
+ | branch: ^(?!stable/(?:kilo|liberty)).*$ | ||
+ | |||
+ | === Using Job Templates === | ||
+ | There are predefined job templates that can be applied to a project. This will define multiple jobs that will run for check, gate, and/or post queues. Here is a common template (zuul/layout.yaml): | ||
+ | project-templates: | ||
+ | - name: python-jobs | ||
+ | check: | ||
+ | - 'gate-{name}-pep8' | ||
+ | - 'gate-{name}-docs' | ||
+ | - 'gate-{name}-python27' | ||
+ | gate: | ||
+ | - 'gate-{name}-docs' | ||
+ | - 'gate-{name}-pep8' | ||
+ | - 'gate-{name}-python27' | ||
+ | post: | ||
+ | - '{name}-branch-tarball' | ||
+ | |||
+ | This defines pep8, docs, and python27 jobs for the check and gate queues, and tarball for post queue. The '''name''' is replaced with the project name, upon use. For example (zuul/layout.yaml): | ||
+ | projects: | ||
+ | .... | ||
+ | - name: openstack/neutron-vpnaas | ||
+ | template: | ||
+ | ... | ||
+ | - name: python-jobs | ||
+ | - name: python3-jobs | ||
+ | |||
+ | For the '''neutron-vpnaas''' project, the template is being used. This means that for the check queue, gate-neutron-vpnaas-pep8, gate-neutron-vpnaas-docs, and gate-neutron-vpnaas-python27 jobs will be invoked. Similar replacements are done for the other queues defined in the template. The template will reduce the size of the project job definitions, and will introduce consistency, with a single definition for each of the jobs. Just as an example, here is the template that is used for the python27 job (jenkins/jobs/python-jobs.yaml): | ||
+ | - job-template: | ||
+ | name: 'gate-{name}-python27' | ||
+ | |||
+ | wrappers: | ||
+ | - build-timeout: | ||
+ | timeout: 50 | ||
+ | - timestamps | ||
+ | |||
+ | builders: | ||
+ | - print-template-name: | ||
+ | template-name: "{template-name}" | ||
+ | - revoke-sudo | ||
+ | - gerrit-git-prep | ||
+ | - python27 | ||
+ | - assert-no-extra-files | ||
+ | |||
+ | publishers: | ||
+ | - test-results | ||
+ | - console-log | ||
+ | - zuul-swift-test-results-with-console | ||
+ | |||
+ | node: '{node}' |
Latest revision as of 15:03, 11 January 2016
Contents
Setting Up Functional Gate
This wiki contains notes on how to setup functional test gate for a repository. Using the neutron-vpnaas repo as an example.
Preparation
Clone the project-config repo, so that a commit can be formed for changes to support the repo.
git clone https://github.com/openstack-infra/project-config
Setup Non-voting Experimental Test Runs
In the openstack-infra project-config repo, modify the zuul/ayout.yaml for this project. First, there should be a project entry created, under ‘projects:’. In VPNaaS, there was already entry:
projects: ... - name: openstack/neutron-vpnaas template: - name: check-requirements - name: integrated-gate-neutron - name: merge-check - name: python-jobs - name: openstack-server-release-jobs
With this project, there are only the templates, and no check, gate, post, or experimental sections. For now, we’ll add a new test, under the experimental section, so that the test will only run when the user adds a Gerrit comment with “check experimental"
experimental: - check-neutron-vpnaas-dsvm-functional
In the “jobs:” section, we need to mark this test as non-voting: for the check queue:
jobs: ... - name: check-neutron-vpnaas-dsvm-functional voting: false
Next, a job must be created for the VPNaaS repo in jenkins/jobs/. To speed things up, and because the needs are similar, the neutron-functional.yaml is copied to neutron-vpnaas-functional.yaml. The changes are made to the name, which is at the top of the file:
- job-template: name: '{pipeline}-neutron-vpnaas-dsvm-functional{branch-designator}'
and change the path for the gate_hook.sh and post_test_hook.sh, from neutron/neutron/ to neutron-vpnaas/neutron_vpnaas/. Since the VPN repo only has master branch, and not older branches that have hooks in different paths, the conditional logic for the hook scripts is also removed. The whole file looks like this (in case you don’t have access to the Neutron YAML file):
- job-template: name: '{pipeline}-neutron-vpnaas-dsvm-functional{branch-designator}' node: '{node}' wrappers: - build-timeout: timeout: 125 - timestamps builders: - link-logs - net-info - devstack-checkout - shell: | #!/bin/bash -xe export PYTHONUNBUFFERED=true export DEVSTACK_GATE_UNSTACK=1 export DEVSTACK_GATE_TIMEOUT=120 export DEVSTACK_GATE_TEMPEST=0 export DEVSTACK_GATE_EXERCISES=0 export DEVSTACK_GATE_NEUTRON=1 export DEVSTACK_GATE_INSTALL_TESTONLY=1 export BRANCH_OVERRIDE={branch-override} if [ "$BRANCH_OVERRIDE" != "default" ] ; then export OVERRIDE_ZUUL_BRANCH=$BRANCH_OVERRIDE fi function gate_hook { bash -xe $BASE/new/neutron-vpnaas/neutron_vpnaas/tests/contrib/gate_hook.sh dsvm-functional } export -f gate_hook function post_test_hook { bash -xe $BASE/new/neutron-vpnaas/neutron_vpnaas/tests/contrib/post_test_hook.sh dsvm-functional } export -f post_test_hook cp devstack-gate/devstack-vm-gate-wrap.sh ./safe-devstack-vm-gate-wrap.sh ./safe-devstack-vm-gate-wrap.sh publishers: - test-results - devstack-logs - console-log
A quick check was made in openstack-infra/devstack-gate/features.yaml to make sure that the q-vpn service was started up for neutron. It was, no nothing was needed there.
To check the job definitions, you can run tox -e zuul, and it will show the jobs run, and any conditions (branches, exclusions, etc).
Pushed up for review (it might be preferable to set the title of the commit message to "Create experimental job for <your-project-here>" such that reviewers understand the experimental nature). Did get failure of gate-project-config-layout test. Log was truncated, so didn’t know what the error was and had to ask infra folks. They pointed me to console log to see the error (can go to https://jenkins06.openstack.org/job/gate-project-config-laylout/. There is not a check-neutron-vpnaas-dsvm-functional defined. Needed to add to project.yaml in jenkins/jobs/. Under the neutron-vpnaas project, I added job definitions:
- project: name: neutron-vpnaas node: 'bare-precise || bare-trusty' tarball-site: tarballs.openstack.org doc-publisher-site: docs.openstack.org jobs: - python-jobs - openstack-publish-jobs - '{pipeline}-neutron-vpnaas-dsvm-functional{branch-designator}': pipeline: gate node: 'devstack-precise || devstack-trusty' branch-designator: branch-override: default - '{pipeline}-neutron-vpnaas-dsvm-functional{branch-designator}': pipeline: check node: 'devstack-precise || devstack-trusty' branch-designator: branch-override: default
Made sure that Jenkins passes for the review and get approval from Infra folks and the Neutron liaisons for infra (https://wiki.openstack.org/wiki/CrossProjectLiaisons#Infra). This goes under openstack-infra review obviously.
Ref: Review 148616
UPDATE
If you have multiple similar jobs (like was eventually added for neutron-vpnaas), you can create one template that is shared. In this example, we have neutron-vpnaas-dsvm-functional and neutron-vpnaas-dsvm-functioanl-sswan jobs defined. The template was modified to look like this:
- job-template: name: '{pipeline}-neutron-vpnaas-dsvm-{vpnaastest}{job-suffix}' node: '{node}'
wrappers: - build-timeout: timeout: 125 - timestamps
builders: - print-template-name: template-name: "{template-name}" - link-logs - net-info - devstack-checkout - shell: | #!/bin/bash -xe export PYTHONUNBUFFERED=true export DEVSTACK_GATE_UNSTACK=1 export DEVSTACK_GATE_TIMEOUT=120 export DEVSTACK_GATE_TEMPEST=0 export DEVSTACK_GATE_EXERCISES=0 export DEVSTACK_GATE_NEUTRON=1 export DEVSTACK_GATE_INSTALL_TESTONLY=1 export BRANCH_OVERRIDE={branch-override} if [ "$BRANCH_OVERRIDE" != "default" ] ; then export OVERRIDE_ZUUL_BRANCH=$BRANCH_OVERRIDE fi
function gate_hook { bash -xe $BASE/new/neutron-vpnaas/neutron_vpnaas/tests/contrib/gate_hook.sh dsvm-{vpnaastest} } export -f gate_hook
function post_test_hook { bash -xe $BASE/new/neutron-vpnaas/neutron_vpnaas/tests/contrib/post_test_hook.sh dsvm-{vpnaastest} } export -f post_test_hook
cp devstack-gate/devstack-vm-gate-wrap.sh ./safe-devstack-vm-gate-wrap.sh ./safe-devstack-vm-gate-wrap.sh
publishers: - test-results - devstack-logs - console-log
As you can see, vpnaastest is added in the template name, and is passed into the hook scripts, as an argument. The hook scripts will get either dsvm-functional, or dsvm-functional-sswan, and can perform actions for the two jobs based on that argument.
In addition to these changes (and adding the jobs to the experimental queue), the following changes are needed to jenkins/jobs/projects.yaml:
- project: name: neutron-vpnaas node: bare-trusty tarball-site: tarballs.openstack.org doc-publisher-site: docs.openstack.org
jobs: - python-jobs - translation-jobs - python-constraints-jobs - python-kilo-bitrot-jobs - python-liberty-bitrot-jobs - openstack-publish-jobs - openstack-releasenotes-jobs - '{pipeline}-neutron-vpnaas-dsvm-{vpnaastest}{job-suffix}': pipeline: gate node: devstack-trusty vpnaastest: functional job-suffix: branch-override: default - '{pipeline}-neutron-vpnaas-dsvm-{vpnaastest}{job-suffix}': pipeline: gate node: devstack-trusty vpnaastest: functional-sswan job-suffix: branch-override: default
This defines the vpnaastest argument used in the template, for each of the jobs.
Quick Test On Local Host (optional)
In a neutron-vpnaas view, make sure the development environment is setup for tox (e.g. “tox -r -e py27 —notest”). Then, a functional.filter can be added to /etc/neutron/rootwrap.d/ with perms 644 and containing:
# neutron-rootwrap command filters to support functional testing. It # is NOT intended to be used outside of a test environment. # # This file should be owned by (and only-writeable by) the root user [Filters] # '$BASE_PATH' is intended to be replaced with the expected tox path # (e.g. /opt/stack/new/neutron/.tox/dsvm-functional) by the neutron # functional jenkins job. This ensures that tests can kill the # processes that they launch with their containing tox environment's # python. kill_tox_python: KillFilter, root, /opt/stack/neutron-vpnaas/.tox/dsvm-functional/bin/python, -9 # enable ping from namespace ping_filter: CommandFilter, ping, root # enable curl from namespace curl_filter: CommandFilter, curl, root
Ran ‘tox -e dsvm-functional’ on a repo with a functional test added. This worked, however, I did get a warning about test command found, but not installed in testenv.
When the project-config changes are used up for review, several tests are run on the changes, and the Jenkins results should be checked for any failures.
Add Hooks and Test
The easiest way here, is to copy the files from neutron/neutron/tests/contrib, into the repo (neutron-vpnaas/neutron_vpnaas/tests/contrib), and tweak them (yeah, cut and paste :). The devstack-vm-gate.sh and functions.sh from Neutron should not be used. The gate_hook.sh should run devstack-vm-gate.sh in openstack-infra/devstack-gate/ project area.
For the neutron-vpnaas repo, the following changes were made to the files (note: both Neutron and VPN repos have been simplified and the updated VPN code is shown here):
- Indicated neutron-vpnaas repo in README
- Set project name to neutron-vpnaas in gate-hook.sh
- Handle both the dsvm-functional and dsvm-functional-sswan jobs (the same way) in gate_hook.sh and post_test_hook.sh.
- Added NEUTRON_VPN_PATH in gate_hook.sh so that can point to both the Neutron and VPN repo areas.
- Source configure_for_vpn_func_testing.sh script, which in turn uses the configure_for_func_testing.sh script from Neutron.
- Use NEUTRON_VPN_PATH in post_test_hook.sh to set file ownership.
Here is gate_hook.sh (see repo for latest):
#!/usr/bin/env bash set -ex VENV=${1:-"dsvm-functional"} case $VENV in dsvm-functional | dsvm-functional-sswan) # The following need to be set before sourcing # configure_for_func_testing. GATE_DEST=$BASE/new GATE_STACK_USER=stack NEUTRON_PATH=$GATE_DEST/neutron PROJECT_NAME=neutron-vpnaas NEUTRON_VPN_PATH=$GATE_DEST/$PROJECT_NAME DEVSTACK_PATH=$GATE_DEST/devstack IS_GATE=True source $NEUTRON_VPN_PATH/tools/configure_for_vpn_func_testing.sh # Make the workspace owned by the stack user sudo chown -R $STACK_USER:$STACK_USER $BASE configure_host_for_vpn_func_testing ;; api) $BASE/new/devstack-gate/devstack-vm-gate.sh ;; esac
Here is the tools/configure_for_vpn_func_testing.sh for VPN:
#!/usr/bin/env bash # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. set -e IS_GATE=${IS_GATE:-False} PROJECT_NAME=${PROJECT_NAME:-neutron-vpnaas} REPO_BASE=${GATE_DEST:-$(cd $(dirname "$BASH_SOURCE")/../.. && pwd)} source $REPO_BASE/neutron/tools/configure_for_func_testing.sh function _install_vpn_package { if [ "$VENV" == "dsvm-functional-sswan" ] then IPSEC_PACKAGE=strongswan else IPSEC_PACKAGE=openswan fi echo_summary "Installing $IPSEC_PACKAGE" neutron_vpn_install_agent_packages } function _configure_vpn_ini_file { echo_summary "Configuring VPN ini file" local temp_ini=$(mktemp) cp $REPO_BASE/$PROJECT_NAME/etc/vpn_agent.ini $temp_ini if [ "$IPSEC_PACKAGE" == "strongswan" ]; then iniset_multiline $temp_ini vpnagent vpn_device_driver neutron_vpnaas.services.vpn.device_drivers.strongswan_ipsec.StrongSwanDriver if is_fedora; then iniset $temp_ini strongswan default_config_area /usr/share/strongswan/templates/config/strongswan.d fi else iniset_multiline $temp_ini vpnagent vpn_device_driver neutron_vpnaas.services.vpn.device_drivers.ipsec.OpenSwanDriver fi sudo install -d -o $STACK_USER /etc/neutron/ sudo mv $temp_ini $Q_VPN_CONF_FILE } function configure_host_for_vpn_func_testing { echo_summary "Configuring for VPN functional testing" if [ "$IS_GATE" == "True" ]; then configure_host_for_func_testing fi _install_vpn_package _configure_vpn_ini_file } if [ "$IS_GATE" != "True" ]; then configure_host_for_vpn_func_testing fi
It essentially does additional setup for VPN and then performs the Neutron configure_host_for_func_testing() method.
For the post_test_hook.sh script VPN changed the Neutron script to look like this:
#!/usr/bin/env bash set -xe NEUTRON_VPNAAS_DIR="$BASE/new/neutron-vpnaas" TEMPEST_DIR="$BASE/new/tempest" SCRIPTS_DIR="/usr/local/jenkins/slave_scripts" VENV=${1:-"dsvm-functional"} function generate_testr_results { # Give job user rights to access tox logs sudo -H -u $owner chmod o+rw . sudo -H -u $owner chmod o+rw -R .testrepository if [ -f ".testrepository/0" ] ; then .tox/$VENV/bin/subunit-1to2 < .testrepository/0 > ./testrepository.subunit .tox/$VENV/bin/python $SCRIPTS_DIR/subunit2html.py ./testrepository.subunit testr_results.html gzip -9 ./testrepository.subunit gzip -9 ./testr_results.html sudo mv ./*.gz /opt/stack/logs/ fi } case $VENV in dsvm-functional | dsvm-functional-sswan) owner=stack sudo_env= ;; api) owner=tempest # Configure the api tests to use the tempest.conf set by devstack. sudo_env="TEMPEST_CONFIG_DIR=$TEMPEST_DIR/etc" ;; esac # Set owner permissions according to job's requirements. cd $NEUTRON_VPNAAS_DIR sudo chown -R $owner:stack $NEUTRON_VPNAAS_DIR # Run tests echo "Running neutron $VENV test suite" set +e sudo -H -u $owner $sudo_env tox -e $VENV testr_exit_code=$? set -e # Collect and parse results generate_testr_results exit $testr_exit_code
In addition to the hook support, create a dummy functional test in tests/functional. Push this for review, and from Gerrit, enter in a comment with ‘check experimental’ to run the functional test from the experimental pipeline.
Ref: Review 150990
Move To Check Queue
Once the the experimental queue run works and the hooks commit is merged, the check-neutron-vpnaas-dsvm-functional job can be moved to the check queue (still as non-voting). It will now run on all commits.
This is as simple as changing experimental entry to check for the project in zuul/layout.yaml
diff --git a/zuul/layout.yaml b/zuul/layout.yaml index d587a43..684c035 100755 --- a/zuul/layout.yaml +++ b/zuul/layout.yaml @@ -1350,7 +1350,7 @@ projects: - name: merge-check - name: python-jobs - name: openstack-server-release-jobs - experimental: + check: - check-neutron-vpnaas-dsvm-functional - name: openstack/nova
Now, the test will run for every patch reviewed for the repo, but the testing will not be voting. Again, ensure that neutron liaisons for infra review the commit (Infra will withhold +A, until a liaison reviews).
Ref: Review 152602
Add to Gate and Check Queue With Voting
Once everything looks good, the check-neutron-vpnaas-dsvm-functional job can be made voting and a gate-neutron-vpnaas-dsvm-functional job can be added to the gate pipeline.
For this, zulu/layout.yaml is again modified to remove the lines that make the check non-voting, and add a gate entry for the project.
diff --git a/zuul/layout.yaml b/zuul/layout.yaml index d8ca382..578dd32 100755 --- a/zuul/layout.yaml +++ b/zuul/layout.yaml @@ -666,8 +666,6 @@ jobs: branch: ^(?!stable/icehouse).*$ - name: gate-designate-dsvm-bind9 branch: ^(?!stable/(?:icehouse|juno)).*$ - - name: check-neutron-vpnaas-dsvm-functional - voting: false - name: ^(gate|check)-tempest-dsvm-networking-odl branch: ^(?!stable) voting: false @@ -1431,6 +1429,7 @@ projects: - check-neutron-vpnaas-dsvm-functional gate: - gate-neutron-vpnaas-pylint + - gate-neutron-vpnaas-dsvm-functional - name: openstack/nova template:
As before, get a neutron liaison to review, so that infra can +A the review.
Ref: Review 156281
Odds & Ends
There are a few other things to consider, when creating jobs (both functional and non-functional). This section will highlight them.
Improving Performance
On thing that can be done to improve the performance, is to skip running jobs based on the change set. A common one is to skip running some jobs when the changes were only for documentation files. For example (zuul/layout.yaml):
# Skip non-doc jobs for doc-only neutron-vpnaas changes. The dsvm # tests area already skipped by another check. - name: ^.*neutron-vpnaas-python(27|34)(-constraints)?$ skip-if: - project: ^openstack/neutron-vpnaas$ all-files-match-any: - ^.*\.rst$ - ^doc/.*$ - ^releasenotes/.*$
This will skip the python27 and python34 constraints and non-constraints jobs, if the changes are only to .rst files or files in the doc or releasenotes areas. You can confirm this filter, by running tox -e zuul and examining the output.
Older Releases
When adding in new jobs, you may want to exclude from running them for older releases, as the jobs may not be supported, may not have hook support, and/or may not have targets implemented in tox.ini. To exclude a job from a release, you can add the following (zuul/layout.yaml):
# Don't run python34 on Kilo - name: ^gate-neutron-vpnaas-python34$ branch: ^(?!stable/kilo).*$
LIkewise, if you are replacing an existing job with another job, you may not want to run the older job with current releases, so you can specify that it runs with only specific releases, like this (zuul/layout.yaml):
# Run the non-constraints coverage on Kilo - name: neutron-vpnaas-coverage branch: ^(?:stable/kilo).*$
You can specify multiple releases. For example, to only run a job if it is NOT a Kilo and NOT a Liberty release, you'd specify:
branch: ^(?!stable/(?:kilo|liberty)).*$
Using Job Templates
There are predefined job templates that can be applied to a project. This will define multiple jobs that will run for check, gate, and/or post queues. Here is a common template (zuul/layout.yaml):
project-templates: - name: python-jobs check: - 'gate-{name}-pep8' - 'gate-{name}-docs' - 'gate-{name}-python27' gate: - 'gate-{name}-docs' - 'gate-{name}-pep8' - 'gate-{name}-python27' post: - '{name}-branch-tarball'
This defines pep8, docs, and python27 jobs for the check and gate queues, and tarball for post queue. The name is replaced with the project name, upon use. For example (zuul/layout.yaml):
projects: .... - name: openstack/neutron-vpnaas template: ... - name: python-jobs - name: python3-jobs
For the neutron-vpnaas project, the template is being used. This means that for the check queue, gate-neutron-vpnaas-pep8, gate-neutron-vpnaas-docs, and gate-neutron-vpnaas-python27 jobs will be invoked. Similar replacements are done for the other queues defined in the template. The template will reduce the size of the project job definitions, and will introduce consistency, with a single definition for each of the jobs. Just as an example, here is the template that is used for the python27 job (jenkins/jobs/python-jobs.yaml):
- job-template: name: 'gate-{name}-python27'
wrappers: - build-timeout: timeout: 50 - timestamps
builders: - print-template-name: template-name: "{template-name}" - revoke-sudo - gerrit-git-prep - python27 - assert-no-extra-files
publishers: - test-results - console-log - zuul-swift-test-results-with-console
node: '{node}'