Difference between revisions of "StarlingX/Containers/K8sUpgradesTesting"

Revision as of 12:56, 29 January 2020

Testing Kubernetes Upgrades

In order to test a kubernetes upgrade, the recommended approach is to:

• Build a StarlingX load with a previous kubernetes version
• Build a set of patches for the new kubernetes version
• Install the load and perform the upgrade

For example, if the release of StarlingX you are testing is currently using kubernetes v1.16.2, you could build a load using kubernetes v1.16.1.

Build a StarlingX load with a previous kubernetes version

The following changes must be made and then a load must be built.

Project: config

File: sysinv/sysinv/sysinv/sysinv/common/kubernetes.py

• Change get_kube_versions function to specify previous version. If the previous version was v1.16.1 it would look like this:

 def get_kube_versions():
     """Provides a list of supported kubernetes versions."""
     return [
         {'version': 'v1.16.1',
          'upgrade_from': [],
          'downgrade_to': [],
          'applied_patches': [],
          'available_patches': [],
          },
     ]

Project: integ

File: kubernetes/kubernetes/centos/build_srpm.data

• Change VERSION to previous version.

File: kubernetes/kubernetes/centos/kubernetes.spec

• Change %global commit to previous version.
• Change %global kube_version to previous version.

Project: stx-puppet

File: puppet-manifests/src/modules/platform/templates/kubeadm.yaml.erb

• Change kubernetesVersion to previous version.

Retrieve tarball for kubernetes

The tarball for the previous kubernetes version must be available in the downloads directory for your workspace ($MY_REPO/stx/downloads).

Download the tarball for the previous kubernetes version: https://github.com/kubernetes/kubernetes/archive/v1.16.1.tar.gz

Copy this to your workspace and rename it as follows:

 mv <source file> $MY_REPO/stx/downloads/kubernetes-v1.16.1.tar.gz

Build a set of patches for the new kubernetes version

Update get_kube_versions in sysinv/common/kubernetes.py:

Project: config

File: sysinv/sysinv/sysinv/sysinv/common/kubernetes.py

• Change get_kube_versions function to specify both the old and new version and the patches that will be used for the upgrade. If the previous version was v1.16.1 it would look like this:

 def get_kube_versions():
   return [
       {'version': 'v1.16.1',
        'upgrade_from': [],
        'downgrade_to': [],
        'applied_patches': [],
        'available_patches': [],
        },
       {'version': 'v1.16.2',
        'upgrade_from': ['v1.16.1'],
        'downgrade_to': [],
        'applied_patches': ['KUBE.1'],
        'available_patches': ['KUBE.2'],
        },
   ]

File: sysinv/sysinv/centos/build_srpm.data

• Increment the TIS_PATCH_VER

Project: integ

File: kubernetes/kubernetes/centos/build_srpm.data

• Undo change to VERSION
• Increment the TIS_PATCH_VER

File: kubernetes/kubernetes/centos/kubernetes.spec

• Undo all changes

Build the patches

The patches are built from the same workspace used to build the load with the previous kubernetes version.

First build the updated RPMS:

 # Build packages
 cd $MY_WORKSPACE
 build-pkgs --no-build-info --no-descendants sysinv
 build-pkgs --no-build-info --no-descendants kubernetes

Several scripts have been provided to build the patches - they can be found in the update project at patch-scripts/kubernetes-upgrades. Copy these files to a local directory, make any necessary edits and then invoke the make_kube_patch.sh script.

Install the load and perform the upgrade

Follow the steps described in the spec for Kubernetes Upgrades.