Containerizing StarlingX Infrastructure

= Initiative =

Introduction

 * The first release of StarlingX provided a hardened OpenStack platform
 * Evolution plan to move to a cloud native (Kubernetes) platform was presented at the Vancouver Summit
 * Run the infrastructure including OpenStack services as containerized applications on Kubernetes
 * Containerization work is planned to be done in phases having OpenStack and dependencies the initial focus

Container Platform

 * Kubernetes master configuration on two nodes with high availability (HA)
 * Run on existing StarlingX nodes
 * Deployed by StarlingX system configuration
 * Calico CNI plugin
 * Docker runtime
 * CEPH as persistent storage backend
 * Leverage existing bare metal CEPH cluster
 * Extend CEPH support to one- and two-node configuration
 * Authentication/authorization of Kubernetes APIs with Keystone
 * Local Docker image registry, authentication with Keystone
 * Helm as package manager
 * Airship Armada for orchestrating the deployment of multiple Helm charts (ex. OpenStack)
 * Initial Kubernetes hosting environment for applications, the infrastructure including OpenStack will also be containerized



Overview

 * The "Flock" services with the exceptions called out later are not planned to be containerized for the initial phase
 * An instance of Keystone, RabbitMQ, and PostgreSQL will remain on bare metal and will be used for the "Flock" service
 * An instance of Horizon will remain on bare metal for the "Flock" services only



OpenStack and other services

 * All OpenStack services currently integrated by StarlingX are planned to be containerized
 * Including dependencies such as MariaDB
 * Pike based
 * nova-api proxy from stx-nfv will be containerized
 * nova-compute will be containerized
 * Neutron will be containerized
 * cilometer-poller will be containerized
 * Keystone container will run in the 'openstack' namespace for the OpenStack services only
 * rbd-provisioner pod: enabling pvc's from the CEPH cluster
 * An instance of the Fault Management service will be containerized for alarming in OpenStack
 * Current StarlingX compute nodes are re-purposed as Kubernetes worker nodes
 * OVS-DPDK is not containerized
 * No Helm chart is currently available



Deployment/Lifecycle Management

 * OpenStack Helm to be used as a starting point
 * Extensions are required (e.g. CentOS support) with the plan of working with the OpenStack Helm team upstream
 * Airship Armada to orchestrate the set of Helm charts that comprise OpenStack
 * Extensions to the StarlingX Configuration Management service
 * Generation of Helm chart overrides
 * Kubernetes node label assignments
 * Upload application
 * Push images to local docker registry
 * Install Helm charts and Armada manifests
 * Install application
 * Interface with Configuration Management for override generation and invoking Armada to deploy
 * Update application
 * Allow the end user to specify overrides to particular charts and the trigger an update via Armada

Docker Images

 * The build system will have to be enhanced to generate StarlingX based Docker images for the containerized services
 * Images to be hosted on a public repository

Containerized OpenStack Network Architecture

 * Management network for platform services only and isolated from the cluster
 * OpenStack services to be exposed on the cluster network via a Kubernetes ingress controller
 * Demarcation point for public APIs and TLS/SSL termination
 * NFV-VIM APIs need to be accessible from both the OAM and cluster networks
 * Demarcation point for public APIs and TLS/SSL termination via ha-proxy
 * Platform services need access to several OpenStack APIs
 * VIM: Neutron, Nova
 * Sysinv: Neutron, Nova
 * CEPH service must be accessible from the cluster network



Software Updates

 * Software updates for containerized services will drive change in strategy
 * A software update will look like the following:
 * A new Docker image for a service(s) to be delivered to the system
 * An Armada update to be invoked to update the applicable charts

Future Phases Planned

 * Infrastructure services
 * Containerize the "Flock" services
 * Containerize OVS-DPDK
 * Containerize CEPH
 * Container infrastructure
 * Accelerated container networking with SR-IOV and DPDK
 * Multi-tenancy support for containers
 * Support for additional container runtimes including Kata Containers