Jump to: navigation, search

Containerizing StarlingX Infrastructure

Revision as of 18:42, 1 March 2019 by Ildiko (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)



  • The first release of StarlingX provided a hardened OpenStack platform
  • Evolution plan to move to a cloud native (Kubernetes) platform was presented at the Vancouver Summit
    • Run the infrastructure including OpenStack services as containerized applications on Kubernetes
    • Containerization work is planned to be done in phases having OpenStack and dependencies the initial focus

Container Platform

  • Kubernetes master configuration on two nodes with high availability (HA)
    • Run on existing StarlingX nodes
    • Deployed by StarlingX system configuration
  • Calico CNI plugin
  • Docker runtime
  • CEPH as persistent storage backend
    • Leverage existing bare metal CEPH cluster
    • Extend CEPH support to one- and two-node configuration
  • Authentication/authorization of Kubernetes APIs with Keystone
  • Local Docker image registry, authentication with Keystone
  • Helm as package manager
  • Airship Armada for orchestrating the deployment of multiple Helm charts (ex. OpenStack)
  • Initial Kubernetes hosting environment for applications, the infrastructure including OpenStack will also be containerized

StarlingX Container Platform.png

Initial Infrastructure Containerization


  • The "Flock" services with the exceptions called out later are not planned to be containerized for the initial phase
  • An instance of Keystone, RabbitMQ, and PostgreSQL will remain on bare metal and will be used for the "Flock" service
  • An instance of Horizon will remain on bare metal for the "Flock" services only

StarlingX Initial Containerized Infrastructure.png

OpenStack and other services

  • All OpenStack services currently integrated by StarlingX are planned to be containerized
    • Including dependencies such as MariaDB
    • Pike based
      • nova-api proxy from stx-nfv will be containerized
      • nova-compute will be containerized
      • Neutron will be containerized
      • cilometer-poller will be containerized
    • Keystone container will run in the 'openstack' namespace for the OpenStack services only
  • rbd-provisioner pod: enabling pvc's from the CEPH cluster
  • An instance of the Fault Management service will be containerized for alarming in OpenStack
  • Current StarlingX compute nodes are re-purposed as Kubernetes worker nodes
  • OVS-DPDK is not containerized
    • No Helm chart is currently available

StarlingX Initial Containerized Infrastructure 2.png

Deployment/Lifecycle Management

  • OpenStack Helm to be used as a starting point
    • Extensions are required (e.g. CentOS support) with the plan of working with the OpenStack Helm team upstream
    • Airship Armada to orchestrate the set of Helm charts that comprise OpenStack
  • Extensions to the StarlingX Configuration Management service
    • Generation of Helm chart overrides
    • Kubernetes node label assignments
    • Upload application
      • Push images to local docker registry
      • Install Helm charts and Armada manifests
    • Install application
      • Interface with Configuration Management for override generation and invoking Armada to deploy
    • Update application
      • Allow the end user to specify overrides to particular charts and the trigger an update via Armada

Docker Images

  • The build system will have to be enhanced to generate StarlingX based Docker images for the containerized services
  • Images to be hosted on a public repository

Containerized OpenStack Network Architecture

  • Management network for platform services only and isolated from the cluster
  • OpenStack services to be exposed on the cluster network via a Kubernetes ingress controller
    • Demarcation point for public APIs and TLS/SSL termination
  • NFV-VIM APIs need to be accessible from both the OAM and cluster networks
    • Demarcation point for public APIs and TLS/SSL termination via ha-proxy
  • Platform services need access to several OpenStack APIs
    • VIM: Neutron, Nova
    • Sysinv: Neutron, Nova
  • CEPH service must be accessible from the cluster network

Containerized OpenStack Network Architecture.png

Software Updates

  • Software updates for containerized services will drive change in strategy
  • A software update will look like the following:
    • A new Docker image for a service(s) to be delivered to the system
    • An Armada update to be invoked to update the applicable charts

Future Phases Planned

  • Infrastructure services
    • Containerize the "Flock" services
    • Containerize OVS-DPDK
    • Containerize CEPH
  • Container infrastructure
    • Accelerated container networking with SR-IOV and DPDK
    • Multi-tenancy support for containers
    • Support for additional container runtimes including Kata Containers