Solum/Pipeline

The design of Solum's API in release 2014.1.1 focuses primarily on resources to enable Application Lifecycle Management. It is suitable for expressing how to deploy an application, but it's not as useful for modeling a custom build/test/deploy pipeline for a given application taking into account everything needed in order to produce a CI/CD environment. This proposed design addresses that concern by detailing how the default behavior of Solum can be customized to accommodate a variety of different CI workflows.

From https://wiki.openstack.org/wiki/Solum: “An OpenStack Related Stackforge project designed to make cloud services easier to consume and integrate into your application development process. “

Solum currently allows integration with simple development process using Git, and a pre-defined workflow. We plan to add components that will allow customization of events that happen before the application's deployment, such as testing, image building, and advancing between various Environments.

Proposal

Empower the Application Developer to use his/her source tree (git repo) as a tool for building images, running tests and deploying into a variety of Environments.

Allow the following inputs: from the Application Developer:

Application source code
Application Requirements (both build and runtime)
Desired Pipeline Steps (eg. build, test, deploy)
Target Deployment Environment (how and where the application is run “dev”, “staging” & “production”)

Use Case 1: Commit Triggers Unit/Build/Stage/Approve/Prod

Given a git repo, whenever there is a commit pushed to the repo:

Run unit tests
Build an image from the app (with any requirements pre-built)
Deploy to a “staging” environment (where manual testing is done)
Await an approval of successful testing
Deploy to the production environment

Definitions

Three concepts are needed to instruct Solum how to handle the requested sequence:

Applications (aka plan)

git url

required runtime services

extra config:

Environments (still under discussion)

how and where to deploy resources

e.g. credentils, endpoints, location (region & AZ), compute type (VM, container) and scaling info.

Lifecycle

Jobs advance the application from a git commit to a deployed heat stack

Example YAML Input

The example would be split into application/environment/lifecycle.

applications:
 thingy:
   git_url: git://x.me/a.git 
   services: 
     - trove
       remote_logging
   config:
     custom_env: foo

environments:
 default:
   compute: docker
   trust_token: bla-bla
   region: SYD
   flavor: m1.small
 production:
   trust_token: foo-fee
   region: US1
   flavor: m1.large

lifecycle:
 check:
   type: task
   cmd: tox
 template_builder:
   type: build_template
   depends_on: check
   target_environment: {find_in_catalog: {name: production, type: environment}}
 image_builder:
   type: task
   depends_on: check
   cmd: “solum-build-app .”
 deploy:
   type: deploy_stack
   environment: {find_in_catalog: {name: production, type: environment}}
   template: {get_output: [template_builder, created_template_id]}
   image: {get_output: [image_builder, created_image_id]}

Templates like this one will be be stored as artifacts in Glance, pending completion of it's Artifact Repository feature, currently in design stage. Note: the environment might have to be stored in Barbacan if it has sensitive data (trust_token).

REST API

This API extension allows Solum to facilitate test/build/stage/approve/prod automation for the development of a given application. We refer to this as a Pipeline.

Synopsis

Get a list of app pipelines:

GET pipelines/

Create a new pipeline resource: Request:

POST pipelines/ HTTP/1.1
Content-Type: application/json
Length: ...

{ lifecycle: <glance_id>,
  application: <glance_id>,
  create_git_trigger: true }

Response:

HTTP/1.1 200 OK
Content Type: application/json
Content-Length: ...

{ pipeline_uuid: bla-bla,
  git_trigger_url: http://example.com/solum/triggers/1234 }

Get/Update a specific pipeline resource:

GET pipelines/<uuid>
PUT pipelines/<uuid>

Return a list of heat stacks for a pipeline, normally one:

GET pipelines/<uuid>/stacks/

Show the job history for a given pipeline (This could be links to Mistral jobs)

GET pipelines/<uuid>/job_history/

Show the details of a specific job in the job history:

GET pipelines/<uuid>/job_history/<uuid>