Solum/ApiModel

The design of Solum's API in release 2014.1.1 focuses primarily on resources to enable Application Lifecycle Management. It is suitable for expressing how to deploy an application, but it's not as useful for modeling a custom build/test/deploy pipeline for a given application taking into account everything needed in order to produce a CI/CD environment. This proposed design addresses that concern by detailing how the default behavior of Solum can be customized to accommodate a variety of different CI workflows.

From https://wiki.openstack.org/wiki/Solum: “An OpenStack Related Stackforge project designed to make cloud services easier to consume and integrate into your application development process. “

Solum currently allows integration with simple development process using Git, and a pre-defined workflow. We plan to add components that will allow customization of events that happen before the application's deployment, such as testing, image building, and advancing between various environments.

Proposal

Empower the Application Developer to use his/her source tree (git repo) as a tool for building images, running tests and deploying into a variety of environments.

Allow the following inputs: from the Application Developer:

Application source code
Application Requirements (both build and runtime)
Desired Pipeline Steps (eg. build, test, deploy)
Target Deployment Environment (how and where the application is run “dev”, “staging” & “production”)

A usecase

Given a git repo, whenever there is a commit pushed to the repo:

run local unit tests

build an image from the app (with any requirements pre-built)
deploy to a “staging” environment (where manual testing is done)
deploy to the production environment

Definitions

I see 3 main entities:

application (aka plan)

git url

required runtime services

extra config:

environments (still under discussion)

how and where to deploy resources

e.g. credentils, endpoints, location (region & AZ), compute type (VM, container) and scaling info.

lifecycle

the jobs to get the application from a git commit to a deployed heat stack

Example

applications:
 thingy:
   git_url: git://x.me/a.git 
   services: 
     - trove
       remote_logging
   config:
     custom_env: foo
environments:
 default:
   compute: docker
   trust_token: bla-bla
   region: SYD
   flavor: m1.small
 production:
   trust_token: foo-fee
   region: US1
   flavor: m1.large
lifecycle:
 check:
   type: task
   cmd: tox
 template_builder:
   type: build_template
   depends_on: check
   target_environment: {find_in_catalog: {name: production, type: environment}}
 image_builder:
   type: task
   depends_on: check
   cmd: “solum-build-app .”
 deploy:
   type: deploy_stack
   environment: {find_in_catalog: {name: production, type: environment}}
   template: {get_output: [template_builder, created_template_id]}
   image: {get_output: [image_builder, created_image_id]}

REST API

All the above applications/environments/lifecycle templates could be stored in glance

Then we just need the following API

Note: I can't think of a better name than "project", any suggestions?

project/
project/<uuid>
project/<uuid>/stacks/  [return a list of links to heat stacks, normally one]
project/<uuid>/job_history/  [once we have mistral, this could be links to mistral jobs]
project/<uuid>/job_history/<uuid>

POST project/

{ lifecycle: <glance_id>,
  application: <glance_id>,
  need_git_trigger: true }

response:

{project_uuid: bla-abl,
 git_trigger_url: http://}

Old stuff

For DSL, we can also look at Mistral, Taskflow, Murano. It will be great if more than one folks participate in the DSL discussions being proposed by Mistral guys (ref. Email on Openstack-dev list).

Builder subsystem/service:

Investigate Jenkins
Investigate Mistral

Additional Issues:

Long running workflows would need Solum to use trusts (and renew tokens periodically).

Build jobs need to be run on disposable VM's to be secure.