Difference between revisions of "Solum/FeatureBlueprints/GitIntegration"

Revision as of 20:01, 6 October 2013

Blueprint

Git Integration

Functionality

As an application developer, I want to push my code to my git repository, and have the platform automatically detect the code push and generate a running application using the new code.

I would be able to use git push for deploying a new application, or to update an existing application.

I would be able to specify the the meta-data for the application (AppName, app instance size, instance count, auto scale policies, etc) via an application manifest file. The application manifest file will be stored at the root location of my code project's directory, and will be automatically picked up by the platform in response to a git push.

Implementation

We posit that the git integration workflow would work as follows. To setup the git repository within Solum for a user's application, we propose that Solum API provide a 'register' resource. A user would invoke this resource and pass in the application name (AppName) as part of the request body. The Solum system will create a git repository for that user and the specified AppName. A 'post-receive-hook' will be created in this git repository (contents described below). The git repository URL will be sent back to the user in the response of the 'register' call. Users will have to add this URL as a 'remote' for their local repository. Let's call this as an application's solum remote. Adding a solum remote will be a one time step, which will need to be performed only once per application. Once the solum remote is setup, whenever the user pushes the code to it, the configured post-receive-hook will get triggered and perform actions which will lead to deployment of the application in the Solum system. The URL to access the application will be sent back to the user as part of the response of git push.

At a high-level, the steps that need to happen for creating a running application from the pushed code include, (a) detecting the application's type (Java vs. Python, etc.), (b) compiling the application (if required), (c) adding/updating any required data services (for instance, a database), (d) deploying the application code to the appropriate runtime environment (like a Docker container) after setting up the required application specific environment on it.

In designing the contents of the post-receive-hook we have to answer the following question: which of the above steps are good candidates to be included within a post-receive-hook? We argue that steps (a) and (b) can potentially be performed in a post-receive-hook. The remaining steps (step (c) and step (d)) require knowledge about parts of the Solum system, such as available container resources, that may not be easily accessible within a post-receive-hook. Also, from design perspective post-receive-hook does not seem to be the right place to be performing steps (c) and (d).

Now, even if steps (a) and (b) can be done in a post-receive-hook, one of the proposed Solum requirements is that users would be able to deploy their code to the Solum system directly, without having to register the application first (Put the deploy application API and blueprint link). This means that, in the workflow where users choose not to first register their applications with the Solum system, steps (a) and (b) need to occur as part of the implementation of the deploy application API call. So, if we implement steps (a) and (b) in the post-receive-hook, it will lead to duplication of functionality. This leads to the decision that neither of step (a) or (b) are good candidates for being included in a post-receive-hook as well.

So, if neither of the above steps can be part of a post-receive-hook then what should be in it?

The answer to this is provided by the above mentioned Solum's proposed functionality of supporting a resource endpoint to deploy application directly. The post-receive-hook will then just be a call to invoke this resource. The structure of this call will be exactly similar to the call that a user would make in order to directly invoke the deploy application resource. The deploy application API call is discussed in blueprint (put deploy app blueprint link here).

Other aspects that need to be kept in mind when designing the Git integration are: (a) The API server and the Solum's Git server will typically be located on different hosts, and (b) In order for a user to perform git push on the solum remote, the user's account needs to be provisioned on the Solum Git server.

The fact that API server and the Solum's Git server can be separate means that we will need to have some sort of an agent on the Git server. Let's call this SolumGitAgent. We will also need a communication channel between the API server and the SolumGitAgent. Upon receiving the register call, the API server can send a message to this agent to create the application's git repository and a post-receive-hook in it. Towards addressing the issue of provisioning a user account on the Solum Git server, there are at least two options. We could send a message to the SolumGitAgent to provision a user account on the Git Server, or we could manage user accounts in a centralized directory server, such as OpenLDAP, and setup the ssh agent on the Git Server to use such a centralized directory server for authenticating users' git push commands.