TripleO/ImageBuildingAPI

Please edit this page, anyone who is interested!

Blueprint is at https://blueprints.launchpad.net/tripleo/+spec/image-building-api

Use Cases

• Heat would be able to use this API to generate images automatically to be used in instances that it orchestrates.
•  ???

Summary

Design and implement an API that allows building of OpenStack images from scratch.

It should provide multiple ways of building images.

1. building images "statically", i.e. with diskimage-builder
2. building images by booting a base image, running a configuration management tool, and then snapshotting
3. building images by booting an installer ISO, "kickstarting" the install, and performing additional post-install configuration (???)

As a first pass, only the diskimage-builder support is necessary, but an eye should be aimed at extensibility.

Discussion

Figuring out how image building and heat /should/ interact given an idea CI/CD story is nontrivial. We discussed on IRC: 09:59 <+lifeless> radix: SpamapS: haven't read that yet, but one important thing IMO is decoupling: for CI/CD you /don't/ want to rebuild the image in each deployment environment. 09:59 <+lifeless> so yes, API - cool. But tying it into heat templates so heat does it - mmmm, I think the coupling is wrong 10:00 < radix> lifeless: it depends what abstraction level the API is at, I guess. I think there is an important thing missing from Heat right now 10:00 < radix> lifeless: or maybe I don't know what you mean. 10:01 < radix> lifeless: are you talking about the thing SpamapS said about having the API invoke heat to "spin up an instance and build"? 10:01 <+lifeless> radix: so, consider a C binary. You build it and test it. 10:01 <+lifeless> radix: do you recompile it when you deploy it somewhere else? If so, are your tests still valid ? 10:02 < radix> maybe? if you want to? 10:02 <+lifeless> radix: In general, they *aren't valid* if any of the build inputs could have changed. 10:02 < radix> or maybe you build the image with that binary baked in from somewhere else 10:02 <+lifeless> radix: now, consider that images are basically just a compile; they have all the tradeoffs of static C binaries, but at the machine scale. 10:05 < radix> lifeless: hmm. so are you arguing against using prebaked images at all in orchestrated deployments? 10:05 <+lifeless> radix: so my point is that when you do CI/CD, it removes a significant range of failure modes if you build the image, deploy to test environment, test, deploy to prod environment. 10:05 <+lifeless> radix: I'm arguing for using prebaked images in both the CI environment and CD environment, vs having the heat in the CI environment build an image and then the heat in the CD environment build an image. 10:05 <+lifeless> radix: I'm the guy that argued for prebaked images for tripleo, so no, I'm not contradicting myself :) 10:06 < radix> lifeless: I think a workflow can be designed that allows you to use the same image in test and prod 10:06 <+lifeless> radix: sure; I'm just highlighting that you've got two distinct heats there 10:07 <+lifeless> radix: so if heat is the one making the image, it doesn't follow naturally 10:07 <+lifeless> radix: vs e.g. zuul or something managing the CI/CD loop being the thing that builds it 10:07 <+lifeless> radix: where it would just fall out of the design 10:08 < radix> let me think about that. 10:08 < radix> lifeless: what if you're not actually building the software you're deploying? 10:08 <+lifeless> or perhaps you need a metaheat 10:08 < radix> what if it's just a collection of existing third-party software and you're just configuring it to do what you want? 10:08 <+lifeless> radix: you might receive a prebaked image from a company like HP or Rackspace 10:09 <+lifeless> radix: or you might be making a custon collection of third party software 10:09 <+lifeless> radix: in which case how do you know they all play nice together unless you test that ? 10:09 < radix> lifeless: you can test it. you have a Heat in your test environment, and a Heat in your prod environment, and you can use the same template in both. 10:10 < radix> lifeless: *something* in the pipeline can determine "oh, I already have this image built, I'll just use the same one", so you don't rebuild the image between test and prod. 10:13 < radix> The layer of abstraction I'm trying to get at is that you have a Heat resource that says "give me an image with os=ubuntu, packages=[...], pip_packages=[...], custom_files=[...]", and then my Instance will have a {"ref": "that_image"} as its ImageId. 10:13 < radix> (or, if you're using a different image-builder backend, maybe it will be something like os="ubuntu", config_system="chef", recipe="foo") 10:15 < radix> my dream is to specify my entire deployment in a set of templates, some written by me, and some coming from elsewhere, without me having to prepare anything else like images up front. 10:19 <+lifeless> radix: sorry, elocal for a bit 10:19 < radix> noproblem :) 10:23 <+lifeless> right, daughter's toy fixed 10:24 < SpamapS> lifeless: I had forgotten about that aspect, and I agree that the two must stay de-coupled 10:24 <+lifeless> radix: so, I have the same goal 10:24 <+lifeless> radix: and in fact we're working up the stack to that in tripleo, improving heat etc as needed 10:24 <+lifeless> radix: it is our *mission statement* to do that with the specific use case of deploying OpenStack as the application 10:25 < radix> I see :) 10:25 * SpamapS wishes he had more time for the "improving heat" aspect :) 10:26 < SpamapS> rwsu: hm, I have not seen that 10:26 <+lifeless> radix: I don't think in terms of heat templates directly though - heat is a tool in the stack of of template-like-things-that-combine-for-a-full-lifecycle-story 10:26 <+lifeless> radix: so the bit I'm quibbling on is saying 'heat should know about <..>' 10:27 < radix> Heat must call all the APIs! alll offf themmmmm!!!! 10:27 <+lifeless> radix: I'm not sure it shouldn't! But I'm not sure it should - as it seems problematic from the model I have in my head. 10:27 <+lifeless> radix: my head may be wrong