Difference between revisions of "TripleO/ImageBuildingAPI"
< TripleO
Oleg Gelbukh (talk | contribs) m |
|||
| Line 26: | Line 26: | ||
Figuring out how image building and heat /should/ interact given an idea CI/CD story is nontrivial. We discussed on IRC: | Figuring out how image building and heat /should/ interact given an idea CI/CD story is nontrivial. We discussed on IRC: | ||
| − | < | + | <pre> |
09:59 <+lifeless> radix: SpamapS: haven't read that yet, but one important thing IMO is decoupling: for CI/CD you /don't/ want to rebuild the image in each deployment environment. | 09:59 <+lifeless> radix: SpamapS: haven't read that yet, but one important thing IMO is decoupling: for CI/CD you /don't/ want to rebuild the image in each deployment environment. | ||
09:59 <+lifeless> so yes, API - cool. But tying it into heat templates so heat does it - mmmm, I think the coupling is wrong | 09:59 <+lifeless> so yes, API - cool. But tying it into heat templates so heat does it - mmmm, I think the coupling is wrong | ||
| Line 56: | Line 56: | ||
10:09 < radix> lifeless: you can test it. you have a Heat in your test environment, and a Heat in your prod environment, and you can use the same template in both. | 10:09 < radix> lifeless: you can test it. you have a Heat in your test environment, and a Heat in your prod environment, and you can use the same template in both. | ||
10:10 < radix> lifeless: *something* in the pipeline can determine "oh, I already have this image built, I'll just use the same one", so you don't rebuild the image between test and prod. | 10:10 < radix> lifeless: *something* in the pipeline can determine "oh, I already have this image built, I'll just use the same one", so you don't rebuild the image between test and prod. | ||
| − | 10:13 < radix> The layer of abstraction I'm trying to get at is that you have a Heat resource that says "give me an image with os=ubuntu, packages=[...], pip_packages=[...], custom_files=[...]", and then my Instance will have | + | 10:13 < radix> The layer of abstraction I'm trying to get at is that you have a Heat resource that says "give me an image with os=ubuntu, packages=[...], pip_packages=[...], custom_files=[...]", and then my Instance will have a {"ref": "that_image"} as its ImageId. |
| − | |||
10:13 < radix> (or, if you're using a different image-builder backend, maybe it will be something like os="ubuntu", config_system="chef", recipe="foo") | 10:13 < radix> (or, if you're using a different image-builder backend, maybe it will be something like os="ubuntu", config_system="chef", recipe="foo") | ||
10:15 < radix> my dream is to specify my entire deployment in a set of templates, some written by me, and some coming from elsewhere, without me having to prepare anything else like images up front. | 10:15 < radix> my dream is to specify my entire deployment in a set of templates, some written by me, and some coming from elsewhere, without me having to prepare anything else like images up front. | ||
| Line 75: | Line 74: | ||
10:27 <+lifeless> radix: I'm not sure it shouldn't! But I'm not sure it should - as it seems problematic from the model I have in my head. | 10:27 <+lifeless> radix: I'm not sure it shouldn't! But I'm not sure it should - as it seems problematic from the model I have in my head. | ||
10:27 <+lifeless> radix: my head may be wrong | 10:27 <+lifeless> radix: my head may be wrong | ||
| − | </ | + | </pre> |
Latest revision as of 12:57, 17 October 2013
Please edit this page, anyone who is interested!
Blueprint is at https://blueprints.launchpad.net/tripleo/+spec/image-building-api
Use Cases
- Heat would be able to use this API to generate images automatically to be used in instances that it orchestrates.
- ???
Summary
Design and implement an API that allows building of OpenStack images from scratch.
It should provide multiple ways of building images.
- building images "statically", i.e. with diskimage-builder
- building images by booting a base image, running a configuration management tool, and then snapshotting
- building images by booting an installer ISO, "kickstarting" the install, and performing additional post-install configuration (???)
As a first pass, only the diskimage-builder support is necessary, but an eye should be aimed at extensibility.
Discussion
Figuring out how image building and heat /should/ interact given an idea CI/CD story is nontrivial. We discussed on IRC:
09:59 <+lifeless> radix: SpamapS: haven't read that yet, but one important thing IMO is decoupling: for CI/CD you /don't/ want to rebuild the image in each deployment environment.
09:59 <+lifeless> so yes, API - cool. But tying it into heat templates so heat does it - mmmm, I think the coupling is wrong
10:00 < radix> lifeless: it depends what abstraction level the API is at, I guess. I think there is an important thing missing from Heat right now
10:00 < radix> lifeless: or maybe I don't know what you mean.
10:01 < radix> lifeless: are you talking about the thing SpamapS said about having the API invoke heat to "spin up an instance and build"?
10:01 <+lifeless> radix: so, consider a C binary. You build it and test it.
10:01 <+lifeless> radix: do you recompile it when you deploy it somewhere else? If so, are your tests still valid ?
10:02 < radix> maybe? if you want to?
10:02 <+lifeless> radix: In general, they *aren't valid* if any of the build inputs could have changed.
10:02 < radix> or maybe you build the image with that binary baked in from somewhere else
10:02 <+lifeless> radix: now, consider that images are basically just a compile; they have all the tradeoffs of static C binaries, but at the machine scale.
10:05 < radix> lifeless: hmm. so are you arguing against using prebaked images at all in orchestrated deployments?
10:05 <+lifeless> radix: so my point is that when you do CI/CD, it removes a significant range of failure modes if you build the image, deploy to test environment, test, deploy to prod environment.
10:05 <+lifeless> radix: I'm arguing for using prebaked images in both the CI environment and CD environment, vs having the heat in the CI environment build an image and then the heat in the CD environment build an image.
10:05 <+lifeless> radix: I'm the guy that argued for prebaked images for tripleo, so no, I'm not contradicting myself :)
10:06 < radix> lifeless: I think a workflow can be designed that allows you to use the same image in test and prod
10:06 <+lifeless> radix: sure; I'm just highlighting that you've got two distinct heats there
10:07 <+lifeless> radix: so if heat is the one making the image, it doesn't follow naturally
10:07 <+lifeless> radix: vs e.g. zuul or something managing the CI/CD loop being the thing that builds it
10:07 <+lifeless> radix: where it would just fall out of the design
10:08 < radix> let me think about that.
10:08 < radix> lifeless: what if you're not actually building the software you're deploying?
10:08 <+lifeless> or perhaps you need a metaheat
10:08 < radix> what if it's just a collection of existing third-party software and you're just configuring it to do what you want?
10:08 <+lifeless> radix: you might receive a prebaked image from a company like HP or Rackspace
10:09 <+lifeless> radix: or you might be making a custon collection of third party software
10:09 <+lifeless> radix: in which case how do you know they all play nice together unless you test that ?
10:09 < radix> lifeless: you can test it. you have a Heat in your test environment, and a Heat in your prod environment, and you can use the same template in both.
10:10 < radix> lifeless: *something* in the pipeline can determine "oh, I already have this image built, I'll just use the same one", so you don't rebuild the image between test and prod.
10:13 < radix> The layer of abstraction I'm trying to get at is that you have a Heat resource that says "give me an image with os=ubuntu, packages=[...], pip_packages=[...], custom_files=[...]", and then my Instance will have a {"ref": "that_image"} as its ImageId.
10:13 < radix> (or, if you're using a different image-builder backend, maybe it will be something like os="ubuntu", config_system="chef", recipe="foo")
10:15 < radix> my dream is to specify my entire deployment in a set of templates, some written by me, and some coming from elsewhere, without me having to prepare anything else like images up front.
10:19 <+lifeless> radix: sorry, elocal for a bit
10:19 < radix> noproblem :)
10:23 <+lifeless> right, daughter's toy fixed
10:24 < SpamapS> lifeless: I had forgotten about that aspect, and I agree that the two must stay de-coupled
10:24 <+lifeless> radix: so, I have the same goal
10:24 <+lifeless> radix: and in fact we're working up the stack to that in tripleo, improving heat etc as needed
10:24 <+lifeless> radix: it is our *mission statement* to do that with the specific use case of deploying OpenStack as the application
10:25 < radix> I see :)
10:25 * SpamapS wishes he had more time for the "improving heat" aspect :)
10:26 < SpamapS> rwsu: hm, I have not seen that
10:26 <+lifeless> radix: I don't think in terms of heat templates directly though - heat is a tool in the stack of of template-like-things-that-combine-for-a-full-lifecycle-story
10:26 <+lifeless> radix: so the bit I'm quibbling on is saying 'heat should know about <..>'
10:27 < radix> Heat must call all the APIs! alll offf themmmmm!!!!
10:27 <+lifeless> radix: I'm not sure it shouldn't! But I'm not sure it should - as it seems problematic from the model I have in my head.
10:27 <+lifeless> radix: my head may be wrong
