https://wiki.openstack.org/w/api.php?action=feedcontributions&feedformat=atom&user=Atiwari 2026-07-08T17:54:53Z User contributions MediaWiki 1.28.2 https://wiki.openstack.org/w/index.php?title=Marshal&diff=93835 2015-10-21T16:20:48Z

<p>Atiwari: /* Contributors */</p> <hr /> <div>==Contributors==<br /> # Arvind Tiwari (atiwari)<br /> # Dave McCowan (dave-mccowam)<br /> <br /> ==Introduction==<br /> Over the last 5 years, the OpenStack community has provided support for core cloud services such as Nova, Glance, Cinder, Neutron, Keystone, and Barbican. These core services provide the fundamental features for most OpenStack deployments. The community has also also done a great job, providing support services such as Heat, Horizon, Ceilometer, and Monasca, which helps cloud providers and consumers to interact with and manage cloud resources. <br /> One area which has not received as much attention is the area of volume encryption. The import of data security is evident now more than ever. With scrutiny of OpenStack’s current volume encryption offering, it has become apparent that large gaps exist and this is an area deserving of more attention. Project Marshal is a new initiative in the community aimed at addressing those gaps, and improving overall data security within the OpenStack platform.<br /> <br /> ==Recent Evolution Of OpenStack Volume Encryption==<br /> Recently, OpenStack Barbican has evolved from a key storage facility to more of a key management facility. It has received many useful features and some are still in the pipeline, but it is still not used in handling many of the desired volume encryption use cases. In the current flow of volume encryption, when a user chooses to create an encrypted volume using the Cinder API, an associated encryption key will be created in Barbican. A key reference is stored in Cinder along with the disk metadata. The key is later retrieved by Nova from Barbican and provided to the disk encryption subsystem (eg, dm-crypt) while attaching the disk to VM, causing all subsequent I/O to the disk to be encrypted.<br /> There are some gaps in the current flow. E.g., it is not very transparent to end users; only one key can be used at a time and there is no way users can change the encryption key. This flow is very tightly integrated with OpenStack components. This is good for users of those OpenStack components, but there are cases where, e.g., users want to use a non-OpenStack KMS. As well, the current volume encryption implementation focus mostly on Linux based VMs with little support for Windows VMs. On-premise KMS (vs cloud-based KMS) is another use case which is not covered in the current flow.<br /> <br /> ==What is Marshal? ==<br /> Marshal is a new service/tool in the OpenStack community aimed at filling the above-mentioned gaps; it approaches volume encryption in a new way. Marshal is an agent service running inside guest virtual machines (VMs), which will be responsible for securely fetching encryption keys from the KMS (e.g., Barbican). This agent will interface with the disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O. In the case of Linux VMs, this agent will be interfacing with dm-crypt and for Microsoft VMs, it will be interfacing with Bit-locker. The Marshal agent provides an abstraction service and can be integrated with other encryption subsystem if needed. The agent will read keys from the KMS – storing them only in temporary/volatile storage – then forgets the keys immediately after providing them as needed to the disk encryption subsystem.<br /> The foreseen benefits of this approach are outlined below under the “Benefits” section.<br /> <br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by Marshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> # In very basic user will provide the key id using Cloud-Config Scripting. Key id will be stored in certain location of the new VM to be consumed by Marshal agent. <br /> # User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM using Customization Script Source. This KAT will be stored in VM temporary directory for consumed by Marshal. This approach need some enhancement in Barbican.<br /> # User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. Marshal will pick the key info from volume meta-data while configuration.<br /> <br /> User will use command provided by Marshal agent to configure disk encryption. User will get ability to encrypt <br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # KMS interface for Applications.<br /> # KMS interface and key injection for containers. <br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> # Encryption agent for IoT devices.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret Key Access Ticket (KAT) to access the secret.<br /> <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Use of generic image (Linux/Win) with the Marshal agent.<br /> # Use of deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> &lt;TDB&gt;</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=93834 2015-10-21T16:19:58Z

<p>Atiwari: </p> <hr /> <div>==Contributors==<br /> Arvind Tiwari (atiwari)<br /> Dave McCowan (dave-mccowam)<br /> <br /> ==Introduction==<br /> Over the last 5 years, the OpenStack community has provided support for core cloud services such as Nova, Glance, Cinder, Neutron, Keystone, and Barbican. These core services provide the fundamental features for most OpenStack deployments. The community has also also done a great job, providing support services such as Heat, Horizon, Ceilometer, and Monasca, which helps cloud providers and consumers to interact with and manage cloud resources. <br /> One area which has not received as much attention is the area of volume encryption. The import of data security is evident now more than ever. With scrutiny of OpenStack’s current volume encryption offering, it has become apparent that large gaps exist and this is an area deserving of more attention. Project Marshal is a new initiative in the community aimed at addressing those gaps, and improving overall data security within the OpenStack platform.<br /> <br /> ==Recent Evolution Of OpenStack Volume Encryption==<br /> Recently, OpenStack Barbican has evolved from a key storage facility to more of a key management facility. It has received many useful features and some are still in the pipeline, but it is still not used in handling many of the desired volume encryption use cases. In the current flow of volume encryption, when a user chooses to create an encrypted volume using the Cinder API, an associated encryption key will be created in Barbican. A key reference is stored in Cinder along with the disk metadata. The key is later retrieved by Nova from Barbican and provided to the disk encryption subsystem (eg, dm-crypt) while attaching the disk to VM, causing all subsequent I/O to the disk to be encrypted.<br /> There are some gaps in the current flow. E.g., it is not very transparent to end users; only one key can be used at a time and there is no way users can change the encryption key. This flow is very tightly integrated with OpenStack components. This is good for users of those OpenStack components, but there are cases where, e.g., users want to use a non-OpenStack KMS. As well, the current volume encryption implementation focus mostly on Linux based VMs with little support for Windows VMs. On-premise KMS (vs cloud-based KMS) is another use case which is not covered in the current flow.<br /> <br /> ==What is Marshal? ==<br /> Marshal is a new service/tool in the OpenStack community aimed at filling the above-mentioned gaps; it approaches volume encryption in a new way. Marshal is an agent service running inside guest virtual machines (VMs), which will be responsible for securely fetching encryption keys from the KMS (e.g., Barbican). This agent will interface with the disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O. In the case of Linux VMs, this agent will be interfacing with dm-crypt and for Microsoft VMs, it will be interfacing with Bit-locker. The Marshal agent provides an abstraction service and can be integrated with other encryption subsystem if needed. The agent will read keys from the KMS – storing them only in temporary/volatile storage – then forgets the keys immediately after providing them as needed to the disk encryption subsystem.<br /> The foreseen benefits of this approach are outlined below under the “Benefits” section.<br /> <br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by Marshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> # In very basic user will provide the key id using Cloud-Config Scripting. Key id will be stored in certain location of the new VM to be consumed by Marshal agent. <br /> # User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM using Customization Script Source. This KAT will be stored in VM temporary directory for consumed by Marshal. This approach need some enhancement in Barbican.<br /> # User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. Marshal will pick the key info from volume meta-data while configuration.<br /> <br /> User will use command provided by Marshal agent to configure disk encryption. User will get ability to encrypt <br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # KMS interface for Applications.<br /> # KMS interface and key injection for containers. <br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> # Encryption agent for IoT devices.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret Key Access Ticket (KAT) to access the secret.<br /> <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Use of generic image (Linux/Win) with the Marshal agent.<br /> # Use of deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> &lt;TDB&gt;</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=92581 2015-10-14T20:57:22Z

<p>Atiwari: /* Roadmap */</p> <hr /> <div>==Introduction==<br /> Over the last 5 years, the OpenStack community has provided support for core cloud services such as Nova, Glance, Cinder, Neutron, Keystone, and Barbican. These core services provide the fundamental features for most OpenStack deployments. The community has also also done a great job, providing support services such as Heat, Horizon, Ceilometer, and Monasca, which helps cloud providers and consumers to interact with and manage cloud resources. <br /> One area which has not received as much attention is the area of volume encryption. The import of data security is evident now more than ever. With scrutiny of OpenStack’s current volume encryption offering, it has become apparent that large gaps exist and this is an area deserving of more attention. Project Marshal is a new initiative in the community aimed at addressing those gaps, and improving overall data security within the OpenStack platform.<br /> <br /> ==Recent Evolution Of OpenStack Volume Encryption==<br /> Recently, OpenStack Barbican has evolved from a key storage facility to more of a key management facility. It has received many useful features and some are still in the pipeline, but it is still not used in handling many of the desired volume encryption use cases. In the current flow of volume encryption, when a user chooses to create an encrypted volume using the Cinder API, an associated encryption key will be created in Barbican. A key reference is stored in Cinder along with the disk metadata. The key is later retrieved by Nova from Barbican and provided to the disk encryption subsystem (eg, dm-crypt) while attaching the disk to VM, causing all subsequent I/O to the disk to be encrypted.<br /> There are some gaps in the current flow. E.g., it is not very transparent to end users; only one key can be used at a time and there is no way users can change the encryption key. This flow is very tightly integrated with OpenStack components. This is good for users of those OpenStack components, but there are cases where, e.g., users want to use a non-OpenStack KMS. As well, the current volume encryption implementation focus mostly on Linux based VMs with little support for Windows VMs. On-premise KMS (vs cloud-based KMS) is another use case which is not covered in the current flow.<br /> <br /> ==What is Marshal? ==<br /> Marshal is a new service/tool in the OpenStack community aimed at filling the above-mentioned gaps; it approaches volume encryption in a new way. Marshal is an agent service running inside guest virtual machines (VMs), which will be responsible for securely fetching encryption keys from the KMS (e.g., Barbican). This agent will interface with the disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O. In the case of Linux VMs, this agent will be interfacing with dm-crypt and for Microsoft VMs, it will be interfacing with Bit-locker. The Marshal agent provides an abstraction service and can be integrated with other encryption subsystem if needed. The agent will read keys from the KMS – storing them only in temporary/volatile storage – then forgets the keys immediately after providing them as needed to the disk encryption subsystem.<br /> The foreseen benefits of this approach are outlined below under the “Benefits” section.<br /> <br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by Marshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> # In very basic user will provide the key id using Cloud-Config Scripting. Key id will be stored in certain location of the new VM to be consumed by Marshal agent. <br /> # User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM using Customization Script Source. This KAT will be stored in VM temporary directory for consumed by Marshal. This approach need some enhancement in Barbican.<br /> # User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. Marshal will pick the key info from volume meta-data while configuration.<br /> <br /> User will use command provided by Marshal agent to configure disk encryption. User will get ability to encrypt <br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # KMS interface for Applications.<br /> # KMS interface and key injection for containers. <br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> # Encryption agent for IoT devices.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret Key Access Ticket (KAT) to access the secret.<br /> <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Use of generic image (Linux/Win) with the Marshal agent.<br /> # Use of deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> &lt;TDB&gt;</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=92580 2015-10-14T20:55:03Z

<p>Atiwari: /* Roadmap */</p> <hr /> <div>==Introduction==<br /> Over the last 5 years, the OpenStack community has provided support for core cloud services such as Nova, Glance, Cinder, Neutron, Keystone, and Barbican. These core services provide the fundamental features for most OpenStack deployments. The community has also also done a great job, providing support services such as Heat, Horizon, Ceilometer, and Monasca, which helps cloud providers and consumers to interact with and manage cloud resources. <br /> One area which has not received as much attention is the area of volume encryption. The import of data security is evident now more than ever. With scrutiny of OpenStack’s current volume encryption offering, it has become apparent that large gaps exist and this is an area deserving of more attention. Project Marshal is a new initiative in the community aimed at addressing those gaps, and improving overall data security within the OpenStack platform.<br /> <br /> ==Recent Evolution Of OpenStack Volume Encryption==<br /> Recently, OpenStack Barbican has evolved from a key storage facility to more of a key management facility. It has received many useful features and some are still in the pipeline, but it is still not used in handling many of the desired volume encryption use cases. In the current flow of volume encryption, when a user chooses to create an encrypted volume using the Cinder API, an associated encryption key will be created in Barbican. A key reference is stored in Cinder along with the disk metadata. The key is later retrieved by Nova from Barbican and provided to the disk encryption subsystem (eg, dm-crypt) while attaching the disk to VM, causing all subsequent I/O to the disk to be encrypted.<br /> There are some gaps in the current flow. E.g., it is not very transparent to end users; only one key can be used at a time and there is no way users can change the encryption key. This flow is very tightly integrated with OpenStack components. This is good for users of those OpenStack components, but there are cases where, e.g., users want to use a non-OpenStack KMS. As well, the current volume encryption implementation focus mostly on Linux based VMs with little support for Windows VMs. On-premise KMS (vs cloud-based KMS) is another use case which is not covered in the current flow.<br /> <br /> ==What is Marshal? ==<br /> Marshal is a new service/tool in the OpenStack community aimed at filling the above-mentioned gaps; it approaches volume encryption in a new way. Marshal is an agent service running inside guest virtual machines (VMs), which will be responsible for securely fetching encryption keys from the KMS (e.g., Barbican). This agent will interface with the disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O. In the case of Linux VMs, this agent will be interfacing with dm-crypt and for Microsoft VMs, it will be interfacing with Bit-locker. The Marshal agent provides an abstraction service and can be integrated with other encryption subsystem if needed. The agent will read keys from the KMS – storing them only in temporary/volatile storage – then forgets the keys immediately after providing them as needed to the disk encryption subsystem.<br /> The foreseen benefits of this approach are outlined below under the “Benefits” section.<br /> <br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by Marshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> # In very basic user will provide the key id using Cloud-Config Scripting. Key id will be stored in certain location of the new VM to be consumed by Marshal agent. <br /> # User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM using Customization Script Source. This KAT will be stored in VM temporary directory for consumed by Marshal. This approach need some enhancement in Barbican.<br /> # User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. Marshal will pick the key info from volume meta-data while configuration.<br /> <br /> User will use command provided by Marshal agent to configure disk encryption. User will get ability to encrypt <br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # Key interface for containers. <br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> # Containers Integration.<br /> # Encryption agent for IoT device.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret Key Access Ticket (KAT) to access the secret.<br /> <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Use of generic image (Linux/Win) with the Marshal agent.<br /> # Use of deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> &lt;TDB&gt;</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=91732 2015-10-05T16:01:35Z

<p>Atiwari: /* How to inject agent in the VM */</p> <hr /> <div>==Introduction==<br /> Over the last 5 years, the OpenStack community has provided support for core cloud services such as Nova, Glance, Cinder, Neutron, Keystone, and Barbican. These core services provide the fundamental features for most OpenStack deployments. The community has also also done a great job, providing support services such as Heat, Horizon, Ceilometer, and Monasca, which helps cloud providers and consumers to interact with and manage cloud resources. <br /> One area which has not received as much attention is the area of volume encryption. The import of data security is evident now more than ever. With scrutiny of OpenStack’s current volume encryption offering, it has become apparent that large gaps exist and this is an area deserving of more attention. Project Marshal is a new initiative in the community aimed at addressing those gaps, and improving overall data security within the OpenStack platform.<br /> <br /> ==Recent Evolution Of OpenStack Volume Encryption==<br /> Recently, OpenStack Barbican has evolved from a key storage facility to more of a key management facility. It has received many useful features and some are still in the pipeline, but it is still not used in handling many of the desired volume encryption use cases. In the current flow of volume encryption, when a user chooses to create an encrypted volume using the Cinder API, an associated encryption key will be created in Barbican. A key reference is stored in Cinder along with the disk metadata. The key is later retrieved by Nova from Barbican and provided to the disk encryption subsystem (eg, dm-crypt) while attaching the disk to VM, causing all subsequent I/O to the disk to be encrypted.<br /> There are some gaps in the current flow. E.g., it is not very transparent to end users; only one key can be used at a time and there is no way users can change the encryption key. This flow is very tightly integrated with OpenStack components. This is good for users of those OpenStack components, but there are cases where, e.g., users want to use a non-OpenStack KMS. As well, the current volume encryption implementation focus mostly on Linux based VMs with little support for Windows VMs. On-premise KMS (vs cloud-based KMS) is another use case which is not covered in the current flow.<br /> <br /> ==What is Marshal? ==<br /> Marshal is a new service/tool in the OpenStack community aimed at filling the above-mentioned gaps; it approaches volume encryption in a new way. Marshal is an agent service running inside guest virtual machines (VMs), which will be responsible for securely fetching encryption keys from the KMS (e.g., Barbican). This agent will interface with the disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O. In the case of Linux VMs, this agent will be interfacing with dm-crypt and for Microsoft VMs, it will be interfacing with Bit-locker. The Marshal agent provides an abstraction service and can be integrated with other encryption subsystem if needed. The agent will read keys from the KMS – storing them only in temporary/volatile storage – then forgets the keys immediately after providing them as needed to the disk encryption subsystem.<br /> The foreseen benefits of this approach are outlined below under the “Benefits” section.<br /> <br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by Marshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> # In very basic user will provide the key id using Cloud-Config Scripting. Key id will be stored in certain location of the new VM to be consumed by Marshal agent. <br /> # User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM using Customization Script Source. This KAT will be stored in VM temporary directory for consumed by Marshal. This approach need some enhancement in Barbican.<br /> # User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. Marshal will pick the key info from volume meta-data while configuration.<br /> <br /> User will use command provided by Marshal agent to configure disk encryption. User will get ability to encrypt <br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # Key interface for containers. <br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret Key Access Ticket (KAT) to access the secret.<br /> <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Use of generic image (Linux/Win) with the Marshal agent.<br /> # Use of deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> &lt;TDB&gt;</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=91731 2015-10-05T16:00:22Z

<p>Atiwari: /* Roadmap */</p> <hr /> <div>==Introduction==<br /> Over the last 5 years, the OpenStack community has provided support for core cloud services such as Nova, Glance, Cinder, Neutron, Keystone, and Barbican. These core services provide the fundamental features for most OpenStack deployments. The community has also also done a great job, providing support services such as Heat, Horizon, Ceilometer, and Monasca, which helps cloud providers and consumers to interact with and manage cloud resources. <br /> One area which has not received as much attention is the area of volume encryption. The import of data security is evident now more than ever. With scrutiny of OpenStack’s current volume encryption offering, it has become apparent that large gaps exist and this is an area deserving of more attention. Project Marshal is a new initiative in the community aimed at addressing those gaps, and improving overall data security within the OpenStack platform.<br /> <br /> ==Recent Evolution Of OpenStack Volume Encryption==<br /> Recently, OpenStack Barbican has evolved from a key storage facility to more of a key management facility. It has received many useful features and some are still in the pipeline, but it is still not used in handling many of the desired volume encryption use cases. In the current flow of volume encryption, when a user chooses to create an encrypted volume using the Cinder API, an associated encryption key will be created in Barbican. A key reference is stored in Cinder along with the disk metadata. The key is later retrieved by Nova from Barbican and provided to the disk encryption subsystem (eg, dm-crypt) while attaching the disk to VM, causing all subsequent I/O to the disk to be encrypted.<br /> There are some gaps in the current flow. E.g., it is not very transparent to end users; only one key can be used at a time and there is no way users can change the encryption key. This flow is very tightly integrated with OpenStack components. This is good for users of those OpenStack components, but there are cases where, e.g., users want to use a non-OpenStack KMS. As well, the current volume encryption implementation focus mostly on Linux based VMs with little support for Windows VMs. On-premise KMS (vs cloud-based KMS) is another use case which is not covered in the current flow.<br /> <br /> ==What is Marshal? ==<br /> Marshal is a new service/tool in the OpenStack community aimed at filling the above-mentioned gaps; it approaches volume encryption in a new way. Marshal is an agent service running inside guest virtual machines (VMs), which will be responsible for securely fetching encryption keys from the KMS (e.g., Barbican). This agent will interface with the disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O. In the case of Linux VMs, this agent will be interfacing with dm-crypt and for Microsoft VMs, it will be interfacing with Bit-locker. The Marshal agent provides an abstraction service and can be integrated with other encryption subsystem if needed. The agent will read keys from the KMS – storing them only in temporary/volatile storage – then forgets the keys immediately after providing them as needed to the disk encryption subsystem.<br /> The foreseen benefits of this approach are outlined below under the “Benefits” section.<br /> <br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by Marshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> # In very basic user will provide the key id using Cloud-Config Scripting. Key id will be stored in certain location of the new VM to be consumed by Marshal agent. <br /> # User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM using Customization Script Source. This KAT will be stored in VM temporary directory for consumed by Marshal. This approach need some enhancement in Barbican.<br /> # User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. Marshal will pick the key info from volume meta-data while configuration.<br /> <br /> User will use command provided by Marshal agent to configure disk encryption. User will get ability to encrypt <br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # Key interface for containers. <br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret Key Access Ticket (KAT) to access the secret.<br /> <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Create image (Linux/Win) with the Marshal agent.<br /> # Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> &lt;TDB&gt;</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=91730 2015-10-05T15:54:15Z

<p>Atiwari: /* Introduction */</p> <hr /> <div>==Introduction==<br /> Over the last 5 years, the OpenStack community has provided support for core cloud services such as Nova, Glance, Cinder, Neutron, Keystone, and Barbican. These core services provide the fundamental features for most OpenStack deployments. The community has also also done a great job, providing support services such as Heat, Horizon, Ceilometer, and Monasca, which helps cloud providers and consumers to interact with and manage cloud resources. <br /> One area which has not received as much attention is the area of volume encryption. The import of data security is evident now more than ever. With scrutiny of OpenStack’s current volume encryption offering, it has become apparent that large gaps exist and this is an area deserving of more attention. Project Marshal is a new initiative in the community aimed at addressing those gaps, and improving overall data security within the OpenStack platform.<br /> <br /> ==Recent Evolution Of OpenStack Volume Encryption==<br /> Recently, OpenStack Barbican has evolved from a key storage facility to more of a key management facility. It has received many useful features and some are still in the pipeline, but it is still not used in handling many of the desired volume encryption use cases. In the current flow of volume encryption, when a user chooses to create an encrypted volume using the Cinder API, an associated encryption key will be created in Barbican. A key reference is stored in Cinder along with the disk metadata. The key is later retrieved by Nova from Barbican and provided to the disk encryption subsystem (eg, dm-crypt) while attaching the disk to VM, causing all subsequent I/O to the disk to be encrypted.<br /> There are some gaps in the current flow. E.g., it is not very transparent to end users; only one key can be used at a time and there is no way users can change the encryption key. This flow is very tightly integrated with OpenStack components. This is good for users of those OpenStack components, but there are cases where, e.g., users want to use a non-OpenStack KMS. As well, the current volume encryption implementation focus mostly on Linux based VMs with little support for Windows VMs. On-premise KMS (vs cloud-based KMS) is another use case which is not covered in the current flow.<br /> <br /> ==What is Marshal? ==<br /> Marshal is a new service/tool in the OpenStack community aimed at filling the above-mentioned gaps; it approaches volume encryption in a new way. Marshal is an agent service running inside guest virtual machines (VMs), which will be responsible for securely fetching encryption keys from the KMS (e.g., Barbican). This agent will interface with the disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O. In the case of Linux VMs, this agent will be interfacing with dm-crypt and for Microsoft VMs, it will be interfacing with Bit-locker. The Marshal agent provides an abstraction service and can be integrated with other encryption subsystem if needed. The agent will read keys from the KMS – storing them only in temporary/volatile storage – then forgets the keys immediately after providing them as needed to the disk encryption subsystem.<br /> The foreseen benefits of this approach are outlined below under the “Benefits” section.<br /> <br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by Marshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> # In very basic user will provide the key id using Cloud-Config Scripting. Key id will be stored in certain location of the new VM to be consumed by Marshal agent. <br /> # User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM using Customization Script Source. This KAT will be stored in VM temporary directory for consumed by Marshal. This approach need some enhancement in Barbican.<br /> # User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. Marshal will pick the key info from volume meta-data while configuration.<br /> <br /> User will use command provided by Marshal agent to configure disk encryption. User will get ability to encrypt <br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret Key Access Ticket (KAT) to access the secret.<br /> <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Create image (Linux/Win) with the Marshal agent.<br /> # Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> &lt;TDB&gt;</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=90690 2015-09-21T18:42:54Z

<p>Atiwari: </p> <hr /> <div>==Introduction==<br /> Over the last 5 years, the OpenStack community has provided support for core cloud services such as Nova, Glance, Cinder, Neutron, Keystone, and Barbican. These core services provide the fundamental features for most OpenStack deployments. The community has also provided cloud support services such as Heat, Horizon, Ceilometer, and Monasca, which helps cloud providers and consumers to interact with and manage cloud resources. <br /> One area which has not received as much attention is the area of volume encryption. The import of data security is evident now more than ever. With scrutiny of OpenStack’s current volume encryption offering, it has become apparent that large gaps exist and this is an area deserving of more attention. Project Marshal is a new initiative in the community aimed at addressing those gaps, and improving overall data security within the OpenStack platform. <br /> <br /> ==Recent Evolution Of OpenStack Volume Encryption==<br /> Recently, OpenStack Barbican has evolved from a key storage facility to more of a key management facility. It has received many useful features and some are still in the pipeline, but it is still not used in handling many of the desired volume encryption use cases. In the current flow of volume encryption, when a user chooses to create an encrypted volume using the Cinder API, an associated encryption key will be created in Barbican. A key reference is stored in Cinder along with the disk metadata. The key is later retrieved by Nova from Barbican and provided to the disk encryption subsystem (eg, dm-crypt) while attaching the disk to VM, causing all subsequent I/O to the disk to be encrypted.<br /> There are some gaps in the current flow. E.g., it is not very transparent to end users; only one key can be used at a time and there is no way users can change the encryption key. This flow is very tightly integrated with OpenStack components. This is good for users of those OpenStack components, but there are cases where, e.g., users want to use a non-OpenStack KMS. As well, the current volume encryption implementation focus mostly on Linux based VMs with little support for Windows VMs. On-premise KMS (vs cloud-based KMS) is another use case which is not covered in the current flow.<br /> <br /> ==What is Marshal? ==<br /> Marshal is a new service/tool in the OpenStack community aimed at filling the above-mentioned gaps; it approaches volume encryption in a new way. Marshal is an agent service running inside guest virtual machines (VMs), which will be responsible for securely fetching encryption keys from the KMS (e.g., Barbican). This agent will interface with the disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O. In the case of Linux VMs, this agent will be interfacing with dm-crypt and for Microsoft VMs, it will be interfacing with Bit-locker. The Marshal agent provides an abstraction service and can be integrated with other encryption subsystem if needed. The agent will read keys from the KMS – storing them only in temporary/volatile storage – then forgets the keys immediately after providing them as needed to the disk encryption subsystem.<br /> The foreseen benefits of this approach are outlined below under the “Benefits” section.<br /> <br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by Marshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> # In very basic user will provide the key id using Cloud-Config Scripting. Key id will be stored in certain location of the new VM to be consumed by Marshal agent. <br /> # User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM using Customization Script Source. This KAT will be stored in VM temporary directory for consumed by Marshal. This approach need some enhancement in Barbican.<br /> # User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. Marshal will pick the key info from volume meta-data while configuration.<br /> <br /> User will use command provided by Marshal agent to configure disk encryption. User will get ability to encrypt <br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret Key Access Ticket (KAT) to access the secret.<br /> <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Create image (Linux/Win) with the Marshal agent.<br /> # Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> &lt;TDB&gt;</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Inter_Cloud_Resource_Federation&diff=90008 2015-09-11T11:53:34Z

<p>Atiwari: </p> <hr /> <div>=Intercloud Resource Federation ('''Alliance''')=<br /> <br /> ===Arvind Tiwari===<br /> <br /> <br /> Cloud based services are a growing business trend in the IT industry, where service providers establish cloud and offer computing resources (Infrastructure, Platform and Software) to consumers. Consumers often require computing resources across multiple regions, to address their application needs. Single cloud provider may not be able to address such requests due to lack of presence or capacity in multiple regions. This blueprint proposes a solution to address this concern by introducing a concept of InterCloud Resource Federation (a.k.a. Alliance). Using this technical approach multiple cloud entities can work in alliance to form a bigger cloud entity with massive resource capacities.<br /> <br /> = Problem =<br /> This blueprint is trying to address following problems<br /> <br /> === Multi Region Capability ===<br /> Enterprises are adopting cloud based service to address their growing computing workload and they often want resources across multiple regions. These regions sometimes span across multiple geographical boundaries. <br /> <br /> Single providers may not be able to address such requests as setting up huge cloud across multiple regions is a costly and risky investment. Providers often want to lease resources from other provider cross regions to fulfill customer computing workload.<br /> <br /> === Virtual Private Cloud or Cloud Leasing ===<br /> Vendors want to lease resources to other service providers or customers who want have VPC (Virtual Private Cloud). Unlike other service providers these vendors want to target large consumers. <br /> <br /> Cloud service providers have idle computing resources which are not making revenue. Service provider wants to lease these resources to another provider who are in need of these resources.<br /> <br /> === Cloud Bursting ===<br /> In private in-house cloud deployments, enterprises want to bursts into a public/separate cloud when the demand for computing capacity spikes.<br /> <br /> === Joint Venture Cloud ===<br /> Setting up a Cloud with massive resource capacity is time consuming and very costly venture; not everyone would like to invest in such a big venture. Business partners like to invest in joint ventures to make a bigger cloud entity with massive resource capacities.<br /> <br /> === Merger and Acquisitions ===<br /> Merger of business entities are common, integration/utilization of cloud enabled computing resources of two business enterprises are troublesome due to lack of technology. An enterprise expects seamless integration of two cloud entities after merger with anticipation of effective and prompt resource utilization.<br /> <br /> === Expensive Architecture ===<br /> Building multi region architecture to join multiple data centers using dedicated/leased lines is costly and creates tight coupling. This solution is also not a scalable solution for incremental growth.<br /> <br /> = Solution =<br /> OpenStack provides an open source cloud computing platform which is getting huge industry adoption for building up private/public/hybrid cloud. OpenStack based cloud deployments provide Infrastructure and Platform services capabilities to cloud providers. OpenStack inherited cloud entities are ideal candidates to work in alliances to form InterCloud Federation, as their API and Resource models are identical.<br /> <br /> To address these problems, a new service will be added to the OpenStack services stack which will provide “InterCloud Federation” (a.k.a. '''Alliance''') services to participating cloud entities. This service will be responsible for interconnecting multiple cloud instances and provide an abstraction layer to hide interoperability and integration complexities from the provider and end users. <br /> <br /> Cloud providers have to enable this service on their cloud deployments and configure the required artifacts for partner or host cloud entities.<br /> <br /> The picture below depicts the resource consumption flow from Host cloud to Partner cloud.<br /> <br /> [[File:ICRF01.jpg]]<br /> <br /> The picture ‘a’ (below), shows integration of multiple data centers with dedicated leased line for data replication. This architecture tightly couples the data centers and it is not cost effective. <br /> <br /> Picture ‘b’ portrays the interlinked cloudified data centers through Alliance service. In this model, data exchanges between the participating clouds will be happening on secured (PtoP VPN) Internet channel not on dedicated lines.<br /> <br /> [[File:ICRF04.jpg]]<br /> <br /> =Design Considerations=<br /> Following are the high level design consideration and functional components of InterCloud Federation Services (a.k.a. '''Alliance'''). <br /> <br /> === Host Cloud (HC) ===<br /> A cloud entity consumes resources from different cloud providers. This cloud entity plays a role of “resource consumer” in cloud federation business.<br /> <br /> === Partner Cloud (PC) ===<br /> Cloud instance(s) offer its services to a Host Cloud entity. These cloud entities plays a role of “resource provider” in cloud federation business.<br /> <br /> === Cloud Pairing ===<br /> The pairing process is the initial step taken by two cloud providers to work in alliance. This step is required to establish trust between two cloud instances and configure required artifacts. These artifacts include cryptographic keys, cloud identifiers, region/zone information, service availability and InterCloud federation service endpoints. Alliance service endpoints are required for service discovery and inter-cloud token validations.<br /> <br /> The pairing process can be done manually by two cloud providers (or it can be automated). Host cloud provider has to configure artifacts for partner cloud entities and partner has to configure artifacts for host cloud(s). Cloud pairing can be bidirectional, in this case one cloud instance will be playing as host cloud in one and partner cloud on other side.<br /> <br /> === InterCloud Ticket (ICT) ===<br /> This ticket is used for the purpose of inter-cloud communication (e.g. Resource discovery, user token validation across clouds). This is a PKI ticket and contains data structure to hold context information required for communication. This ticket will be encrypted using partner’s (destination) public key and signed by host’s (source) private key.<br /> <br /> === InterCloud Token Validation ===<br /> This service is used by two participating cloud entities to validate user token which involve signature validation and ticket decryption to get the actual X-Auth-Token. This is a part of Alliance service.<br /> <br /> === Region Discovery ===<br /> This is one of the services provided by ICFS to list available regions (or availability zone). Clients use this service to know what are the available regions or zones offered by a provider. Generally these are static information stored in ICFS system.<br /> <br /> === Service Discovery ===<br /> This service is used to discover available services in a particular region or zone offered by a partner cloud. This will provide dynamic list of available services from particular partner region.<br /> <br /> [[File:ICRF02.jpg]]<br /> <br /> === Remote Resource Provisioning ===<br /> <br /> Resource provisioning is required to allocate limited resource efficiently on the (partner cloud) resource provider cloud. Provisioning is also needed to provide sense of ownership to the end user, for cloud provider (partner/host) provisioning is required to address metering aspect.<br /> <br /> Alliance service is responsible for provisioning remote resources. The resource provisioning will be done on local Keystone project at hosting cloud. It also maintains provisioning information in local database. The provisioning info will be used for the purpose of token generation and validation.<br /> <br /> <br /> [[File:ICRF06.jpg]]<br /> <br /> <br /> In the above picture user maintains his identity at one place (host cloud) and owns resources from remote cloud(s) on a local project. This is another benefit of resource federation, where user can use a single project in host cloud to scope all the remote resources across the cloud(s).<br /> <br /> === Resource Access Across Clouds ===<br /> Resource access process start by getting an “X-Auth-Token” scoped (1) to local Keystone project of &quot;host&quot; cloud. Keystone service at HC will talk to local Alliance to get information about remote resources associated with project (2).<br /> <br /> As part of token response (3) client gets a service catalog containing endpoints to the remote (federated) resources. Client uses the remote resource endpoint to access the resource (4), it provides (X-Host-Cloud-Id) host cloud identifier in request header and the X-Auth-Token{hc} got from host cloud. <br /> <br /> Auth middle-ware protecting the resource at partner cloud intercepts the request and makes a call (5) to Keystone for token validation. Keystone delegates (6) such the token validation request to Alliance service which is not issued by it (foreign token) and have X-Host-Cloud-Id header associated.<br /> <br /> Alliance uses the cloud identifier (X-Host-Cloud-Id) from the header to lookup the paired host cloud and it's peer Alliance endpoint. Using the X-Auth-Token{hc}, it forms an InterCloud Federation Ticket and uses paired Alliance endpoint to validate user token (7). <br /> <br /> Alliance at HC will coordinate with local Keystone to validate the token (8). <br /> <br /> After successful inter-cloud token validation (9), Alliance service provide (10) the validate response to Keystone service running at PC. Keystone will caches the token in locale system and respond to middle-ware(11). Keystone will use the cached token for future token validations.<br /> <br /> [[File:ICRF03.jpg]]<br /> <br /> === X-Auth-Token processing=== <br /> <br /> Clients won't like to deal with multiple X-Auth-Tokens to access their resources across clouds (regions). <br /> <br /> Following are the options to solve this issue.<br /> <br /> ====PKI tokens====<br /> PKI tokens can be used by clients to access resources across clouds. There won't be inter-cloud token validation required to validate the PKI tokens.<br /> PKI token are proven to be heavy, Federated Token can be a better solution.<br /> <br /> ====Federated Tokens====<br /> Instead of generating new X-Auth-Token{pc} by partner cloud, partner cloud may choose to use the same X-Auth-Token{hc} issued by host cloud. <br /> <br /> After successful inter-cloud token validation (as explained above) Alliance will cache the token locally and utilize the same X-Auth-Token for future communication. This option can be set as part of cloud pairing depending of level of trust between two cloud providers. <br /> <br /> Note: Inter-cloud token validation is one time process or can be done multiple times over the period of communication by clients.<br /> <br /> ====Federated Tokens by Eager Propagation====<br /> To support federated tokens partner cloud has to do inter-cloud token validation and cache the validate token response to make the future token validation more efficient. Another approach to solve the performance of inter-cloud token validation is to propagate the tokens to partner cloud in push mode. Host cloud will propagate token to the relevant partner using notification route.<br /> <br /> === SSO Across Cloud===<br /> <br /> In this mode, clients chooses to use PC's identity (Keystone) endpoint to make auth token request. Client provide credentials, project_id and cloud_id to the PC's identity service. Keystone will coordinate with Alliance service to get the token from remote cloud.<br /> <br /> [[File:ICRF05.jpg]]<br /> <br /> === SSOut Across Cloud (or InterCloud Token Revocation) ===<br /> <br /> Token revocation in an important aspect to maintain the security and system integrity. In case of resource federation use case, tokens revocation become more important as an stale token can cause bigger harm specially to the resource provider clouds.<br /> <br /> Inter-cloud token revocation will allow token revocation across cloud, e.g. Host cloud can initiate the token revocation for a token issued by itself or partner clouds can request/initiate the token revocation of a federated token.<br /> <br /> Alliance service is will be the interface between clouds to make the token revocation happen.<br /> <br /> === Notification ===<br /> Is part of InterCloud Federation Service and used to notify certain events (e.g. Meter, resource de-provisioning etc...) to and from participating clouds.<br /> <br /> <br /> ==Issue and Concerns ==<br /> <br /> === Homogeneous vs Heterogeneous Cloud Partners===<br /> TBD<br /> <br /> =Impact=<br /> <br /> Following are the high level impact on existing OpenStack service <br /> <br /> * Keystone token management subsystem will be impacted to support endpoints for partner services. <br /> * Keystone service REST API will make an extra call to Alliance service to get list of available remote services.<br /> * Horizon will be enhanced to integrate with Inter Alliance Service.<br /> * Other OS Services - TBD<br /> <br /> =Note=<br /> * This page is in &quot;work in progress&quot; mode.<br /> * Please excuse any typographical error.</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Inter_Cloud_Resource_Federation&diff=90007 2015-09-11T11:50:19Z

<p>Atiwari: </p> <hr /> <div>=InterCloud Resource Federation ('''Alliance''')=<br /> <br /> ===Arvind Tiwari===<br /> <br /> <br /> Cloud based services are a growing business trend in the IT industry, where service providers establish cloud and offer computing resources (Infrastructure, Platform and Software) to consumers. Consumers often require computing resources across multiple regions, to address their application needs. Single cloud provider may not be able to address such requests due to lack of presence or capacity in multiple regions. This blueprint proposes a solution to address this concern by introducing a concept of InterCloud Resource Federation (a.k.a. Alliance). Using this technical approach multiple cloud entities can work in alliance to form a bigger cloud entity with massive resource capacities.<br /> <br /> = Problem =<br /> This blueprint is trying to address following problems<br /> <br /> === Multi Region Capability ===<br /> Enterprises are adopting cloud based service to address their growing computing workload and they often want resources across multiple regions. These regions sometimes span across multiple geographical boundaries. <br /> <br /> Single providers may not be able to address such requests as setting up huge cloud across multiple regions is a costly and risky investment. Providers often want to lease resources from other provider cross regions to fulfill customer computing workload.<br /> <br /> === Virtual Private Cloud or Cloud Leasing ===<br /> Vendors want to lease resources to other service providers or customers who want have VPC (Virtual Private Cloud). Unlike other service providers these vendors want to target large consumers. <br /> <br /> Cloud service providers have idle computing resources which are not making revenue. Service provider wants to lease these resources to another provider who are in need of these resources.<br /> <br /> === Cloud Bursting ===<br /> In private in-house cloud deployments, enterprises want to bursts into a public/separate cloud when the demand for computing capacity spikes.<br /> <br /> === Joint Venture Cloud ===<br /> Setting up a Cloud with massive resource capacity is time consuming and very costly venture; not everyone would like to invest in such a big venture. Business partners like to invest in joint ventures to make a bigger cloud entity with massive resource capacities.<br /> <br /> === Merger and Acquisitions ===<br /> Merger of business entities are common, integration/utilization of cloud enabled computing resources of two business enterprises are troublesome due to lack of technology. An enterprise expects seamless integration of two cloud entities after merger with anticipation of effective and prompt resource utilization.<br /> <br /> === Expensive Architecture ===<br /> Building multi region architecture to join multiple data centers using dedicated/leased lines is costly and creates tight coupling. This solution is also not a scalable solution for incremental growth.<br /> <br /> = Solution =<br /> OpenStack provides an open source cloud computing platform which is getting huge industry adoption for building up private/public/hybrid cloud. OpenStack based cloud deployments provide Infrastructure and Platform services capabilities to cloud providers. OpenStack inherited cloud entities are ideal candidates to work in alliances to form InterCloud Federation, as their API and Resource models are identical.<br /> <br /> To address these problems, a new service will be added to the OpenStack services stack which will provide “InterCloud Federation” (a.k.a. '''Alliance''') services to participating cloud entities. This service will be responsible for interconnecting multiple cloud instances and provide an abstraction layer to hide interoperability and integration complexities from the provider and end users. <br /> <br /> Cloud providers have to enable this service on their cloud deployments and configure the required artifacts for partner or host cloud entities.<br /> <br /> The picture below depicts the resource consumption flow from Host cloud to Partner cloud.<br /> <br /> [[File:ICRF01.jpg]]<br /> <br /> The picture ‘a’ (below), shows integration of multiple data centers with dedicated leased line for data replication. This architecture tightly couples the data centers and it is not cost effective. <br /> <br /> Picture ‘b’ portrays the interlinked cloudified data centers through Alliance service. In this model, data exchanges between the participating clouds will be happening on secured (PtoP VPN) Internet channel not on dedicated lines.<br /> <br /> [[File:ICRF04.jpg]]<br /> <br /> =Design Considerations=<br /> Following are the high level design consideration and functional components of InterCloud Federation Services (a.k.a. '''Alliance'''). <br /> <br /> === Host Cloud (HC) ===<br /> A cloud entity consumes resources from different cloud providers. This cloud entity plays a role of “resource consumer” in cloud federation business.<br /> <br /> === Partner Cloud (PC) ===<br /> Cloud instance(s) offer its services to a Host Cloud entity. These cloud entities plays a role of “resource provider” in cloud federation business.<br /> <br /> === Cloud Pairing ===<br /> The pairing process is the initial step taken by two cloud providers to work in alliance. This step is required to establish trust between two cloud instances and configure required artifacts. These artifacts include cryptographic keys, cloud identifiers, region/zone information, service availability and InterCloud federation service endpoints. Alliance service endpoints are required for service discovery and inter-cloud token validations.<br /> <br /> The pairing process can be done manually by two cloud providers (or it can be automated). Host cloud provider has to configure artifacts for partner cloud entities and partner has to configure artifacts for host cloud(s). Cloud pairing can be bidirectional, in this case one cloud instance will be playing as host cloud in one and partner cloud on other side.<br /> <br /> === InterCloud Ticket (ICT) ===<br /> This ticket is used for the purpose of inter-cloud communication (e.g. Resource discovery, user token validation across clouds). This is a PKI ticket and contains data structure to hold context information required for communication. This ticket will be encrypted using partner’s (destination) public key and signed by host’s (source) private key.<br /> <br /> === InterCloud Token Validation ===<br /> This service is used by two participating cloud entities to validate user token which involve signature validation and ticket decryption to get the actual X-Auth-Token. This is a part of Alliance service.<br /> <br /> === Region Discovery ===<br /> This is one of the services provided by ICFS to list available regions (or availability zone). Clients use this service to know what are the available regions or zones offered by a provider. Generally these are static information stored in ICFS system.<br /> <br /> === Service Discovery ===<br /> This service is used to discover available services in a particular region or zone offered by a partner cloud. This will provide dynamic list of available services from particular partner region.<br /> <br /> [[File:ICRF02.jpg]]<br /> <br /> === Remote Resource Provisioning ===<br /> <br /> Resource provisioning is required to allocate limited resource efficiently on the (partner cloud) resource provider cloud. Provisioning is also needed to provide sense of ownership to the end user, for cloud provider (partner/host) provisioning is required to address metering aspect.<br /> <br /> Alliance service is responsible for provisioning remote resources. The resource provisioning will be done on local Keystone project at hosting cloud. It also maintains provisioning information in local database. The provisioning info will be used for the purpose of token generation and validation.<br /> <br /> <br /> [[File:ICRF06.jpg]]<br /> <br /> <br /> In the above picture user maintains his identity at one place (host cloud) and owns resources from remote cloud(s) on a local project. This is another benefit of resource federation, where user can use a single project in host cloud to scope all the remote resources across the cloud(s).<br /> <br /> === Resource Access Across Clouds ===<br /> Resource access process start by getting an “X-Auth-Token” scoped (1) to local Keystone project of &quot;host&quot; cloud. Keystone service at HC will talk to local Alliance to get information about remote resources associated with project (2).<br /> <br /> As part of token response (3) client gets a service catalog containing endpoints to the remote (federated) resources. Client uses the remote resource endpoint to access the resource (4), it provides (X-Host-Cloud-Id) host cloud identifier in request header and the X-Auth-Token{hc} got from host cloud. <br /> <br /> Auth middle-ware protecting the resource at partner cloud intercepts the request and makes a call (5) to Keystone for token validation. Keystone delegates (6) such the token validation request to Alliance service which is not issued by it (foreign token) and have X-Host-Cloud-Id header associated.<br /> <br /> Alliance uses the cloud identifier (X-Host-Cloud-Id) from the header to lookup the paired host cloud and it's peer Alliance endpoint. Using the X-Auth-Token{hc}, it forms an InterCloud Federation Ticket and uses paired Alliance endpoint to validate user token (7). <br /> <br /> Alliance at HC will coordinate with local Keystone to validate the token (8). <br /> <br /> After successful inter-cloud token validation (9), Alliance service provide (10) the validate response to Keystone service running at PC. Keystone will caches the token in locale system and respond to middle-ware(11). Keystone will use the cached token for future token validations.<br /> <br /> [[File:ICRF03.jpg]]<br /> <br /> === X-Auth-Token processing=== <br /> <br /> Clients won't like to deal with multiple X-Auth-Tokens to access their resources across clouds (regions). <br /> <br /> Following are the options to solve this issue.<br /> <br /> ====PKI tokens====<br /> PKI tokens can be used by clients to access resources across clouds. There won't be inter-cloud token validation required to validate the PKI tokens.<br /> PKI token are proven to be heavy, Federated Token can be a better solution.<br /> <br /> ====Federated Tokens====<br /> Instead of generating new X-Auth-Token{pc} by partner cloud, partner cloud may choose to use the same X-Auth-Token{hc} issued by host cloud. <br /> <br /> After successful inter-cloud token validation (as explained above) Alliance will cache the token locally and utilize the same X-Auth-Token for future communication. This option can be set as part of cloud pairing depending of level of trust between two cloud providers. <br /> <br /> Note: Inter-cloud token validation is one time process or can be done multiple times over the period of communication by clients.<br /> <br /> ====Federated Tokens by Eager Propagation====<br /> To support federated tokens partner cloud has to do inter-cloud token validation and cache the validate token response to make the future token validation more efficient. Another approach to solve the performance of inter-cloud token validation is to propagate the tokens to partner cloud in push mode. Host cloud will propagate token to the relevant partner using notification route.<br /> <br /> === SSO Across Cloud===<br /> <br /> In this mode, clients chooses to use PC's identity (Keystone) endpoint to make auth token request. Client provide credentials, project_id and cloud_id to the PC's identity service. Keystone will coordinate with Alliance service to get the token from remote cloud.<br /> <br /> [[File:ICRF05.jpg]]<br /> <br /> === SSOut Across Cloud (or InterCloud Token Revocation) ===<br /> <br /> Token revocation in an important aspect to maintain the security and system integrity. In case of resource federation use case, tokens revocation become more important as an stale token can cause bigger harm specially to the resource provider clouds.<br /> <br /> Inter-cloud token revocation will allow token revocation across cloud, e.g. Host cloud can initiate the token revocation for a token issued by itself or partner clouds can request/initiate the token revocation of a federated token.<br /> <br /> Alliance service is will be the interface between clouds to make the token revocation happen.<br /> <br /> === Notification ===<br /> Is part of InterCloud Federation Service and used to notify certain events (e.g. Meter, resource de-provisioning etc...) to and from participating clouds.<br /> <br /> <br /> ==Issue and Concerns ==<br /> <br /> === Homogeneous vs Heterogeneous Cloud Partners===<br /> TBD<br /> <br /> =Impact=<br /> <br /> Following are the high level impact on existing OpenStack service <br /> <br /> * Keystone token management subsystem will be impacted to support endpoints for partner services. <br /> * Keystone service REST API will make an extra call to Alliance service to get list of available remote services.<br /> * Horizon will be enhanced to integrate with Inter Alliance Service.<br /> * Other OS Services - TBD<br /> <br /> =Note=<br /> * This page is in &quot;work in progress&quot; mode.<br /> * Please excuse any typographical error.</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Inter_Cloud_Resource_Federation&diff=90006 2015-09-11T11:48:55Z

<p>Atiwari: </p> <hr /> <div>=InterCloud Resource Federation ('''Alliance''')=<br /> <br /> ==== Tiwari===''<br /> <br /> <br /> Cloud based services are a growing business trend in the IT industry, where service providers establish cloud and offer computing resources (Infrastructure, Platform and Software) to consumers. Consumers often require computing resources across multiple regions, to address their application needs. Single cloud provider may not be able to address such requests due to lack of presence or capacity in multiple regions. This blueprint proposes a solution to address this concern by introducing a concept of InterCloud Resource Federation (a.k.a. Alliance). Using this technical approach multiple cloud entities can work in alliance to form a bigger cloud entity with massive resource capacities.<br /> <br /> = Problem =<br /> This blueprint is trying to address following problems<br /> <br /> === Multi Region Capability ===<br /> Enterprises are adopting cloud based service to address their growing computing workload and they often want resources across multiple regions. These regions sometimes span across multiple geographical boundaries. <br /> <br /> Single providers may not be able to address such requests as setting up huge cloud across multiple regions is a costly and risky investment. Providers often want to lease resources from other provider cross regions to fulfill customer computing workload.<br /> <br /> === Virtual Private Cloud or Cloud Leasing ===<br /> Vendors want to lease resources to other service providers or customers who want have VPC (Virtual Private Cloud). Unlike other service providers these vendors want to target large consumers. <br /> <br /> Cloud service providers have idle computing resources which are not making revenue. Service provider wants to lease these resources to another provider who are in need of these resources.<br /> <br /> === Cloud Bursting ===<br /> In private in-house cloud deployments, enterprises want to bursts into a public/separate cloud when the demand for computing capacity spikes.<br /> <br /> === Joint Venture Cloud ===<br /> Setting up a Cloud with massive resource capacity is time consuming and very costly venture; not everyone would like to invest in such a big venture. Business partners like to invest in joint ventures to make a bigger cloud entity with massive resource capacities.<br /> <br /> === Merger and Acquisitions ===<br /> Merger of business entities are common, integration/utilization of cloud enabled computing resources of two business enterprises are troublesome due to lack of technology. An enterprise expects seamless integration of two cloud entities after merger with anticipation of effective and prompt resource utilization.<br /> <br /> === Expensive Architecture ===<br /> Building multi region architecture to join multiple data centers using dedicated/leased lines is costly and creates tight coupling. This solution is also not a scalable solution for incremental growth.<br /> <br /> = Solution =<br /> OpenStack provides an open source cloud computing platform which is getting huge industry adoption for building up private/public/hybrid cloud. OpenStack based cloud deployments provide Infrastructure and Platform services capabilities to cloud providers. OpenStack inherited cloud entities are ideal candidates to work in alliances to form InterCloud Federation, as their API and Resource models are identical.<br /> <br /> To address these problems, a new service will be added to the OpenStack services stack which will provide “InterCloud Federation” (a.k.a. '''Alliance''') services to participating cloud entities. This service will be responsible for interconnecting multiple cloud instances and provide an abstraction layer to hide interoperability and integration complexities from the provider and end users. <br /> <br /> Cloud providers have to enable this service on their cloud deployments and configure the required artifacts for partner or host cloud entities.<br /> <br /> The picture below depicts the resource consumption flow from Host cloud to Partner cloud.<br /> <br /> [[File:ICRF01.jpg]]<br /> <br /> The picture ‘a’ (below), shows integration of multiple data centers with dedicated leased line for data replication. This architecture tightly couples the data centers and it is not cost effective. <br /> <br /> Picture ‘b’ portrays the interlinked cloudified data centers through Alliance service. In this model, data exchanges between the participating clouds will be happening on secured (PtoP VPN) Internet channel not on dedicated lines.<br /> <br /> [[File:ICRF04.jpg]]<br /> <br /> =Design Considerations=<br /> Following are the high level design consideration and functional components of InterCloud Federation Services (a.k.a. '''Alliance'''). <br /> <br /> === Host Cloud (HC) ===<br /> A cloud entity consumes resources from different cloud providers. This cloud entity plays a role of “resource consumer” in cloud federation business.<br /> <br /> === Partner Cloud (PC) ===<br /> Cloud instance(s) offer its services to a Host Cloud entity. These cloud entities plays a role of “resource provider” in cloud federation business.<br /> <br /> === Cloud Pairing ===<br /> The pairing process is the initial step taken by two cloud providers to work in alliance. This step is required to establish trust between two cloud instances and configure required artifacts. These artifacts include cryptographic keys, cloud identifiers, region/zone information, service availability and InterCloud federation service endpoints. Alliance service endpoints are required for service discovery and inter-cloud token validations.<br /> <br /> The pairing process can be done manually by two cloud providers (or it can be automated). Host cloud provider has to configure artifacts for partner cloud entities and partner has to configure artifacts for host cloud(s). Cloud pairing can be bidirectional, in this case one cloud instance will be playing as host cloud in one and partner cloud on other side.<br /> <br /> === InterCloud Ticket (ICT) ===<br /> This ticket is used for the purpose of inter-cloud communication (e.g. Resource discovery, user token validation across clouds). This is a PKI ticket and contains data structure to hold context information required for communication. This ticket will be encrypted using partner’s (destination) public key and signed by host’s (source) private key.<br /> <br /> === InterCloud Token Validation ===<br /> This service is used by two participating cloud entities to validate user token which involve signature validation and ticket decryption to get the actual X-Auth-Token. This is a part of Alliance service.<br /> <br /> === Region Discovery ===<br /> This is one of the services provided by ICFS to list available regions (or availability zone). Clients use this service to know what are the available regions or zones offered by a provider. Generally these are static information stored in ICFS system.<br /> <br /> === Service Discovery ===<br /> This service is used to discover available services in a particular region or zone offered by a partner cloud. This will provide dynamic list of available services from particular partner region.<br /> <br /> [[File:ICRF02.jpg]]<br /> <br /> === Remote Resource Provisioning ===<br /> <br /> Resource provisioning is required to allocate limited resource efficiently on the (partner cloud) resource provider cloud. Provisioning is also needed to provide sense of ownership to the end user, for cloud provider (partner/host) provisioning is required to address metering aspect.<br /> <br /> Alliance service is responsible for provisioning remote resources. The resource provisioning will be done on local Keystone project at hosting cloud. It also maintains provisioning information in local database. The provisioning info will be used for the purpose of token generation and validation.<br /> <br /> <br /> [[File:ICRF06.jpg]]<br /> <br /> <br /> In the above picture user maintains his identity at one place (host cloud) and owns resources from remote cloud(s) on a local project. This is another benefit of resource federation, where user can use a single project in host cloud to scope all the remote resources across the cloud(s).<br /> <br /> === Resource Access Across Clouds ===<br /> Resource access process start by getting an “X-Auth-Token” scoped (1) to local Keystone project of &quot;host&quot; cloud. Keystone service at HC will talk to local Alliance to get information about remote resources associated with project (2).<br /> <br /> As part of token response (3) client gets a service catalog containing endpoints to the remote (federated) resources. Client uses the remote resource endpoint to access the resource (4), it provides (X-Host-Cloud-Id) host cloud identifier in request header and the X-Auth-Token{hc} got from host cloud. <br /> <br /> Auth middle-ware protecting the resource at partner cloud intercepts the request and makes a call (5) to Keystone for token validation. Keystone delegates (6) such the token validation request to Alliance service which is not issued by it (foreign token) and have X-Host-Cloud-Id header associated.<br /> <br /> Alliance uses the cloud identifier (X-Host-Cloud-Id) from the header to lookup the paired host cloud and it's peer Alliance endpoint. Using the X-Auth-Token{hc}, it forms an InterCloud Federation Ticket and uses paired Alliance endpoint to validate user token (7). <br /> <br /> Alliance at HC will coordinate with local Keystone to validate the token (8). <br /> <br /> After successful inter-cloud token validation (9), Alliance service provide (10) the validate response to Keystone service running at PC. Keystone will caches the token in locale system and respond to middle-ware(11). Keystone will use the cached token for future token validations.<br /> <br /> [[File:ICRF03.jpg]]<br /> <br /> === X-Auth-Token processing=== <br /> <br /> Clients won't like to deal with multiple X-Auth-Tokens to access their resources across clouds (regions). <br /> <br /> Following are the options to solve this issue.<br /> <br /> ====PKI tokens====<br /> PKI tokens can be used by clients to access resources across clouds. There won't be inter-cloud token validation required to validate the PKI tokens.<br /> PKI token are proven to be heavy, Federated Token can be a better solution.<br /> <br /> ====Federated Tokens====<br /> Instead of generating new X-Auth-Token{pc} by partner cloud, partner cloud may choose to use the same X-Auth-Token{hc} issued by host cloud. <br /> <br /> After successful inter-cloud token validation (as explained above) Alliance will cache the token locally and utilize the same X-Auth-Token for future communication. This option can be set as part of cloud pairing depending of level of trust between two cloud providers. <br /> <br /> Note: Inter-cloud token validation is one time process or can be done multiple times over the period of communication by clients.<br /> <br /> ====Federated Tokens by Eager Propagation====<br /> To support federated tokens partner cloud has to do inter-cloud token validation and cache the validate token response to make the future token validation more efficient. Another approach to solve the performance of inter-cloud token validation is to propagate the tokens to partner cloud in push mode. Host cloud will propagate token to the relevant partner using notification route.<br /> <br /> === SSO Across Cloud===<br /> <br /> In this mode, clients chooses to use PC's identity (Keystone) endpoint to make auth token request. Client provide credentials, project_id and cloud_id to the PC's identity service. Keystone will coordinate with Alliance service to get the token from remote cloud.<br /> <br /> [[File:ICRF05.jpg]]<br /> <br /> === SSOut Across Cloud (or InterCloud Token Revocation) ===<br /> <br /> Token revocation in an important aspect to maintain the security and system integrity. In case of resource federation use case, tokens revocation become more important as an stale token can cause bigger harm specially to the resource provider clouds.<br /> <br /> Inter-cloud token revocation will allow token revocation across cloud, e.g. Host cloud can initiate the token revocation for a token issued by itself or partner clouds can request/initiate the token revocation of a federated token.<br /> <br /> Alliance service is will be the interface between clouds to make the token revocation happen.<br /> <br /> === Notification ===<br /> Is part of InterCloud Federation Service and used to notify certain events (e.g. Meter, resource de-provisioning etc...) to and from participating clouds.<br /> <br /> <br /> ==Issue and Concerns ==<br /> <br /> === Homogeneous vs Heterogeneous Cloud Partners===<br /> TBD<br /> <br /> =Impact=<br /> <br /> Following are the high level impact on existing OpenStack service <br /> <br /> * Keystone token management subsystem will be impacted to support endpoints for partner services. <br /> * Keystone service REST API will make an extra call to Alliance service to get list of available remote services.<br /> * Horizon will be enhanced to integrate with Inter Alliance Service.<br /> * Other OS Services - TBD<br /> <br /> =Note=<br /> * This page is in &quot;work in progress&quot; mode.<br /> * Please excuse any typographical error.</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Hierarchical_administrative_boundary&diff=90005 2015-09-11T11:46:38Z

<p>Atiwari: </p> <hr /> <div><br /> ===Arvind Tiwari===<br /> <br /> == Abstract: ==<br /> <br /> In large scale OpenStack based cloud deployments, cloud provider (CP) may wants to sell their services in White Label (AKA Reseller or value-added reseller) business model. This is a symbiotic relation as CP sells its services in bulk and Reseller gets appropriate discounts.<br /> <br /> In such business model CP setup a &quot;virtual private cloud&quot; (VPS) for each Resellers on its infrastructure or hosting cloud (HC). A reseller acquires special privileges to administer its customers and resources allocated on its customer's disposal.<br /> <br /> <br /> == Requirements: ==<br /> <br /> === CP perspective: ===<br /> <br /> # Cloud provider want to have capabilities to manage multiple Resellers accounts.<br /> # Ability to spin up multiple reseller accounts on demand.<br /> # Ability to administer resellers accounts including their direct resources (IAM and NON-IAM).<br /> # Ability to administer sub-customers (direct customer to resellers) including their resources (IAM and NON-IAM)..<br /> <br /> <br /> e.g. Cloud Provider Inc. (CPI) has large scale OpenStack based deployment and sells it services in Reseller mode.<br /> <br /> === Reseller perspective: ===<br /> # A reseller want to have it own realm (vpc) over CP's cloud infrastructure and expect re-branding aspects.<br /> # Ability to spin up multiple accounts on behalf of my customers without interventions from CP.<br /> # Ability to administer my customers without interventions from CP.<br /> # Ability to administer my customers resources (IAM and NON-IAM) without interventions from CP.<br /> <br /> <br /> e.g. Services Provider Inc. (SPI) a service provider which sells its services to multiple customers over 3rd party cloud establishments.<br /> <br /> == Potential Solution: ==<br /> <br /> Note: The scope of this proposal is low-level, and is focused on the architectural changes limited to Identity management (Keystone) system only.<br /> <br /> To solve these problem, We need an ability to setup hierarchical administrative boundaries within a cloud deployment, so that multiple service providers (like SPI) can be created and have administrative privileges across domains which falls under their administrative boundary. In this model CP will be the root of the hierarchical administrative boundary.<br /> <br /> Keystone has concept of domains which is nothing but a notion of an administrative boundary. An admin of one domain is allowed to manage resources (Computing and non computing) within a specific domain but not across domains (provided correct policy is in place). This feature is already in place so there will be no change needed.<br /> <br /> We can extend the notion of domain further to establish root (parent/super) and leaf (child/sub) domain relationship, one set of root and leaf domains will define a hierarchical administrative boundary. Below diagram depicts a domain hierarchy where the root represents the cloud providers (root administrative boundary) and each levels below represents different hierarchical administrative boundaries.<br /> <br /> [[File:Hierarchical_administrative_boundary_1.jpg]]<br /> <br /> Keystone provide robust role based access control (RBAC) capabilities where roles are associated with access control rules through policy. Subjects are associated with roles scoped to a domain or projects within a domains. Keystone also support a notion of inherited roles which allows roles (mostly project) to be inherited from owning domain. We need to extend the current role inheritance model so that subject's (identity in keystone) inherited roles (project and/or domain) can be propagated down in the domain hierarchy. Diagram below shows the role inheritance propagation.<br /> <br /> Note: Role inheritance propagation will go downwards only but not upwards or side wise. <br /> <br /> [[File:Hierarchical_administrative_boundary_2.jpg]]<br /> <br /> <br /> Cloud provider (or reseller) administrator has to scope his/her security token to a particular child domain (or a project in child domain) to acquire the inherited administrative roles. Malicious and unauthorized access to external resources will be prevented by Keystone RBAC policy enforcement. Diagram below shows how cloud admin can scope token to a project in leaf domain to manage its comuting resource.<br /> <br /> <br /> [[File:Hierarchical_administrative_boundary_3.jpg]]<br /> <br /> == Impact: ==<br /> <br /> Most of the changes are scoped to Keystone only, apparently there is no change required to external (Nova, Swift, ....) services.<br /> <br /> * Domain model need to be extended to support notion of domain hierarchy.<br /> * Role subsystem will be impacted to handle hierarchical inherited roles.<br /> * Role evaluation subsystem will be impacted to handle hierarchical inherited roles.<br /> * Auth middle-ware and token validation has to be enhanced.<br /> * Domain API will be impacted to capture the domain hierarchy.</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Hierarchical_administrative_boundary&diff=90004 2015-09-11T11:44:41Z

<p>Atiwari: </p> <hr /> <div>'''Hierarchical administrative boundary'''<br /> <br /> Arvind Tiwari<br /> <br /> == Abstract: ==<br /> <br /> In large scale OpenStack based cloud deployments, cloud provider (CP) may wants to sell their services in White Label (AKA Reseller or value-added reseller) business model. This is a symbiotic relation as CP sells its services in bulk and Reseller gets appropriate discounts.<br /> <br /> In such business model CP setup a &quot;virtual private cloud&quot; (VPS) for each Resellers on its infrastructure or hosting cloud (HC). A reseller acquires special privileges to administer its customers and resources allocated on its customer's disposal.<br /> <br /> <br /> == Requirements: ==<br /> <br /> === CP perspective: ===<br /> <br /> # Cloud provider want to have capabilities to manage multiple Resellers accounts.<br /> # Ability to spin up multiple reseller accounts on demand.<br /> # Ability to administer resellers accounts including their direct resources (IAM and NON-IAM).<br /> # Ability to administer sub-customers (direct customer to resellers) including their resources (IAM and NON-IAM)..<br /> <br /> <br /> e.g. Cloud Provider Inc. (CPI) has large scale OpenStack based deployment and sells it services in Reseller mode.<br /> <br /> === Reseller perspective: ===<br /> # A reseller want to have it own realm (vpc) over CP's cloud infrastructure and expect re-branding aspects.<br /> # Ability to spin up multiple accounts on behalf of my customers without interventions from CP.<br /> # Ability to administer my customers without interventions from CP.<br /> # Ability to administer my customers resources (IAM and NON-IAM) without interventions from CP.<br /> <br /> <br /> e.g. Services Provider Inc. (SPI) a service provider which sells its services to multiple customers over 3rd party cloud establishments.<br /> <br /> == Potential Solution: ==<br /> <br /> Note: The scope of this proposal is low-level, and is focused on the architectural changes limited to Identity management (Keystone) system only.<br /> <br /> To solve these problem, We need an ability to setup hierarchical administrative boundaries within a cloud deployment, so that multiple service providers (like SPI) can be created and have administrative privileges across domains which falls under their administrative boundary. In this model CP will be the root of the hierarchical administrative boundary.<br /> <br /> Keystone has concept of domains which is nothing but a notion of an administrative boundary. An admin of one domain is allowed to manage resources (Computing and non computing) within a specific domain but not across domains (provided correct policy is in place). This feature is already in place so there will be no change needed.<br /> <br /> We can extend the notion of domain further to establish root (parent/super) and leaf (child/sub) domain relationship, one set of root and leaf domains will define a hierarchical administrative boundary. Below diagram depicts a domain hierarchy where the root represents the cloud providers (root administrative boundary) and each levels below represents different hierarchical administrative boundaries.<br /> <br /> [[File:Hierarchical_administrative_boundary_1.jpg]]<br /> <br /> Keystone provide robust role based access control (RBAC) capabilities where roles are associated with access control rules through policy. Subjects are associated with roles scoped to a domain or projects within a domains. Keystone also support a notion of inherited roles which allows roles (mostly project) to be inherited from owning domain. We need to extend the current role inheritance model so that subject's (identity in keystone) inherited roles (project and/or domain) can be propagated down in the domain hierarchy. Diagram below shows the role inheritance propagation.<br /> <br /> Note: Role inheritance propagation will go downwards only but not upwards or side wise. <br /> <br /> [[File:Hierarchical_administrative_boundary_2.jpg]]<br /> <br /> <br /> Cloud provider (or reseller) administrator has to scope his/her security token to a particular child domain (or a project in child domain) to acquire the inherited administrative roles. Malicious and unauthorized access to external resources will be prevented by Keystone RBAC policy enforcement. Diagram below shows how cloud admin can scope token to a project in leaf domain to manage its comuting resource.<br /> <br /> <br /> [[File:Hierarchical_administrative_boundary_3.jpg]]<br /> <br /> == Impact: ==<br /> <br /> Most of the changes are scoped to Keystone only, apparently there is no change required to external (Nova, Swift, ....) services.<br /> <br /> * Domain model need to be extended to support notion of domain hierarchy.<br /> * Role subsystem will be impacted to handle hierarchical inherited roles.<br /> * Role evaluation subsystem will be impacted to handle hierarchical inherited roles.<br /> * Auth middle-ware and token validation has to be enhanced.<br /> * Domain API will be impacted to capture the domain hierarchy.</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88331 2015-08-19T13:23:12Z

<p>Atiwari: /* Next Steps */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by Marshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> # In very basic user will provide the key id using Cloud-Config Scripting. Key id will be stored in certain location of the new VM to be consumed by Marshal agent. <br /> # User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM using Customization Script Source. This KAT will be stored in VM temporary directory for consumed by Marshal. This approach need some enhancement in Barbican.<br /> # User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. Marshal will pick the key info from volume meta-data while configuration.<br /> <br /> User will use command provided by Marshal agent to configure disk encryption. User will get ability to encrypt <br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret Key Access Ticket (KAT) to access the secret.<br /> <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Create image (Linux/Win) with the Marshal agent.<br /> # Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> &lt;TDB&gt;</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88330 2015-08-19T13:22:41Z

<p>Atiwari: /* Boot VM and setup Marshal */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by Marshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> # In very basic user will provide the key id using Cloud-Config Scripting. Key id will be stored in certain location of the new VM to be consumed by Marshal agent. <br /> # User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM using Customization Script Source. This KAT will be stored in VM temporary directory for consumed by Marshal. This approach need some enhancement in Barbican.<br /> # User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. Marshal will pick the key info from volume meta-data while configuration.<br /> <br /> User will use command provided by Marshal agent to configure disk encryption. User will get ability to encrypt <br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret Key Access Ticket (KAT) to access the secret.<br /> <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Create image (Linux/Win) with the Marshal agent.<br /> # Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88328 2015-08-19T13:15:12Z

<p>Atiwari: /* Boot VM and setup Marshal */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by MArshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> # In very basic user will provide the key id using Cloud-Config Scripting, key id will be stored in certain location of the new VM. <br /> # User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM.<br /> # User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret Key Access Ticket (KAT) to access the secret.<br /> <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Create image (Linux/Win) with the Marshal agent.<br /> # Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88327 2015-08-19T13:14:37Z

<p>Atiwari: /* Agent authentication to Barbican (or any KMS). */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by MArshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> 1. In very basic user will provide the key id using Cloud-Config Scripting, key id will be stored in certain location of the new VM. <br /> 2. User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM.<br /> 3. User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret Key Access Ticket (KAT) to access the secret.<br /> <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Create image (Linux/Win) with the Marshal agent.<br /> # Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88326 2015-08-19T13:14:07Z

<p>Atiwari: /* Agent authentication to Barbican (or any KMS). */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by MArshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> 1. In very basic user will provide the key id using Cloud-Config Scripting, key id will be stored in certain location of the new VM. <br /> 2. User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM.<br /> 3. User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret Key ticket to access the secret.<br /> <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Create image (Linux/Win) with the Marshal agent.<br /> # Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88325 2015-08-19T13:12:33Z

<p>Atiwari: /* Challenges and Workarounds */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by MArshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> 1. In very basic user will provide the key id using Cloud-Config Scripting, key id will be stored in certain location of the new VM. <br /> 2. User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM.<br /> 3. User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach along with possible workaround<br /> === Agent authentication to Barbican (or any KMS). ===<br /> # Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> # Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> # Use per secret policy, which is in Barbican roadmap.<br /> # Per secret license/grant ticket to access the secret. <br /> ===Security of keys in transit===<br /> # Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> ===How to inject agent in the VM===<br /> # Create image (Linux/Win) with the Marshal agent.<br /> # Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88323 2015-08-19T12:42:46Z

<p>Atiwari: /* Challenges and Workarounds */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by MArshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> 1. In very basic user will provide the key id using Cloud-Config Scripting, key id will be stored in certain location of the new VM. <br /> 2. User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM.<br /> 3. User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> # Agent authentication to Barbican (or any KMS).<br /> * Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> * Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> * Use per secret policy, which is in Barbican roadmap.<br /> * Per secret license/grant ticket to access the secret. <br /> # Security of keys in transit<br /> Agent will be talking to Barbican on secured and trusted network over SSL.<br /> # Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> # How to inject agent in the VM<br /> # Create image (Linux/Win) with the Marshal agent.<br /> # Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88322 2015-08-19T12:41:17Z

<p>Atiwari: /* Roadmap */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by MArshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> 1. In very basic user will provide the key id using Cloud-Config Scripting, key id will be stored in certain location of the new VM. <br /> 2. User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM.<br /> 3. User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88321 2015-08-19T12:35:32Z

<p>Atiwari: /* Roadmap */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by MArshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> 1. In very basic user will provide the key id using Cloud-Config Scripting, key id will be stored in certain location of the new VM. <br /> 2. User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM.<br /> 3. User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * dm-crypt<br /> * bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88320 2015-08-19T12:35:11Z

<p>Atiwari: /* Roadmap */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by MArshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> 1. In very basic user will provide the key id using Cloud-Config Scripting, key id will be stored in certain location of the new VM. <br /> 2. User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM.<br /> 3. User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> # Entire disk encryption with one key stored in KMS.<br /> * # dm-crypt<br /> * # bitlocker<br /> # Fine grained (root disk, individual partition) encryption support with one key.<br /> # Multi key encryption support and rekey data in volume.<br /> # Ability to fetch keys from on-prem KMS.<br /> # LBaaS certificate monitoring<br /> # Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88319 2015-08-19T12:33:07Z

<p>Atiwari: /* Boot VM and setup Marshal */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent, alternatively user can use an exiting VM. User will provide key information stored in Barbican to be used by MArshal agent, there are multiple way to provide keys to the new VM as below<br /> <br /> 1. In very basic user will provide the key id using Cloud-Config Scripting, key id will be stored in certain location of the new VM. <br /> 2. User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM.<br /> 3. User will use a encrypted volume which he/she has created before and volume meta-data will provide the key id. <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> 1. Entire disk encryption with one key stored in KMS.<br /> 1. dm-crypt<br /> 2. bitlocker<br /> 2. Fine grained (root disk, individual partition) encryption support with one key.<br /> 3. Multi key encryption support and rekey data in volume.<br /> 4. Ability to fetch keys from on-prem KMS.<br /> 5. LBaaS certificate monitoring<br /> 6. Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88283 2015-08-19T02:19:23Z

<p>Atiwari: /* Boot VM and setup Marshal */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> User would choose to boot a new VM from an image which is build with Marshal agent providing key information stored in Barbican. There are multiple way to provide keys to the new VM as below<br /> <br /> 1. In very basic user will provide the key id using Cloud-Config Scripting, key id will be stored in certain location of the new VM. <br /> 2. User will generate key access ticket (KAT) for particular keys and provide the KAT to the VM. <br /> <br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> 1. Entire disk encryption with one key stored in KMS.<br /> 1. dm-crypt<br /> 2. bitlocker<br /> 2. Fine grained (root disk, individual partition) encryption support with one key.<br /> 3. Multi key encryption support and rekey data in volume.<br /> 4. Ability to fetch keys from on-prem KMS.<br /> 5. LBaaS certificate monitoring<br /> 6. Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88266 2015-08-18T22:26:46Z

<p>Atiwari: /* Volume Creation (Plain) */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Disk creation in Cinder ===<br /> Disk/Volume are managed in Cinder and user will interface with Cinder through Horizon to provision virtual disks. In the flow given below, we are creating a plain disk which means encryption key will not be created in the Barbican. I an alternative flow user will opt to create encrypted disk and key will automatically created in the Barbican. <br /> <br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> 1. Entire disk encryption with one key stored in KMS.<br /> 1. dm-crypt<br /> 2. bitlocker<br /> 2. Fine grained (root disk, individual partition) encryption support with one key.<br /> 3. Multi key encryption support and rekey data in volume.<br /> 4. Ability to fetch keys from on-prem KMS.<br /> 5. LBaaS certificate monitoring<br /> 6. Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88265 2015-08-18T22:22:08Z

<p>Atiwari: /* Key provisioning in Barbican */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> Key provisioning will be done by the user by log-in in to the Horizon dashboard and create secret. User may choose to come with his own key or he/she can order one from Barbican. Once the key is created he should define certain policy on that secret to avoid illegal access. <br /> <br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Volume Creation (Plain)===<br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> 1. Entire disk encryption with one key stored in KMS.<br /> 1. dm-crypt<br /> 2. bitlocker<br /> 2. Fine grained (root disk, individual partition) encryption support with one key.<br /> 3. Multi key encryption support and rekey data in volume.<br /> 4. Ability to fetch keys from on-prem KMS.<br /> 5. LBaaS certificate monitoring<br /> 6. Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=File:SetDiskForEncryption.jpg&diff=88262 2015-08-18T20:55:50Z

<p>Atiwari: Atiwari uploaded a new version of File:SetDiskForEncryption.jpg</p> <hr /> <div></div>

Atiwari https://wiki.openstack.org/w/index.php?title=File:SetDiskForEncryption.jpg&diff=88260 2015-08-18T20:50:01Z

<p>Atiwari: Atiwari uploaded a new version of File:SetDiskForEncryption.jpg</p> <hr /> <div></div>

Atiwari https://wiki.openstack.org/w/index.php?title=File:CreateCinderDisk.jpg&diff=88259 2015-08-18T20:49:03Z

<p>Atiwari: Atiwari uploaded a new version of File:CreateCinderDisk.jpg</p> <hr /> <div></div>

Atiwari https://wiki.openstack.org/w/index.php?title=File:KeyProvisioningInBarbican.jpg&diff=88258 2015-08-18T20:48:36Z

<p>Atiwari: Atiwari uploaded a new version of File:KeyProvisioningInBarbican.jpg</p> <hr /> <div></div>

Atiwari https://wiki.openstack.org/w/index.php?title=File:KeyProvisioningInBarbican.jpg&diff=88257 2015-08-18T20:45:58Z

<p>Atiwari: Atiwari uploaded a new version of File:KeyProvisioningInBarbican.jpg</p> <hr /> <div></div>

Atiwari https://wiki.openstack.org/w/index.php?title=File:SetDiskForEncryption.jpg&diff=88256 2015-08-18T20:41:48Z

<p>Atiwari: Atiwari uploaded a new version of File:SetDiskForEncryption.jpg</p> <hr /> <div></div>

Atiwari https://wiki.openstack.org/w/index.php?title=File:CreateCinderDisk.jpg&diff=88255 2015-08-18T20:41:25Z

<p>Atiwari: Atiwari uploaded a new version of File:CreateCinderDisk.jpg</p> <hr /> <div></div>

Atiwari https://wiki.openstack.org/w/index.php?title=File:KeyProvisioningInBarbican.jpg&diff=88254 2015-08-18T20:40:26Z

<p>Atiwari: Atiwari uploaded a new version of File:KeyProvisioningInBarbican.jpg</p> <hr /> <div></div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88253 2015-08-18T20:34:27Z

<p>Atiwari: /* Boot VM and setup Marshal */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Volume Creation (Plain)===<br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> [[File:SetDiskForEncryption.jpg]]<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> 1. Entire disk encryption with one key stored in KMS.<br /> 1. dm-crypt<br /> 2. bitlocker<br /> 2. Fine grained (root disk, individual partition) encryption support with one key.<br /> 3. Multi key encryption support and rekey data in volume.<br /> 4. Ability to fetch keys from on-prem KMS.<br /> 5. LBaaS certificate monitoring<br /> 6. Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88252 2015-08-18T20:34:10Z

<p>Atiwari: /* Volume Creation (Plain) */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Volume Creation (Plain)===<br /> [[File:CreateCinderDisk.jpg]]<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> 1. Entire disk encryption with one key stored in KMS.<br /> 1. dm-crypt<br /> 2. bitlocker<br /> 2. Fine grained (root disk, individual partition) encryption support with one key.<br /> 3. Multi key encryption support and rekey data in volume.<br /> 4. Ability to fetch keys from on-prem KMS.<br /> 5. LBaaS certificate monitoring<br /> 6. Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88251 2015-08-18T20:33:55Z

<p>Atiwari: /* Key provisioning in Barbican */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> [[File:KeyProvisioningInBarbican.jpg]]<br /> <br /> === Volume Creation (Plain)===<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> 1. Entire disk encryption with one key stored in KMS.<br /> 1. dm-crypt<br /> 2. bitlocker<br /> 2. Fine grained (root disk, individual partition) encryption support with one key.<br /> 3. Multi key encryption support and rekey data in volume.<br /> 4. Ability to fetch keys from on-prem KMS.<br /> 5. LBaaS certificate monitoring<br /> 6. Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=File:SetDiskForEncryption.jpg&diff=88250 2015-08-18T20:32:06Z

<p>Atiwari: </p> <hr /> <div></div>

Atiwari https://wiki.openstack.org/w/index.php?title=File:CreateCinderDisk.jpg&diff=88249 2015-08-18T20:31:28Z

<p>Atiwari: </p> <hr /> <div></div>

Atiwari https://wiki.openstack.org/w/index.php?title=File:KeyProvisioningInBarbican.jpg&diff=88248 2015-08-18T20:26:06Z

<p>Atiwari: </p> <hr /> <div></div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88247 2015-08-18T20:24:57Z

<p>Atiwari: /* Key provisioning in Barbican */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> === Volume Creation (Plain)===<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> 1. Entire disk encryption with one key stored in KMS.<br /> 1. dm-crypt<br /> 2. bitlocker<br /> 2. Fine grained (root disk, individual partition) encryption support with one key.<br /> 3. Multi key encryption support and rekey data in volume.<br /> 4. Ability to fetch keys from on-prem KMS.<br /> 5. LBaaS certificate monitoring<br /> 6. Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88246 2015-08-18T20:02:05Z

<p>Atiwari: </p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> &lt;gallery&gt;<br /> Example.jpg|Caption1<br /> Example.jpg|Caption2<br /> &lt;/gallery&gt;<br /> === Volume Creation (Plain)===<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> 1. Entire disk encryption with one key stored in KMS.<br /> 1. dm-crypt<br /> 2. bitlocker<br /> 2. Fine grained (root disk, individual partition) encryption support with one key.<br /> 3. Multi key encryption support and rekey data in volume.<br /> 4. Ability to fetch keys from on-prem KMS.<br /> 5. LBaaS certificate monitoring<br /> 6. Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88245 2015-08-18T19:58:44Z

<p>Atiwari: /* Activity Diagrams */</p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> === Key provisioning in Barbican ===<br /> <br /> === Volume Creation (Plain)===<br /> <br /> === Boot VM and setup Marshal===<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> 1. Entire disk encryption with one key stored in KMS.<br /> 1. dm-crypt<br /> 2. bitlocker<br /> 2. Fine grained (root disk, individual partition) encryption support with one key.<br /> 3. Multi key encryption support and rekey data in volume.<br /> 4. Ability to fetch keys from on-prem KMS.<br /> 5. LBaaS certificate monitoring<br /> 6. Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88244 2015-08-18T19:54:29Z

<p>Atiwari: </p> <hr /> <div>==Introduction==<br /> Over the period of last 5 years OpenStack community has provided support for core cloud (e. g. Nova, Glance, Cinder, Neutron, Keystone, Barbican etc …) services which are mandatory for any infrastructure cloud. It has also provided supporting services (Heat, Horizon, Ceilometer, Monasca), which helps cloud providers and consumers to interact and manage cloud resources. Volume encryption is one of the areas, which still needs more attentions, as the current model has some gaps. Project Marshal is new initiative in community, which support and streamline volume encryption use cases and provide data security a rest. <br /> <br /> ==What is Marshal? ==<br /> OpenStack Barbican has gone through long ways from key storage to key manager. We have implemented many cool features in it and some are in pipeline, but still it is not well integrated to support volume encryption use cases, which is one of the basic needs. In the current flow of volume encryption, user chooses to create an encrypted volume using cinder API, an encryption key will be created in Barbican. Key reference will be stored in Cinder with disk metadata. Key will be retrieved by Nova from the Barbican and provided to dm-crypt while attaching the disk to VM, all I/O to the disk will be encrypted.<br /> <br /> There are some gaps in the current flow, e.g. it is not very transparent to the end users, only one key can be used at a time and there is no way users can change the encryption keys. This flow is also very tightly integrated with OpenStack components but there are cases where users want to use their existing KMS. The current volume encryption implementation focus mostly on Linux based VM but not support for Windows VM. On-premise KMS integration is another use case, which is not, yet solves.<br /> <br /> Marshal is a new service (or tool) in community to fill the above-mentioned gaps; it solves the volume encryption problem with different approach. Marshal is an agent service running inside virtual machines, which will be responsible for securely fetching encryption key from KMS (Barbican). This agent will be interfacing with disk encryption subsystem of underlying operating system to encrypt/decrypt the disk I/O.<br /> <br /> In case of Linux based virtual machines this agent will be interfacing with dm-crypt and for Microsoft it will be interfacing with Bit-locker. The agent provides abstraction service and can be integrated with other encryption subsystem if needed. This agent reads key from KMS and create a temp file volatile memory, it forgets the key immediately after providing to disk encryption subsystem.<br /> <br /> ==Activity Diagrams==<br /> <br /> 1. Key provisioning in Barbican<br /> 2. Volume Creation (Plain)<br /> 3. Boot VM and setup Marshal<br /> <br /> ==Benefits==<br /> <br /> Current OpenStack volume encryption is very static in nature, only one key can be used and it won't be changed. Marshal agent can able to change the encryption keys and re-key the data. Marshal agent will provide ability to choose the keys from KMS system to be used for disk encryption. <br /> OpenStack volume encryption flow is very rigidly defined, where Cinder creates key in KMS and Nova provide that key to the VM encryption subsystem. This process is mostly abstracted from users.<br /> This agent will help older OpenStack deployment to integrate with Barbican (or any KMS) system to consume keys and use them for the disk encryption.<br /> This agent provides abstraction and can be integrate with customer's on-prem KMS systems. It can also be integrated with HSM devises directly using Castellan, KMIP or PKCS11 interface.<br /> Marshal agent will provide more flexibility to the end users by providing fine-grained control. Marshal will allow features like full disk encryption, root disk encrypt and different partitions of the disk, which is not covered in current flow.<br /> <br /> ==Roadmap==<br /> 1. Entire disk encryption with one key stored in KMS.<br /> 1. dm-crypt<br /> 2. bitlocker<br /> 2. Fine grained (root disk, individual partition) encryption support with one key.<br /> 3. Multi key encryption support and rekey data in volume.<br /> 4. Ability to fetch keys from on-prem KMS.<br /> 5. LBaaS certificate monitoring<br /> 6. Provisioning certificate from CA and installation.<br /> <br /> ==Challenges and Workarounds==<br /> Following are the challenges we are seeing in this approach <br /> 1. Agent authentication to Barbican (or any KMS).<br /> 1. Use of certificate based auth from Keystone, which is on the way for Liberty. Agent will be assigned a certificate in this mode.<br /> 2. Use of trust (Keystone concept) to authenticate, trust will be provided to agent.<br /> 3. Use per secret policy, which is in Barbican roadmap.<br /> 4. Per secret license/grant ticket to access the secret. <br /> 2. Security of keys in transit<br /> 1. Agent will be talking to Barbican on secured and trusted network over SSL.<br /> 2. Transport keys can be used to securely fetch key from KMS. Specially when collecting keys from on-prem KMS.<br /> 3. How to inject agent in the VM<br /> 1. Create image (Linux/Win) with the Marshal agent.<br /> 2. Provide deb/zip package<br /> <br /> ==Next Steps==<br /> <br /> TDB</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Marshal&diff=88243 2015-08-18T19:49:13Z

<p>Atiwari: Created page with &quot;=Marshal= Introduction == Heading text ==&quot;</p> <hr /> <div>=Marshal=<br /> <br /> <br /> Introduction<br /> == Heading text ==</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Inter_Cloud_Resource_Federation&diff=80560 2015-05-06T23:00:38Z

<p>Atiwari: /* InterCloud Resource Federation (Alliance) */</p> <hr /> <div>=InterCloud Resource Federation ('''Alliance''')=<br /> <br /> '''Arvind Tiwari'''<br /> <br /> <br /> Cloud based services are a growing business trend in the IT industry, where service providers establish cloud and offer computing resources (Infrastructure, Platform and Software) to consumers. Consumers often require computing resources across multiple regions, to address their application needs. Single cloud provider may not be able to address such requests due to lack of presence or capacity in multiple regions. This blueprint proposes a solution to address this concern by introducing a concept of InterCloud Resource Federation (a.k.a. Alliance). Using this technical approach multiple cloud entities can work in alliance to form a bigger cloud entity with massive resource capacities.<br /> <br /> = Problem =<br /> This blueprint is trying to address following problems<br /> <br /> === Multi Region Capability ===<br /> Enterprises are adopting cloud based service to address their growing computing workload and they often want resources across multiple regions. These regions sometimes span across multiple geographical boundaries. <br /> <br /> Single providers may not be able to address such requests as setting up huge cloud across multiple regions is a costly and risky investment. Providers often want to lease resources from other provider cross regions to fulfill customer computing workload.<br /> <br /> === Virtual Private Cloud or Cloud Leasing ===<br /> Vendors want to lease resources to other service providers or customers who want have VPC (Virtual Private Cloud). Unlike other service providers these vendors want to target large consumers. <br /> <br /> Cloud service providers have idle computing resources which are not making revenue. Service provider wants to lease these resources to another provider who are in need of these resources.<br /> <br /> === Cloud Bursting ===<br /> In private in-house cloud deployments, enterprises want to bursts into a public/separate cloud when the demand for computing capacity spikes.<br /> <br /> === Joint Venture Cloud ===<br /> Setting up a Cloud with massive resource capacity is time consuming and very costly venture; not everyone would like to invest in such a big venture. Business partners like to invest in joint ventures to make a bigger cloud entity with massive resource capacities.<br /> <br /> === Merger and Acquisitions ===<br /> Merger of business entities are common, integration/utilization of cloud enabled computing resources of two business enterprises are troublesome due to lack of technology. An enterprise expects seamless integration of two cloud entities after merger with anticipation of effective and prompt resource utilization.<br /> <br /> === Expensive Architecture ===<br /> Building multi region architecture to join multiple data centers using dedicated/leased lines is costly and creates tight coupling. This solution is also not a scalable solution for incremental growth.<br /> <br /> = Solution =<br /> OpenStack provides an open source cloud computing platform which is getting huge industry adoption for building up private/public/hybrid cloud. OpenStack based cloud deployments provide Infrastructure and Platform services capabilities to cloud providers. OpenStack inherited cloud entities are ideal candidates to work in alliances to form InterCloud Federation, as their API and Resource models are identical.<br /> <br /> To address these problems, a new service will be added to the OpenStack services stack which will provide “InterCloud Federation” (a.k.a. '''Alliance''') services to participating cloud entities. This service will be responsible for interconnecting multiple cloud instances and provide an abstraction layer to hide interoperability and integration complexities from the provider and end users. <br /> <br /> Cloud providers have to enable this service on their cloud deployments and configure the required artifacts for partner or host cloud entities.<br /> <br /> The picture below depicts the resource consumption flow from Host cloud to Partner cloud.<br /> <br /> [[File:ICRF01.jpg]]<br /> <br /> The picture ‘a’ (below), shows integration of multiple data centers with dedicated leased line for data replication. This architecture tightly couples the data centers and it is not cost effective. <br /> <br /> Picture ‘b’ portrays the interlinked cloudified data centers through Alliance service. In this model, data exchanges between the participating clouds will be happening on secured (PtoP VPN) Internet channel not on dedicated lines.<br /> <br /> [[File:ICRF04.jpg]]<br /> <br /> =Design Considerations=<br /> Following are the high level design consideration and functional components of InterCloud Federation Services (a.k.a. '''Alliance'''). <br /> <br /> === Host Cloud (HC) ===<br /> A cloud entity consumes resources from different cloud providers. This cloud entity plays a role of “resource consumer” in cloud federation business.<br /> <br /> === Partner Cloud (PC) ===<br /> Cloud instance(s) offer its services to a Host Cloud entity. These cloud entities plays a role of “resource provider” in cloud federation business.<br /> <br /> === Cloud Pairing ===<br /> The pairing process is the initial step taken by two cloud providers to work in alliance. This step is required to establish trust between two cloud instances and configure required artifacts. These artifacts include cryptographic keys, cloud identifiers, region/zone information, service availability and InterCloud federation service endpoints. Alliance service endpoints are required for service discovery and inter-cloud token validations.<br /> <br /> The pairing process can be done manually by two cloud providers (or it can be automated). Host cloud provider has to configure artifacts for partner cloud entities and partner has to configure artifacts for host cloud(s). Cloud pairing can be bidirectional, in this case one cloud instance will be playing as host cloud in one and partner cloud on other side.<br /> <br /> === InterCloud Ticket (ICT) ===<br /> This ticket is used for the purpose of inter-cloud communication (e.g. Resource discovery, user token validation across clouds). This is a PKI ticket and contains data structure to hold context information required for communication. This ticket will be encrypted using partner’s (destination) public key and signed by host’s (source) private key.<br /> <br /> === InterCloud Token Validation ===<br /> This service is used by two participating cloud entities to validate user token which involve signature validation and ticket decryption to get the actual X-Auth-Token. This is a part of Alliance service.<br /> <br /> === Region Discovery ===<br /> This is one of the services provided by ICFS to list available regions (or availability zone). Clients use this service to know what are the available regions or zones offered by a provider. Generally these are static information stored in ICFS system.<br /> <br /> === Service Discovery ===<br /> This service is used to discover available services in a particular region or zone offered by a partner cloud. This will provide dynamic list of available services from particular partner region.<br /> <br /> [[File:ICRF02.jpg]]<br /> <br /> === Remote Resource Provisioning ===<br /> <br /> Resource provisioning is required to allocate limited resource efficiently on the (partner cloud) resource provider cloud. Provisioning is also needed to provide sense of ownership to the end user, for cloud provider (partner/host) provisioning is required to address metering aspect.<br /> <br /> Alliance service is responsible for provisioning remote resources. The resource provisioning will be done on local Keystone project at hosting cloud. It also maintains provisioning information in local database. The provisioning info will be used for the purpose of token generation and validation.<br /> <br /> <br /> [[File:ICRF06.jpg]]<br /> <br /> <br /> In the above picture user maintains his identity at one place (host cloud) and owns resources from remote cloud(s) on a local project. This is another benefit of resource federation, where user can use a single project in host cloud to scope all the remote resources across the cloud(s).<br /> <br /> === Resource Access Across Clouds ===<br /> Resource access process start by getting an “X-Auth-Token” scoped (1) to local Keystone project of &quot;host&quot; cloud. Keystone service at HC will talk to local Alliance to get information about remote resources associated with project (2).<br /> <br /> As part of token response (3) client gets a service catalog containing endpoints to the remote (federated) resources. Client uses the remote resource endpoint to access the resource (4), it provides (X-Host-Cloud-Id) host cloud identifier in request header and the X-Auth-Token{hc} got from host cloud. <br /> <br /> Auth middle-ware protecting the resource at partner cloud intercepts the request and makes a call (5) to Keystone for token validation. Keystone delegates (6) such the token validation request to Alliance service which is not issued by it (foreign token) and have X-Host-Cloud-Id header associated.<br /> <br /> Alliance uses the cloud identifier (X-Host-Cloud-Id) from the header to lookup the paired host cloud and it's peer Alliance endpoint. Using the X-Auth-Token{hc}, it forms an InterCloud Federation Ticket and uses paired Alliance endpoint to validate user token (7). <br /> <br /> Alliance at HC will coordinate with local Keystone to validate the token (8). <br /> <br /> After successful inter-cloud token validation (9), Alliance service provide (10) the validate response to Keystone service running at PC. Keystone will caches the token in locale system and respond to middle-ware(11). Keystone will use the cached token for future token validations.<br /> <br /> [[File:ICRF03.jpg]]<br /> <br /> === X-Auth-Token processing=== <br /> <br /> Clients won't like to deal with multiple X-Auth-Tokens to access their resources across clouds (regions). <br /> <br /> Following are the options to solve this issue.<br /> <br /> ====PKI tokens====<br /> PKI tokens can be used by clients to access resources across clouds. There won't be inter-cloud token validation required to validate the PKI tokens.<br /> PKI token are proven to be heavy, Federated Token can be a better solution.<br /> <br /> ====Federated Tokens====<br /> Instead of generating new X-Auth-Token{pc} by partner cloud, partner cloud may choose to use the same X-Auth-Token{hc} issued by host cloud. <br /> <br /> After successful inter-cloud token validation (as explained above) Alliance will cache the token locally and utilize the same X-Auth-Token for future communication. This option can be set as part of cloud pairing depending of level of trust between two cloud providers. <br /> <br /> Note: Inter-cloud token validation is one time process or can be done multiple times over the period of communication by clients.<br /> <br /> ====Federated Tokens by Eager Propagation====<br /> To support federated tokens partner cloud has to do inter-cloud token validation and cache the validate token response to make the future token validation more efficient. Another approach to solve the performance of inter-cloud token validation is to propagate the tokens to partner cloud in push mode. Host cloud will propagate token to the relevant partner using notification route.<br /> <br /> === SSO Across Cloud===<br /> <br /> In this mode, clients chooses to use PC's identity (Keystone) endpoint to make auth token request. Client provide credentials, project_id and cloud_id to the PC's identity service. Keystone will coordinate with Alliance service to get the token from remote cloud.<br /> <br /> [[File:ICRF05.jpg]]<br /> <br /> === SSOut Across Cloud (or InterCloud Token Revocation) ===<br /> <br /> Token revocation in an important aspect to maintain the security and system integrity. In case of resource federation use case, tokens revocation become more important as an stale token can cause bigger harm specially to the resource provider clouds.<br /> <br /> Inter-cloud token revocation will allow token revocation across cloud, e.g. Host cloud can initiate the token revocation for a token issued by itself or partner clouds can request/initiate the token revocation of a federated token.<br /> <br /> Alliance service is will be the interface between clouds to make the token revocation happen.<br /> <br /> === Notification ===<br /> Is part of InterCloud Federation Service and used to notify certain events (e.g. Meter, resource de-provisioning etc...) to and from participating clouds.<br /> <br /> <br /> ==Issue and Concerns ==<br /> <br /> === Homogeneous vs Heterogeneous Cloud Partners===<br /> TBD<br /> <br /> =Impact=<br /> <br /> Following are the high level impact on existing OpenStack service <br /> <br /> * Keystone token management subsystem will be impacted to support endpoints for partner services. <br /> * Keystone service REST API will make an extra call to Alliance service to get list of available remote services.<br /> * Horizon will be enhanced to integrate with Inter Alliance Service.<br /> * Other OS Services - TBD<br /> <br /> =Note=<br /> * This page is in &quot;work in progress&quot; mode.<br /> * Please excuse any typographical error.</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Inter_Cloud_Resource_Federation&diff=73724 2015-02-16T17:17:28Z

<p>Atiwari: /* InterCloud Resource Federation (Alliance) */</p> <hr /> <div>=InterCloud Resource Federation ('''Alliance''')=<br /> <br /> '''Arvind Tiwari (Cisco)'''<br /> <br /> <br /> Cloud based services are a growing business trend in the IT industry, where service providers establish cloud and offer computing resources (Infrastructure, Platform and Software) to consumers. Consumers often require computing resources across multiple regions, to address their application needs. Single cloud provider may not be able to address such requests due to lack of presence or capacity in multiple regions. This blueprint proposes a solution to address this concern by introducing a concept of InterCloud Resource Federation (a.k.a. Alliance). Using this technical approach multiple cloud entities can work in alliance to form a bigger cloud entity with massive resource capacities.<br /> <br /> = Problem =<br /> This blueprint is trying to address following problems<br /> <br /> === Multi Region Capability ===<br /> Enterprises are adopting cloud based service to address their growing computing workload and they often want resources across multiple regions. These regions sometimes span across multiple geographical boundaries. <br /> <br /> Single providers may not be able to address such requests as setting up huge cloud across multiple regions is a costly and risky investment. Providers often want to lease resources from other provider cross regions to fulfill customer computing workload.<br /> <br /> === Virtual Private Cloud or Cloud Leasing ===<br /> Vendors want to lease resources to other service providers or customers who want have VPC (Virtual Private Cloud). Unlike other service providers these vendors want to target large consumers. <br /> <br /> Cloud service providers have idle computing resources which are not making revenue. Service provider wants to lease these resources to another provider who are in need of these resources.<br /> <br /> === Cloud Bursting ===<br /> In private in-house cloud deployments, enterprises want to bursts into a public/separate cloud when the demand for computing capacity spikes.<br /> <br /> === Joint Venture Cloud ===<br /> Setting up a Cloud with massive resource capacity is time consuming and very costly venture; not everyone would like to invest in such a big venture. Business partners like to invest in joint ventures to make a bigger cloud entity with massive resource capacities.<br /> <br /> === Merger and Acquisitions ===<br /> Merger of business entities are common, integration/utilization of cloud enabled computing resources of two business enterprises are troublesome due to lack of technology. An enterprise expects seamless integration of two cloud entities after merger with anticipation of effective and prompt resource utilization.<br /> <br /> === Expensive Architecture ===<br /> Building multi region architecture to join multiple data centers using dedicated/leased lines is costly and creates tight coupling. This solution is also not a scalable solution for incremental growth.<br /> <br /> = Solution =<br /> OpenStack provides an open source cloud computing platform which is getting huge industry adoption for building up private/public/hybrid cloud. OpenStack based cloud deployments provide Infrastructure and Platform services capabilities to cloud providers. OpenStack inherited cloud entities are ideal candidates to work in alliances to form InterCloud Federation, as their API and Resource models are identical.<br /> <br /> To address these problems, a new service will be added to the OpenStack services stack which will provide “InterCloud Federation” (a.k.a. '''Alliance''') services to participating cloud entities. This service will be responsible for interconnecting multiple cloud instances and provide an abstraction layer to hide interoperability and integration complexities from the provider and end users. <br /> <br /> Cloud providers have to enable this service on their cloud deployments and configure the required artifacts for partner or host cloud entities.<br /> <br /> The picture below depicts the resource consumption flow from Host cloud to Partner cloud.<br /> <br /> [[File:ICRF01.jpg]]<br /> <br /> The picture ‘a’ (below), shows integration of multiple data centers with dedicated leased line for data replication. This architecture tightly couples the data centers and it is not cost effective. <br /> <br /> Picture ‘b’ portrays the interlinked cloudified data centers through Alliance service. In this model, data exchanges between the participating clouds will be happening on secured (PtoP VPN) Internet channel not on dedicated lines.<br /> <br /> [[File:ICRF04.jpg]]<br /> <br /> =Design Considerations=<br /> Following are the high level design consideration and functional components of InterCloud Federation Services (a.k.a. '''Alliance'''). <br /> <br /> === Host Cloud (HC) ===<br /> A cloud entity consumes resources from different cloud providers. This cloud entity plays a role of “resource consumer” in cloud federation business.<br /> <br /> === Partner Cloud (PC) ===<br /> Cloud instance(s) offer its services to a Host Cloud entity. These cloud entities plays a role of “resource provider” in cloud federation business.<br /> <br /> === Cloud Pairing ===<br /> The pairing process is the initial step taken by two cloud providers to work in alliance. This step is required to establish trust between two cloud instances and configure required artifacts. These artifacts include cryptographic keys, cloud identifiers, region/zone information, service availability and InterCloud federation service endpoints. Alliance service endpoints are required for service discovery and inter-cloud token validations.<br /> <br /> The pairing process can be done manually by two cloud providers (or it can be automated). Host cloud provider has to configure artifacts for partner cloud entities and partner has to configure artifacts for host cloud(s). Cloud pairing can be bidirectional, in this case one cloud instance will be playing as host cloud in one and partner cloud on other side.<br /> <br /> === InterCloud Ticket (ICT) ===<br /> This ticket is used for the purpose of inter-cloud communication (e.g. Resource discovery, user token validation across clouds). This is a PKI ticket and contains data structure to hold context information required for communication. This ticket will be encrypted using partner’s (destination) public key and signed by host’s (source) private key.<br /> <br /> === InterCloud Token Validation ===<br /> This service is used by two participating cloud entities to validate user token which involve signature validation and ticket decryption to get the actual X-Auth-Token. This is a part of Alliance service.<br /> <br /> === Region Discovery ===<br /> This is one of the services provided by ICFS to list available regions (or availability zone). Clients use this service to know what are the available regions or zones offered by a provider. Generally these are static information stored in ICFS system.<br /> <br /> === Service Discovery ===<br /> This service is used to discover available services in a particular region or zone offered by a partner cloud. This will provide dynamic list of available services from particular partner region.<br /> <br /> [[File:ICRF02.jpg]]<br /> <br /> === Remote Resource Provisioning ===<br /> <br /> Resource provisioning is required to allocate limited resource efficiently on the (partner cloud) resource provider cloud. Provisioning is also needed to provide sense of ownership to the end user, for cloud provider (partner/host) provisioning is required to address metering aspect.<br /> <br /> Alliance service is responsible for provisioning remote resources. The resource provisioning will be done on local Keystone project at hosting cloud. It also maintains provisioning information in local database. The provisioning info will be used for the purpose of token generation and validation.<br /> <br /> <br /> [[File:ICRF06.jpg]]<br /> <br /> <br /> In the above picture user maintains his identity at one place (host cloud) and owns resources from remote cloud(s) on a local project. This is another benefit of resource federation, where user can use a single project in host cloud to scope all the remote resources across the cloud(s).<br /> <br /> === Resource Access Across Clouds ===<br /> Resource access process start by getting an “X-Auth-Token” scoped (1) to local Keystone project of &quot;host&quot; cloud. Keystone service at HC will talk to local Alliance to get information about remote resources associated with project (2).<br /> <br /> As part of token response (3) client gets a service catalog containing endpoints to the remote (federated) resources. Client uses the remote resource endpoint to access the resource (4), it provides (X-Host-Cloud-Id) host cloud identifier in request header and the X-Auth-Token{hc} got from host cloud. <br /> <br /> Auth middle-ware protecting the resource at partner cloud intercepts the request and makes a call (5) to Keystone for token validation. Keystone delegates (6) such the token validation request to Alliance service which is not issued by it (foreign token) and have X-Host-Cloud-Id header associated.<br /> <br /> Alliance uses the cloud identifier (X-Host-Cloud-Id) from the header to lookup the paired host cloud and it's peer Alliance endpoint. Using the X-Auth-Token{hc}, it forms an InterCloud Federation Ticket and uses paired Alliance endpoint to validate user token (7). <br /> <br /> Alliance at HC will coordinate with local Keystone to validate the token (8). <br /> <br /> After successful inter-cloud token validation (9), Alliance service provide (10) the validate response to Keystone service running at PC. Keystone will caches the token in locale system and respond to middle-ware(11). Keystone will use the cached token for future token validations.<br /> <br /> [[File:ICRF03.jpg]]<br /> <br /> === X-Auth-Token processing=== <br /> <br /> Clients won't like to deal with multiple X-Auth-Tokens to access their resources across clouds (regions). <br /> <br /> Following are the options to solve this issue.<br /> <br /> ====PKI tokens====<br /> PKI tokens can be used by clients to access resources across clouds. There won't be inter-cloud token validation required to validate the PKI tokens.<br /> PKI token are proven to be heavy, Federated Token can be a better solution.<br /> <br /> ====Federated Tokens====<br /> Instead of generating new X-Auth-Token{pc} by partner cloud, partner cloud may choose to use the same X-Auth-Token{hc} issued by host cloud. <br /> <br /> After successful inter-cloud token validation (as explained above) Alliance will cache the token locally and utilize the same X-Auth-Token for future communication. This option can be set as part of cloud pairing depending of level of trust between two cloud providers. <br /> <br /> Note: Inter-cloud token validation is one time process or can be done multiple times over the period of communication by clients.<br /> <br /> ====Federated Tokens by Eager Propagation====<br /> To support federated tokens partner cloud has to do inter-cloud token validation and cache the validate token response to make the future token validation more efficient. Another approach to solve the performance of inter-cloud token validation is to propagate the tokens to partner cloud in push mode. Host cloud will propagate token to the relevant partner using notification route.<br /> <br /> === SSO Across Cloud===<br /> <br /> In this mode, clients chooses to use PC's identity (Keystone) endpoint to make auth token request. Client provide credentials, project_id and cloud_id to the PC's identity service. Keystone will coordinate with Alliance service to get the token from remote cloud.<br /> <br /> [[File:ICRF05.jpg]]<br /> <br /> === SSOut Across Cloud (or InterCloud Token Revocation) ===<br /> <br /> Token revocation in an important aspect to maintain the security and system integrity. In case of resource federation use case, tokens revocation become more important as an stale token can cause bigger harm specially to the resource provider clouds.<br /> <br /> Inter-cloud token revocation will allow token revocation across cloud, e.g. Host cloud can initiate the token revocation for a token issued by itself or partner clouds can request/initiate the token revocation of a federated token.<br /> <br /> Alliance service is will be the interface between clouds to make the token revocation happen.<br /> <br /> === Notification ===<br /> Is part of InterCloud Federation Service and used to notify certain events (e.g. Meter, resource de-provisioning etc...) to and from participating clouds.<br /> <br /> <br /> ==Issue and Concerns ==<br /> <br /> === Homogeneous vs Heterogeneous Cloud Partners===<br /> TBD<br /> <br /> =Impact=<br /> <br /> Following are the high level impact on existing OpenStack service <br /> <br /> * Keystone token management subsystem will be impacted to support endpoints for partner services. <br /> * Keystone service REST API will make an extra call to Alliance service to get list of available remote services.<br /> * Horizon will be enhanced to integrate with Inter Alliance Service.<br /> * Other OS Services - TBD<br /> <br /> =Note=<br /> * This page is in &quot;work in progress&quot; mode.<br /> * Please excuse any typographical error.</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Meetings/Barbican&diff=62617 2014-09-15T19:36:37Z

<p>Atiwari: /* Agenda */</p> <hr /> <div><br /> = Weekly Barbican Meeting =<br /> <br /> The [https://github.com/cloudkeep/barbican Barbican] project team holds a weekly team meeting in &lt;code&gt;&lt;nowiki&gt;#openstack-meeting-alt&lt;/nowiki&gt;&lt;/code&gt;:<br /> <br /> * Weekly on Mondays at [http://www.timeanddate.com/worldclock/fixedtime.html?iso=20130502T2000 2000 UTC]<br /> * The blueprints that are used as a basis for the [https://launchpad.net/barbican Barbican project] can be found at https://blueprints.launchpad.net/barbican<br /> * Notes for previous meetings can be found [http://eavesdrop.openstack.org/meetings/barbican here].<br /> * Chair (to contact for more information): jraim (#openstack-barbican @ Freenode)<br /> <br /> == Agenda ==<br /> * Sep 15, 2014<br /> ** jenkins.cloudkeep.io<br /> ** Metadata Storage: https://etherpad.openstack.org/p/barbican_metadata<br /> ** API Stability <br /> ** Documentation sync up with new API<br /> ** [https://etherpad.openstack.org/p/barbican-juno-final-roadmap Juno roadmap addition]: Refactor Barbican python client to use new Keystone auth components<br /> ** Various additions made to the [https://etherpad.openstack.org/p/barbican-kilo-design-sessions Kilo design etherpad].<br /> ** Need someone to look at https://review.openstack.org/#/c/118697/ comment from Nathan. Based on vote I made this change.<br /> <br /> * Sep 8, 2014<br /> ** Kilo Design Sessions<br /> ** Juno roadmap discussions: https://etherpad.openstack.org/p/barbican-juno-final-roadmap<br /> * Sep 1, 2014<br /> ** Kilo Design Sessions<br /> * Aug 25, 2014<br /> ** CR Sizes (jvrbanac)<br /> ** String interpolation in debug logging (redrobot, rellerreller)<br /> ** Python 3 support (rellerreller)<br /> * Aug 18, 2014<br /> ** Juno Home Stretch (woodster)<br /> * Aug 11, 2014<br /> ** [[Barbican/Integration|Barbican Integration]] <br /> ** Barbican as a Keystone service<br /> * Aug 4, 2014<br /> ** #openstack-barbican on eavesdrop<br /> * July 28, 2014<br /> ** Add more type in order post (https://review.openstack.org/#/c/87405/) waiting for review.<br /> ** Kilo conference presentation submissions<br /> <br /> * July 21, 2014<br /> ** (redrobot) Expiring Launchpad BPs after 5 days<br /> <br /> * July 14, 2014<br /> ** barbican-core nominations vote count<br /> ** can we plan better to make such change https://review.openstack.org/#/c/103431?<br /> ** python-barbicanclient release schedule<br /> <br /> * June 30, 2014<br /> ** Mid-cycle meetup next week<br /> ** Keystone events blueprint<br /> ** (atiwari) https://review.openstack.org/#/c/98174/ (Do we need to worry about backward compatibility?)<br /> <br /> * June 23, 2014<br /> ** Mid-cycle meetup in two weeks.<br /> <br /> * June 16, 2014<br /> ** Mid-cycle meetup<br /> <br /> * June 9, 2014<br /> ** barbican-specs repo <br /> ** juno-1 release coming up<br /> ** mid-cycle meetup<br /> ** https://review.openstack.org/#/c/98473 (is it merge ready? kind of blocker)<br /> ** https://review.openstack.org/#/c/97844/ (is it merge ready?)<br /> ** https://review.openstack.org/#/c/98174 (is it merge ready?)<br /> ** Testing code pattern<br /> ** Any progress on eventing system (atiwari).<br /> ** Can tenant_id removal from uri deserve v2 api version(atiwari).<br /> <br /> * June 2, 2014<br /> ** Hacking enabled on pep8 gate<br /> ** New barbican-specs repository<br /> ** Order model changes for https://blueprints.launchpad.net/barbican/+spec/api-orders-add-more-types<br /> *** New field &quot;meta of type JsonBlob&quot; and &quot;container_id of type String&quot;<br /> ** Meera is adding Barbican tempest tests https://blueprints.launchpad.net/tempest/+spec/add-basic-tests-for-barbican<br /> *** Comments, Suggestions or disagreement?<br /> <br /> * May 5, 2014<br /> ** https://review.openstack.org/#/c/82189/ (is it merge ready?)<br /> ** https://review.openstack.org/#/c/88463/ (review required- API change proposal Key generation)<br /> ** https://review.openstack.org/#/c/81310/ (review required- Adding target support for policy enforcement.)<br /> ** https://review.openstack.org/#/c/90613/ (review required- API change proposal certificate generation order)<br /> **The Advanced Services Common Requirements team wanted to discuss the current status of the Barbican and its readiness to utilize the Barbican for Certificate/key generation and for storing the tenant's certificates sercurely in the system. <br /> **Can we get an update or talk about this in today's meeting.<br /> *** https://blueprints.launchpad.net/barbican/+spec/add-wrapping-key-to-barbican-server<br /> <br /> * April 28, 2014<br /> ** Action items:<br /> *** all: Review/contribute to Malini's etherpad: https://etherpad.openstack.org/p/juno-key-manager-chapter<br /> *** all: Review atiwari's CR to modify the crypto plugin interface: https://review.openstack.org/#/c/82189/<br /> *** core: Consider beefing up barbican docs related to current crypto plugin operation, as prep for the OpenStack summit<br /> <br /> * April 7, 2014<br /> ** malini - update on Secuirty Guide documentation<br /> ** alee_/atiwari - Crypto plugin changes<br /> ** arunkant - Target support in barbican policy enforcement [https://blueprints.launchpad.net/barbican/+spec/policy-target-support]<br /> ** jraim - Support for debug mode start in barbican [https://review.openstack.org/#/c/82265/], can be merged?<br /> <br /> == Meeting organizers ==<br /> <br /> * Publish the agenda 24h in advance<br /> * Mail the agenda to the list and invite participants<br /> * Ask each person responsible for an action from the previous meeting to prepare a line of the form, for each action item: . #info nickname description of the action link to the diff / mailing list thread etc. describing the implementation of the action<br /> * Use http://meetbot.debian.net/Manual.html to get an automatic summary<br /> * Prepare an outline for the meeting to speed things up (see http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-10-16.00.log.html for an actual example)<br /> * Record decisions and commitments; review in the next meeting</div>

Atiwari https://wiki.openstack.org/w/index.php?title=Meetings/Barbican&diff=62454 2014-09-11T15:15:30Z

<p>Atiwari: /* Agenda */</p> <hr /> <div><br /> = Weekly Barbican Meeting =<br /> <br /> The [https://github.com/cloudkeep/barbican Barbican] project team holds a weekly team meeting in &lt;code&gt;&lt;nowiki&gt;#openstack-meeting-alt&lt;/nowiki&gt;&lt;/code&gt;:<br /> <br /> * Weekly on Mondays at [http://www.timeanddate.com/worldclock/fixedtime.html?iso=20130502T2000 2000 UTC]<br /> * The blueprints that are used as a basis for the [https://launchpad.net/barbican Barbican project] can be found at https://blueprints.launchpad.net/barbican<br /> * Notes for previous meetings can be found [http://eavesdrop.openstack.org/meetings/barbican here].<br /> * Chair (to contact for more information): jraim (#openstack-barbican @ Freenode)<br /> <br /> == Agenda ==<br /> * Sep 15, 2014<br /> ** jenkins.cloudkeep.io<br /> ** Metadata Storage: https://etherpad.openstack.org/p/barbican_metadata<br /> ** API Stability <br /> ** Documentation sync up with new API<br /> <br /> * Sep 8, 2014<br /> ** Kilo Design Sessions<br /> ** Juno roadmap discussions: https://etherpad.openstack.org/p/barbican-juno-final-roadmap<br /> * Sep 1, 2014<br /> ** Kilo Design Sessions<br /> * Aug 25, 2014<br /> ** CR Sizes (jvrbanac)<br /> ** String interpolation in debug logging (redrobot, rellerreller)<br /> ** Python 3 support (rellerreller)<br /> * Aug 18, 2014<br /> ** Juno Home Stretch (woodster)<br /> * Aug 11, 2014<br /> ** [[Barbican/Integration|Barbican Integration]] <br /> ** Barbican as a Keystone service<br /> * Aug 4, 2014<br /> ** #openstack-barbican on eavesdrop<br /> * July 28, 2014<br /> ** Add more type in order post (https://review.openstack.org/#/c/87405/) waiting for review.<br /> ** Kilo conference presentation submissions<br /> <br /> * July 21, 2014<br /> ** (redrobot) Expiring Launchpad BPs after 5 days<br /> <br /> * July 14, 2014<br /> ** barbican-core nominations vote count<br /> ** can we plan better to make such change https://review.openstack.org/#/c/103431?<br /> ** python-barbicanclient release schedule<br /> <br /> * June 30, 2014<br /> ** Mid-cycle meetup next week<br /> ** Keystone events blueprint<br /> ** (atiwari) https://review.openstack.org/#/c/98174/ (Do we need to worry about backward compatibility?)<br /> <br /> * June 23, 2014<br /> ** Mid-cycle meetup in two weeks.<br /> <br /> * June 16, 2014<br /> ** Mid-cycle meetup<br /> <br /> * June 9, 2014<br /> ** barbican-specs repo <br /> ** juno-1 release coming up<br /> ** mid-cycle meetup<br /> ** https://review.openstack.org/#/c/98473 (is it merge ready? kind of blocker)<br /> ** https://review.openstack.org/#/c/97844/ (is it merge ready?)<br /> ** https://review.openstack.org/#/c/98174 (is it merge ready?)<br /> ** Testing code pattern<br /> ** Any progress on eventing system (atiwari).<br /> ** Can tenant_id removal from uri deserve v2 api version(atiwari).<br /> <br /> * June 2, 2014<br /> ** Hacking enabled on pep8 gate<br /> ** New barbican-specs repository<br /> ** Order model changes for https://blueprints.launchpad.net/barbican/+spec/api-orders-add-more-types<br /> *** New field &quot;meta of type JsonBlob&quot; and &quot;container_id of type String&quot;<br /> ** Meera is adding Barbican tempest tests https://blueprints.launchpad.net/tempest/+spec/add-basic-tests-for-barbican<br /> *** Comments, Suggestions or disagreement?<br /> <br /> * May 5, 2014<br /> ** https://review.openstack.org/#/c/82189/ (is it merge ready?)<br /> ** https://review.openstack.org/#/c/88463/ (review required- API change proposal Key generation)<br /> ** https://review.openstack.org/#/c/81310/ (review required- Adding target support for policy enforcement.)<br /> ** https://review.openstack.org/#/c/90613/ (review required- API change proposal certificate generation order)<br /> **The Advanced Services Common Requirements team wanted to discuss the current status of the Barbican and its readiness to utilize the Barbican for Certificate/key generation and for storing the tenant's certificates sercurely in the system. <br /> **Can we get an update or talk about this in today's meeting.<br /> *** https://blueprints.launchpad.net/barbican/+spec/add-wrapping-key-to-barbican-server<br /> <br /> * April 28, 2014<br /> ** Action items:<br /> *** all: Review/contribute to Malini's etherpad: https://etherpad.openstack.org/p/juno-key-manager-chapter<br /> *** all: Review atiwari's CR to modify the crypto plugin interface: https://review.openstack.org/#/c/82189/<br /> *** core: Consider beefing up barbican docs related to current crypto plugin operation, as prep for the OpenStack summit<br /> <br /> * April 7, 2014<br /> ** malini - update on Secuirty Guide documentation<br /> ** alee_/atiwari - Crypto plugin changes<br /> ** arunkant - Target support in barbican policy enforcement [https://blueprints.launchpad.net/barbican/+spec/policy-target-support]<br /> ** jraim - Support for debug mode start in barbican [https://review.openstack.org/#/c/82265/], can be merged?<br /> <br /> == Meeting organizers ==<br /> <br /> * Publish the agenda 24h in advance<br /> * Mail the agenda to the list and invite participants<br /> * Ask each person responsible for an action from the previous meeting to prepare a line of the form, for each action item: . #info nickname description of the action link to the diff / mailing list thread etc. describing the implementation of the action<br /> * Use http://meetbot.debian.net/Manual.html to get an automatic summary<br /> * Prepare an outline for the meeting to speed things up (see http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-10-16.00.log.html for an actual example)<br /> * Record decisions and commitments; review in the next meeting</div>

Atiwari