Jump to: navigation, search

Difference between revisions of "Ironic/Drivers/iLODrivers/Kilo"

(Boot modes)
(Known Issues)
 
(31 intermediate revisions by 2 users not shown)
Line 3: Line 3:
 
=== Overview ===  
 
=== Overview ===  
  
iLO drivers enable to take advantage of features of iLO management engine in HP Proliant servers.  iLO drivers are targetted for HP Proliant Gen 8 systems and above which have iLO 4 management engine. [1]
+
iLO drivers enable to take advantage of features of iLO management engine in HP Proliant servers.  iLO drivers are targetted for HP Proliant Gen 8 systems and above which have [http://www8.hp.com/us/en/products/servers/ilo/ iLO 4 management engine].
  
 
Currently there are 3 iLO drivers:
 
Currently there are 3 iLO drivers:
  
* '''iscsi_ilo'''
+
* [[Ironic/Drivers/iLODrivers/Kilo#iscsi_ilo_driver|iscsi_ilo]]
* '''agent_ilo'''
+
* [[Ironic/Drivers/iLODrivers/Kilo#agent_ilo_driver|agent_ilo]]
* '''pxe_ilo'''.
+
* [[Ironic/Drivers/iLODrivers/Kilo#pxe_ilo_driver|pxe_ilo]]
  
The '''iscsi_ilo''' and '''agent_ilo''' drivers provide security enhanced PXE-less deployment by using iLO virtual media to boot up the baremetal node. These drivers send management info through management channel and separates it from data channel which is used for deployment. '''iscsi_ilo''' driver uses deployment ramdisk built from '''diskimage-builder''', deploys from Ironic conductor node and can do both net-boot and localboot.  By default, '''iscsi_ilo''' driver will do net-boot. '''agent_ilo''' driver uses deployment ramdisk built from IPA, deploys from baremetal node and always does local boot.
+
The '''iscsi_ilo''' and '''agent_ilo''' drivers provide security enhanced PXE-less deployment by using iLO virtual media to boot up the bare metal node. These drivers send management info through management channel and separates it from data channel which is used for deployment. '''iscsi_ilo''' driver deploys from Ironic conductor node and can do both net-boot and lcaol-boot. '''agent_ilo''' driver deploys from bare metal node and always does local-boot.
  
 
'''pxe_ilo''' driver uses PXE/iSCSI for deployment (just like normal PXE driver), but support automatic setting of requested boot mode from nova. This driver doesn't require iLO Advanced license.
 
'''pxe_ilo''' driver uses PXE/iSCSI for deployment (just like normal PXE driver), but support automatic setting of requested boot mode from nova. This driver doesn't require iLO Advanced license.
Line 17: Line 17:
 
=== Prerequisites ===  
 
=== Prerequisites ===  
  
* '''proliantutils''' is a python package which contains a set of modules for managing HP Proliant hardware.  Install '''proliantutils''' [2] module on the Ironic conductor node. Minimum version required is 2.1.3.
+
* '''proliantutils''' is a python package which contains a set of modules for managing HP Proliant hardware.  Install [https://pypi.python.org/pypi/proliantutils proliantutils] module on the Ironic conductor node. Minimum version required is 2.1.3. Recommended version is 2.1.5
 +
 
 +
 
 +
  $ pip install "proliantutils>=2.1.5"
  
  $ pip install "proliantutils>=2.1.3"
 
  
 
* '''ipmitool''' command must be present on the service node(s) where  '''ironic-conductor''' is running. On most distros, this is provided as part of the '''ipmitool''' package. Source code is available at http://ipmitool.sourceforge.net/.
 
* '''ipmitool''' command must be present on the service node(s) where  '''ironic-conductor''' is running. On most distros, this is provided as part of the '''ipmitool''' package. Source code is available at http://ipmitool.sourceforge.net/.
Line 29: Line 31:
  
 
===== Overview =====
 
===== Overview =====
'''iscsi_ilo''' driver was introduced as an alternative to '''pxe_ipmitool''' and '''pxe_ipminative''' drivers for HP Proliant servers. '''iscsi_ilo''' uses virtual media feature in iLO to boot up the baremetal node instead of using PXE or iPXE.
+
'''iscsi_ilo''' driver was introduced as an alternative to '''pxe_ipmitool''' and '''pxe_ipminative''' drivers for HP Proliant servers. '''iscsi_ilo''' uses virtual media feature in iLO to boot up the bare metal node instead of using PXE or iPXE.
  
 
===== Target Users =====
 
===== Target Users =====
Line 47: Line 49:
  
 
===== Features =====
 
===== Features =====
* PXE-less deploy with Virtual Media.
+
* PXE-less deployment with virtual media.
 
* Automatic detection of current boot mode.
 
* Automatic detection of current boot mode.
 
* Automatic setting of the required boot mode if UEFI boot mode is requested by the nova flavor's extra spec.
 
* Automatic setting of the required boot mode if UEFI boot mode is requested by the nova flavor's extra spec.
* Supports booting the instance from Virtual Media as well as booting locally from disk.  Default is booting from Virtual Media.
+
* Supports booting the instance from virtual media as well as booting locally from disk.  Default is booting from virtual media.
 
* UEFI Boot
 
* UEFI Boot
 
* UEFI Secure Boot
 
* UEFI Secure Boot
* Passing management information via secure, encrypted management network (Virtual Media) if Swift proxy server has an HTTPS endpoint. See [[Ironic/Drivers/iLODrivers/Kilo#Enabling HTTPS in Swift|Enabling HTTPS in Swift]] for more info.  Provisioning is done using iSCSI over data network, so this driver has the  benefit of security enhancement with the same performance. Hence it segregates management info from data channel.
+
* Passing management information via secure, encrypted management network (virtual media) if Swift proxy server has an HTTPS endpoint. See [[Ironic/Drivers/iLODrivers/Kilo#Enabling HTTPS in Swift|Enabling HTTPS in Swift]] for more info.  Provisioning is done using iSCSI over data network, so this driver has the  benefit of security enhancement with the same performance. It segregates management info from data channel.
* Remote Console
+
* Remote Console (based on IPMI)
 
* HW Sensors
 
* HW Sensors
 
* Works well for machines with resource constraints (lesser amount of memory).
 
* Works well for machines with resource constraints (lesser amount of memory).
 +
* Local boot (both BIOS and UEFI)
 +
* Supports deployment of whole disk image.
 
* Support for out-of-band hardware inspection.
 
* Support for out-of-band hardware inspection.
 
* Node cleaning.
 
* Node cleaning.
Line 63: Line 67:
  
 
* '''iLO 4 Advanced License''' needs to be installed on iLO to enable Virtual Media feature.
 
* '''iLO 4 Advanced License''' needs to be installed on iLO to enable Virtual Media feature.
* **Swift Object Storage Service** - iLO driver uses Swift to store temporary FAT images as well as boot ISO images.
+
* '''Swift Object Storage Service''' - iLO driver uses Swift to store temporary FAT images as well as boot ISO images.
* **Glance Image Service with Swift configured as its backend** - When using '''iscsi_ilo''' driver, the image containing the deploy ramdisk is retrieved from Swift directly by the iLO.
+
* '''Glance Image Service with Swift configured as its backend''' - When using '''iscsi_ilo''' driver, the image containing the deploy ramdisk is retrieved from Swift directly by the iLO.
  
 
===== Deploy Process =====
 
===== Deploy Process =====
Line 77: Line 81:
 
* The deploy kernel/ramdisk is booted on the baremetal node.  The ramdisk exposes the local disk over iSCSI and requests Ironic conductor to complete the deployment.
 
* The deploy kernel/ramdisk is booted on the baremetal node.  The ramdisk exposes the local disk over iSCSI and requests Ironic conductor to complete the deployment.
 
* The driver on the Ironic conductor writes the glance image to the baremetal node's disk.
 
* The driver on the Ironic conductor writes the glance image to the baremetal node's disk.
* The driver bundles the boot kernel/ramdisk for the Glance deploy image into an ISO and then uploads it to Swift. This ISO image will be used for booting the deployed instance.
+
* If local-boot is requested, Ironic conductor asks the deployment ramdisk to install the boot loader.
 +
* If it's a netboot (default), the driver bundles the boot kernel/ramdisk for the Glance deploy image into an ISO and then uploads it to Swift. This ISO image will be used for booting the deployed instance.
 
* The driver reboots the node.
 
* The driver reboots the node.
* On the first and subsequent reboots '''iscsi_ilo''' driver attaches this boot ISO image in Swift as Virtual Media CDROM and then sets iLO to boot from it.  If '''boot_option''' was set to local, then the instance is booted from disk.
+
* For netboot, on the first and subsequent reboots '''iscsi_ilo''' driver attaches this boot ISO image in Swift as Virtual Media CDROM and then sets iLO to boot from it.  If '''boot_option''' was set to local, then the instance is booted from disk.
  
 
===== Configuring and Enabling the driver =====
 
===== Configuring and Enabling the driver =====
Line 85: Line 90:
 
'''deploy-ramdisk.initramfs''' and '''deploy-ramdisk.iso''' in the current working directory
 
'''deploy-ramdisk.initramfs''' and '''deploy-ramdisk.iso''' in the current working directory
  
     cd <path-to-diskimage-builder>
+
     pip install "diskimage-builder"
     ./bin/ramdisk-image-create -o deploy-ramdisk ubuntu deploy-ironic iso
+
     ramdisk-image-create -o deploy-ramdisk ubuntu deploy-ironic iso
  
 
2. Upload this image to Glance.::
 
2. Upload this image to Glance.::
Line 92: Line 97:
 
     glance image-create --name deploy-ramdisk.iso --disk-format iso --container-format bare < deploy-ramdisk.iso
 
     glance image-create --name deploy-ramdisk.iso --disk-format iso --container-format bare < deploy-ramdisk.iso
  
3. Configure Glance image service with its storage backend as Swift. See [4] for configuration instructions.
+
3. Configure Glance image service with its storage backend as Swift. [http://docs.openstack.org/developer/glance/configuring.html#configuring-the-swift-storage-backend See here] for configuration instructions.
  
 
4. Set a temp-url key for Glance user in Swift. For example, if you have configured Glance with user '''glance-swift''' and tenant as '''service''',  then run the below command::
 
4. Set a temp-url key for Glance user in Swift. For example, if you have configured Glance with user '''glance-swift''' and tenant as '''service''',  then run the below command::
Line 120: Line 125:
 
     admin_user = ironic
 
     admin_user = ironic
 
     admin_tenant_name = service
 
     admin_tenant_name = service
 
+
    auth_version = 2
Ensure '''auth_version''' in '''keystone_authtoken''' to 2.
 
  
 
Then, the below command should work.::
 
Then, the below command should work.::
Line 162: Line 166:
 
'''iscsi_ilo''' driver supports automatic detection of boot mode (Legacy BIOS or UEFI) and setting of boot mode from BIOS to UEFI.  Please see Note below for details.
 
'''iscsi_ilo''' driver supports automatic detection of boot mode (Legacy BIOS or UEFI) and setting of boot mode from BIOS to UEFI.  Please see Note below for details.
  
* When no boot mode setting is provided, '''iscsi_ilo''' driver preserves the current boot mode on the deployed instance.
+
* When no boot mode setting is provided, '''iscsi_ilo''' driver preserves the current boot mode of the bare metal on the deployed instance.
* A requirement of a specific boot mode may be provided by adding '''boot_mode:bios''' or '''boot_mode:uefi''' to '''capabilities''' property within the '''properties''' field of an Ironic node. Then '''iscsi_ilo''' driver will deploy and configure the instance in the appropriate boot mode.
+
* A requirement of a specific boot mode may be provided by adding '''boot_mode:bios''' or '''boot_mode:uefi''' to '''capabilities''' property within the '''properties''' field of an Ironic node. iscsi_ilo''' driver will then deploy and configure the instance in the specified boot mode.
  
For example, to make a Proliant baremetal node boot in UEFI mode, run the following command::
+
For example, to make a Proliant baremetal node boot always in UEFI mode, run the following command::
  
 
   ironic node-update <node-id> add properties/capabilities='boot_mode:uefi'
 
   ironic node-update <node-id> add properties/capabilities='boot_mode:uefi'
Line 172: Line 176:
  
 
* We recommend setting the '''boot_mode''' property on systems that support both UEFI and legacy modes if user wants facility in Nova to choose a baremetal node with appropriate boot mode. This is for Gen8 (ProLiant DL580 only) and Gen9 systems.
 
* We recommend setting the '''boot_mode''' property on systems that support both UEFI and legacy modes if user wants facility in Nova to choose a baremetal node with appropriate boot mode. This is for Gen8 (ProLiant DL580 only) and Gen9 systems.
* '''iscsi_ilo''' driver automatically set boot mode from BIOS to UEFI, if the requested boot mode in nova boot is UEFI. However, users will need to pre-configure boot mode to Legacy on Gen8 (ProLiant DL580 only) and Gen9 servers if they want to deploy the node in legacy mode.
+
* '''iscsi_ilo''' driver automatically sets boot mode from BIOS to UEFI, if the requested boot mode in nova boot is UEFI. However, users will need to pre-configure boot mode to Legacy on Gen8 (ProLiant DL580 only) and Gen9 servers if they want to deploy the node in legacy mode.
 
* The automatic boot ISO creation for UEFI boot mode has been enabled in Kilo. The manual creation of boot ISO for UEFI boot mode is also supported. For the latter, the boot ISO for the deploy image needs to be built separately and the deploy image's '''boot_iso''' property in Glance should contain the Glance UUID of the boot ISO.  For building boot ISO, add the '''iso''' element after adding the baremetal element while building disk images with diskimage-builder
 
* The automatic boot ISO creation for UEFI boot mode has been enabled in Kilo. The manual creation of boot ISO for UEFI boot mode is also supported. For the latter, the boot ISO for the deploy image needs to be built separately and the deploy image's '''boot_iso''' property in Glance should contain the Glance UUID of the boot ISO.  For building boot ISO, add the '''iso''' element after adding the baremetal element while building disk images with diskimage-builder
  
Line 201: Line 205:
  
 
===== Features =====
 
===== Features =====
* PXE-less deploy with Virtual Media using Ironic Python Agent.
+
* PXE-less deploy with virtual media using Ironic Python Agent.
 
* Remote Console
 
* Remote Console
 
* HW Sensors
 
* HW Sensors
Line 208: Line 212:
 
* UEFI Boot
 
* UEFI Boot
 
* UEFI Secure Boot
 
* UEFI Secure Boot
* IPA runs on the baremetal node and pulls the image directly from Swift.
+
* IPA runs on the bare metal node and pulls the image directly from Swift.
* IPA deployed instances always boots from local disk.
+
* IPA deployed instances always boot from local disk.
 +
* Supports deployment of whole disk image.
 
* Segregates management info from data channel.
 
* Segregates management info from data channel.
 
* Support for out-of-band hardware inspection.
 
* Support for out-of-band hardware inspection.
Line 215: Line 220:
  
 
===== Requirements =====
 
===== Requirements =====
* **iLO 4 Advanced License** needs to be installed on iLO to enable Virtual Media feature.
+
* '''iLO 4 Advanced License''' needs to be installed on iLO to enable virtual media feature.
* **Swift Object Storage Service** - iLO driver uses Swift to store temporary FAT images as well as boot ISO images.
+
* '''Swift Object Storage Service''' - iLO driver uses Swift to store temporary FAT images as well as boot ISO images.
* **Glance Image Service with Swift configured as its backend** - When using '''agent_ilo''' driver, the image containing the agent is retrieved from Swift directly by the iLO.
+
* '''Glance Image Service with Swift configured as its backend''' - When using '''agent_ilo''' driver, the image containing the agent is retrieved from Swift directly by the iLO.
  
 
===== Deploy Process =====
 
===== Deploy Process =====
Line 227: Line 232:
 
* Driver sets the node to boot one-time from CDROM.
 
* Driver sets the node to boot one-time from CDROM.
 
* Driver powers on the baremetal node.
 
* Driver powers on the baremetal node.
* The deploy kernel/ramdisk containing the agent is booted on the baremetal node.  The agent ramdisk talks to the Ironic conductor, downloads the image directly from Swift and writes the node's disk.
+
* The deploy kernel/ramdisk containing the agent is booted on the baremetal node.  The agent ramdisk talks to the Ironic conductor, downloads the image directly from Swift and writes the image to chosen disk on the node.
 
* Driver sets the node to permanently boot from disk and then reboots the node.
 
* Driver sets the node to permanently boot from disk and then reboots the node.
  
Line 233: Line 238:
 
1. Prepare an ISO deploy Ironic Python Agent image containing the agent [5]_.  This can be done by using the iso-image-create script found within the agent. The below set of commands will create a file '''ipa-ramdisk.iso''' in the below directory '''UPLOAD'''::
 
1. Prepare an ISO deploy Ironic Python Agent image containing the agent [5]_.  This can be done by using the iso-image-create script found within the agent. The below set of commands will create a file '''ipa-ramdisk.iso''' in the below directory '''UPLOAD'''::
  
     $ cd <directory-containing-ironic-python-agent>
+
     $ pip install "diskimage-builder"
     $ cd ./imagebuild/coreos
+
     $ disk-image-create -o ipa-ramdisk fedora ironic-agent iso
    $ make iso
 
    $ cd UPLOAD
 
    $ ls
 
    $ coreos_production_pxe_image-oem.cpio.gz  coreos_production_pxe.vmlinuz  ipa-coreos.iso
 
 
 
  
 
2. Upload the IPA ramdisk image to Glance.::
 
2. Upload the IPA ramdisk image to Glance.::
  
     glance image-create --name ipa-ramdisk.iso --disk-format iso --container-format bare < ipa-coreos.iso
+
     glance image-create --name ipa-ramdisk.iso --disk-format iso --container-format bare < ipa-ramdisk.iso
  
 
3. Configure Glance image service with its storage backend as Swift. See [4]_ for configuration instructions.
 
3. Configure Glance image service with its storage backend as Swift. See [4]_ for configuration instructions.
Line 271: Line 271:
 
     admin_user = ironic
 
     admin_user = ironic
 
     admin_tenant_name = service
 
     admin_tenant_name = service
 
+
    auth_version = 2
Ensure '''auth_version''' in '''keystone_authtoken''' to 2.
 
  
 
Then, the below command should work.::
 
Then, the below command should work.::
Line 297: Line 296:
  
 
     $ service ironic-conductor restart
 
     $ service ironic-conductor restart
 
 
  
 
===== Registering Proliant node in Ironic =====
 
===== Registering Proliant node in Ironic =====
Line 350: Line 347:
  
 
* ProLiant DL380e Gen8
 
* ProLiant DL380e Gen8
* ProLiant DL380e Gen8
+
* ProLiant DL580e Gen8
* ProLiant DL580 Gen8
+
* ProLiant DL180 Gen9 UEFI
* ProLiant DL180 Gen9
+
* ProLiant DL360 Gen9 UEFI
 
+
* ProLiant DL380 Gen9 UEFI
  
 
===== Features =====
 
===== Features =====
Line 359: Line 356:
 
* Automatic detection of current boot mode.
 
* Automatic detection of current boot mode.
 
* Automatic setting of the required boot mode if UEFI boot mode is requested by the nova flavor's extra spec.
 
* Automatic setting of the required boot mode if UEFI boot mode is requested by the nova flavor's extra spec.
 +
* Remote Console
 +
* HW Sensors
 +
* UEFI Boot
 +
* Local boot (both BIOS and UEFI)
 +
* Supports deployment of whole disk image.
 +
* Support for out-of-band hardware inspection.
 +
* Node cleaning
  
 
===== Requirements =====
 
===== Requirements =====
Line 450: Line 454:
 
flavor
 
flavor
  
Use element  '''ubuntu-signed''' or '''fedora''' to build signed ubuntu deploy iso and user images from  '''diskimage-builder''' [3]_. The below command creates files named '''deploy-ramdisk.kernel''',
+
Use element  '''ubuntu-signed''' or '''fedora''' to build signed ubuntu deploy iso and user images from  [https://pypi.python.org/pypi/diskimage-builder diskimage-builder]_. The below command creates files named '''deploy-ramdisk.kernel''',
 
'''deploy-ramdisk.initramfs''' and '''deploy-ramdisk.iso''' in the current working directory
 
'''deploy-ramdisk.initramfs''' and '''deploy-ramdisk.iso''' in the current working directory
  
     cd <path-to-diskimage-builder>
+
     pip install "diskimage-builder"
     ./bin/ramdisk-image-create -o deploy-ramdisk ubuntu-signed deploy-ironic iso
+
     ramdisk-image-create -o deploy-ramdisk ubuntu-signed deploy-ironic iso
  
 
The below command creates files named '''cloud-image-boot.iso''', '''cloud-image.initrd''', '''cloud-image.vmlinuz''' and '''cloud-image.qcow2''' in the current working directory
 
The below command creates files named '''cloud-image-boot.iso''', '''cloud-image.initrd''', '''cloud-image.vmlinuz''' and '''cloud-image.qcow2''' in the current working directory
  
     cd <path-to-diskimage-builder>
+
     disk-image-create -o cloud-image ubuntu-signed baremetal iso
    ./bin/disk-image-create -o cloud-image ubuntu-signed baremetal iso
 
  
 
'''''NOTE''''':  
 
'''''NOTE''''':  
  
In UEFI secure boot, digitally signed bootloader should be able to validate digital signatures of kernel during boot process. This requires that the bootloader contains the digital signatures of the kernel. For '''iscsi_ilo''' driver, it is recommended that '''boot_iso''' property for user image contains the Glance UUID of the boot ISO. If '''boot_iso''' property is not updated in Glance for the user image, it would create the '''boot_iso''' using bootloader from the deploy iso. This '''boot_iso''' will be able to boot the user image in UEFI secure boot environment only if the bootloader is signed and can validate digital signatures of user  image kernel.
+
* UEFI secure boot is enabled when instance image is getting booted. The bare metal deploy happens in UEFI boot mode.
  
Ensure the public key of the signed image is loaded into baremetal to deploy signed images. For HP Proliant Gen9
+
* In UEFI secure boot, digitally signed bootloader should be able to validate digital signatures of kernel during boot process. This requires that the bootloader contains the digital signatures of the kernel. For '''iscsi_ilo''' driver, it is recommended that '''boot_iso''' property for user image contains the Glance UUID of the boot ISO. If '''boot_iso''' property is not updated in Glance for the user image, it would create the '''boot_iso''' using bootloader from the deploy iso. This '''boot_iso''' will be able to boot the user image in UEFI secure boot environment only if the bootloader is signed and can validate digital signatures of user  image kernel.
servers, one can enroll public key using iLO System Utilities UI. Please refer to section '''Accessing Secure Boot options'''
+
 
in HP UEFI System Utilities User Guide. [7] One can also refer to white paper on Secure Boot for Linux on HP Proliant
+
* Ensure the public key of the signed image is loaded into baremetal to deploy signed images. For HP Proliant Gen9 servers, one can enroll public key using iLO System Utilities UI. Please refer to section '''Accessing Secure Boot options''' in [http://www.hp.com/ctg/Manual/c04398276.pdf HP UEFI System Utilities User Guide]. One can also refer to white paper on [http://h20195.www2.hp.com/V2/getpdf.aspx/4AA5-4496ENW.pdf  Secure Boot for Linux on HP Proliant servers] for additional details.
servers for additional details. [8]
 
  
 
===Hardware Inspection===
 
===Hardware Inspection===
Line 484: Line 486:
  
 
NOTE:  
 
NOTE:  
* The RAID shall be pre-configured prior to inspection otherwise proliantutils returns '''0 (zero)''' for '''disk size'''.
+
* The RAID should be pre-configured prior to inspection otherwise proliantutils returns '''0 (zero)''' for '''disk size'''.
** The disk size can be retrieved only for '''real SmartArray Controllers''' after RAID configuration.  
+
** The disk size can be retrieved only for real Smart Array controllers with RAID configured.
** For '''direct storage''' and '''dynamic SmartArray Controllers''' operator has to manually enter the disk size after inspection.
+
** For direct storage and Dynamic Smart Array controllers, operator has to manually enter the disk size after inspection.
* The '''iLO firmware version''' shall be '''2.10''' or above for '''nic_capacity''' to be discovered.
+
* The '''iLO firmware version''' should be '''2.10''' or above for '''nic_capacity''' to be discovered.
  
 
The inspection process will discover the following essential properties
 
The inspection process will discover the following essential properties
Line 516: Line 518:
  
  
The operator can specify these capabilities in nova flavor for node to be selected for scheduling::
+
The operator can specify these capabilities in nova flavor for node to be selected for scheduling:
  
 
   nova flavor-key my-baremetal-flavor set capabilities:server_model="<in> Gen8"
 
   nova flavor-key my-baremetal-flavor set capabilities:server_model="<in> Gen8"
 
 
   nova flavor-key my-baremetal-flavor set capabilities:pci_gpu_devices="> 0"
 
   nova flavor-key my-baremetal-flavor set capabilities:pci_gpu_devices="> 0"
 
 
   nova flavor-key my-baremetal-flavor set capabilities:nic_capacity="10Gb"
 
   nova flavor-key my-baremetal-flavor set capabilities:nic_capacity="10Gb"
 
 
   nova flavor-key my-baremetal-flavor set capabilities:ilo_firmware_version="<in> 2.10"
 
   nova flavor-key my-baremetal-flavor set capabilities:ilo_firmware_version="<in> 2.10"
 
 
   nova flavor-key my-baremetal-flavor set capabilities:secure_boot="true"
 
   nova flavor-key my-baremetal-flavor set capabilities:secure_boot="true"
  
Line 587: Line 585:
 
===Instance Images===
 
===Instance Images===
  
All iLO drivers support deployment of whole disk images. Not all Liniux distributions support hybrid images (single image that can boot in BIOS and UEFI boot mode). If the image can be booted only in a specific boot mode then User needs to add 'boot_mode' capability in nova flavor's extra_spec.  
+
All iLO drivers support deployment of whole disk images. The whole disk images
 +
could be one of following types:
 +
 
 +
1. BIOS only image.  An image having only MBR partition and will boot only in BIOS boot mode.
 +
 
 +
2. UEFI only image. An image having GPT partition and will boot only in UEFI boot mode.
 +
 
 +
3. Hybrid image. An image that has GPT and MBR partition and will boot in both BIOS and UEFI boot mode.
 +
 
 +
4. Signed UEFI image. An UEFI image wherein bootloader and kernel are signed which could be used in UEFI secure boot environment.
 +
 
 +
Few of the linux distros provide whole disk images. Examples are:
 +
 
 +
1. Ubuntu - https://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-uefi1.img
 +
 
 +
2. CoreOS - http://stable.release.core-os.net/amd64-usr/current/coreos_production_openstack_image.img.bz2
 +
 
 +
3. OpenSuse - https://susestudio.com (It lets you build the image through the browser)
 +
 
 +
Following table summarizes the whole disk image capabilities:
 +
{| class="wikitable"
 +
|-
 +
! scope="col"; | Image Type
 +
! scope="col"; | Boot Mode
 +
! scope="col"; | Config Drive
 +
! scope="col"; | UEFI Secure Boot
 +
|-
 +
| BIOS only
 +
| BIOS
 +
| Yes
 +
| NA
 +
|-
 +
| UEFI only
 +
| UEFI
 +
| Yes
 +
| No
 +
|-
 +
| UEFI Signed
 +
| UEFI
 +
| Yes
 +
| Yes
 +
|-
 +
| Hybrid
 +
| BIOS and UEFI
 +
| See note below
 +
| Yes, if signed
 +
|}
 +
 
 +
'''Note''' : Config Drive feature of Ironic may not work on all the whole disk images, especially hybrid images wherein partition information may get lost when config drive partition is being created leading to failure during provisioning or instance may not boot.
 +
 
 +
Not all Linux distributions support hybrid images (single image that can boot in BIOS and UEFI boot mode). If the image can be booted only in a specific boot mode then user needs to add 'boot_mode' capability in nova flavor's extra_spec.  
 
From nova, specific boot mode may be requested by using the ComputeCapabilitesFilter. For example:-
 
From nova, specific boot mode may be requested by using the ComputeCapabilitesFilter. For example:-
 
   nova flavor-key ironic-test-3 set capabilities:boot_mode="uefi"
 
   nova flavor-key ironic-test-3 set capabilities:boot_mode="uefi"
 
   nova boot --flavor ironic-test-3 --image test-image instance-1
 
   nova boot --flavor ironic-test-3 --image test-image instance-1
  
For pxe-ilo driver, to deploy a whole disk image in UEFI boot mode, User needs to add boot_option="local" capability in nova flavor's extra_spec. For example:-
+
For pxe-ilo driver, to deploy a whole disk image in UEFI boot mode, user needs to add boot_option="local" capability in nova flavor's extra_spec. For example:-
 
   nova flavor-key ironic-test-3 set capabilities:boot_mode="uefi" capabilities:boot_option="local"  
 
   nova flavor-key ironic-test-3 set capabilities:boot_mode="uefi" capabilities:boot_option="local"  
 
   nova boot --flavor ironic-test-3 --image test-image instance-1
 
   nova boot --flavor ironic-test-3 --image test-image instance-1
  
 
=== Known Issues===
 
=== Known Issues===
'''Issue:''' Deploy on Gen9 servers fails as iLO do not honour one time boot device settings and tries to boot from the persistent boot device.
 
 
'''Solution:''' It is caused due to a defect in BIOS System ROM. The fix for the same is available since firmware version 1.32_03-05-2015 (B)  13 May 2015 onward.
 
 
'''Issue:''' HP Fedora based IPA deploy ramdisk ISO fails to boot with error "error: can't allocate initrd" if the P220 based smart array controller is attached to the Proliant server.
 
  
'''Solution:''' It is a Fishman driver issue in firmware for P220 based smart arrays. The defect has been filed on Fishman firmware. The driver patch would be made available shortly.
+
{| class="wikitable"
 +
|-
 +
! scope="col"; | Sr No
 +
! scope="col"| Firmware Version
 +
! scope="col"| Known Issues
 +
! scope="col"| Resolutions
 +
|-
 +
|1
 +
| BIOS System ROM version 1.20
 +
| Deploy on Gen9 servers fails as iLO do not honour one time boot device settings and tries to boot from the persistent boot device.
 +
| It is caused due to a defect in BIOS System ROM. The fix for the same is available since firmware version 1.32_03-05-2015 13 May 2015 onward.
 +
|-
 +
| 2
 +
| Smart Array SAS Driver v8.03
 +
| Fedora based IPA deploy ramdisk ISO fails to boot with error "error: can't allocate initrd" if the P220 based smart array controller is attached to the ProLiant server
 +
| It is a Fishman driver issue in firmware for P220 based smart arrays. The defect has been filed on Fishman firmware. The driver patch would be made available shortly.
 +
|-
 +
| 3
 +
| iLO version 2.20
 +
| Deploy using any of the iLO drivers can fail on Gen9 servers with error in conductor logs as "Invalid Device Choice" while setting persistent boot device. This issue happens only when Gen9 servers are running with iLO firmware version 2.20
 +
| This issue is in iLO firmware wherein if RIBCL is used to update persistent boot devices in UEFI boot mode on Gen9 servers, it fails with error message mentioned above. This issue can be resolved by using one of the methods given below:-
  
'''Issue:''' Deploy using any of the iLO drivers can fail on Gen9 servers with error in conductor logs as "Invalid Device Choice" while setting the persistent boot device. This issue happens only when Gen9 servers are running with iLO firmware version 2.20
+
A. Downgrading the iLO firmware version to 2.10 or upgrading it to version higher than 2.20
  
'''Solution:''' This issue is in iLO firmware wherein if RIBCL is used to update persistent boot devices in UEFI boot mode on Gen9 servers, it fails with error message mentioned above. This issue can be resolved by using one of the methods given below:-
+
B. Upgrading python package 'proliantutils' to version greater or equal to 2.1.3, This issue has been fixed in 'proliantutils' by enhancing it to use HP REST interface to update persistent boot devices for Gen9 servers.
 
 
1. Downgrading the iLO firmware version to 2.10 or upgrading it to version higher than 2.20
 
 
 
2. Upgrading python package 'proliantutils' to version greater or equal to 2.1.3, This issue has been fixed in 'proliantutils' by enhancing it to use HP REST interface to update persistent boot devices for Gen9 servers.
 
 
   $ sudo pip install "proliantutils>=2.1.3"
 
   $ sudo pip install "proliantutils>=2.1.3"
 
+
|-
'''Issue:''' Openstack documentation (http://docs.openstack.org/developer/ironic/) does not document '''Enabling HTTPS in Swift'''
+
| 4
 
+
| NA
'''Solution:''' Refer to '''Enabling HTTPS in Swift''' section of the iLO driver wiki (https://wiki.openstack.org/wiki/Ironic/Drivers/iLODrivers/Kilo#Enabling_HTTPS_in_Swift) to get information and steps to enable it.
+
| Openstack documentation (http://docs.openstack.org/developer/ironic/) does not document '''Enabling HTTPS in Swift'''
 +
| Refer to '''Enabling HTTPS in Swift''' section of the iLO driver wiki (https://wiki.openstack.org/wiki/Ironic/Drivers/iLODrivers/Kilo#Enabling_HTTPS_in_Swift) to get information and steps to enable it.
 +
|-
 +
| 5
 +
| NA
 +
| When SSL is enabled in OpenStack environment and images to be attached to iLO virtual media are based on 'https', iLO is unable to read/boot using such images.
 +
| iLO firmware version may not support the ciphers being enabled at the SSL server hosting the images. Please refer to iLO firmware documentation to ensure that the ciphers being used are supported http://h10032.www1.hp.com/ctg/Manual/c03334051. It is also recommended to refer to  'Release Notes' of iLO firmware version being used for more details.
 +
|}
  
 
=== References ===
 
=== References ===

Latest revision as of 03:56, 22 March 2016

iLO drivers (Kilo)

Overview

iLO drivers enable to take advantage of features of iLO management engine in HP Proliant servers. iLO drivers are targetted for HP Proliant Gen 8 systems and above which have iLO 4 management engine.

Currently there are 3 iLO drivers:

The iscsi_ilo and agent_ilo drivers provide security enhanced PXE-less deployment by using iLO virtual media to boot up the bare metal node. These drivers send management info through management channel and separates it from data channel which is used for deployment. iscsi_ilo driver deploys from Ironic conductor node and can do both net-boot and lcaol-boot. agent_ilo driver deploys from bare metal node and always does local-boot.

pxe_ilo driver uses PXE/iSCSI for deployment (just like normal PXE driver), but support automatic setting of requested boot mode from nova. This driver doesn't require iLO Advanced license.

Prerequisites

  • proliantutils is a python package which contains a set of modules for managing HP Proliant hardware. Install proliantutils module on the Ironic conductor node. Minimum version required is 2.1.3. Recommended version is 2.1.5


  $ pip install "proliantutils>=2.1.5"


  • ipmitool command must be present on the service node(s) where ironic-conductor is running. On most distros, this is provided as part of the ipmitool package. Source code is available at http://ipmitool.sourceforge.net/.

Drivers

iscsi_ilo driver

Overview

iscsi_ilo driver was introduced as an alternative to pxe_ipmitool and pxe_ipminative drivers for HP Proliant servers. iscsi_ilo uses virtual media feature in iLO to boot up the bare metal node instead of using PXE or iPXE.

Target Users
  • Users who do not want to use PXE/TFTP protocol on their data centres.
  • Users who have concerns on PXE driver's security issues and want to have a security enhanced PXE-less deployment mechanism - The PXE driver passes management information in clear-text to the baremetal node. However, if Swift proxy server has an HTTPS endpoint (See Enabling HTTPS in Swift for more information), the iscsi_ilo driver provides enhanced security by passing management information to and from Swift endpoint over HTTPS. The management information and boot image will be retrieved over encrypted management network via iLO virtual media.
Tested Platforms

This driver should work on HP Proliant Gen8 Servers and above with iLO 4.

It has been tested with the following servers:

  • ProLiant DL380e Gen8
  • ProLiant DL580e Gen8
  • ProLiant DL180 Gen9 UEFI
  • ProLiant DL360 Gen9 UEFI
  • ProLiant DL380 Gen9 UEFI
Features
  • PXE-less deployment with virtual media.
  • Automatic detection of current boot mode.
  • Automatic setting of the required boot mode if UEFI boot mode is requested by the nova flavor's extra spec.
  • Supports booting the instance from virtual media as well as booting locally from disk. Default is booting from virtual media.
  • UEFI Boot
  • UEFI Secure Boot
  • Passing management information via secure, encrypted management network (virtual media) if Swift proxy server has an HTTPS endpoint. See Enabling HTTPS in Swift for more info. Provisioning is done using iSCSI over data network, so this driver has the benefit of security enhancement with the same performance. It segregates management info from data channel.
  • Remote Console (based on IPMI)
  • HW Sensors
  • Works well for machines with resource constraints (lesser amount of memory).
  • Local boot (both BIOS and UEFI)
  • Supports deployment of whole disk image.
  • Support for out-of-band hardware inspection.
  • Node cleaning.
Requirements
  • iLO 4 Advanced License needs to be installed on iLO to enable Virtual Media feature.
  • Swift Object Storage Service - iLO driver uses Swift to store temporary FAT images as well as boot ISO images.
  • Glance Image Service with Swift configured as its backend - When using iscsi_ilo driver, the image containing the deploy ramdisk is retrieved from Swift directly by the iLO.
Deploy Process
  • Admin configures the Proliant baremetal node for iscsi_ilo driver. The Ironic node configured will have the ilo_deploy_iso property in its driver_info. This will contain the Glance UUID of the ISO deploy ramdisk image.
  • Ironic gets a request to deploy a Glance image on the baremetal node.
  • iscsi_ilo driver powers off the baremetal node.
  • The driver generates a swift-temp-url for the deploy ramdisk image and attaches it as Virtual Media CDROM on the iLO.
  • The driver creates a small FAT32 image containing parameters to the deploy ramdisk. This image is uploaded to Swift and its swift-temp-url is attached as Virtual Media Floppy on the iLO.
  • The driver sets the node to boot one-time from CDROM.
  • The driver powers on the baremetal node.
  • The deploy kernel/ramdisk is booted on the baremetal node. The ramdisk exposes the local disk over iSCSI and requests Ironic conductor to complete the deployment.
  • The driver on the Ironic conductor writes the glance image to the baremetal node's disk.
  • If local-boot is requested, Ironic conductor asks the deployment ramdisk to install the boot loader.
  • If it's a netboot (default), the driver bundles the boot kernel/ramdisk for the Glance deploy image into an ISO and then uploads it to Swift. This ISO image will be used for booting the deployed instance.
  • The driver reboots the node.
  • For netboot, on the first and subsequent reboots iscsi_ilo driver attaches this boot ISO image in Swift as Virtual Media CDROM and then sets iLO to boot from it. If boot_option was set to local, then the instance is booted from disk.
Configuring and Enabling the driver

1. Prepare an ISO deploy ramdisk image from diskimage-builder [3]_. This can be done by adding the iso element to the ramdisk-image-create command. This command creates the deploy kernel/ramdisk as well as a bootable ISO image containing the deploy kernel and ramdisk. The below command creates files named deploy-ramdisk.kernel, deploy-ramdisk.initramfs and deploy-ramdisk.iso in the current working directory

   pip install "diskimage-builder"
   ramdisk-image-create -o deploy-ramdisk ubuntu deploy-ironic iso

2. Upload this image to Glance.::

   glance image-create --name deploy-ramdisk.iso --disk-format iso --container-format bare < deploy-ramdisk.iso

3. Configure Glance image service with its storage backend as Swift. See here for configuration instructions.

4. Set a temp-url key for Glance user in Swift. For example, if you have configured Glance with user glance-swift and tenant as service, then run the below command::

   swift --os-username=service:glance-swift post -m temp-url-key:mysecretkeyforglance

5. Fill the required parameters in the [glance] section in /etc/ironic/ironic.conf. Normally you would be required to fill in the following details.::

   [glance]
   swift_temp_url_key=mysecretkeyforglance
   swift_endpoint_url=http://10.10.1.10:8080
   swift_api_version=v1
   swift_account=AUTH_51ea2fb400c34c9eb005ca945c0dc9e1
   swift_container=glance

The details can be retrieved by running the below command:

  $ swift --os-username=service:glance-swift stat -v | grep -i url
  StorageURL:     http://10.10.1.10:8080/v1/AUTH_51ea2fb400c34c9eb005ca945c0dc9e1
  Meta Temp-Url-Key: mysecretkeyforglance


6. Swift must be accessible with the same admin credentials configured in Ironic. For example, if Ironic is configured with the below credentials in /etc/ironic/ironic.conf.::

   [keystone_authtoken]
   admin_password = password
   admin_user = ironic
   admin_tenant_name = service
   auth_version = 2

Then, the below command should work.::

   $ swift --os-username ironic --os-password password --os-tenant-name service --auth-version 2 stat
                        Account: AUTH_22af34365a104e4689c46400297f00cb
                     Containers: 2
                        Objects: 18
                          Bytes: 1728346241
   Objects in policy "policy-0": 18
     Bytes in policy "policy-0": 1728346241
              Meta Temp-Url-Key: mysecretkeyforglance
                    X-Timestamp: 1409763763.84427
                     X-Trans-Id: tx51de96a28f27401eb2833-005433924b
                   Content-Type: text/plain; charset=utf-8
                  Accept-Ranges: bytes


7. Add iscsi_ilo to the list of enabled_drivers in /etc/ironic/ironic.conf. For example:::

   enabled_drivers = fake,pxe_ssh,pxe_ipmitool,iscsi_ilo

8. Restart the Ironic conductor service.

   $ service ironic-conductor restart
Registering Proliant node in Ironic

Nodes configured for iLO driver should have the driver property set to iscsi_ilo. The following configuration values are also required in driver_info:

  • ilo_address: IP address or hostname of the iLO.
  • ilo_username: Username for the iLO with administrator privileges.
  • ilo_password: Password for the above iLO user.
  • ilo_deploy_iso: The Glance UUID of the deploy ramdisk ISO image.
  • client_port: (optional) Port to be used for iLO operations if you are using a custom port on the iLO. Default port used is 443.
  • client_timeout: (optional) Timeout for iLO operations. Default timeout is 60 seconds.
  • console_port: (optional) Node's UDP port for console access. Any unused port on the Ironic conductor node may be used.
Boot modes

iscsi_ilo driver supports automatic detection of boot mode (Legacy BIOS or UEFI) and setting of boot mode from BIOS to UEFI. Please see Note below for details.

  • When no boot mode setting is provided, iscsi_ilo driver preserves the current boot mode of the bare metal on the deployed instance.
  • A requirement of a specific boot mode may be provided by adding boot_mode:bios or boot_mode:uefi to capabilities property within the properties field of an Ironic node. iscsi_ilo driver will then deploy and configure the instance in the specified boot mode.

For example, to make a Proliant baremetal node boot always in UEFI mode, run the following command::

  ironic node-update <node-id> add properties/capabilities='boot_mode:uefi'

NOTE:

  • We recommend setting the boot_mode property on systems that support both UEFI and legacy modes if user wants facility in Nova to choose a baremetal node with appropriate boot mode. This is for Gen8 (ProLiant DL580 only) and Gen9 systems.
  • iscsi_ilo driver automatically sets boot mode from BIOS to UEFI, if the requested boot mode in nova boot is UEFI. However, users will need to pre-configure boot mode to Legacy on Gen8 (ProLiant DL580 only) and Gen9 servers if they want to deploy the node in legacy mode.
  • The automatic boot ISO creation for UEFI boot mode has been enabled in Kilo. The manual creation of boot ISO for UEFI boot mode is also supported. For the latter, the boot ISO for the deploy image needs to be built separately and the deploy image's boot_iso property in Glance should contain the Glance UUID of the boot ISO. For building boot ISO, add the iso element after adding the baremetal element while building disk images with diskimage-builder
   disk-image-create ubuntu baremetal iso
  • From nova, specific boot mode may be requested by using the ComputeCapabilitesFilter. For example, it can be set in a flavor like below::
  nova flavor-key ironic-test-3 set capabilities:boot_mode="uefi"
  nova boot --flavor ironic-test-3 --image test-image instance-1

agent_ilo driver

Overview

agent_ilo driver was introduced as an alternative to agent_ipmitool and agent_ipminative drivers for HP Proliant servers. agent_ilo driver uses virtual media feature in HP Proliant baremetal servers to boot up the Ironic Python Agent (IPA) on the baremetal node instead of using PXE. For more information on IPA, refer https://wiki.openstack.org/wiki/Ironic-python-agent.

Target Users
  • Users who do not want to use PXE/TFTP protocol on their data centres.
Tested Platforms

This driver should work on HP Proliant Gen8 Servers and above with iLO 4.

It has been tested with the following servers:

  • ProLiant DL380e Gen8
  • ProLiant DL580e Gen8
  • ProLiant DL180 Gen9 UEFI
  • ProLiant DL360 Gen9 UEFI
  • ProLiant DL380 Gen9 UEFI
Features
  • PXE-less deploy with virtual media using Ironic Python Agent.
  • Remote Console
  • HW Sensors
  • Automatic detection of current boot mode.
  • Automatic setting of the required boot mode if UEFI boot mode is requested by the nova flavor's extra spec.
  • UEFI Boot
  • UEFI Secure Boot
  • IPA runs on the bare metal node and pulls the image directly from Swift.
  • IPA deployed instances always boot from local disk.
  • Supports deployment of whole disk image.
  • Segregates management info from data channel.
  • Support for out-of-band hardware inspection.
  • Node cleaning.
Requirements
  • iLO 4 Advanced License needs to be installed on iLO to enable virtual media feature.
  • Swift Object Storage Service - iLO driver uses Swift to store temporary FAT images as well as boot ISO images.
  • Glance Image Service with Swift configured as its backend - When using agent_ilo driver, the image containing the agent is retrieved from Swift directly by the iLO.
Deploy Process
  • Admin configures the Proliant baremetal node for agent_ilo driver. The Ironic node configured will have the ilo_deploy_iso property in its driver_info. This will contain the Glance UUID of the ISO deploy agent image containing the agent.
  • Ironic gets a request to deploy a Glance image on the baremetal node.
  • Driver powers off the baremetal node.
  • Driver generates a swift-temp-url for the deploy agent image and attaches it as Virtual Media CDROM on the iLO.
  • Driver creates a small FAT32 image containing parameters to the agent ramdisk. This image is uploaded to Swift and its swift-temp-url is attached as Virtual Media Floppy on the iLO.
  • Driver sets the node to boot one-time from CDROM.
  • Driver powers on the baremetal node.
  • The deploy kernel/ramdisk containing the agent is booted on the baremetal node. The agent ramdisk talks to the Ironic conductor, downloads the image directly from Swift and writes the image to chosen disk on the node.
  • Driver sets the node to permanently boot from disk and then reboots the node.
Configuring and Enabling the driver

1. Prepare an ISO deploy Ironic Python Agent image containing the agent [5]_. This can be done by using the iso-image-create script found within the agent. The below set of commands will create a file ipa-ramdisk.iso in the below directory UPLOAD::

   $ pip install "diskimage-builder"
   $ disk-image-create -o ipa-ramdisk fedora ironic-agent iso

2. Upload the IPA ramdisk image to Glance.::

   glance image-create --name ipa-ramdisk.iso --disk-format iso --container-format bare < ipa-ramdisk.iso

3. Configure Glance image service with its storage backend as Swift. See [4]_ for configuration instructions. 4. Set a temp-url key for Glance user in Swift. For example, if you have configured Glance with user glance-swift and tenant as service, then run the below command::

   swift --os-username=service:glance-swift post -m temp-url-key:mysecretkeyforglance

5. Fill the required parameters in the [glance] section in /etc/ironic/ironic.conf. Normally you would be required to fill in the following details.::

   [glance]
   swift_temp_url_key=mysecretkeyforglance
   swift_endpoint_url=http://10.10.1.10:8080
   swift_api_version=v1
   swift_account=AUTH_51ea2fb400c34c9eb005ca945c0dc9e1
   swift_container=glance
 The details can be retrieved by running the below command:::
  $ swift --os-username=service:glance-swift stat -v | grep -i url
  StorageURL:     http://10.10.1.10:8080/v1/AUTH_51ea2fb400c34c9eb005ca945c0dc9e1
  Meta Temp-Url-Key: mysecretkeyforglance

6. Swift must be accessible with the same admin credentials configured in Ironic. For example, if Ironic is configured with the below credentials in /etc/ironic/ironic.conf.::

   [keystone_authtoken]
   admin_password = password
   admin_user = ironic
   admin_tenant_name = service
   auth_version = 2

Then, the below command should work.::

   $ swift --os-username ironic --os-password password --os-tenant-name service --auth-version 2 stat
                        Account: AUTH_22af34365a104e4689c46400297f00cb
                     Containers: 2
                        Objects: 18
                          Bytes: 1728346241
   Objects in policy "policy-0": 18
     Bytes in policy "policy-0": 1728346241
              Meta Temp-Url-Key: mysecretkeyforglance
                    X-Timestamp: 1409763763.84427
                     X-Trans-Id: tx51de96a28f27401eb2833-005433924b
                   Content-Type: text/plain; charset=utf-8
                  Accept-Ranges: bytes


7. Add agent_ilo to the list of enabled_drivers in /etc/ironic/ironic.conf. For example:::

   enabled_drivers = fake,pxe_ssh,pxe_ipmitool,agent_ilo

8. Restart the Ironic conductor service.::

   $ service ironic-conductor restart
Registering Proliant node in Ironic

Nodes configured for iLO driver should have the driver property set to agent_ilo. The following configuration values are also required in driver_info:

  • ilo_address: IP address or hostname of the iLO.
  • ilo_username: Username for the iLO with administrator privileges.
  • ilo_password: Password for the above iLO user.
  • ilo_deploy_iso: The Glance UUID of the deploy agent ISO image containing the agent.
  • client_port: (optional) Port to be used for iLO operations if you are using a custom port on the iLO. Default port used is 443.
  • client_timeout: (optional) Timeout for iLO operations. Default timeout is 60 seconds.
  • console_port: (optional) Node's UDP port for console access. Any unused port on the Ironic conductor node may be used.
Boot modes

agent_ilo driver supports automatic detection of boot mode (Legacy BIOS or UEFI) and setting of boot mode from BIOS to UEFI. Please see Note below for details.

  • When no boot mode setting is provided, agent_ilo driver preserves the current boot mode on the deployed instance.
  • A requirement of a specific boot mode may be provided by adding boot_mode:bios or boot_mode:uefi to capabilities property within the properties field of an Ironic node. Then agent_ilo driver will deploy and configure the instance in the appropriate boot mode.

For example, to make a Proliant baremetal node boot in UEFI mode, run the following command::

  ironic node-update <node-id> add properties/capabilities='boot_mode:uefi'

NOTE:

  • We recommend setting the boot_mode property on systems that support both UEFI and legacy modes if user wants facility in Nova to choose a baremetal node with appropriate boot mode. This is for ProLiant DL580 Gen8 and Gen9 systems.
  • agent_ilo driver automatically set boot mode from BIOS to UEFI, if the requested boot mode in nova boot is UEFI. However, users will need to pre-configure boot mode to Legacy on Gen8 (ProLiant DL580 only) and Gen9 servers if they want to deploy the node in legacy mode.
  • From nova, specific boot mode may be requested by using the ComputeCapabilitesFilter. For example, it can be set in a flavor like below::
  nova flavor-key ironic-test-3 set capabilities:boot_mode="uefi"
  nova boot --flavor ironic-test-3 --image test-image instance-1

pxe_ilo driver

Overview

pxe_ilo driver uses PXE/iSCSI (just like pxe_ipmitool driver) to deploy the image and uses iLO to do all management operations on the baremetal node(instead of using IPMI).

Target Users
  • Users who want to use PXE/iSCSI for deployment in their environment or who don't have Advanced License in their iLO.
  • Users who don't want to configure boot mode and boot device manually on the baremetal node.
  • User who wants to use iLO driver value-add features such as boot mode management, out-of-band node cleaning and hardware introspection.
Tested Platforms

This driver should work on HP Proliant Gen8 Servers and above with iLO 4. It has been tested with the following servers:

  • ProLiant DL380e Gen8
  • ProLiant DL580e Gen8
  • ProLiant DL180 Gen9 UEFI
  • ProLiant DL360 Gen9 UEFI
  • ProLiant DL380 Gen9 UEFI
Features
  • Automatic detection of current boot mode.
  • Automatic setting of the required boot mode if UEFI boot mode is requested by the nova flavor's extra spec.
  • Remote Console
  • HW Sensors
  • UEFI Boot
  • Local boot (both BIOS and UEFI)
  • Supports deployment of whole disk image.
  • Support for out-of-band hardware inspection.
  • Node cleaning
Requirements

None.

Configuring and Enabling the driver

1. Prepare an ISO deploy ramdisk image from diskimage-builder [3]. The below command creates a file named deploy-ramdisk.kernel and deploy-ramdisk.initramfs in the current working directory::

   ramdisk-image-create -o deploy-ramdisk ubuntu deploy-ironic

2. Upload this image to Glance.::

   glance image-create --name deploy-ramdisk.kernel --disk-format aki --container-format aki < deploy-ramdisk.kernel
   glance image-create --name deploy-ramdisk.initramfs --disk-format ari --container-format ari < deploy-ramdisk.initramfs

7. Add pxe_ilo to the list of enabled_drivers in /etc/ironic/ironic.conf. For example:::

   enabled_drivers = fake,pxe_ssh,pxe_ipmitool,pxe_ilo

8. Restart the Ironic conductor service.::

   service ironic-conductor restart
Registering Proliant node in Ironic

Nodes configured for iLO driver should have the driver property set to pxe_ilo. The following configuration values are also required in driver_info:

  • ilo_address: IP address or hostname of the iLO.
  • ilo_username: Username for the iLO with administrator privileges.
  • ilo_password: Password for the above iLO user.
  • pxe_deploy_kernel: The Glance UUID of the deployment kernel.
  • pxe_deploy_ramdisk: The Glance UUID of the deployment ramdisk.
  • client_port: (optional) Port to be used for iLO operations if you are using a custom port on the iLO. Default port used is 443.
  • client_timeout: (optional) Timeout for iLO operations. Default timeout is 60 seconds.
  • console_port: (optional) Node's UDP port for console access. Any unused port on the Ironic conductor node may be used.
Boot modes

pxe_ilo driver supports automatic detection of boot mode (Legacy BIOS or UEFI) and setting of boot mode from BIOS to UEFI. Please see Note below for details.

  • When no boot mode setting is provided, pxe_ilo driver preserves the current boot mode on the deployed instance.
  • A requirement of a specific boot mode may be provided by adding boot_mode:bios or boot_mode:uefi to capabilities property within the properties field of an Ironic node. Then pxe_ilo driver will deploy and configure the instance in the appropriate boot mode.::
  ironic node-update <NODE-ID> add properties/capabilities='boot_mode:uefi'

NOTE:

  • We recommend setting the boot_mode property on systems that support both UEFI and legacy modes if user wants facility in Nova to choose a baremetal node with appropriate boot mode. This is for ProLiant DL580 Gen8 and Gen9 systems.
  • pxe_ilo driver automatically set boot mode from BIOS to UEFI, if the requested boot mode in nova boot is UEFI. However, users will need to pre-configure boot mode to Legacy on DL580 Gen8 and Gen9 servers if they want to deploy the node in legacy mode.
  • From nova, specific boot mode may be requested by using the ComputeCapabilitesFilter. For example, it can be set in a flavor like below::
  nova flavor-key ironic-test-3 set capabilities:boot_mode="uefi"
  nova boot --flavor ironic-test-3 --image test-image instance-1

UEFI Secure Boot support

  • The following drivers support UEFI secure boot deploy:
    • iscsi_ilo
    • agent_ilo


Tested Platforms: This feature is available on HP Proliant Gen9 servers and above with iLO 4. It has been tested with the following servers:

  • ProLiant DL360 Gen9 UEFI
  • ProLiant DL380 Gen9 UEFI


The UEFI secure boot mode can be configured in Ironic by adding secure_boot parameter in the capabilities parameter within properties field of an Ironic node.

secure_boot is a boolean parameter and takes value as true or false.

To enable secure_boot on a node add it to capabilities as below::

 ironic node-update <node-uuid> add properties/capabilities='secure_boot:true'

or, alternatively use hardware inspection to populate the secure boot capability.

Nodes having secure_boot set to true may be requested by adding an extra_spec to the Nova flavor::

 nova flavor-key ironic-test-3 set capabilities:secure_boot="true"
 nova boot --flavor ironic-test-3 --image test-image instance-1

If capabilities is used in extra_spec as above, Nova scheduler (ComputeCapabilitiesFilter) will match only Ironic nodes which have the secure_boot set appropriately in properties/capabilities. It will filter out rest of the nodes.

The above facility for matching in Nova can be used in heterogeneous environments where there is a mix of machines supporting and not supporting UEFI secure boot, and operator wants to provide a choice to the user regarding secure boot. If the flavor doesn't contain secure_boot then Nova scheduler will not consider secure boot mode as a placement criteria, hence user may get a secure boot capable machine that matches with user specified flavors but deployment would not use its secure boot capability. Secure boot deploy would happen only when it is explicitly specified through flavor

Use element ubuntu-signed or fedora to build signed ubuntu deploy iso and user images from diskimage-builder_. The below command creates files named deploy-ramdisk.kernel, deploy-ramdisk.initramfs and deploy-ramdisk.iso in the current working directory

   pip install "diskimage-builder"
   ramdisk-image-create -o deploy-ramdisk ubuntu-signed deploy-ironic iso

The below command creates files named cloud-image-boot.iso, cloud-image.initrd, cloud-image.vmlinuz and cloud-image.qcow2 in the current working directory

   disk-image-create -o cloud-image ubuntu-signed baremetal iso

NOTE:

  • UEFI secure boot is enabled when instance image is getting booted. The bare metal deploy happens in UEFI boot mode.
  • In UEFI secure boot, digitally signed bootloader should be able to validate digital signatures of kernel during boot process. This requires that the bootloader contains the digital signatures of the kernel. For iscsi_ilo driver, it is recommended that boot_iso property for user image contains the Glance UUID of the boot ISO. If boot_iso property is not updated in Glance for the user image, it would create the boot_iso using bootloader from the deploy iso. This boot_iso will be able to boot the user image in UEFI secure boot environment only if the bootloader is signed and can validate digital signatures of user image kernel.

Hardware Inspection

Hardware inspection is supported by following drivers:

  • pxe_ilo
  • iscsi_ilo
  • agent_ilo
  • The inspection can be initiated by using following commands:
    • Move node to manageable state:
   ironic node-set-provision-state <node_UUID> manage
    • Initiate inspection:
   ironic node-set-provision-state <node_UUID> inspect

NOTE:

  • The RAID should be pre-configured prior to inspection otherwise proliantutils returns 0 (zero) for disk size.
    • The disk size can be retrieved only for real Smart Array controllers with RAID configured.
    • For direct storage and Dynamic Smart Array controllers, operator has to manually enter the disk size after inspection.
  • The iLO firmware version should be 2.10 or above for nic_capacity to be discovered.

The inspection process will discover the following essential properties (properties required for scheduling deployment):

  • memory_mb: memory size
  • cpus: number of cpus
  • cpu_arch: cpu architecture
  • local_gb: disk size


Inspection can also discover the following extra capabilities for iLO drivers:

  • ilo_firmware_version: iLO firmware version
  • rom_firmware_version: System ROM firmware version
  • secure_boot: secure boot is supported or not. The possible values are 'true' or 'false'. The value is returned as 'true' if secure boot is supported by the server.
  • server_model: server model
  • pci_gpu_devices: number of gpu devices connected to the baremetal.
  • nic_capacity: the max speed of the embedded NIC adapter.


The operator can specify these capabilities in nova flavor for node to be selected for scheduling:

 nova flavor-key my-baremetal-flavor set capabilities:server_model="<in> Gen8"
 nova flavor-key my-baremetal-flavor set capabilities:pci_gpu_devices="> 0"
 nova flavor-key my-baremetal-flavor set capabilities:nic_capacity="10Gb"
 nova flavor-key my-baremetal-flavor set capabilities:ilo_firmware_version="<in> 2.10"
 nova flavor-key my-baremetal-flavor set capabilities:secure_boot="true"

The above are just the examples of using the capabilities in nova flavor.

Enabling HTTPS in Swift

iLO drivers iscsi_ilo and agent_ilo use Swift for storing boot images and management information. By default, HTTPS is not enabled in Swift. HTTPS is required to encrypt all communication between Ironic Conductor and Swift proxy server, thereby preventing eavesdropping of network packets. It can be enabled in one of the following ways:

   cd /etc/swift
   openssl req -new -x509 -nodes -out cert.crt -keyout cert.key
    • Add the following lines to /etc/swift/proxy-server.conf under [DEFAULT]
 bind_port = 443
 cert_file = /etc/swift/cert.crt
 key_file = /etc/swift/cert.key
    • Restart the Swift proxy server.

Node Cleaning

  • The following drivers support node cleaning:
    • pxe_ilo
    • iscsi_ilo
    • agent_ilo

Node cleaning is enabled by default. This setting can be changed in ironic.conf.

   [conductor]
   clean_nodes=true

OR

   [conductor]
   clean_nodes=false

Nodes are set to cleaning state in either of the following -

  • During deletion of an existing instance
   ironic node-set-provision-state <node-uuid> deleted
  • Or while moving the node from MANAGEABLE state to AVAILABLE state
   ironic node-set-provision-state <node-uuid> provide

Currently, supported out-of-band iLO cleaning operations are:

  • reset_ilo : Resets the iLO. By default, enabled with priority 1.
  • reset_ilo_credential : Resets the iLO password, if ‘ilo_change_password’ is specified as part of node’s driver_info. By default, enabled with priority 30.
  • reset_bios_to_default : Resets BIOS Settings to default. By default, enabled with priority 10. This clean step is supported only on Gen9 and above servers.
  • reset_secure_boot_keys: Resets secure boot keys to manufacturer’s defaults. This step is supported only on Gen9 and above servers. By default, enabled with priority 20 .
  • clear_secure_boot_keys: Clears all secure boot keys. This step is supported only on Gen9 and above servers. By default, this step is disabled.

Additionally, agent_ilo driver supports inband disk erase operation.

To disable or change the priority of the particular clean step, respective configuration options to be updated in ironic.conf

   [ilo]
   clean_priority_reset_ilo=1
   clean_priority_reset_bios_to_default=10
   clean_priority_reset_secure_boot_keys_to_default=20
   clean_priority_clear_secure_boot_keys=0
   clean_priority_reset_ilo_credential=30
   clean_priority_erase_devices=10

To disable a particular clean step, update the priority of step to 0.

Instance Images

All iLO drivers support deployment of whole disk images. The whole disk images could be one of following types:

1. BIOS only image. An image having only MBR partition and will boot only in BIOS boot mode.

2. UEFI only image. An image having GPT partition and will boot only in UEFI boot mode.

3. Hybrid image. An image that has GPT and MBR partition and will boot in both BIOS and UEFI boot mode.

4. Signed UEFI image. An UEFI image wherein bootloader and kernel are signed which could be used in UEFI secure boot environment.

Few of the linux distros provide whole disk images. Examples are:

1. Ubuntu - https://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-uefi1.img

2. CoreOS - http://stable.release.core-os.net/amd64-usr/current/coreos_production_openstack_image.img.bz2

3. OpenSuse - https://susestudio.com (It lets you build the image through the browser)

Following table summarizes the whole disk image capabilities:

Image Type Boot Mode Config Drive UEFI Secure Boot
BIOS only BIOS Yes NA
UEFI only UEFI Yes No
UEFI Signed UEFI Yes Yes
Hybrid BIOS and UEFI See note below Yes, if signed

Note : Config Drive feature of Ironic may not work on all the whole disk images, especially hybrid images wherein partition information may get lost when config drive partition is being created leading to failure during provisioning or instance may not boot.

Not all Linux distributions support hybrid images (single image that can boot in BIOS and UEFI boot mode). If the image can be booted only in a specific boot mode then user needs to add 'boot_mode' capability in nova flavor's extra_spec. From nova, specific boot mode may be requested by using the ComputeCapabilitesFilter. For example:-

 nova flavor-key ironic-test-3 set capabilities:boot_mode="uefi"
 nova boot --flavor ironic-test-3 --image test-image instance-1

For pxe-ilo driver, to deploy a whole disk image in UEFI boot mode, user needs to add boot_option="local" capability in nova flavor's extra_spec. For example:-

 nova flavor-key ironic-test-3 set capabilities:boot_mode="uefi" capabilities:boot_option="local" 
 nova boot --flavor ironic-test-3 --image test-image instance-1

Known Issues

Sr No Firmware Version Known Issues Resolutions
1 BIOS System ROM version 1.20 Deploy on Gen9 servers fails as iLO do not honour one time boot device settings and tries to boot from the persistent boot device. It is caused due to a defect in BIOS System ROM. The fix for the same is available since firmware version 1.32_03-05-2015 13 May 2015 onward.
2 Smart Array SAS Driver v8.03 Fedora based IPA deploy ramdisk ISO fails to boot with error "error: can't allocate initrd" if the P220 based smart array controller is attached to the ProLiant server It is a Fishman driver issue in firmware for P220 based smart arrays. The defect has been filed on Fishman firmware. The driver patch would be made available shortly.
3 iLO version 2.20 Deploy using any of the iLO drivers can fail on Gen9 servers with error in conductor logs as "Invalid Device Choice" while setting persistent boot device. This issue happens only when Gen9 servers are running with iLO firmware version 2.20 This issue is in iLO firmware wherein if RIBCL is used to update persistent boot devices in UEFI boot mode on Gen9 servers, it fails with error message mentioned above. This issue can be resolved by using one of the methods given below:-

A. Downgrading the iLO firmware version to 2.10 or upgrading it to version higher than 2.20

B. Upgrading python package 'proliantutils' to version greater or equal to 2.1.3, This issue has been fixed in 'proliantutils' by enhancing it to use HP REST interface to update persistent boot devices for Gen9 servers.

  $ sudo pip install "proliantutils>=2.1.3"
4 NA Openstack documentation (http://docs.openstack.org/developer/ironic/) does not document Enabling HTTPS in Swift Refer to Enabling HTTPS in Swift section of the iLO driver wiki (https://wiki.openstack.org/wiki/Ironic/Drivers/iLODrivers/Kilo#Enabling_HTTPS_in_Swift) to get information and steps to enable it.
5 NA When SSL is enabled in OpenStack environment and images to be attached to iLO virtual media are based on 'https', iLO is unable to read/boot using such images. iLO firmware version may not support the ciphers being enabled at the SSL server hosting the images. Please refer to iLO firmware documentation to ensure that the ciphers being used are supported http://h10032.www1.hp.com/ctg/Manual/c03334051. It is also recommended to refer to 'Release Notes' of iLO firmware version being used for more details.

References