Difference between revisions of "Ironic/NovaBaremetalIronicMigration"

Revision as of 22:09, 15 September 2014

Migration from Nova Baremetal to Ironic

The purpose of this wiki to to provide an understanding of the migration process from Nova's "baremetal" driver to its "ironic" driver. First is a list of assumptions we took to write this document and the physical requirements for the migration to successfully complete. Next, a high-level description of the upgrade process is provided, followed by a detailed example of each step.

Assumptions / Requirements

This section describes assumptions made about the operating environment in which the upgrade can be performed, and is a reflection of the environment used in the continuous upgrade testing that will be performed on the Juno stable branch. An upgrade may be possible under other circumstances as well.

Only a "sideways" migration within the Juno release is supported. In other words:
- migrating from Juno "baremetal" to Juno "ironic" is supported, but:
- migrating from Icehouse "baremetal" to Juno "ironic" is not supported.
Migrating live instances between virt drivers in Nova is not tested, and therefore is not recommended.
- That being said, we have had some success in migrating a live "baremetal" instance to "ironic" and then deleting it.
Upgrading from Icehouse "baremetal" to Juno "baremetal" is not tested upstream, but we are not aware of any bugs affecting this upgrade.

High Level Walk Through

This section outlines the recommended approach for an operator to perform a sideways migration from the Juno release of Nova "baremetal" driver to the Juno release of the Nova "ironic" driver.

Install Ironic by following the steps described in these two sections of Ironic's install guide:
- Install and Configure Prerequisites
- Configure the Bare Metal Service. Do not start the Ironic services (the last step) just yet.
Prepare the "deploy-ironic" kernel and initrd for use with the PXE driver
- Build the kernel and initrd images using diskimage-builder (Example)
- Load these images into Glance (Example)
- If you plan to use another driver, prepare the appropriate image(s) for that driver instead.
Begin maintenance period, preventing user access to Nova, which will last until the migration is complete.
Stop Nova's compute and scheduler services (Example)
Update nova-compute configuration to use Ironic:
- follow steps 1 and 2 of Configure Compute Service to use the Bare Metal Service to change some settings and add an [ironic] section
- Remove the entire [baremetal] section
Update Nova baremetal's flavor metadata to reference Ironic's deploy kernel & ramdisk (Example)
Run the migrate_db script (Example)
Start Ironic services (Example)
Ensure proper service start-up (Example)
(Re)start Nova's compute and scheduler services (Example)
End maintenance period

Detailed examples

Below are detailed examples of the steps required to migrate Nova baremetal to Ironic.

Note these are general examples and may differ slightly from your configuration.

Build kernel and ram-disk

Use diskimage-builder to create the Ironic deploy kernel and ram-disk. For example:

 diskimage-builder/bin/ramdisk-image-create -a $NODE_ARCH $NODE_DIST deploy-ironic -o deploy-ironic $DIB_ELEMENTS

Load kernel and ram-disk

Load the Ironic deploy kernel and ramdisk built above into Glance.

 dkid=$(glance image-create --name "deploy-ironic-kernel" --public --disk-format aki --file deploy-ironic.kernel | awk ' / id / {print $4}')
 drid=$(glance image-create --name "deploy-ironic-ramdisk" --public --disk-format ari --file deploy-ironic.initramfs | awk ' / id / {print $4}')

Stop services

 sudo service nova-compute stop
 sudo service nova-scheduler stop

Update flavor

Update the Nova flavor to reference the deploy-ironic kernel and ram-disk loaded into glance.

 nova flavor-key baremetal set "baremetal:deploy_kernel_id"="$dkid"
 nova flavor-key baremetal set "baremetal:deploy_ramdisk_id"="$drid"

Run migrate script

 ironic-nova-bm-migrate  --nova-bm-db $database_connection_url_nova_bm \
     --ironic-db $database_connection_url_ironic \
     --node-arch x86_64 \
     --nova-conf $NOVA_CONF_FILE

Start Ironic services

 sudo service ironic-api start
 sudo service ironic-conductor start

Ensure service start-up

Before running the command below be sure that the required OS_* environment variables are set / sourced.

 ironic driver-list

Which should produce output like this sample:

 +--------------+--------+
 | name         | hosts  |
 +--------------+--------+
 | pxe_ipmitool | ubuntu |
 | pxe_ssh      | ubuntu |
 +--------------+--------+

Note that the above list may not match your out but exactly depending on the drivers you've enabled. If this is the first startup for Ironic it would also be advisable to check the ironic logs for errors too.

Then run a node list to ensure that the nodes have migrated correctly.

 ironic node-list

This should produce a list such as:

 +--------------------------------------+---------------+-------------+-----------------+-------------+
 | uuid                                 | instance_uuid | power_state | provision_state | maintenance |
 +--------------------------------------+---------------+-------------+-----------------+-------------+
 | 36f24e67-aaec-41ba-ba20-509bdd548974 | None          | power off   | None            | False       |
 | 77024bdb-a360-419f-a0b5-bc18a97571c1 | None          | power off   | None            | False       |
 +--------------------------------------+---------------+-------------+-----------------+-------------+

Inspect each node in detail with the node-show command:

 ironic node-show

 +------------------------+-------------------------------------------------------------------------+
 | Property               | Value                                                                   |
 +------------------------+-------------------------------------------------------------------------+
 | instance_uuid          | None                                                                    |
 | target_power_state     | None                                                                    |
 | properties             | {u'memory_mb': u'2048', u'cpu_arch': u'i386', u'local_gb': u'40',       |
 |                        | u'cpus': u'1'}                                                          |
 | maintenance            | False                                                                   |
 | driver_info            | {u'ssh_username': u'nobodycam', u'ssh_virt_type': u'virsh',             |
 |                        | u'ssh_address': u'192.168.122.1', u'ssh_key_contents': u'-----BEGIN RSA |
 |                        | PRIVATE KEY-----                                                        |
 |                        | <SSH KEY REMOVED                                                                      |
 |                        | -----END RSA PRIVATE KEY-----'}                                         |
 | extra                  | {}                                                                      |
 | last_error             | None                                                                    |
 | created_at             | 2014-09-15T17:50:59+00:00                                               |
 | target_provision_state | None                                                                    |
 | driver                 | pxe_ssh                                                                 |
 | updated_at             | 2014-09-15T18:24:15+00:00                                               |
 | instance_info          | {}                                                                      |
 | chassis_uuid           | None                                                                    |
 | provision_state        | None                                                                    |
 | reservation            | None                                                                    |
 | power_state            | power off                                                               |
 | console_enabled        | False                                                                   |
 | uuid                   | 36f24e67-aaec-41ba-ba20-509bdd548974                                    |
 +------------------------+-------------------------------------------------------------------------+

Start Nova services

 sudo service nova-compute start
 sudo service nova-scheduler start

References

Ironic: https://wiki.openstack.org/wiki/Ironic
Nova Baremetal: https://wiki.openstack.org/wiki/Baremetal

@@ Line 20: / Line 20: @@
 * Install Ironic by following the steps described in these two sections of Ironic's [http://docs.openstack.org/developer/ironic/deploy/install-guide.html install guide]:
 ** [http://docs.openstack.org/developer/ironic/deploy/install-guide.html#install-and-configure-prerequisites Install and Configure Prerequisites]
-** [http://docs.openstack.org/developer/ironic/deploy/install-guide.html#configure-the-bare-metal-service Configure the Bare Metal Service]
+** [http://docs.openstack.org/developer/ironic/deploy/install-guide.html#configure-the-bare-metal-service Configure the Bare Metal Service]. Do not start the Ironic services (the last step) just yet.
 * Prepare the "deploy-ironic" kernel and initrd for use with the PXE driver
 ** Build the kernel and initrd images using ''diskimage-builder'' ([[#Build_kernel_and_ram-disk|Example]])
-** Load these images into glance ([[#Load_kernel_and_ram-disk|Example]])
+** Load these images into Glance ([[#Load_kernel_and_ram-disk|Example]])
 ** If you plan to use another driver, prepare the appropriate image(s) for that driver instead.
 * Begin maintenance period, preventing user access to Nova, which will last until the migration is complete.
-* Stop Nova compute and scheduler services ([[#Stop_services|Example]])
+* Stop Nova's compute and scheduler services ([[#Stop_services|Example]])
-* Update nova-compute configuration  (nova.conf)
+* Update nova-compute configuration to use Ironic:
-** '''Change''' settings ([[#Changes|Example]])
+** follow steps 1 and 2 of [http://docs.openstack.org/developer/ironic/deploy/install-guide.html#configure-compute-service-to-use-the-bare-metal-service Configure Compute Service to use the Bare Metal Service] to change some settings and add an [ironic] section
-** '''Add''' an [ironic] section ([[#Ironic_section|Example]])
+** Remove the entire [baremetal] section
-** '''Remove''' the entire [baremetal] section
+* Update Nova baremetal's flavor metadata to reference Ironic's deploy kernel & ramdisk  ([[#Update_flavor|Example]])
-* Update flavor metadata to reference Ironic's deploy kernel & ramdisk  ([[#Update_flavor|Example]])
+* Run the [[#https://github.com/openstack/ironic/blob/master/ironic/migrate_nova/migrate_db.py|migrate_db]] script  ([[#Run_migrate_script|Example]])
-* Run [[#https://github.com/openstack/ironic/blob/master/ironic/migrate_nova/migrate_db.py|migrate_db]] script  ([[#Run_migrate_script|Example]])
 * Start Ironic services  ([[#Start_Ironic_services|Example]])
 * Ensure proper service start-up ([[#Ensure_service_start-up|Example]])
-* (Re)start Nova compute and scheduler services ([[#Start_Nova_services|Example]])
+* (Re)start Nova's compute and scheduler services ([[#Start_Nova_services|Example]])
 * End maintenance period
@@ Line 45: / Line 44: @@
 Use [https://github.com/openstack/diskimage-builder diskimage-builder] to create the Ironic deploy kernel and ram-disk. For example:
    diskimage-builder/bin/ramdisk-image-create -a $NODE_ARCH $NODE_DIST deploy-ironic -o deploy-ironic $DIB_ELEMENTS
+===== Load kernel and ram-disk =====
+Load the Ironic deploy kernel and ramdisk built above into Glance.
+  dkid=$(glance image-create --name "deploy-ironic-kernel" --public --disk-format aki --file deploy-ironic.kernel | awk ' / id / {print $4}')
+  drid=$(glance image-create --name "deploy-ironic-ramdisk" --public --disk-format ari --file deploy-ironic.initramfs | awk ' / id / {print $4}')
 ===== Stop services =====
    sudo service nova-compute stop
    sudo service nova-scheduler stop
-===== Changes =====
-'''Change''' the current compute_driver and scheduler_host_manager to match the example below:
-  compute_driver = nova.virt.ironic.IronicDriver
-  scheduler_host_manager = nova.scheduler.ironic_host_manager.IronicHostManager
-===== Ironic section =====
-'''Add''' the Ironic section to the bottom of the nova.conf file: (adjusting the sample below to match your settings)
-  [ironic]
-  admin_username = ironic
-  admin_password = password
-  admin_url = http://keystone.host:35357/v2.0
-  admin_tenant_name = service
-===== Load kernel and ram-disk =====
-Load the Ironic deploy kernel and ramdisk built above into glance.
-  dkid=$(glance image-create --name "deploy-ironic-kernel" --public --disk-format aki --file deploy-ironic.kernel | awk ' / id / {print $4}')
-  drid=$(glance image-create --name "deploy-ironic-ramdisk" --public --disk-format ari --file deploy-ironic.initramfs | awk ' / id / {print $4}')
 ===== Update flavor =====
 Update the Nova flavor to reference the deploy-ironic kernel and ram-disk loaded into glance.