Jump to: navigation, search

Difference between revisions of "StarlingX/Installation Guide Virtual Environment/Duplex"

(→‎Provisioning Network Interfaces on Controller-1)
(106 intermediate revisions by 4 users not shown)
Line 1: Line 1:
−
==Simplex==
+
== Preparing Servers ==  
 +
 
 +
=== Bare Metal ===
 +
 
 +
Required Servers:
 +
 
 +
* Combined Servers (Controller + Compute): 2
 +
 
 +
==== Hardware Requirements ====
 +
 
 +
The recommended minimum requirements for the physical servers where StarlingX Duplex will be deployed, include:
 +
 
 +
* ‘Minimum’ Processor:
 +
** Dual-CPU Intel® Xeon® E5 26xx Family (SandyBridge) 8 cores/socket
 +
* Memory: 64 GB
 +
* BIOS:
 +
** Hyper-Threading Tech: Enabled
 +
** Virtualization Technology: Enabled
 +
** VT for Directed I/O: Enabled
 +
** CPU Power and Performance Policy: Performance
 +
** CPU C State Control: Disabled
 +
** Plug & Play BMC Detection: Disabled
 +
* Primary Disk:
 +
** 500 GB SDD or NVMe
 +
* Additional Disks:
 +
** 1 or more 500 GB disks (min. 10K RPM)
 +
* Network Ports
 +
** Management: 10GE
 +
** OAM: 10GE
 +
** Data: n x 10GE
 +
 
 +
=== Virtual Environment ===
 +
 
 +
Run the libvirt qemu setup scripts. Setting up virtualized OAM and Management networks:
 +
<pre><nowiki>
 +
$ bash setup_network.sh
 +
</nowiki></pre>
 +
 
 +
Building xmls for definition of virtual servers:
 +
<pre><nowiki>
 +
$ bash setup_allinone.sh -i <starlingx iso image>
 +
</nowiki></pre>
 +
 
 +
==== Accessing Virtual Server Consoles ====
 +
 
 +
The xml for virtual servers in stx-tools repo, deployment/libvirt, provides both graphical and text consoles.
 +
 
 +
Access the graphical console in virt-manager by right-click on the domain (the server) and selecting "Open".
 +
 
 +
Access the textual console with the command "virsh console $DOMAIN", where DOMAIN is the name of the server shown in virsh.
 +
 
 +
When booting the controller-0 for the first time, both the serial and graphical consoles will present the initial configuration menu for the cluster.  One can select serial or graphical console for controller-0.  For the other nodes however only serial is used, regardless of which option is selected.
 +
 
 +
Open the graphic console on all servers before powering them on to observe the boot device selection and PXI boot progress.  Run "virsh console $DOMAIN" command promptly after power on to see the initial boot sequence which follows the boot device selection.  One has a few seconds to do this.
  
 
==Controller-0 Host Installation==
 
==Controller-0 Host Installation==
  
−
Installing controller-0 involves initializing a host with software and then applying a configuration from the command line. The configured host becomes Controller-0. <br>
+
Installing controller-0 involves initializing a host with software and then applying a bootstrap configuration from the command line. The configured bootstrapped host becomes Controller-0. <br>
 
Procedure:
 
Procedure:
  
−
# Using an ISO image of StarlingX, initialize the controller host via Libvirt/QEMU.
+
# Power on the server that will be controller-0 with the StarlingX ISO on a USB in a bootable USB slot.
 
# Configure the controller using the config_controller script.
 
# Configure the controller using the config_controller script.
  
 
===Initializing Controller-0===
 
===Initializing Controller-0===
−
This section describes how to initialize StarlingX in host Controller-0. Except where noted, all the commands must be executed from a console of the Workstation. Make sure Virtual Machine Manager is open.
+
This section describes how to initialize StarlingX in host Controller-0. Except where noted, all the commands must be executed from a console of the host.
−
 
 
−
Run the libvirt qemu setup script:
 
−
<pre><nowiki>
 
−
$ bash setup_all_in_one.sh
 
−
</nowiki></pre>
 
−
 
 
−
From the KVM/VirtManager window, power on the host to be configured as Controller-0 and show the virtual machine console and details:
 
−
* When the installer is loaded and the installer welcome screen appears in the Controller-0 host, select the type of installation "All-in-one Controller Configuration".
 
−
* Select the "Graphical Console" as the console to use during installation.
 
−
* Select "Standard Security Boot Profile" as the Security Profile.
 
−
* Monitor the initialization until it is complete. When initialization is complete, a reboot is initiated on the Controller-0 host, briefly displays a GNU GRUB screen, and then boots automatically into the StarlingX image.
 
  
 +
Power on the host to be configured as Controller-0, with the StarlingX ISO on a USB in a bootable USB slot. Wait for the console to show the StarlingX ISO booting options:
 +
* '''All-in-one Controller Configuration'''
 +
** When the installer is loaded and the installer welcome screen appears in the Controller-0 host, select the type of installation "All-in-one Controller Configuration".
 +
* '''Graphical Console'''
 +
** Select the "Graphical Console" as the console to use during installation.
 +
* '''Standard Security Boot Profile'''
 +
** Select "Standard Security Boot Profile" as the Security Profile.
 +
<br>
 +
Monitor the initialization. When it is complete, a reboot is initiated on the Controller-0 host, briefly displays a GNU GRUB screen, and then boots automatically into the StarlingX image.
 +
<br>
 
Log into Controller-0 as user wrsroot, with password wrsroot. The first time you log in as wrsroot, you are required to change your password. Enter the current password (wrsroot):
 
Log into Controller-0 as user wrsroot, with password wrsroot. The first time you log in as wrsroot, you are required to change your password. Enter the current password (wrsroot):
 
<pre><nowiki>
 
<pre><nowiki>
Line 43: Line 95:
 
===Configuring Controller-0===
 
===Configuring Controller-0===
  
−
This section describes how to perform the Controller-0 configuration interactively. Except where noted, all the commands must be executed from the console of the active controller (here assumed to be controller-0.
+
This section describes how to perform the Controller-0 configuration interactively just to bootstrap system with minimum critical data. Except where noted, all the commands must be executed from the console of the active controller (here assumed to be controller-0).
  
−
When run interactively, the config_controller script presents a series of prompts for initial configuration of StarlingX. The prompts are grouped by configuration area. To start the script interactively, use the following command, in the System mode select Simplex as the configuration option and accept all the default values for the rest of the options:
+
When run interactively, the config_controller script presents a series of prompts for initial configuration of StarlingX:
 +
* For the Virtual Environment, you can accept all the default values immediately after ‘system date and time’.
 +
* For a Physical Deployment, answer the bootstrap configuration questions with answers applicable to your particular physical setup.
 +
<br>
 +
The script is used to configure the first controller in the StarlingX cluster as controller-0. The prompts are grouped by configuration area. To start the script interactively, use the following command with no parameters:
  
 
<pre><nowiki>
 
<pre><nowiki>
 
controller-0:~$ sudo config_controller
 
controller-0:~$ sudo config_controller
 +
System Configuration
 +
================
 +
Enter ! at any prompt to abort...
 +
...
 
</nowiki></pre>
 
</nowiki></pre>
  
−
The output when config_controller script is run interactively is:
+
Select [y] for System Date and Time:
 +
 
 +
<pre><nowiki>
 +
System date and time:
 +
-----------------------------
 +
 
 +
Is the current date and time correct?  [y/N]: y
 +
</nowiki></pre>
 +
 
 +
Accept all the default values immediately after ‘system date and time’, for System mode choose "duplex":
  
 
<pre><nowiki>
 
<pre><nowiki>
Line 63: Line 132:
 
System mode [duplex-direct]: 2
 
System mode [duplex-direct]: 2
 
...
 
...
−
WARNING: Command should only be run from the console. Continuing with this
 
−
terminal may cause loss of connectivity and configuration failure
 
−
...
 
−
Apply the above configuration? [y/n]: y
 
−
 
 
Applying configuration (this will take several minutes):
 
Applying configuration (this will take several minutes):
  
Line 81: Line 145:
 
Configuration was applied
 
Configuration was applied
  
−
Please complete any out of service comissioning steps with system commands and unlock controller to proceed.
+
Please complete any out of service commissioning steps with system commands and unlock controller to proceed.
 
</nowiki></pre>
 
</nowiki></pre>
 +
 +
After config_controller bootstrap configuration, REST API, CLI and Horizon interfaces are enabled on the controller-0 OAM IP Address.  The remaining installation instructions will use the CLI.
  
 
==Controller-0 Host Provision==
 
==Controller-0 Host Provision==
Line 92: Line 158:
 
</nowiki></pre>
 
</nowiki></pre>
  
−
On Controller-0, create a newshell script file, copy and paste the content below and run:
+
===Configuring Provider Networks at Installation===
 +
 
 +
Set up one provider network of the vlan type, named providernet-a:
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-create providernet-a --type=vlan
 +
[wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-range-create --name providernet-a-range1 --range 100-400 providernet-a
 +
</nowiki></pre>
 +
 
 +
===Providing Data Interfaces on Controller-0===
 +
 
 +
List all interfaces
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-if-list -a controller-0
 +
+--------------------------------------+---------+----------+...+------+--------------+------+---------+------------+..
 +
| uuid                                | name    | class    |...| vlan | ports        | uses | used by | attributes |..
 +
|                                      |        |          |...| id  |              | i/f  | i/f    |            |..
 +
+--------------------------------------+---------+----------+...+------+--------------+------+---------+------------+..
 +
| 49fd8938-e76f-49f1-879e-83c431a9f1af | enp0s3  | platform |...| None | [u'enp0s3']  | []  | []      | MTU=1500  |..
 +
| 8957bb2c-fec3-4e5d-b4ed-78071f9f781c | eth1000 | None    |...| None | [u'eth1000'] | []  | []      | MTU=1500  |..
 +
| bf6f4cad-1022-4dd7-962b-4d7c47d16d54 | eth1001 | None    |...| None | [u'eth1001'] | []  | []      | MTU=1500  |..
 +
| f59b9469-7702-4b46-bad5-683b95f0a1cb | enp0s8  | platform |...| None | [u'enp0s8']  | []  | []      | MTU=1500  |..
 +
+--------------------------------------+---------+----------+...+------+--------------+------+---------+------------+..
 +
</nowiki></pre>
 +
 
 +
Configure the data interfaces
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -c data controller-0 eth1000 -p providernet-a
 +
+------------------+--------------------------------------+
 +
| Property        | Value                                |
 +
+------------------+--------------------------------------+
 +
| ifname          | eth1000                              |
 +
| iftype          | ethernet                            |
 +
| ports            | [u'eth1000']                        |
 +
| providernetworks | providernet-a                       |
 +
| imac            | 08:00:27:c4:ad:3e                    |
 +
| imtu            | 1500                                |
 +
| ifclass          | data                                |
 +
| aemode          | None                                |
 +
| schedpolicy      | None                                |
 +
| txhashpolicy    | None                                |
 +
| uuid            | 8957bb2c-fec3-4e5d-b4ed-78071f9f781c |
 +
| ihost_uuid      | 9c332b27-6f22-433b-bf51-396371ac4608 |
 +
| vlan_id          | None                                |
 +
| uses            | []                                  |
 +
| used_by          | []                                  |
 +
| created_at      | 2018-08-28T12:50:51.820151+00:00    |
 +
| updated_at      | 2018-08-28T14:46:18.333109+00:00    |
 +
| sriov_numvfs    | 0                                    |
 +
| ipv4_mode        | disabled                            |
 +
| ipv6_mode        | disabled                            |
 +
| accelerated      | [True]                              |
 +
+------------------+--------------------------------------+
 +
</nowiki></pre>
 +
 
 +
===Configuring Cinder on Controller Disk===
 +
 
 +
Review the available disk space and capacity and obtain the uuid of the physical disk
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0
 +
+--------------------------------------+-----------+---------+---------+---------+------------+...
 +
| uuid                                | device_no | device_ | device_ | size_mi | available_ |...
 +
|                                      | de        | num    | type    | b      | mib        |...
 +
+--------------------------------------+-----------+---------+---------+---------+------------+...
 +
| 6b42c9dc-f7c0-42f1-a410-6576f5f069f1 | /dev/sda  | 2048    | HDD    | 600000  | 434072    |...
 +
|                                      |          |        |        |        |            |...
 +
|                                      |          |        |        |        |            |...
 +
| 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 | /dev/sdb  | 2064    | HDD    | 16240  | 16237      |...
 +
|                                      |          |        |        |        |            |...
 +
|                                      |          |        |        |        |            |...
 +
| 146195b2-f3d7-42f9-935d-057a53736929 | /dev/sdc  | 2080    | HDD    | 16240  | 16237      |...
 +
|                                      |          |        |        |        |            |...
 +
|                                      |          |        |        |        |            |...
 +
+--------------------------------------+-----------+---------+---------+---------+------------+...
 +
</nowiki></pre>
 +
 
 +
Create the 'cinder-volumes' local volume group
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-0 cinder-volumes
 +
+-----------------+--------------------------------------+
 +
| lvm_vg_name    | cinder-volumes                      |
 +
| vg_state        | adding                              |
 +
| uuid            | 61cb5cd2-171e-4ef7-8228-915d3560cdc3 |
 +
| ihost_uuid      | 9c332b27-6f22-433b-bf51-396371ac4608 |
 +
| lvm_vg_access  | None                                |
 +
| lvm_max_lv      | 0                                    |
 +
| lvm_cur_lv      | 0                                    |
 +
| lvm_max_pv      | 0                                    |
 +
| lvm_cur_pv      | 0                                    |
 +
| lvm_vg_size    | 0.00                                |
 +
| lvm_vg_total_pe | 0                                    |
 +
| lvm_vg_free_pe  | 0                                    |
 +
| created_at      | 2018-08-28T13:45:20.218905+00:00    |
 +
| updated_at      | None                                |
 +
| parameters      | {u'lvm_type': u'thin'}              |
 +
+-----------------+--------------------------------------+
 +
</nowiki></pre>
 +
 
 +
Create a disk partition to add to the volume group
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-0 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 16237 -t lvm_phys_vol                     
 +
+-------------+--------------------------------------------------+
 +
| Property    | Value                                            |
 +
+-------------+--------------------------------------------------+
 +
| device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 |
 +
| device_node | /dev/sdb1                                        |
 +
| type_guid  | ba5eba11-0000-1111-2222-000000000001            |
 +
| type_name  | None                                            |
 +
| start_mib  | None                                            |
 +
| end_mib    | None                                            |
 +
| size_mib    | 16237                                            |
 +
| uuid        | 0494615f-bd79-4490-84b9-dcebbe5f377a            |
 +
| ihost_uuid  | 9c332b27-6f22-433b-bf51-396371ac4608            |
 +
| idisk_uuid  | 534352d8-fec2-4ca5-bda7-0e0abe5a8e17            |
 +
| ipv_uuid    | None                                            |
 +
| status      | Creating                                        |
 +
| created_at  | 2018-08-28T13:45:48.512226+00:00                |
 +
| updated_at  | None                                            |
 +
+-------------+--------------------------------------------------+
 +
</nowiki></pre>
 +
 
 +
Wait for the new partition to be created (i.e. status=Ready)
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-0 --disk  534352d8-fec2-4ca5-bda7-0e0abe5a8e17
 +
+--------------------------------------+...+------------+...+---------------------+----------+--------+
 +
| uuid                                |...| device_nod |...| type_name          | size_mib | status |
 +
|                                      |...| e          |...|                    |          |        |
 +
+--------------------------------------+...+------------+...+---------------------+----------+--------+
 +
| 0494615f-bd79-4490-84b9-dcebbe5f377a |...| /dev/sdb1  |...| LVM Physical Volume | 16237    | Ready  |
 +
|                                      |...|            |...|                    |          |        |
 +
|                                      |...|            |...|                    |          |        |
 +
+--------------------------------------+...+------------+...+---------------------+----------+--------+
 +
</nowiki></pre>
 +
 
 +
Add the partition to the volume group
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-0 cinder-volumes 0494615f-bd79-4490-84b9-dcebbe5f377a
 +
+--------------------------+--------------------------------------------------+
 +
| Property                | Value                                            |
 +
+--------------------------+--------------------------------------------------+
 +
| uuid                    | 9a0ad568-0ace-4d57-9e03-e7a63f609cf2            |
 +
| pv_state                | adding                                          |
 +
| pv_type                  | partition                                        |
 +
| disk_or_part_uuid        | 0494615f-bd79-4490-84b9-dcebbe5f377a            |
 +
| disk_or_part_device_node | /dev/sdb1                                        |
 +
| disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 |
 +
| lvm_pv_name              | /dev/sdb1                                        |
 +
| lvm_vg_name              | cinder-volumes                                  |
 +
| lvm_pv_uuid              | None                                            |
 +
| lvm_pv_size              | 0                                                |
 +
| lvm_pe_total            | 0                                                |
 +
| lvm_pe_alloced          | 0                                                |
 +
| ihost_uuid              | 9c332b27-6f22-433b-bf51-396371ac4608            |
 +
| created_at              | 2018-08-28T13:47:39.450763+00:00                |
 +
| updated_at              | None                                            |
 +
+--------------------------+--------------------------------------------------+
 +
</nowiki></pre>
 +
 
 +
===Adding an LVM Storage Backend at Installation===
 +
 
 +
Ensure requirements are met to add LVM storage
  
−
<source lang="sh">
+
<pre><nowiki>
−
#!/usr/bin/env bash
+
[wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-add lvm -s cinder
  
−
system host-disk-list controller-0
+
WARNING : THIS OPERATION IS NOT REVERSIBLE AND CANNOT BE CANCELLED.
  
−
NODE=controller-0
+
By confirming this operation, the LVM backend will be created.
−
DEVICE=/dev/sdb
 
−
SIZE=$(system host-disk-list $NODE | grep $DEVICE | awk '{print $12}')
 
−
DISK=$(system host-disk-list $NODE | grep $DEVICE | awk '{print $2}')
 
−
# Create a partition for Cinder
 
−
system host-disk-partition-add $NODE $DISK $SIZE -t lvm_phys_vol
 
−
# Create the Volume Group
 
−
system host-lvg-add $NODE cinder-volumes
 
−
# Wait for partition to be created
 
−
while true; do
 
−
    system host-disk-partition-list $NODE --nowrap | grep $DEVICE | grep Ready;
 
−
    if [ $? -eq 0 ]; then
 
−
        break;
 
−
    fi;
 
−
    sleep 1;
 
−
    echo "Waiting for Disk Partition for $DEVICE:$NODE"
 
−
done
 
  
−
PARTITION=$(system host-disk-partition-list $NODE --disk $DISK --nowrap | grep part1 | awk '{print $2}')
+
Please refer to the system admin guide for minimum spec for LVM
−
# Create the PV
+
storage. Set the 'confirmed' field to execute this operation
−
sleep 1
+
for the lvm backend.
−
system host-pv-add $NODE cinder-volumes $PARTITION
+
</nowiki></pre>
−
sleep 1
 
−
#Enable LVM Backend.
 
  
−
system storage-backend-add lvm -s cinder --confirmed
+
Add the LVM storage backend
  
−
#Wait for backend to be configured:
+
<pre><nowiki>
−
echo " This can take a few minutes..."
+
[wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-add lvm -s cinder --confirmed
−
while true; do
 
−
    system storage-backend-list | grep lvm | grep configured;
 
−
    if [ $? -eq 0 ]; then
 
−
        break;
 
−
    else sleep 10;
 
−
    fi;
 
−
    echo "Waiting for backend to be configured"
 
−
done
 
−
system storage-backend-list
 
  
−
# Add provider networks and assign segmentation ranges
+
System configuration has changed.
−
PHYSNET0='providernet-a'
+
Please follow the administrator guide to complete configuring the system.
−
PHYSNET1='providernet-b'
 
−
neutron providernet-create ${PHYSNET0} --type vlan
 
−
neutron providernet-create ${PHYSNET1} --type vlan
 
−
neutron providernet-range-create ${PHYSNET0} --name ${PHYSNET0}-a --range 400-499
 
−
neutron providernet-range-create ${PHYSNET0} --name ${PHYSNET0}-b --range 10-10 --shared
 
−
neutron providernet-range-create ${PHYSNET1} --name ${PHYSNET1}-a --range 500-599
 
  
−
# Create data interfaces
+
+--------------------------------------+------------+---------+-------------+...+----------+--------------+
−
DATA0IF=eth1000
+
| uuid                                | name      | backend | state      |...| services | capabilities |
−
DATA1IF=eth1001
+
+--------------------------------------+------------+---------+-------------+...+----------+--------------+
−
COMPUTE='controller-0'
+
| 6d750a68-115a-4c26-adf4-58d6e358a00d | file-store | file    | configured |...| glance  | {}           |
−
system host-list --nowrap &> /dev/null && NOWRAP="--nowrap"
+
| e2697426-2d79-4a83-beb7-2eafa9ceaee5 | lvm-store  | lvm    | configuring |...| cinder  | {}           |
−
SPL=/tmp/tmp-system-port-list
+
+--------------------------------------+------------+---------+-------------+...+----------+--------------+
−
SPIL=/tmp/tmp-system-host-if-list
+
</nowiki></pre>
−
system host-port-list ${COMPUTE} $NOWRAP > ${SPL}
 
−
system host-if-list -a ${COMPUTE} $NOWRAP > ${SPIL}
 
−
DATA0PCIADDR=$(cat $SPL | grep $DATA0IF |awk '{print $8}')
 
−
DATA1PCIADDR=$(cat $SPL | grep $DATA1IF |awk '{print $8}')
 
−
DATA0PORTUUID=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $2}')
 
−
DATA1PORTUUID=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $2}')
 
−
DATA0PORTNAME=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $4}')
 
−
DATA1PORTNAME=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $4}')
 
−
DATA0IFUUID=$(cat $SPIL | awk -v DATA0PORTNAME=$DATA0PORTNAME '($12 ~ DATA0PORTNAME) {print $2}')
 
−
DATA1IFUUID=$(cat $SPIL | awk -v DATA1PORTNAME=$DATA1PORTNAME '($12 ~ DATA1PORTNAME) {print $2}')
 
−
system host-if-modify -m 1500 -n data0 -p ${PHYSNET0} -nt data ${COMPUTE} ${DATA0IFUUID}
 
−
system host-if-modify -m 1500 -n data1 -p ${PHYSNET1} -nt data ${COMPUTE} ${DATA1IFUUID}
 
  
−
# Add nova local backend
+
Wait for the LVM storage backend to be configured (i.e. state=Configured)  
−
system host-lvg-add ${COMPUTE} nova-local
 
−
ROOT_DISK=$(system host-show ${COMPUTE} | grep rootfs | awk '{print $4}')
 
−
ROOT_DISK_UUID=$(system host-disk-list ${COMPUTE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}')
 
−
PARTITION_SIZE=$(($(system host-disk-list ${COMPUTE} --nowrap  | grep ${ROOT_DISK} | awk '{print $12;}')/2))
 
−
CGTS_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${COMPUTE} ${ROOT_DISK_UUID} ${PARTITION_SIZE})
 
  
−
while true; do
+
<pre><nowiki>
−
    system host-disk-partition-list ${COMPUTE} | grep /dev/sda5 | grep Ready
+
[wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-list
−
    if [ $? -eq 0 ]; then
+
+--------------------------------------+------------+---------+------------+------+----------+--------------+
−
        break;
+
| uuid                                | name      | backend | state      | task | services | capabilities |
−
     else sleep 2;
+
+--------------------------------------+------------+---------+------------+------+----------+--------------+
−
    fi;
+
| 6d750a68-115a-4c26-adf4-58d6e358a00d | file-store | file    | configured | None | glance  | {}          |
−
    echo "Waiting to add disk partition"
+
| e2697426-2d79-4a83-beb7-2eafa9ceaee5 | lvm-store  | lvm     | configured | None | cinder  | {}          |
−
done
+
+--------------------------------------+------------+---------+------------+------+----------+--------------+
−
system host-disk-partition-list ${COMPUTE}
+
</nowiki></pre>
  
−
CGTS_PARTITION_UUID=$(echo ${CGTS_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}')
+
===Configuring VM Local Storage on Controller Disk===
−
sleep 1
 
−
system host-pv-add ${COMPUTE} cgts-vg ${CGTS_PARTITION_UUID}
 
−
sleep 1
 
−
NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${COMPUTE} ${ROOT_DISK_UUID} ${PARTITION_SIZE})
 
  
−
while true; do
+
Review the available disk space and capacity and obtain the uuid of the physical disk  
−
    system host-disk-partition-list ${COMPUTE} | grep /dev/sda6 | grep Ready
 
−
    if [ $? -eq 0 ]; then
 
−
        break;
 
−
    else sleep 2;
 
−
    fi;
 
−
    echo "Waiting to add disk partition"
 
−
done
 
−
system host-disk-partition-list ${COMPUTE}
 
  
−
NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}')
+
<pre><nowiki>
−
system host-pv-add ${COMPUTE} nova-local ${NOVA_PARTITION_UUID}
+
[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0
−
sleep 1
+
+--------------------------------------+-----------+---------+---------+---------+------------+...
−
system host-lvg-modify -b image -s 10240 ${COMPUTE} nova-local
+
| uuid                                | device_no | device_ | device_ | size_mi | available_ |...
−
sleep 10
+
|                                      | de        | num    | type    | b      | mib        |...
 +
+--------------------------------------+-----------+---------+---------+---------+------------+...
 +
| 6b42c9dc-f7c0-42f1-a410-6576f5f069f1 | /dev/sda  | 2048    | HDD    | 600000  | 434072    |...
 +
|                                      |          |        |        |        |            |...
 +
|                                      |          |        |        |        |            |...
 +
| 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 | /dev/sdb  | 2064    | HDD    | 16240  | 0          |...
 +
|                                      |          |        |        |        |            |...
 +
|                                      |          |        |        |        |           |...
 +
| 146195b2-f3d7-42f9-935d-057a53736929 | /dev/sdc  | 2080    | HDD    | 16240  | 16237      |...
 +
|                                      |          |        |        |        |            |...
 +
|                                      |          |        |        |        |            |...
 +
+--------------------------------------+-----------+---------+---------+---------+------------+...
 +
</nowiki></pre>
  
−
### This will result in a reboot.
+
Create the 'noval-local' volume group
−
system host-unlock controller-0
+
 
−
echo " Watch CONSOLE to see progress. You will see things like "
+
<pre><nowiki>
−
echo "   Applying manifest 127.168.204.3_patching.pp..."
+
[wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-0 nova-local
−
echo "    [DONE]"
+
+-----------------+-------------------------------------------------------------------+
−
echo " Tailing /var/log/platform.log until reboot..."
+
| Property        | Value                                                            |
−
tail -f /var/log/platform.log
+
+-----------------+-------------------------------------------------------------------+
−
</source>
+
| lvm_vg_name    | nova-local                                                        |
 +
| vg_state        | adding                                                            |
 +
| uuid            | 517d313e-8aa0-4b4d-92e6-774b9085f336                              |
 +
| ihost_uuid      | 9c332b27-6f22-433b-bf51-396371ac4608                              |
 +
| lvm_vg_access  | None                                                              |
 +
| lvm_max_lv      | 0                                                                |
 +
| lvm_cur_lv      | 0                                                                |
 +
| lvm_max_pv      | 0                                                                |
 +
| lvm_cur_pv      | 0                                                                |
 +
| lvm_vg_size    | 0.00                                                              |
 +
| lvm_vg_total_pe | 0                                                                |
 +
| lvm_vg_free_pe  | 0                                                                |
 +
| created_at      | 2018-08-28T14:02:58.486716+00:00                                  |
 +
| updated_at      | None                                                              |
 +
| parameters      | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} |
 +
+-----------------+-------------------------------------------------------------------+
 +
</nowiki></pre>
 +
 
 +
Create a disk partition to add to the volume group
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-0 146195b2-f3d7-42f9-935d-057a53736929 16237 -t lvm_phys_vol
 +
+-------------+--------------------------------------------------+
 +
| Property    | Value                                            |
 +
+-------------+--------------------------------------------------+
 +
| device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 |
 +
| device_node | /dev/sdc1                                        |
 +
| type_guid  | ba5eba11-0000-1111-2222-000000000001            |
 +
| type_name  | None                                            |
 +
| start_mib  | None                                            |
 +
| end_mib    | None                                            |
 +
| size_mib    | 16237                                            |
 +
| uuid        | 009ce3b1-ed07-46e9-9560-9d2371676748            |
 +
| ihost_uuid  | 9c332b27-6f22-433b-bf51-396371ac4608            |
 +
| idisk_uuid  | 146195b2-f3d7-42f9-935d-057a53736929            |
 +
| ipv_uuid    | None                                            |
 +
| status      | Creating                                        |
 +
| created_at  | 2018-08-28T14:04:29.714030+00:00                |
 +
| updated_at  | None                                            |
 +
+-------------+--------------------------------------------------+
 +
</nowiki></pre>
 +
 
 +
Wait for the new partition to be created (i.e. status=Ready)
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-0 --disk 146195b2-f3d7-42f9-935d-057a53736929
 +
+--------------------------------------+...+------------+...+---------------------+----------+--------+
 +
| uuid                                |...| device_nod |...| type_name          | size_mib | status |
 +
|                                      |...| e          |...|                    |          |        |
 +
+--------------------------------------+...+------------+...+---------------------+----------+--------+
 +
| 009ce3b1-ed07-46e9-9560-9d2371676748 |...| /dev/sdc1  |...| LVM Physical Volume | 16237   | Ready  |
 +
|                                      |...|            |...|                    |          |        |
 +
|                                      |...|            |...|                    |          |        |
 +
+--------------------------------------+...+------------+...+---------------------+----------+--------+
 +
</nowiki></pre>
 +
 
 +
Add the partition to the volume group
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-0 nova-local 009ce3b1-ed07-46e9-9560-9d2371676748
 +
+--------------------------+--------------------------------------------------+
 +
| Property                | Value                                            |
 +
+--------------------------+--------------------------------------------------+
 +
| uuid                    | 830c9dc8-c71a-4cb2-83be-c4d955ef4f6b            |
 +
| pv_state                | adding                                          |
 +
| pv_type                  | partition                                        |
 +
| disk_or_part_uuid        | 009ce3b1-ed07-46e9-9560-9d2371676748            |
 +
| disk_or_part_device_node | /dev/sdc1                                        |
 +
| disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 |
 +
| lvm_pv_name              | /dev/sdc1                                        |
 +
| lvm_vg_name              | nova-local                                      |
 +
| lvm_pv_uuid              | None                                            |
 +
| lvm_pv_size              | 0                                                |
 +
| lvm_pe_total            | 0                                                |
 +
| lvm_pe_alloced          | 0                                                |
 +
| ihost_uuid              | 9c332b27-6f22-433b-bf51-396371ac4608            |
 +
| created_at              | 2018-08-28T14:06:05.705546+00:00                |
 +
| updated_at              | None                                            |
 +
+--------------------------+--------------------------------------------------+
 +
[wrsroot@controller-0 ~(keystone_admin)]$
 +
</nowiki></pre>
 +
 
 +
===Unlocking Controller-0===
 +
 
 +
You must unlock controller-0 so that you can use it to install Controller-1. Use the system host-unlock command:
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-0
 +
</nowiki></pre>
  
 
The host is rebooted. During the reboot, the command line is unavailable, and any ssh connections are dropped. To monitor the progress of the reboot, use the controller-0 console.
 
The host is rebooted. During the reboot, the command line is unavailable, and any ssh connections are dropped. To monitor the progress of the reboot, use the controller-0 console.
Line 225: Line 496:
 
</nowiki></pre>
 
</nowiki></pre>
  
−
Verify that the Titanium Cloud controller services are running:
+
Verify that the controller-0 services are running:
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system service-list
 +
+-----+-------------------------------+--------------+----------------+
 +
| id  | service_name                  | hostname    | state          |
 +
+-----+-------------------------------+--------------+----------------+
 +
...
 +
| 1  | oam-ip                        | controller-0 | enabled-active |
 +
| 2  | management-ip                | controller-0 | enabled-active |
 +
...
 +
+-----+-------------------------------+--------------+----------------+
 +
</nowiki></pre>
 +
 
 +
Verify that controller-0 has controller and compute subfunctions
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-show 1 | grep subfunctions
 +
| subfunctions        | controller,compute                        |
 +
</nowiki></pre>
 +
 
 +
Verify that controller-0 is unlocked, enabled, and available:
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-list
 +
+----+--------------+-------------+----------------+-------------+--------------+
 +
| id | hostname    | personality | administrative | operational | availability |
 +
+----+--------------+-------------+----------------+-------------+--------------+
 +
| 1  | controller-0 | controller  | unlocked      | enabled    | available    |
 +
+----+--------------+-------------+----------------+-------------+--------------+
 +
</nowiki></pre>
 +
 
 +
==Controller-1 Host Installation==
 +
 
 +
After initializing and configuring controller-0, you can add and configure a backup controller controller-1.
 +
 
 +
===Initializing Controller-1 Host===
 +
 
 +
==== Bare Metal ====
 +
 
 +
Power on Controller-1 node
 +
 
 +
==== Virtual Environment ====
 +
 
 +
Start Controller-1 in the workstation:
 +
 
 +
<pre><nowiki>
 +
$ sudo virsh start controller-1
 +
</nowiki></pre>
 +
 
 +
In Controller-1 console you will see:
 +
 
 +
<pre><nowiki>
 +
Waiting for this node to be configured.
 +
 
 +
Please configure the personality for this node from the
 +
controller node in order to proceed.
 +
</nowiki></pre>
 +
 
 +
===Updating Controller-1 Host Host Name and Personality===
 +
 
 +
On Controller-0, acquire Keystone administrative privileges:
 +
 
 +
<pre><nowiki>
 +
controller-0:~$ source /etc/nova/openrc
 +
</nowiki></pre>
 +
 
 +
Wait for Controller-0 to discover new host, list the host until new UNKNOWN host shows up in table:
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-list
 +
+----+--------------+-------------+----------------+-------------+--------------+
 +
| id | hostname    | personality | administrative | operational | availability |
 +
+----+--------------+-------------+----------------+-------------+--------------+
 +
| 1  | controller-0 | controller  | unlocked      | enabled    | available    |
 +
| 2  | None        | None        | locked        | disabled    | offline      |
 +
+----+--------------+-------------+----------------+-------------+--------------+
 +
</nowiki></pre>
 +
 
 +
Use the system host-update to update Contoller-1 host personality attribute:
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-update 2 personality=controller hostname=controller-1
 +
+---------------------+--------------------------------------+
 +
| Property            | Value                                |
 +
+---------------------+--------------------------------------+
 +
| action              | none                                |
 +
| administrative      | locked                              |
 +
| availability        | offline                              |
 +
| bm_ip              | None                                |
 +
| bm_type            | None                                |
 +
| bm_username        | None                                |
 +
| boot_device        | sda                                  |
 +
| capabilities        | {}                                  |
 +
| config_applied      | None                                |
 +
| config_status      | None                                |
 +
| config_target      | None                                |
 +
| console            | ttyS0,115200                        |
 +
| created_at          | 2018-08-28T15:09:13.812906+00:00    |
 +
| hostname            | controller-1                        |
 +
| id                  | 2                                    |
 +
| install_output      | text                                |
 +
| install_state      | None                                |
 +
| install_state_info  | None                                |
 +
| invprovision        | None                                |
 +
| location            | {}                                  |
 +
| mgmt_ip            | 192.168.204.4                        |
 +
| mgmt_mac            | 08:00:27:3d:e2:66                    |
 +
| operational        | disabled                            |
 +
| personality        | controller                          |
 +
| reserved            | False                                |
 +
| rootfs_device      | sda                                  |
 +
| serialid            | None                                |
 +
| software_load      | 18.03                                |
 +
| subfunction_avail  | not-installed                        |
 +
| subfunction_oper    | disabled                            |
 +
| subfunctions        | controller,compute                  |
 +
| task                | None                                |
 +
| tboot              | false                                |
 +
| ttys_dcd            | None                                |
 +
| updated_at          | None                                |
 +
| uptime              | 0                                    |
 +
| uuid                | be0f35f7-31d9-4145-bf6a-0c2556cf398c |
 +
| vim_progress_status | None                                |
 +
+---------------------+--------------------------------------+
 +
</nowiki></pre>
 +
 
 +
===Monitoring Controller-1 Host===
 +
 
 +
On Controller-0, you can monitor the installation progress by running the system host-show command for the host periodically. Progress is shown in the install_state field.
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-show controller-1 | grep install
 +
| install_output      | text                                |
 +
| install_state      | booting                              |
 +
| install_state_info  | None                                |
 +
</nowiki></pre>
 +
 
 +
Wait while the Controller-1 is configured and rebooted. Up to 20 minutes may be required for a reboot, depending on hardware. When the reboot is complete, the Controller-1 is reported as Locked, Disabled, and Online.
 +
 
 +
===Listing Controller-1 Host===
 +
 
 +
Once Controller-1 has been installed, configured and rebooted, on Controller-0 list the hosts:
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-list
 +
+----+--------------+-------------+----------------+-------------+--------------+
 +
| id | hostname    | personality | administrative | operational | availability |
 +
+----+--------------+-------------+----------------+-------------+--------------+
 +
| 1  | controller-0 | controller  | unlocked      | enabled    | available    |
 +
| 2  | controller-1 | controller  | locked        | disabled    | online      |
 +
+----+--------------+-------------+----------------+-------------+--------------+
 +
</nowiki></pre>
 +
 
 +
==Controller-1 Provisioning==
 +
 
 +
On Controller-0, list hosts
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-list
 +
+----+--------------+-------------+----------------+-------------+--------------+
 +
| id | hostname    | personality | administrative | operational | availability |
 +
+----+--------------+-------------+----------------+-------------+--------------+
 +
...
 +
| 2  | controller-1 | controller  | locked        | disabled    | online      |
 +
+----+--------------+-------------+----------------+-------------+--------------+
 +
</nowiki></pre>
 +
 
 +
===Provisioning Network Interfaces on Controller-1===
 +
 
 +
In order to list out hardware port names, types, pci-addresses that have been discovered:
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-port-list controller-1
 +
</nowiki></pre>
 +
 
 +
Provision the Controller-1 oam interface
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -n ens6 -c platform --networks oam controller-1 ens6
 +
+------------------+--------------------------------------+
 +
| Property        | Value                                |
 +
+------------------+--------------------------------------+
 +
| ifname          | ens6                                |
 +
| iftype          | ethernet                            |
 +
| ports            | [u'ens6']                            |
 +
| providernetworks | None                                |
 +
| imac            | 08:00:27:ba:3c:38                    |
 +
| imtu            | 1500                                |
 +
| ifclass          | oam                                  |
 +
| aemode          | None                                |
 +
| schedpolicy      | None                                |
 +
| txhashpolicy    | None                                |
 +
| uuid            | 160bfede-0950-42ba-acef-d0eb31e7fc49 |
 +
| ihost_uuid      | be0f35f7-31d9-4145-bf6a-0c2556cf398c |
 +
| vlan_id          | None                                |
 +
| uses            | []                                  |
 +
| used_by          | []                                  |
 +
| created_at      | 2018-08-28T15:59:10.701772+00:00    |
 +
| updated_at      | 2018-08-29T05:44:38.585642+00:00    |
 +
| sriov_numvfs    | 0                                    |
 +
| ipv4_mode        | static                              |
 +
| ipv6_mode        | disabled                            |
 +
| accelerated      | [False]                              |
 +
+------------------+--------------------------------------+
 +
</nowiki></pre>
 +
 
 +
===Providing Data Interfaces on Controller-1===
 +
 
 +
List all interfaces
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-if-list -a controller-1
 +
+--------------------------------------+---------+---------+...+------+--------------+------+------+------------+..
 +
| uuid                                | name    | network |...| vlan | ports        | uses | used | attributes |..
 +
|                                      |        | type    |...| id  |              | i/f  | by  |            |..
 +
|                                      |        |        |...|      |              |      | i/f  |            |..
 +
+--------------------------------------+---------+---------+...+------+--------------+------+------+------------+..
 +
| 160bfede-0950-42ba-acef-d0eb31e7fc49 | ens6    | oam    |...| None | [u'enp0s3']  | []  | []  | MTU=1500  |..
 +
| 37b3abcd-bcbe-44d5-b2fb-759c34efec89 | eth1001 | None    |...| None | [u'eth1001'] | []  | []  | MTU=1500  |..
 +
| 81d66feb-b23c-4533-bd4b-129f9b75ecd6 | mgmt0  | mgmt    |...| None | [u'enp0s8']  | []  | []  | MTU=1500  |..
 +
| e78ad9a9-e74d-4c6c-9de8-0e41aad8d7b7 | eth1000 | None    |...| None | [u'eth1000'] | []  | []  | MTU=1500  |..
 +
+--------------------------------------+---------+---------+...+------+--------------+------+------+------------+..
 +
</nowiki></pre>
 +
 
 +
Configure the data interfaces
  
 
<pre><nowiki>
 
<pre><nowiki>
−
[wrsroot@controller-0 ~(keystone_admin)]$ nova service-list
+
[wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -p providernet-a -c data controller-1 eth1000
−
+--------------------------------------+------------------+--------------+----------+---------+-------+ ...
+
+------------------+--------------------------------------+
−
| Id                                  | Binary          | Host         | Zone    | Status  | State | ...
+
| Property         | Value                                |
−
+--------------------------------------+------------------+--------------+----------+---------+-------+ ...
+
+------------------+--------------------------------------+
−
| d7cdfaf0-9394-4053-b752-d609c837b729 | nova-conductor  | controller-0 | internal | enabled | up    | ...
+
| ifname          | eth1000                              |
−
| 692c2659-7188-42ec-9ad1-c88ac3020dc6 | nova-scheduler  | controller-0 | internal | enabled | up    | ...
+
| networktype      | data                                |
−
| 5c7c9aad-696f-4dd1-a497-464bdd525e0c | nova-consoleauth | controller-0 | internal | enabled | up    | ...
+
| iftype          | ethernet                            |
−
+--------------------------------------+------------------+--------------+----------+---------+-------+ ...
+
| ports            | [u'eth1000']                        |
 +
| providernetworks | providernet-a                        |
 +
| imac            | 08:00:27:39:61:6e                    |
 +
| imtu            | 1500                                |
 +
| aemode          | None                                |
 +
| schedpolicy      | None                                |
 +
| txhashpolicy    | None                                |
 +
| uuid            | e78ad9a9-e74d-4c6c-9de8-0e41aad8d7b7 |
 +
| ihost_uuid      | be0f35f7-31d9-4145-bf6a-0c2556cf398c |
 +
| vlan_id          | None                                |
 +
| uses            | []                                  |
 +
| used_by          | []                                  |
 +
| created_at      | 2018-08-28T15:59:17.667899+00:00    |
 +
| updated_at      | 2018-08-29T06:04:55.116653+00:00    |
 +
| sriov_numvfs    | 0                                   |
 +
| ipv4_mode        | disabled                            |
 +
| ipv6_mode        | disabled                            |
 +
| accelerated      | [True]                              |
 +
+------------------+--------------------------------------+
 
</nowiki></pre>
 
</nowiki></pre>
  
−
If one of the listed services is down please reenable
+
===Provisioning Storage on Controller-1===
  
−
<source lang="sh">
+
Review the available disk space and capacity and obtain the uuid of the physical disk
−
#!/usr/bin/env bash
 
  
−
source /etc/nova/openrc
+
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1
 +
+--------------------------------------+-------------+------------+-------------+----------+---------------+...
 +
| uuid                                | device_node | device_num | device_type | size_mib | available_mib |...
 +
+--------------------------------------+-------------+------------+-------------+----------+---------------+...
 +
| 2dcc3d33-ba43-4b5a-b0b0-2b5a2e5737b7 | /dev/sda    | 2048      | HDD        | 600000  | 434072        |...
 +
| f53437c6-77e3-4185-9453-67eaa8b461b1 | /dev/sdb    | 2064      | HDD        | 16240    | 16237        |...
 +
| 623bbfc0-2b38-432a-acf4-a28db6066cce | /dev/sdc    | 2080      | HDD        | 16240    | 16237        |...
 +
+--------------------------------------+-------------+------------+-------------+----------+---------------+...
 +
</nowiki></pre>
  
−
ALL_DISABLED_SERVICES=`nova service-list | grep disabled | awk -F "|" '{print $2}'`
+
Assign Cinder storage to the physical disk
−
for service in $ALL_DISABLED_SERVICES; do
 
−
    nova service-force-down --unset ${service}
 
−
    nova service-enable ${service}
 
−
done
 
−
</source>
 
  
−
Verify that controller-0 is unlocked, enabled, and available:
+
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-1 cinder-volumes
 +
+-----------------+--------------------------------------+
 +
| Property        | Value                                |
 +
+-----------------+--------------------------------------+
 +
| lvm_vg_name    | cinder-volumes                      |
 +
| vg_state        | adding                              |
 +
| uuid            | 3ccadc5c-e4f7-4b04-8403-af5d2ecef96d |
 +
| ihost_uuid      | be0f35f7-31d9-4145-bf6a-0c2556cf398c |
 +
| lvm_vg_access  | None                                |
 +
| lvm_max_lv      | 0                                    |
 +
| lvm_cur_lv      | 0                                    |
 +
| lvm_max_pv      | 0                                    |
 +
| lvm_cur_pv      | 0                                    |
 +
| lvm_vg_size    | 0.00                                |
 +
| lvm_vg_total_pe | 0                                    |
 +
| lvm_vg_free_pe  | 0                                    |
 +
| created_at      | 2018-08-29T05:51:13.965883+00:00    |
 +
| updated_at      | None                                |
 +
| parameters      | {u'lvm_type': u'thin'}              |
 +
+-----------------+--------------------------------------+
 +
</nowiki></pre>
 +
 
 +
Create a disk partition to add to the volume group based on uuid of the physical disk
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-1 f53437c6-77e3-4185-9453-67eaa8b461b1 16237 -t lvm_phys_vol
 +
+-------------+--------------------------------------------------+
 +
| Property    | Value                                            |
 +
+-------------+--------------------------------------------------+
 +
| device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 |
 +
| device_node | /dev/sdb1                                        |
 +
| type_guid  | ba5eba11-0000-1111-2222-000000000001            |
 +
| type_name  | None                                            |
 +
| start_mib  | None                                            |
 +
| end_mib    | None                                            |
 +
| size_mib    | 16237                                            |
 +
| uuid        | 7a41aab0-6695-4d16-9003-73238adda75b            |
 +
| ihost_uuid  | be0f35f7-31d9-4145-bf6a-0c2556cf398c            |
 +
| idisk_uuid  | f53437c6-77e3-4185-9453-67eaa8b461b1            |
 +
| ipv_uuid    | None                                            |
 +
| status      | Creating (on unlock)                            |
 +
| created_at  | 2018-08-29T05:54:23.326612+00:00                |
 +
| updated_at  | None                                            |
 +
+-------------+--------------------------------------------------+
 +
</nowiki></pre>
 +
 
 +
Wait for the new partition to be created (i.e. status=Ready)
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-1 --disk f53437c6-77e3-4185-9453-67eaa8b461b1
 +
+--------------------------------------+...+-------------+...+-----------+----------+----------------------+
 +
| uuid                                |...| device_node |...| type_name | size_mib | status              |
 +
+--------------------------------------+...+-------------+...+-----------+----------+----------------------+
 +
| 7a41aab0-6695-4d16-9003-73238adda75b |...| /dev/sdb1  |...| None      | 16237    | Creating (on unlock) |
 +
+--------------------------------------+...+-------------+...+-----------+----------+----------------------+
 +
</nowiki></pre>
 +
 
 +
Add the partition to the volume group
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-1 cinder-volumes 7a41aab0-6695-4d16-9003-73238adda75b
 +
+--------------------------+--------------------------------------------------+
 +
| Property                | Value                                            |
 +
+--------------------------+--------------------------------------------------+
 +
| uuid                    | 11628316-56ab-41b2-b2be-a5d72667a5e9            |
 +
| pv_state                | adding                                          |
 +
| pv_type                  | partition                                        |
 +
| disk_or_part_uuid        | 7a41aab0-6695-4d16-9003-73238adda75b            |
 +
| disk_or_part_device_node | /dev/sdb1                                        |
 +
| disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 |
 +
| lvm_pv_name              | /dev/sdb1                                        |
 +
| lvm_vg_name              | cinder-volumes                                  |
 +
| lvm_pv_uuid              | None                                            |
 +
| lvm_pv_size              | 0                                                |
 +
| lvm_pe_total            | 0                                                |
 +
| lvm_pe_alloced          | 0                                                |
 +
| ihost_uuid              | be0f35f7-31d9-4145-bf6a-0c2556cf398c            |
 +
| created_at              | 2018-08-29T06:25:54.550430+00:00                |
 +
| updated_at              | None                                            |
 +
+--------------------------+--------------------------------------------------+
 +
</nowiki></pre>
 +
 
 +
===Configuring VM Local Storage on Controller Disk===
 +
 
 +
Review the available disk space and capacity and obtain the uuid of the physical disk
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1
 +
+--------------------------------------+-------------+------------+-------------+----------+---------------+...
 +
| uuid                                | device_node | device_num | device_type | size_mib | available_mib |...
 +
+--------------------------------------+-------------+------------+-------------+----------+---------------+...
 +
| 2dcc3d33-ba43-4b5a-b0b0-2b5a2e5737b7 | /dev/sda    | 2048      | HDD        | 600000  | 434072        |...
 +
| f53437c6-77e3-4185-9453-67eaa8b461b1 | /dev/sdb    | 2064      | HDD        | 16240    | 16237        |...
 +
| 623bbfc0-2b38-432a-acf4-a28db6066cce | /dev/sdc    | 2080      | HDD        | 16240    | 16237        |...
 +
+--------------------------------------+-------------+------------+-------------+----------+---------------+...
 +
</nowiki></pre>
 +
 
 +
Create the 'cinder-volumes' local volume group
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-1 nova-local
 +
+-----------------+-------------------------------------------------------------------+
 +
| Property        | Value                                                            |
 +
+-----------------+-------------------------------------------------------------------+
 +
| lvm_vg_name    | nova-local                                                        |
 +
| vg_state        | adding                                                            |
 +
| uuid            | d205f839-0930-4e77-9f10-97089150f812                              |
 +
| ihost_uuid      | be0f35f7-31d9-4145-bf6a-0c2556cf398c                              |
 +
| lvm_vg_access  | None                                                              |
 +
| lvm_max_lv      | 0                                                                |
 +
| lvm_cur_lv      | 0                                                                |
 +
| lvm_max_pv      | 0                                                                |
 +
| lvm_cur_pv      | 0                                                                |
 +
| lvm_vg_size    | 0.00                                                              |
 +
| lvm_vg_total_pe | 0                                                                |
 +
| lvm_vg_free_pe  | 0                                                                |
 +
| created_at      | 2018-08-29T06:28:28.226907+00:00                                  |
 +
| updated_at      | None                                                              |
 +
| parameters      | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} |
 +
+-----------------+-------------------------------------------------------------------+
 +
 
 +
</nowiki></pre>
 +
 
 +
Create a disk partition to add to the volume group
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-1 623bbfc0-2b38-432a-acf4-a28db6066cce 16237 -t lvm_phys_vol
 +
+-------------+--------------------------------------------------+
 +
| Property    | Value                                            |
 +
+-------------+--------------------------------------------------+
 +
| device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 |
 +
| device_node | /dev/sdc1                                        |
 +
| type_guid  | ba5eba11-0000-1111-2222-000000000001            |
 +
| type_name  | None                                            |
 +
| start_mib  | None                                            |
 +
| end_mib    | None                                            |
 +
| size_mib    | 16237                                            |
 +
| uuid        | f7bc6095-9375-49fe-83c7-12601c202376            |
 +
| ihost_uuid  | be0f35f7-31d9-4145-bf6a-0c2556cf398c            |
 +
| idisk_uuid  | 623bbfc0-2b38-432a-acf4-a28db6066cce            |
 +
| ipv_uuid    | None                                            |
 +
| status      | Creating (on unlock)                            |
 +
| created_at  | 2018-08-29T06:30:33.176428+00:00                |
 +
| updated_at  | None                                            |
 +
+-------------+--------------------------------------------------+
 +
</nowiki></pre>
 +
 
 +
Wait for the new partition to be created (i.e. status=Ready)
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-1
 +
+--------------------------------------+...+-------------+...+-----------+----------+----------------------+
 +
| uuid                                |...| device_node |...| type_name | size_mib | status              |
 +
+--------------------------------------+...+-------------+...+-----------+----------+----------------------+
 +
| 7a41aab0-6695-4d16-9003-73238adda75b |...| /dev/sdb1  |...| None      | 16237    | Creating (on unlock) |
 +
| f7bc6095-9375-49fe-83c7-12601c202376 |...| /dev/sdc1  |...| None      | 16237    | Creating (on unlock) |
 +
+--------------------------------------+...+-------------+...+-----------+----------+----------------------+
 +
[wrsroot@controller-0 ~(keystone_admin)]$
 +
 
 +
</nowiki></pre>
 +
 
 +
Add the partition to the volume group
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-1 nova-local f7bc6095-9375-49fe-83c7-12601c202376
 +
+--------------------------+--------------------------------------------------+
 +
| Property                | Value                                            |
 +
+--------------------------+--------------------------------------------------+
 +
| uuid                    | 94003d64-4e1b-483a-8a6c-0fc4b1e0e272            |
 +
| pv_state                | adding                                          |
 +
| pv_type                  | partition                                        |
 +
| disk_or_part_uuid        | f7bc6095-9375-49fe-83c7-12601c202376            |
 +
| disk_or_part_device_node | /dev/sdc1                                        |
 +
| disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 |
 +
| lvm_pv_name              | /dev/sdc1                                        |
 +
| lvm_vg_name              | nova-local                                      |
 +
| lvm_pv_uuid              | None                                            |
 +
| lvm_pv_size              | 0                                                |
 +
| lvm_pe_total            | 0                                                |
 +
| lvm_pe_alloced          | 0                                                |
 +
| ihost_uuid              | be0f35f7-31d9-4145-bf6a-0c2556cf398c            |
 +
| created_at              | 2018-08-29T06:33:08.625604+00:00                |
 +
| updated_at              | None                                            |
 +
+--------------------------+--------------------------------------------------+
 +
</nowiki></pre>
 +
 
 +
===Unlocking Controller-1===
 +
 
 +
Unlock Controller-1
 +
 
 +
<pre><nowiki>
 +
[wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-1
 +
</nowiki></pre>
 +
 
 +
Wait while the Controller-1 is rebooted. Up to 10 minutes may be required for a reboot, depending on hardware.
 +
 
 +
REMARK: Controller-1 will remain in 'degraded' state until data-syncing is complete. The duration is dependant on the virtualization host's configuration - i.e., the number and configuration of physical disks used to host the nodes' virtual disks. Also, the management network is expected to have link capacity of 10000 (1000 is not supported due to excessive data-sync time). Use 'fm alarm-list' to confirm status.
  
 
<pre><nowiki>
 
<pre><nowiki>
Line 260: Line 974:
 
+----+--------------+-------------+----------------+-------------+--------------+
 
+----+--------------+-------------+----------------+-------------+--------------+
 
| 1  | controller-0 | controller  | unlocked      | enabled    | available    |
 
| 1  | controller-0 | controller  | unlocked      | enabled    | available    |
 +
| 2  | controller-1 | controller  | unlocked      | enabled    | available    |
 
+----+--------------+-------------+----------------+-------------+--------------+
 
+----+--------------+-------------+----------------+-------------+--------------+
 
</nowiki></pre>
 
</nowiki></pre>
 +
 +
===System Alarm List===
 +
 +
When all nodes are Unlocked, Enabled and Available: check 'fm alarm-list' for issues.
 +
<br>
 +
<br>
 +
Your StarlingX deployment is now up and running with 2x HA Controllers with Cinder Storage and all OpenStack services up and running. You can now proceed with standard OpenStack APIs, CLIs and/or Horizon to load Glance Images, configure Nova Flavors, configure Neutron networks and launch Nova Virtual Machines.

Revision as of 17:43, 11 October 2018

Preparing Servers

Bare Metal

Required Servers:

  • Combined Servers (Controller + Compute): 2

Hardware Requirements

The recommended minimum requirements for the physical servers where StarlingX Duplex will be deployed, include:

  • ‘Minimum’ Processor:
    • Dual-CPU Intel® Xeon® E5 26xx Family (SandyBridge) 8 cores/socket
  • Memory: 64 GB
  • BIOS:
    • Hyper-Threading Tech: Enabled
    • Virtualization Technology: Enabled
    • VT for Directed I/O: Enabled
    • CPU Power and Performance Policy: Performance
    • CPU C State Control: Disabled
    • Plug & Play BMC Detection: Disabled
  • Primary Disk:
    • 500 GB SDD or NVMe
  • Additional Disks:
    • 1 or more 500 GB disks (min. 10K RPM)
  • Network Ports
    • Management: 10GE
    • OAM: 10GE
    • Data: n x 10GE

Virtual Environment

Run the libvirt qemu setup scripts. Setting up virtualized OAM and Management networks: 

$ bash setup_network.sh

Building xmls for definition of virtual servers: 

$ bash setup_allinone.sh -i <starlingx iso image>

Accessing Virtual Server Consoles

The xml for virtual servers in stx-tools repo, deployment/libvirt, provides both graphical and text consoles.

Access the graphical console in virt-manager by right-click on the domain (the server) and selecting "Open".

Access the textual console with the command "virsh console $DOMAIN", where DOMAIN is the name of the server shown in virsh.

When booting the controller-0 for the first time, both the serial and graphical consoles will present the initial configuration menu for the cluster. One can select serial or graphical console for controller-0. For the other nodes however only serial is used, regardless of which option is selected.

Open the graphic console on all servers before powering them on to observe the boot device selection and PXI boot progress. Run "virsh console $DOMAIN" command promptly after power on to see the initial boot sequence which follows the boot device selection. One has a few seconds to do this.

Controller-0 Host Installation

Installing controller-0 involves initializing a host with software and then applying a bootstrap configuration from the command line. The configured bootstrapped host becomes Controller-0.
Procedure:

  1. Power on the server that will be controller-0 with the StarlingX ISO on a USB in a bootable USB slot.
  2. Configure the controller using the config_controller script.

Initializing Controller-0

This section describes how to initialize StarlingX in host Controller-0. Except where noted, all the commands must be executed from a console of the host.

Power on the host to be configured as Controller-0, with the StarlingX ISO on a USB in a bootable USB slot. Wait for the console to show the StarlingX ISO booting options:

  • All-in-one Controller Configuration
    • When the installer is loaded and the installer welcome screen appears in the Controller-0 host, select the type of installation "All-in-one Controller Configuration".
  • Graphical Console
    • Select the "Graphical Console" as the console to use during installation.
  • Standard Security Boot Profile
    • Select "Standard Security Boot Profile" as the Security Profile.


Monitor the initialization. When it is complete, a reboot is initiated on the Controller-0 host, briefly displays a GNU GRUB screen, and then boots automatically into the StarlingX image.
Log into Controller-0 as user wrsroot, with password wrsroot. The first time you log in as wrsroot, you are required to change your password. Enter the current password (wrsroot): 

Changing password for wrsroot.
(current) UNIX Password:

Enter a new password for the wrsroot account: 

New password:

Enter the new password again to confirm it: 

Retype new password:

Controller-0 is initialized with StarlingX, and is ready for configuration.

Configuring Controller-0

This section describes how to perform the Controller-0 configuration interactively just to bootstrap system with minimum critical data. Except where noted, all the commands must be executed from the console of the active controller (here assumed to be controller-0).

When run interactively, the config_controller script presents a series of prompts for initial configuration of StarlingX:

  • For the Virtual Environment, you can accept all the default values immediately after ‘system date and time’.
  • For a Physical Deployment, answer the bootstrap configuration questions with answers applicable to your particular physical setup.


The script is used to configure the first controller in the StarlingX cluster as controller-0. The prompts are grouped by configuration area. To start the script interactively, use the following command with no parameters: 

controller-0:~$ sudo config_controller
System Configuration
================
Enter ! at any prompt to abort...
...

Select [y] for System Date and Time: 

System date and time:
-----------------------------

Is the current date and time correct?  [y/N]: y

Accept all the default values immediately after ‘system date and time’, for System mode choose "duplex": 

...
System mode. Available options are:

1) duplex-direct: two node-redundant configuration. Management and
infrastructure networks are directly connected to peer ports
2) duplex - two node redundant configuration
3) simplex - single node non-redundant configuration
System mode [duplex-direct]: 2
...
Applying configuration (this will take several minutes):

01/08: Creating bootstrap configuration ... DONE
02/08: Applying bootstrap manifest ... DONE
03/08: Persisting local configuration ... DONE
04/08: Populating initial system inventory ... DONE
05:08: Creating system configuration ... DONE
06:08: Applying controller manifest ... DONE
07:08: Finalize controller configuration ... DONE
08:08: Waiting for service activation ... DONE

Configuration was applied

Please complete any out of service commissioning steps with system commands and unlock controller to proceed.

After config_controller bootstrap configuration, REST API, CLI and Horizon interfaces are enabled on the controller-0 OAM IP Address. The remaining installation instructions will use the CLI.

Controller-0 Host Provision

On Controller-0, acquire Keystone administrative privileges: 

controller-0:~$ source /etc/nova/openrc

Configuring Provider Networks at Installation

Set up one provider network of the vlan type, named providernet-a: 

[wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-create providernet-a --type=vlan
[wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-range-create --name providernet-a-range1 --range 100-400 providernet-a

Providing Data Interfaces on Controller-0

List all interfaces 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-if-list -a controller-0
+--------------------------------------+---------+----------+...+------+--------------+------+---------+------------+..
| uuid                                 | name    | class    |...| vlan | ports        | uses | used by | attributes |..
|                                      |         |          |...| id   |              | i/f  | i/f     |            |..
+--------------------------------------+---------+----------+...+------+--------------+------+---------+------------+..
| 49fd8938-e76f-49f1-879e-83c431a9f1af | enp0s3  | platform |...| None | [u'enp0s3']  | []   | []      | MTU=1500   |..
| 8957bb2c-fec3-4e5d-b4ed-78071f9f781c | eth1000 | None     |...| None | [u'eth1000'] | []   | []      | MTU=1500   |..
| bf6f4cad-1022-4dd7-962b-4d7c47d16d54 | eth1001 | None     |...| None | [u'eth1001'] | []   | []      | MTU=1500   |..
| f59b9469-7702-4b46-bad5-683b95f0a1cb | enp0s8  | platform |...| None | [u'enp0s8']  | []   | []      | MTU=1500   |..
+--------------------------------------+---------+----------+...+------+--------------+------+---------+------------+..

Configure the data interfaces 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -c data controller-0 eth1000 -p providernet-a
+------------------+--------------------------------------+
| Property         | Value                                |
+------------------+--------------------------------------+
| ifname           | eth1000                              |
| iftype           | ethernet                             |
| ports            | [u'eth1000']                         |
| providernetworks | providernet-a                        |
| imac             | 08:00:27:c4:ad:3e                    |
| imtu             | 1500                                 |
| ifclass           | data                                 |
| aemode           | None                                 |
| schedpolicy      | None                                 |
| txhashpolicy     | None                                 |
| uuid             | 8957bb2c-fec3-4e5d-b4ed-78071f9f781c |
| ihost_uuid       | 9c332b27-6f22-433b-bf51-396371ac4608 |
| vlan_id          | None                                 |
| uses             | []                                   |
| used_by          | []                                   |
| created_at       | 2018-08-28T12:50:51.820151+00:00     |
| updated_at       | 2018-08-28T14:46:18.333109+00:00     |
| sriov_numvfs     | 0                                    |
| ipv4_mode        | disabled                             |
| ipv6_mode        | disabled                             |
| accelerated      | [True]                               |
+------------------+--------------------------------------+

Configuring Cinder on Controller Disk

Review the available disk space and capacity and obtain the uuid of the physical disk 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0
+--------------------------------------+-----------+---------+---------+---------+------------+...
| uuid                                 | device_no | device_ | device_ | size_mi | available_ |...
|                                      | de        | num     | type    | b       | mib        |...
+--------------------------------------+-----------+---------+---------+---------+------------+...
| 6b42c9dc-f7c0-42f1-a410-6576f5f069f1 | /dev/sda  | 2048    | HDD     | 600000  | 434072     |...
|                                      |           |         |         |         |            |...
|                                      |           |         |         |         |            |...
| 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 | /dev/sdb  | 2064    | HDD     | 16240   | 16237      |...
|                                      |           |         |         |         |            |...
|                                      |           |         |         |         |            |...
| 146195b2-f3d7-42f9-935d-057a53736929 | /dev/sdc  | 2080    | HDD     | 16240   | 16237      |...
|                                      |           |         |         |         |            |...
|                                      |           |         |         |         |            |...
+--------------------------------------+-----------+---------+---------+---------+------------+...

Create the 'cinder-volumes' local volume group 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-0 cinder-volumes
+-----------------+--------------------------------------+
| lvm_vg_name     | cinder-volumes                       |
| vg_state        | adding                               |
| uuid            | 61cb5cd2-171e-4ef7-8228-915d3560cdc3 |
| ihost_uuid      | 9c332b27-6f22-433b-bf51-396371ac4608 |
| lvm_vg_access   | None                                 |
| lvm_max_lv      | 0                                    |
| lvm_cur_lv      | 0                                    |
| lvm_max_pv      | 0                                    |
| lvm_cur_pv      | 0                                    |
| lvm_vg_size     | 0.00                                 |
| lvm_vg_total_pe | 0                                    |
| lvm_vg_free_pe  | 0                                    |
| created_at      | 2018-08-28T13:45:20.218905+00:00     |
| updated_at      | None                                 |
| parameters      | {u'lvm_type': u'thin'}               |
+-----------------+--------------------------------------+

Create a disk partition to add to the volume group 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-0 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 16237 -t lvm_phys_vol                       
+-------------+--------------------------------------------------+
| Property    | Value                                            |
+-------------+--------------------------------------------------+
| device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 |
| device_node | /dev/sdb1                                        |
| type_guid   | ba5eba11-0000-1111-2222-000000000001             |
| type_name   | None                                             |
| start_mib   | None                                             |
| end_mib     | None                                             |
| size_mib    | 16237                                            |
| uuid        | 0494615f-bd79-4490-84b9-dcebbe5f377a             |
| ihost_uuid  | 9c332b27-6f22-433b-bf51-396371ac4608             |
| idisk_uuid  | 534352d8-fec2-4ca5-bda7-0e0abe5a8e17             |
| ipv_uuid    | None                                             |
| status      | Creating                                         |
| created_at  | 2018-08-28T13:45:48.512226+00:00                 |
| updated_at  | None                                             |
+-------------+--------------------------------------------------+

Wait for the new partition to be created (i.e. status=Ready) 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-0 --disk  534352d8-fec2-4ca5-bda7-0e0abe5a8e17
+--------------------------------------+...+------------+...+---------------------+----------+--------+
| uuid                                 |...| device_nod |...| type_name           | size_mib | status |
|                                      |...| e          |...|                     |          |        |
+--------------------------------------+...+------------+...+---------------------+----------+--------+
| 0494615f-bd79-4490-84b9-dcebbe5f377a |...| /dev/sdb1  |...| LVM Physical Volume | 16237    | Ready  |
|                                      |...|            |...|                     |          |        |
|                                      |...|            |...|                     |          |        |
+--------------------------------------+...+------------+...+---------------------+----------+--------+

Add the partition to the volume group 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-0 cinder-volumes 0494615f-bd79-4490-84b9-dcebbe5f377a
+--------------------------+--------------------------------------------------+
| Property                 | Value                                            |
+--------------------------+--------------------------------------------------+
| uuid                     | 9a0ad568-0ace-4d57-9e03-e7a63f609cf2             |
| pv_state                 | adding                                           |
| pv_type                  | partition                                        |
| disk_or_part_uuid        | 0494615f-bd79-4490-84b9-dcebbe5f377a             |
| disk_or_part_device_node | /dev/sdb1                                        |
| disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 |
| lvm_pv_name              | /dev/sdb1                                        |
| lvm_vg_name              | cinder-volumes                                   |
| lvm_pv_uuid              | None                                             |
| lvm_pv_size              | 0                                                |
| lvm_pe_total             | 0                                                |
| lvm_pe_alloced           | 0                                                |
| ihost_uuid               | 9c332b27-6f22-433b-bf51-396371ac4608             |
| created_at               | 2018-08-28T13:47:39.450763+00:00                 |
| updated_at               | None                                             |
+--------------------------+--------------------------------------------------+

Adding an LVM Storage Backend at Installation

Ensure requirements are met to add LVM storage 

[wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-add lvm -s cinder

WARNING : THIS OPERATION IS NOT REVERSIBLE AND CANNOT BE CANCELLED. 

By confirming this operation, the LVM backend will be created.

Please refer to the system admin guide for minimum spec for LVM
storage. Set the 'confirmed' field to execute this operation
for the lvm backend.

Add the LVM storage backend 

[wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-add lvm -s cinder --confirmed

System configuration has changed.
Please follow the administrator guide to complete configuring the system.

+--------------------------------------+------------+---------+-------------+...+----------+--------------+
| uuid                                 | name       | backend | state       |...| services | capabilities |
+--------------------------------------+------------+---------+-------------+...+----------+--------------+
| 6d750a68-115a-4c26-adf4-58d6e358a00d | file-store | file    | configured  |...| glance   | {}           |
| e2697426-2d79-4a83-beb7-2eafa9ceaee5 | lvm-store  | lvm     | configuring |...| cinder   | {}           |
+--------------------------------------+------------+---------+-------------+...+----------+--------------+

Wait for the LVM storage backend to be configured (i.e. state=Configured) 

[wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-list 
+--------------------------------------+------------+---------+------------+------+----------+--------------+
| uuid                                 | name       | backend | state      | task | services | capabilities |
+--------------------------------------+------------+---------+------------+------+----------+--------------+
| 6d750a68-115a-4c26-adf4-58d6e358a00d | file-store | file    | configured | None | glance   | {}           |
| e2697426-2d79-4a83-beb7-2eafa9ceaee5 | lvm-store  | lvm     | configured | None | cinder   | {}           |
+--------------------------------------+------------+---------+------------+------+----------+--------------+

Configuring VM Local Storage on Controller Disk

Review the available disk space and capacity and obtain the uuid of the physical disk 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0
+--------------------------------------+-----------+---------+---------+---------+------------+...
| uuid                                 | device_no | device_ | device_ | size_mi | available_ |...
|                                      | de        | num     | type    | b       | mib        |...
+--------------------------------------+-----------+---------+---------+---------+------------+...
| 6b42c9dc-f7c0-42f1-a410-6576f5f069f1 | /dev/sda  | 2048    | HDD     | 600000  | 434072     |...
|                                      |           |         |         |         |            |...
|                                      |           |         |         |         |            |...
| 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 | /dev/sdb  | 2064    | HDD     | 16240   | 0          |...
|                                      |           |         |         |         |            |...
|                                      |           |         |         |         |            |...
| 146195b2-f3d7-42f9-935d-057a53736929 | /dev/sdc  | 2080    | HDD     | 16240   | 16237      |...
|                                      |           |         |         |         |            |...
|                                      |           |         |         |         |            |...
+--------------------------------------+-----------+---------+---------+---------+------------+...

Create the 'noval-local' volume group 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-0 nova-local
+-----------------+-------------------------------------------------------------------+
| Property        | Value                                                             |
+-----------------+-------------------------------------------------------------------+
| lvm_vg_name     | nova-local                                                        |
| vg_state        | adding                                                            |
| uuid            | 517d313e-8aa0-4b4d-92e6-774b9085f336                              |
| ihost_uuid      | 9c332b27-6f22-433b-bf51-396371ac4608                              |
| lvm_vg_access   | None                                                              |
| lvm_max_lv      | 0                                                                 |
| lvm_cur_lv      | 0                                                                 |
| lvm_max_pv      | 0                                                                 |
| lvm_cur_pv      | 0                                                                 |
| lvm_vg_size     | 0.00                                                              |
| lvm_vg_total_pe | 0                                                                 |
| lvm_vg_free_pe  | 0                                                                 |
| created_at      | 2018-08-28T14:02:58.486716+00:00                                  |
| updated_at      | None                                                              |
| parameters      | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} |
+-----------------+-------------------------------------------------------------------+

Create a disk partition to add to the volume group 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-0 146195b2-f3d7-42f9-935d-057a53736929 16237 -t lvm_phys_vol
+-------------+--------------------------------------------------+
| Property    | Value                                            |
+-------------+--------------------------------------------------+
| device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 |
| device_node | /dev/sdc1                                        |
| type_guid   | ba5eba11-0000-1111-2222-000000000001             |
| type_name   | None                                             |
| start_mib   | None                                             |
| end_mib     | None                                             |
| size_mib    | 16237                                            |
| uuid        | 009ce3b1-ed07-46e9-9560-9d2371676748             |
| ihost_uuid  | 9c332b27-6f22-433b-bf51-396371ac4608             |
| idisk_uuid  | 146195b2-f3d7-42f9-935d-057a53736929             |
| ipv_uuid    | None                                             |
| status      | Creating                                         |
| created_at  | 2018-08-28T14:04:29.714030+00:00                 |
| updated_at  | None                                             |
+-------------+--------------------------------------------------+

Wait for the new partition to be created (i.e. status=Ready) 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-0 --disk 146195b2-f3d7-42f9-935d-057a53736929
+--------------------------------------+...+------------+...+---------------------+----------+--------+
| uuid                                 |...| device_nod |...| type_name           | size_mib | status |
|                                      |...| e          |...|                     |          |        |
+--------------------------------------+...+------------+...+---------------------+----------+--------+
| 009ce3b1-ed07-46e9-9560-9d2371676748 |...| /dev/sdc1  |...| LVM Physical Volume | 16237    | Ready  |
|                                      |...|            |...|                     |          |        |
|                                      |...|            |...|                     |          |        |
+--------------------------------------+...+------------+...+---------------------+----------+--------+

Add the partition to the volume group 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-0 nova-local 009ce3b1-ed07-46e9-9560-9d2371676748
+--------------------------+--------------------------------------------------+
| Property                 | Value                                            |
+--------------------------+--------------------------------------------------+
| uuid                     | 830c9dc8-c71a-4cb2-83be-c4d955ef4f6b             |
| pv_state                 | adding                                           |
| pv_type                  | partition                                        |
| disk_or_part_uuid        | 009ce3b1-ed07-46e9-9560-9d2371676748             |
| disk_or_part_device_node | /dev/sdc1                                        |
| disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 |
| lvm_pv_name              | /dev/sdc1                                        |
| lvm_vg_name              | nova-local                                       |
| lvm_pv_uuid              | None                                             |
| lvm_pv_size              | 0                                                |
| lvm_pe_total             | 0                                                |
| lvm_pe_alloced           | 0                                                |
| ihost_uuid               | 9c332b27-6f22-433b-bf51-396371ac4608             |
| created_at               | 2018-08-28T14:06:05.705546+00:00                 |
| updated_at               | None                                             |
+--------------------------+--------------------------------------------------+
[wrsroot@controller-0 ~(keystone_admin)]$

Unlocking Controller-0

You must unlock controller-0 so that you can use it to install Controller-1. Use the system host-unlock command: 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-0

The host is rebooted. During the reboot, the command line is unavailable, and any ssh connections are dropped. To monitor the progress of the reboot, use the controller-0 console.

Verifying the Controller-0 Configuration

On Controller-0, acquire Keystone administrative privileges: 

controller-0:~$ source /etc/nova/openrc

Verify that the controller-0 services are running: 

[wrsroot@controller-0 ~(keystone_admin)]$ system service-list
+-----+-------------------------------+--------------+----------------+
| id  | service_name                  | hostname     | state          |
+-----+-------------------------------+--------------+----------------+
...
| 1   | oam-ip                        | controller-0 | enabled-active |
| 2   | management-ip                 | controller-0 | enabled-active |
...
+-----+-------------------------------+--------------+----------------+

Verify that controller-0 has controller and compute subfunctions 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-show 1 | grep subfunctions
| subfunctions        | controller,compute                         |

Verify that controller-0 is unlocked, enabled, and available: 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-list
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname     | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
| 1  | controller-0 | controller  | unlocked       | enabled     | available    |
+----+--------------+-------------+----------------+-------------+--------------+

Controller-1 Host Installation

After initializing and configuring controller-0, you can add and configure a backup controller controller-1.

Initializing Controller-1 Host

Bare Metal

Power on Controller-1 node

Virtual Environment

Start Controller-1 in the workstation: 

$ sudo virsh start controller-1

In Controller-1 console you will see: 

Waiting for this node to be configured.

Please configure the personality for this node from the
controller node in order to proceed.

Updating Controller-1 Host Host Name and Personality

On Controller-0, acquire Keystone administrative privileges: 

controller-0:~$ source /etc/nova/openrc

Wait for Controller-0 to discover new host, list the host until new UNKNOWN host shows up in table: 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-list
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname     | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
| 1  | controller-0 | controller  | unlocked       | enabled     | available    |
| 2  | None         | None        | locked         | disabled    | offline      |
+----+--------------+-------------+----------------+-------------+--------------+

Use the system host-update to update Contoller-1 host personality attribute: 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-update 2 personality=controller hostname=controller-1
+---------------------+--------------------------------------+
| Property            | Value                                |
+---------------------+--------------------------------------+
| action              | none                                 |
| administrative      | locked                               |
| availability        | offline                              |
| bm_ip               | None                                 |
| bm_type             | None                                 |
| bm_username         | None                                 |
| boot_device         | sda                                  |
| capabilities        | {}                                   |
| config_applied      | None                                 |
| config_status       | None                                 |
| config_target       | None                                 |
| console             | ttyS0,115200                         |
| created_at          | 2018-08-28T15:09:13.812906+00:00     |
| hostname            | controller-1                         |
| id                  | 2                                    |
| install_output      | text                                 |
| install_state       | None                                 |
| install_state_info  | None                                 |
| invprovision        | None                                 |
| location            | {}                                   |
| mgmt_ip             | 192.168.204.4                        |
| mgmt_mac            | 08:00:27:3d:e2:66                    |
| operational         | disabled                             |
| personality         | controller                           |
| reserved            | False                                |
| rootfs_device       | sda                                  |
| serialid            | None                                 |
| software_load       | 18.03                                |
| subfunction_avail   | not-installed                        |
| subfunction_oper    | disabled                             |
| subfunctions        | controller,compute                   |
| task                | None                                 |
| tboot               | false                                |
| ttys_dcd            | None                                 |
| updated_at          | None                                 |
| uptime              | 0                                    |
| uuid                | be0f35f7-31d9-4145-bf6a-0c2556cf398c |
| vim_progress_status | None                                 |
+---------------------+--------------------------------------+

Monitoring Controller-1 Host

On Controller-0, you can monitor the installation progress by running the system host-show command for the host periodically. Progress is shown in the install_state field. 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-show controller-1 | grep install
| install_output      | text                                 |
| install_state       | booting                              |
| install_state_info  | None                                 |

Wait while the Controller-1 is configured and rebooted. Up to 20 minutes may be required for a reboot, depending on hardware. When the reboot is complete, the Controller-1 is reported as Locked, Disabled, and Online.

Listing Controller-1 Host

Once Controller-1 has been installed, configured and rebooted, on Controller-0 list the hosts: 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-list
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname     | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
| 1  | controller-0 | controller  | unlocked       | enabled     | available    |
| 2  | controller-1 | controller  | locked         | disabled    | online      |
+----+--------------+-------------+----------------+-------------+--------------+

Controller-1 Provisioning

On Controller-0, list hosts 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-list
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname     | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
...
| 2  | controller-1 | controller  | locked         | disabled    | online       |
+----+--------------+-------------+----------------+-------------+--------------+

Provisioning Network Interfaces on Controller-1

In order to list out hardware port names, types, pci-addresses that have been discovered: 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-port-list controller-1

Provision the Controller-1 oam interface 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -n ens6 -c platform --networks oam controller-1 ens6
+------------------+--------------------------------------+
| Property         | Value                                |
+------------------+--------------------------------------+
| ifname           | ens6                                 |
| iftype           | ethernet                             |
| ports            | [u'ens6']                            |
| providernetworks | None                                 |
| imac             | 08:00:27:ba:3c:38                    |
| imtu             | 1500                                 |
| ifclass          | oam                                  |
| aemode           | None                                 |
| schedpolicy      | None                                 |
| txhashpolicy     | None                                 |
| uuid             | 160bfede-0950-42ba-acef-d0eb31e7fc49 |
| ihost_uuid       | be0f35f7-31d9-4145-bf6a-0c2556cf398c |
| vlan_id          | None                                 |
| uses             | []                                   |
| used_by          | []                                   |
| created_at       | 2018-08-28T15:59:10.701772+00:00     |
| updated_at       | 2018-08-29T05:44:38.585642+00:00     |
| sriov_numvfs     | 0                                    |
| ipv4_mode        | static                               |
| ipv6_mode        | disabled                             |
| accelerated      | [False]                              |
+------------------+--------------------------------------+

Providing Data Interfaces on Controller-1

List all interfaces 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-if-list -a controller-1
+--------------------------------------+---------+---------+...+------+--------------+------+------+------------+..
| uuid                                 | name    | network |...| vlan | ports        | uses | used | attributes |..
|                                      |         | type    |...| id   |              | i/f  | by   |            |..
|                                      |         |         |...|      |              |      | i/f  |            |..
+--------------------------------------+---------+---------+...+------+--------------+------+------+------------+..
| 160bfede-0950-42ba-acef-d0eb31e7fc49 | ens6    | oam     |...| None | [u'enp0s3']  | []   | []   | MTU=1500   |..
| 37b3abcd-bcbe-44d5-b2fb-759c34efec89 | eth1001 | None    |...| None | [u'eth1001'] | []   | []   | MTU=1500   |..
| 81d66feb-b23c-4533-bd4b-129f9b75ecd6 | mgmt0   | mgmt    |...| None | [u'enp0s8']  | []   | []   | MTU=1500   |..
| e78ad9a9-e74d-4c6c-9de8-0e41aad8d7b7 | eth1000 | None    |...| None | [u'eth1000'] | []   | []   | MTU=1500   |..
+--------------------------------------+---------+---------+...+------+--------------+------+------+------------+..

Configure the data interfaces 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -p providernet-a -c data controller-1 eth1000
+------------------+--------------------------------------+
| Property         | Value                                |
+------------------+--------------------------------------+
| ifname           | eth1000                              |
| networktype      | data                                 |
| iftype           | ethernet                             |
| ports            | [u'eth1000']                         |
| providernetworks | providernet-a                        |
| imac             | 08:00:27:39:61:6e                    |
| imtu             | 1500                                 |
| aemode           | None                                 |
| schedpolicy      | None                                 |
| txhashpolicy     | None                                 |
| uuid             | e78ad9a9-e74d-4c6c-9de8-0e41aad8d7b7 |
| ihost_uuid       | be0f35f7-31d9-4145-bf6a-0c2556cf398c |
| vlan_id          | None                                 |
| uses             | []                                   |
| used_by          | []                                   |
| created_at       | 2018-08-28T15:59:17.667899+00:00     |
| updated_at       | 2018-08-29T06:04:55.116653+00:00     |
| sriov_numvfs     | 0                                    |
| ipv4_mode        | disabled                             |
| ipv6_mode        | disabled                             |
| accelerated      | [True]                               |
+------------------+--------------------------------------+

Provisioning Storage on Controller-1

Review the available disk space and capacity and obtain the uuid of the physical disk 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1
+--------------------------------------+-------------+------------+-------------+----------+---------------+...
| uuid                                 | device_node | device_num | device_type | size_mib | available_mib |...
+--------------------------------------+-------------+------------+-------------+----------+---------------+...
| 2dcc3d33-ba43-4b5a-b0b0-2b5a2e5737b7 | /dev/sda    | 2048       | HDD         | 600000   | 434072        |...
| f53437c6-77e3-4185-9453-67eaa8b461b1 | /dev/sdb    | 2064       | HDD         | 16240    | 16237         |...
| 623bbfc0-2b38-432a-acf4-a28db6066cce | /dev/sdc    | 2080       | HDD         | 16240    | 16237         |...
+--------------------------------------+-------------+------------+-------------+----------+---------------+...

Assign Cinder storage to the physical disk 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-1 cinder-volumes
+-----------------+--------------------------------------+
| Property        | Value                                |
+-----------------+--------------------------------------+
| lvm_vg_name     | cinder-volumes                       |
| vg_state        | adding                               |
| uuid            | 3ccadc5c-e4f7-4b04-8403-af5d2ecef96d |
| ihost_uuid      | be0f35f7-31d9-4145-bf6a-0c2556cf398c |
| lvm_vg_access   | None                                 |
| lvm_max_lv      | 0                                    |
| lvm_cur_lv      | 0                                    |
| lvm_max_pv      | 0                                    |
| lvm_cur_pv      | 0                                    |
| lvm_vg_size     | 0.00                                 |
| lvm_vg_total_pe | 0                                    |
| lvm_vg_free_pe  | 0                                    |
| created_at      | 2018-08-29T05:51:13.965883+00:00     |
| updated_at      | None                                 |
| parameters      | {u'lvm_type': u'thin'}               |
+-----------------+--------------------------------------+

Create a disk partition to add to the volume group based on uuid of the physical disk 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-1 f53437c6-77e3-4185-9453-67eaa8b461b1 16237 -t lvm_phys_vol
+-------------+--------------------------------------------------+
| Property    | Value                                            |
+-------------+--------------------------------------------------+
| device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 |
| device_node | /dev/sdb1                                        |
| type_guid   | ba5eba11-0000-1111-2222-000000000001             |
| type_name   | None                                             |
| start_mib   | None                                             |
| end_mib     | None                                             |
| size_mib    | 16237                                            |
| uuid        | 7a41aab0-6695-4d16-9003-73238adda75b             |
| ihost_uuid  | be0f35f7-31d9-4145-bf6a-0c2556cf398c             |
| idisk_uuid  | f53437c6-77e3-4185-9453-67eaa8b461b1             |
| ipv_uuid    | None                                             |
| status      | Creating (on unlock)                             |
| created_at  | 2018-08-29T05:54:23.326612+00:00                 |
| updated_at  | None                                             |
+-------------+--------------------------------------------------+

Wait for the new partition to be created (i.e. status=Ready) 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-1 --disk f53437c6-77e3-4185-9453-67eaa8b461b1
+--------------------------------------+...+-------------+...+-----------+----------+----------------------+
| uuid                                 |...| device_node |...| type_name | size_mib | status               |
+--------------------------------------+...+-------------+...+-----------+----------+----------------------+
| 7a41aab0-6695-4d16-9003-73238adda75b |...| /dev/sdb1   |...| None      | 16237    | Creating (on unlock) |
+--------------------------------------+...+-------------+...+-----------+----------+----------------------+

Add the partition to the volume group 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-1 cinder-volumes 7a41aab0-6695-4d16-9003-73238adda75b
+--------------------------+--------------------------------------------------+
| Property                 | Value                                            |
+--------------------------+--------------------------------------------------+
| uuid                     | 11628316-56ab-41b2-b2be-a5d72667a5e9             |
| pv_state                 | adding                                           |
| pv_type                  | partition                                        |
| disk_or_part_uuid        | 7a41aab0-6695-4d16-9003-73238adda75b             |
| disk_or_part_device_node | /dev/sdb1                                        |
| disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 |
| lvm_pv_name              | /dev/sdb1                                        |
| lvm_vg_name              | cinder-volumes                                   |
| lvm_pv_uuid              | None                                             |
| lvm_pv_size              | 0                                                |
| lvm_pe_total             | 0                                                |
| lvm_pe_alloced           | 0                                                |
| ihost_uuid               | be0f35f7-31d9-4145-bf6a-0c2556cf398c             |
| created_at               | 2018-08-29T06:25:54.550430+00:00                 |
| updated_at               | None                                             |
+--------------------------+--------------------------------------------------+

Configuring VM Local Storage on Controller Disk

Review the available disk space and capacity and obtain the uuid of the physical disk 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1
+--------------------------------------+-------------+------------+-------------+----------+---------------+...
| uuid                                 | device_node | device_num | device_type | size_mib | available_mib |...
+--------------------------------------+-------------+------------+-------------+----------+---------------+...
| 2dcc3d33-ba43-4b5a-b0b0-2b5a2e5737b7 | /dev/sda    | 2048       | HDD         | 600000   | 434072        |...
| f53437c6-77e3-4185-9453-67eaa8b461b1 | /dev/sdb    | 2064       | HDD         | 16240    | 16237         |...
| 623bbfc0-2b38-432a-acf4-a28db6066cce | /dev/sdc    | 2080       | HDD         | 16240    | 16237         |...
+--------------------------------------+-------------+------------+-------------+----------+---------------+...

Create the 'cinder-volumes' local volume group 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-1 nova-local
+-----------------+-------------------------------------------------------------------+
| Property        | Value                                                             |
+-----------------+-------------------------------------------------------------------+
| lvm_vg_name     | nova-local                                                        |
| vg_state        | adding                                                            |
| uuid            | d205f839-0930-4e77-9f10-97089150f812                              |
| ihost_uuid      | be0f35f7-31d9-4145-bf6a-0c2556cf398c                              |
| lvm_vg_access   | None                                                              |
| lvm_max_lv      | 0                                                                 |
| lvm_cur_lv      | 0                                                                 |
| lvm_max_pv      | 0                                                                 |
| lvm_cur_pv      | 0                                                                 |
| lvm_vg_size     | 0.00                                                              |
| lvm_vg_total_pe | 0                                                                 |
| lvm_vg_free_pe  | 0                                                                 |
| created_at      | 2018-08-29T06:28:28.226907+00:00                                  |
| updated_at      | None                                                              |
| parameters      | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} |
+-----------------+-------------------------------------------------------------------+

Create a disk partition to add to the volume group 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-1 623bbfc0-2b38-432a-acf4-a28db6066cce 16237 -t lvm_phys_vol
+-------------+--------------------------------------------------+
| Property    | Value                                            |
+-------------+--------------------------------------------------+
| device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 |
| device_node | /dev/sdc1                                        |
| type_guid   | ba5eba11-0000-1111-2222-000000000001             |
| type_name   | None                                             |
| start_mib   | None                                             |
| end_mib     | None                                             |
| size_mib    | 16237                                            |
| uuid        | f7bc6095-9375-49fe-83c7-12601c202376             |
| ihost_uuid  | be0f35f7-31d9-4145-bf6a-0c2556cf398c             |
| idisk_uuid  | 623bbfc0-2b38-432a-acf4-a28db6066cce             |
| ipv_uuid    | None                                             |
| status      | Creating (on unlock)                             |
| created_at  | 2018-08-29T06:30:33.176428+00:00                 |
| updated_at  | None                                             |
+-------------+--------------------------------------------------+

Wait for the new partition to be created (i.e. status=Ready) 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-1
+--------------------------------------+...+-------------+...+-----------+----------+----------------------+
| uuid                                 |...| device_node |...| type_name | size_mib | status               |
+--------------------------------------+...+-------------+...+-----------+----------+----------------------+
| 7a41aab0-6695-4d16-9003-73238adda75b |...| /dev/sdb1   |...| None      | 16237    | Creating (on unlock) |
| f7bc6095-9375-49fe-83c7-12601c202376 |...| /dev/sdc1   |...| None      | 16237    | Creating (on unlock) |
+--------------------------------------+...+-------------+...+-----------+----------+----------------------+
[wrsroot@controller-0 ~(keystone_admin)]$

Add the partition to the volume group 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-1 nova-local f7bc6095-9375-49fe-83c7-12601c202376
+--------------------------+--------------------------------------------------+
| Property                 | Value                                            |
+--------------------------+--------------------------------------------------+
| uuid                     | 94003d64-4e1b-483a-8a6c-0fc4b1e0e272             |
| pv_state                 | adding                                           |
| pv_type                  | partition                                        |
| disk_or_part_uuid        | f7bc6095-9375-49fe-83c7-12601c202376             |
| disk_or_part_device_node | /dev/sdc1                                        |
| disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 |
| lvm_pv_name              | /dev/sdc1                                        |
| lvm_vg_name              | nova-local                                       |
| lvm_pv_uuid              | None                                             |
| lvm_pv_size              | 0                                                |
| lvm_pe_total             | 0                                                |
| lvm_pe_alloced           | 0                                                |
| ihost_uuid               | be0f35f7-31d9-4145-bf6a-0c2556cf398c             |
| created_at               | 2018-08-29T06:33:08.625604+00:00                 |
| updated_at               | None                                             |
+--------------------------+--------------------------------------------------+

Unlocking Controller-1

Unlock Controller-1 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-1

Wait while the Controller-1 is rebooted. Up to 10 minutes may be required for a reboot, depending on hardware.

REMARK: Controller-1 will remain in 'degraded' state until data-syncing is complete. The duration is dependant on the virtualization host's configuration - i.e., the number and configuration of physical disks used to host the nodes' virtual disks. Also, the management network is expected to have link capacity of 10000 (1000 is not supported due to excessive data-sync time). Use 'fm alarm-list' to confirm status. 

[wrsroot@controller-0 ~(keystone_admin)]$ system host-list
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname     | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
| 1  | controller-0 | controller  | unlocked       | enabled     | available    |
| 2  | controller-1 | controller  | unlocked       | enabled     | available    |
+----+--------------+-------------+----------------+-------------+--------------+

System Alarm List

When all nodes are Unlocked, Enabled and Available: check 'fm alarm-list' for issues.

Your StarlingX deployment is now up and running with 2x HA Controllers with Cinder Storage and all OpenStack services up and running. You can now proceed with standard OpenStack APIs, CLIs and/or Horizon to load Glance Images, configure Nova Flavors, configure Neutron networks and launch Nova Virtual Machines.

Retrieved from "https://wiki.openstack.org/w/index.php?title=StarlingX/Installation_Guide_Virtual_Environment/Duplex&oldid=165820"