Difference between revisions of "StarlingX/Packet SIG"

Revision as of 18:07, 5 July 2019

Packet SIG

Packet.com is a baremetal public cloud, and they have donated some resources to the StarlingX project. The resources are available under the STX-PROJECT-01 project on Packet.com.

StarlingX Distributed Cloud on Packet.Com

As a demonstration of the OpenStack Edge Computing Group's Distributed Control Plane MVP Architecture, StarlingX Distributed Cloud has been deployed on Packet.com.

STX R1 : http://mirror.starlingx.cengn.ca/mirror/starlingx/release/2018.10/centos/2018.10.0/outputs/iso/

Horizon for Central Cloud: http://147.75.105.202
SSH to Central Cloud: ssh wrsroot@147.75.105.202

Packet.com Servers deployed

Networking

Setting up the iPXE Boot Server ................................................................

The initial server of a StarlingX cloud, i.e. controller-0, must be installed via a PXE Boot Server. For packet.com, IPXE Booting will actually be used ... however the setup of the ipxe boot server is the same.

Get a small packet.com server (e.g. c1.small.x86) for the IPXE Boot Server. It should use packet.com L3 Networking and any OS can be used ... I used ubuntu 16.04 .

Setup the PXE Boot Server as follows:

apt-get update
apt-install apache2 -y

wget http://mirror.starlingx.cengn.ca/mirror/starlingx/release/2018.10/centos/2018.10.0/outputs/iso/bootimage.iso

mkdir -p /media/iso
mount -o loop ./bootimage.iso  /media/iso
mount -o remount,exec,dev /media/iso

mkdir -p /export/pxeboot
cd /var/www/html
ln -s /export/pxeboot BIOS-Client

# for some reason have to remove pxeboot directory before running pxeboot_setup.sh
cd /export
rmdir pxeboot
cd
/media/iso/pxeboot_setup.sh -u http://<IPADDRESS-OF-IPXE-BOOT-SERVER>/BIOS-Client -t /export/pxeboot 

Installing the Central Cloud ........................................................................

SW Install of initial server (cc-controller-0):
In order to IPXE Install the StarlingX Load onto a packet.com server, it must have L3 connectivity.

Setup a new packet.com server for controller-0 (e.g. m1.xlarge.x86), initially with L3 connectivity and initially with an ubuntu 16.04 load.

After the server boots, 'REINSTALL' the packet.com server using 'Custom IPXE' and specifying the following ipxe.conf file:

#!ipxe
set base-url http://147.75.38.129/BIOS-Client
kernel ${base-url}/vmlinuz console=ttyS1,115200n8 root=live:${base-url}/LiveOS/squashfs.img ip=dhcp ks=${base-url}/pxeboot_controller.cfg boot_device=sda rootfs_device=sda inst.gpt inst.text inst.repo=${base-url} security_profile=standard user_namespace.enable=1 network ksdevice=bootif BOOTIF=01-${netX/mac}
initrd ${base-url}/initrd.img
imgstat
boot

NOTE: I setup a GitHub repo with ipxe.conf files for a StarlingX Standard configuration and a StarlingX All-In-One configuration: https://github.com/gwaines/ipxe-configs

Specify https://raw.githubusercontent.com/gwaines/ipxe-configs/master/ipxe-starlingx-standard-sda.conf as the URL for the ipxe.conf file for the Central Cloud.

The server will reboot a few times, run the StarlingX installer to install the StarlingX load on /dev/sda and then reboot a final time running the StarlingX load.

Bootstrap StarlingX Software:

Configure and Unlock Controller-0

SW Install of second controller (cc-controller-1):

Configure and Unlock Controller-0

Installing the Sub Clouds ............................................................................

SW Install of initial server (cc-controller-0):

Bootstrap StarlingX Software:

Configure and Unlock Controller-0

Problem Tracking

Problems Found with Packet.com:

• Multi-cast Packets appear to be dropped by Packet.com switches
  • Results in StartingX Maintenance reporting a false alarm that it has lost connectivity with other nodes in the StarlingX Cloud. E.g. In Central Cloud, cc-controller-0 claims that it has a heartbeat audit failure with cc-controller-1.
  • WORKAROUND: Change maintenance's heartbeat failure behaviour to simply raise an alarm ... rather than declare the node as failed and reset the node.

system service-parameter-list
system service-parameter-modify platform maintenance heartbeat_failure_action=alarm
system service-parameter-apply platform

• On some interfaces, packets between 1400 and 1500 bytes were dropped by packet.com switch, even though MTU of Interface was 1500 bytes.
  • WORKAROUND: set all MTUs to 1400 bytes

Problems Found with StarlingX R1.0:

• config_subcloud would fail if prior to running the command on controller-0 of the subcloud, the mgmt interface was already configured and there were any unreachable DNS servers configured in /etc/resolv.conf
  • WORKAROUND: ifdown <mgmt-if-dev> and vi /etc/resolv.conf and remove all DNS Server entries
• HORIZON subcloud 'managed' would fail with some error about JSON
  • WORKAROUND: Use CLI to set subcloud to managed