StarlingX/Developer Guide
This section contains the steps for building a StarlingX ISO from Master branch.
Contents
Requirements
The recommended minimum requirements include:
Hardware Requirements
A workstation computer with:
- Processor: x86_64 is the only supported architecture
- Memory: At least 32GB RAM
- Hard Disk: 500GB HDD
- Network: Network adapter with active Internet connection
Software Requirements
A workstation computer with:
- Operating System: Ubuntu 16.04 LTS 64-bit
- Docker
- Android Repo Tool
- Proxy Settings Configured (If Required)
- See http://lists.starlingx.io/pipermail/starlingx-discuss/2018-July/000136.html for more details
- Public SSH Key
Development Environment Setup
This section describes how to set up a StarlingX development system on a workstation computer. After completing these steps, you will be able to build a StarlingX ISO image on the following Linux distribution:
- Ubuntu 16.04 LTS 64-bit
Update Your Operating System
Before proceeding with the build, ensure your OS is up to date. You’ll first need to update the local database list of available packages:
$ sudo apt-get update
Installation Requirements and Dependencies
- Install the required packages in an Ubuntu host system with:
$ sudo apt-get install make git curl
- Make sure to setup your identity
$ git config --global user.name "Name LastName" $ git config --global user.email "Email Address"
Install the required Docker CE packages in an Ubuntu host system. See Get Docker for more information.
Android Repo Tool
- <p>Install the required Android Repo Tool in an Ubuntu host system. Follow the 2 steps in "Installing Repo" section from Installing Repo to have Andriod Repo Tool installed.
- Follow these instructions on GitHub to Generate a Public SSH Key and then upload your public key to your GitHub and Gerrit account profiles:
- Under your $HOME directory, clone the <stx-tools> project
$ cd $HOME $ git clone https://git.starlingx.io/stx-tools
- Create a starlingx workspace directory on your workstation computer. Usually, you’ll want to create it somewhere under your user’s home directory.
$ mkdir -p $HOME/starlingx/
- Navigate to the <$HOME/stx-tools>/centos-mirror-tool project directory:
$ cd $HOME/stx-tools/centos-mirror-tools/
- Temporal! Only Branch Master Check if the following Gerrit Reviews are merged, if not, cherry pick them
None
- Temporal! Only Branch m/2018.07 Check if the following Gerrit Reviews are merged, if not, cherry pick them
None
- If necessary you might have to set http/https proxy in your Dockerfile before building the docker image.
ENV http_proxy " http://your.actual_http_proxy.com:your_port " ENV https_proxy " https://your.actual_https_proxy.com:your_port " ENV ftp_proxy " http://your.actual_ftp_proxy.com:your_port " RUN echo " proxy=http://your-proxy.com:port " >> /etc/yum.conf
- Build your <user>:<tag> base container image with e.g. user:centos-mirror-repository
$ docker build -t <user>:centos-mirror-repository -f Dockerfile .
- Launch a <user> docker container using previously created Docker base container image <user>:<tag> e.g. <user>-centos-mirror-repository. As /localdisk is defined as the workdir of the container, the same folder name should be used to define the volume. The container will start to run and populate a logs and output folders in this directory. The container shall be run from the same directory where the other scripts are stored.
$ docker run -it --name <user>-centos-mirror-repository -v $(pwd):/localdisk <user>:centos-mirror-repository
- Inside Repository Docker container, enter the following command to download the required packages to populate the CentOS Mirror Repository:
# bash download_mirror.sh
- Monitor the download of packages until it is complete. When download is complete, the following message is displayed:
totally 17 files are downloaded! step #3: done successfully IMPORTANT: The following 3 files are just bootstrap versions. Based on them, the workable images for StarlingX could be generated by running "update-pxe-network-installer" command after "build-iso" - out/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img - out/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img - out/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz
- Verify there are no missing or failed packages:
# cat logs/*_missing_*.log # cat logs/*_failmove_*.log
In case there are missing or failed ones due to network instability (or timeout), you should download them manually, to assure you get all RPMs listed in "rpms_from_3rd_parties.lst" and "rpms_from_centos_repo.lst".
Temporal! None
$ None
- From terminal identified as "Two", create a mirror/CentOS directory under your starlingx workspace directory:
$ mkdir -p $HOME/starlingx/mirror/CentOS/
- Copy the built CentOS Mirror Repository built under <stx-tools>/centos-mirror-tool to the $HOME/starlingx/mirror/ workspace directory.
$ cp -r $HOME/stx-tools/centos-mirror-tools/output/stx-r1/ $HOME/starlingx/mirror/CentOS/
- From terminal identified as "Two", Workspace Setup
$ mkdir -p $HOME/starlingx/workspace
- Navigate to the <$HOME/stx-tools> project directory:
$ cd $HOME/stx-tools
- Copy your git options
$ cp ~/.gitconfig toCOPY
- Create a <localrc> file
$ nano localrc
- If necessary you might have to set http/https proxy in your Dockerfile.centos73 before building the docker image.
ENV http_proxy "http://your.actual_http_proxy.com:your_port" ENV https_proxy "https://your.actual_https_proxy.com:your_port" ENV ftp_proxy "http://your.actual_ftp_proxy.com:your_port" RUN echo "proxy=http://your-proxy.com:port" >> /etc/yum.conf
- Base container setup
If you are running in fedora system, you will see " .makeenv:88: *** missing separator. Stop. " error, to continue :
- delete the functions define in the .makeenv ( module () { ... } )
- delete the line 19 in the Makefile and ( NULL := $(shell bash -c "source buildrc ... ).
$ make base-build
- Build container setup
$ make build
- Verify environment variables
$ bash tb.sh env
- Build container run
$ bash tb.sh run
- Execute the built container:
$ bash tb.sh exec
- From terminal identified as "Two", now inside the Building Docker container, Internal environment
$ eval $(ssh-agent) $ ssh-add
- Repo init
$ cd $MY_REPO_ROOT_DIR $ repo init -u https://git.starlingx.io/stx-manifest.git -m default.xml
- Repo sync
$ repo sync
- Temporal! Only Branch Master Check if the following Gerrit Reviews are merged, if not, cherry pick them
None
- Tarballs Repository
$ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/
Alternatively you can run the populate_downloads.sh script to copy the tarballs instead of using a symlink.
$ populate_downloads.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
- From another terminal identified as "Three", Mirror Binaries
$ mkdir -p $HOME/starlingx/mirror/CentOS/tis-installer $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img $HOME/starlingx/mirror/CentOS/tis-installer/initrd.img-stx-0.2 $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz $HOME/starlingx/mirror/CentOS/tis-installer/vmlinuz-stx-0.2 $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img $HOME/starlingx/mirror/CentOS/tis-installer/squashfs.img-stx-0.2
- Back to the Building Docker container, terminal identified as "Two"
- Temporal! Build-Pkgs Errors
Be prepared to have some missing / corrupted rpm and tarball packages generated during StarlingX/Developer_Guide#Build_the_CentOS_Mirror_Repository which will make the next step to fail, if that happens please download manually those missing / corrupted packages.
- Update the symbolic links
$ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
- Build-Pkgs
$ build-pkgs
- Optional! Generate-Cgcs-Tis-Repo
This step is optional but but will improve performance on subsequent builds. The cgcs-tis-repo has the dependency information that sequences the build order; To generate or update the information the following command needs to be executed after building modified or new packages.
- Build-Iso
$ build-iso
- Store the files in the /import/mirror/CentOS/tis-installer/ folder for future use.
- Store it in an arbitrary location and modify the $MY_REPO/stx/stx-metal/installer/pxe-network-installer/centos/build_srpm.data file to point to these files.
- In order to get the first boot working this complete procedure needs to be done. However, once the init files are created, these can be stored in a shared location where different developers can make use of them. Updating these files is not a frequent task and should be done whenever the kernel is upgraded.
- StarlingX is in active development, so it is possible that in the future the 0.2 version will change to a more generic solution.
Git
Docker CE
Install Public SSH Key
Install stx-tools project
Create a Workspace Directory
Build the CentOS Mirror Repository
This section describes how to build the CentOS Mirror Repository.
Setup Repository Docker Container
Run the following commands under a terminal identified as "One".
Download Packages
Verify Packages
Create CentOS Mirror Repository
Outside your Repository Docker container, in another terminal identified as "Two", run the following commands:
Create StarlingX Packages
Setup Building Docker Container
# tbuilder localrc
MYUNAME=user
PROJECT=starlingx
HOST_PREFIX=$HOME/starlingx/workspace
HOST_MIRROR_DIR=$HOME/starlingx/mirror
Download Source Code Repositories
Outside the container
Build Packages
$ generate-cgcs-tis-repo
Build StarlingX ISO
Build installer
To get your StarlingX ISO ready to use, you will need to create the init files that will be used to boot the ISO as well to boot additional controllers and compute nodes. Note that this procedure only is needed in your first build and every time the kernel is upgraded.
Once you had run build-iso, run:
$ build-pkgs --installer
This will build rpm and anaconda packages. Then run:
$ update-pxe-network-installer
The update-pxe-network-installer covers the steps detailed in $MY_REPO/stx/stx-metal/installer/initrd/README. This script will create three files on /localdisk/loadbuild/<user>/<project>/pxe-network-installer/output.
new-initrd.img
new-squashfs.img
new-vmlinuz
Then, rename them to:
initrd.img-stx-0.2
squashfs.img-stx-0.2
vmlinuz-stx-0.2
There are two ways to use these files:
Now, the pxe-network-installer package needs to be recreated and the ISO regenerated.
$ build-pkgs --clean pxe-network-installer
$ build-pkgs pxe-network-installer
$ build-iso
Now your ISO should be able to boot.