Jump to: navigation, search

StarlingX/Developer Guide

< StarlingX
Revision as of 04:30, 13 July 2018 by Abraham.arce.moreno (talk | contribs) (Build Packages)

This section contains the steps for building a StarlingX ISO.


The recommended minimum requirements include:

Hardware Requirements

A workstation computer with:

  • Processor: x86_64 only supported architecture
  • Memory: At least 32GB RAM
  • Hard Disk: 100GB HDD
  • Network: Network adapter with active Internet connection

Software Requirements

A workstation computer with:

Development Environment Setup

This section describes how to set up a StarlingX development system on a workstation computer. After completing these steps, you will be able to build a StarlingX ISO image on the following Linux distribution:

  • Ubuntu 16.04 LTS 64-bit

Update Your Operating System

Before proceeding with the build, ensure your OS is up to date. You’ll first need to update the local database list of available packages:

$ sudo apt-get update

Installation Requirements and Dependencies

  1. Install the required packages in an Ubuntu host system with:
    $ sudo apt-get install git
  2. Install the required Docker CE packages in an Ubuntu host system. See Get Docker for more information.

  3. Install the required Android Repo Tool in an Ubuntu host system. See Installing Repo for more information.

Install Public SSH Key

  1. Follow these instructions on GitHub to Generate a Public SSH Key and then upload your public key to your GitHub and Gerrit account profiles:

Install stx-tools project

  1. Under your $HOME directory, clone the <stx-tools> project
    $ cd $HOME
    $ git clone git://git.openstack.org/openstack/stx-tools

Create a Workspace Directory

  1. Create a starlingx workspace directory on your workstation computer. Usually, you’ll want to create it somewhere under your user’s home directory.
    $ mkdir -p $HOME/starlingx/

Build the CentOS Mirror Repository

This section describes how to build the CentOS Mirror Repository.

Setup Repository Docker Container

Run the following commands under a terminal identified as "one".

  1. Navigate to the <$HOME/stx-tools>/centos-mirror-tool project directory:
    $ cd $HOME/stx-tools/centos-mirror-tools/
  2. Temporal! Cherry pick "Change revision to download TPM kernel module code"
  3. $ git fetch git://git.openstack.org/openstack/stx-tools refs/changes/15/581915/2 && git cherry-pick FETCH_HEAD
  4. If necessary you might have to set http/https proxy in your Dockerfile before building the docker image.
    ENV http_proxy  "http://your.actual_http_proxy.com:your_port" && \
        https_proxy "https://your.actual_https_proxy.com:your_port" && \
        ftp_proxy "http://your.actual_ftp_proxy.com:your_port"
    RUN echo "proxy=http://your-proxy.com:port" >> /etc/yum.conf
  5. Build your <user>:<tag> base container image with e.g. user:centos-mirror-repository
    $ docker build -t user:centos-mirror-repository -f Dockerfile .
  6. Launch a <user> docker container using previously created Docker base container image <user>:<tag> e.g. user-centos-mirror-repository. As /localdisk is defined as the workdir of the container, the same folder name should be used to define the volume. The container will start to run and populate a logs and output folders in this directory. The container shall be run from the same directory where the other scripts are stored.
    $ docker run -it --name user-centos-mirror-repository -v $(pwd):/localdisk user:centos-mirror-repository
  7. Download Packages

    1. Enter the following command to download the required packages to populate the CentOS Mirror Repository:
      # bash download_mirror.sh
    2. Monitor the download of packages until it is complete. When download is complete, the following message is displayed:
      totally 17 files are downloaded!
      step #3: done successfully
      IMPORTANT: The following 3 files are just bootstrap versions. Based on them, the workable images
      for StarlingX could be generated by running "update-pxe-network-installer" command after "build-iso"
          - out/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img
          - out/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img
          - out/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz

    Verify Packages

    1. Verify there are no missing or failed packages:
      # cat output/3rd-centos_rpms_missing_L1.txt output/3rd_rpms_missing_L1.txt output/3rd_srpms_missing_L1.txt output/centos_rpms_missing_L1.txt output/centos_srpms_missing_L1.txt
      # cat output/3rd_rpms_fail_move_L1.txt output/3rd_srpms_fail_move_L1.txt output/centos_rpms_fail_move_L1.txt output/centos_srpms_fail_move_L1.txt
    2. In case there are missing or failed ones due to network instability (or timeout), you should download them manually, to assure you get all RPMs listed in "rpms_from_3rd_parties.lst" and "rpms_from_centos_repo.lst".

    3. After all packages were succesfully downloaded, remove all i686 RPMs packages and change output directory ownership:

      # find ./output -name "*.i686.rpm" | xargs rm -f
      # chown 751:751 -R ./output

    Create CentOS Mirror Repository

    Run the following commands under a terminal identified as "two".

    1. From terminal "two", create a mirror/CentOS directory under your starlingx workspace directory:
      $ mkdir -p $HOME/starlingx/mirror/CentOS/
    2. Copy the built CentOS Mirror Repository built under <stx-tools>/centos-mirror-tool to the $HOME/starlingx/mirror/CentOS workspace directory.
      $ cp -r $HOME/stx-tools/centos-mirror-tools/output/stx-r1/ $HOME/starlingx/mirror/CentOS/

    Create StarlingX Packages

    Setup Building Docker Container

    1. From terminal "two", Workspace Setup
      $ mkdir -p $HOME/starlingx/workspace
    2. Navigate to the <$HOME/stx-tools> project directory:
      $ cd $HOME/stx-tools
    3. Copy your git options
      $ cp ~/.gitconfig toCOPY
    4. Create a <localrc> file
      $ nano localrc
    5. # tbuilder localrc
    6. If necessary you might have to set http/https proxy in your Dockerfile.centos73 before building the docker image.
      ENV http_proxy  "http://your.actual_http_proxy.com:your_port" && \
          https_proxy "https://your.actual_https_proxy.com:your_port" && \
          ftp_proxy "http://your.actual_ftp_proxy.com:your_port"
      RUN echo "proxy=http://your-proxy.com:port" >> /etc/yum.conf
    7. Base container setup If you are running in fedora system, you will see " .makeenv:88: *** missing separator. Stop. " error, to continue :
      • delete the functions define in the .makeenv ( module () { ... } )
      • delete the line 19 in the Makefile and ( NULL := $(shell bash -c "source buildrc ... ).
      $ make base-build
    8. Build container setup
      $ make build
    9. Verify environment variables
      $ bash tb.sh env
    10. Build container run
      $ bash tb.sh run
    11. Execute the built container:
      $ bash tb.sh exec

    Download Source Code Repositories

    1. Internal environment
      $ eval $(ssh-agent)
      $ ssh-add
    2. Repo init
      $ cd $MY_REPO_ROOT_DIR
      $ repo init -u git://git.openstack.org/openstack/stx-manifest.git -m default.xml
    3. Repo sync
      $ repo sync
    4. cgcs-centos-repo
      $ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
    5. cgcs-dl
      $ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/
    6. Outside the container, use another terminal

    7. Mirror Binaries
      $ mkdir -p $HOME/starlingx/mirror/CentOS/tis-installer
      $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img $HOME/starlingx/mirror/CentOS/tis-installer/initrd.img-stx-0.2
      $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz $HOME/starlingx/mirror/CentOS/tis-installer/vmlinuz-stx-0.2
      $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img $HOME/starlingx/mirror/CentOS/tis-installer/squashfs.img-stx-0.2

    Build Packages

    1. Temporal! ERROR: Could not find useradd in chroot, maybe the install failed?
      In case you see this error during the step 2 "build-pkgs --serial", please modify "$MY_REPO/build-tools/modify-build-cfg" to include shadow-utils and rpm-build under chroot_setup_cmd config option and run again, as discussed in the following email:
    2. Temporal! Comps.Xml It could be possible some other dependencies are required and not added, as a temporal fix a comps.xml shall be created using the content from
      under mirror Binary, execute from outside the container:
      $ vi ~/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/comps.xml

      Do not forget to update Symbolic links, execute inside the container:

      $ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
    3. Temporal! Cherry pick "Update TPM patches to be built with upstream code"
      $ cd $MY_REPO/stx/stx-gplv2
      $ git fetch git://git.openstack.org/openstack/stx-gplv2 refs/changes/16/581916/1 && git cherry-pick FETCH_HEAD
    4. Temporal! Cherry pick "Fix malformed patch on cgcs-users/ibsh"
      $ cd $MY_REPO/stx/stx-gplv2
      $ git fetch git://git.openstack.org/openstack/stx-gplv2 refs/changes/59/581159/1 && git cherry-pick FETCH_HEAD
    5. Build-Pkgs --Serial
      $ cd $MY_WORKSPACE
      $ build-pkgs --serial
    6. Generate-Cgcs-Tis-Repo The cgcs-tis-repo has the dependency information that sequences the build order; To generate or update the information the following command needs to be executed after building modified or new packages.
      $ generate-cgcs-tis-repo
    7. Build-Srpms
      $ build-srpms --installer
    8. Build-Rpms
      $ build-rpms --installer

    Build StarlingX ISO

    1. Build-Iso
      $ build-iso


    If you are running in fedora system:

    • delete the line 19 in the Makefile and ( NULL := $(shell bash -c "source buildrc ... ).
    • delete the functions define in the .makeenv