Jump to: navigation, search

Difference between revisions of "StarlingX/Developer Guide"

m (Android Repo Tool)
Line 260: Line 260:
<li> Tarballs Repository
<li> Tarballs Repository
<source lang="sh">$ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/
<source lang="sh">
$ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/
Alternatively you can run the populate_downloads.sh script to copy the tarballs instead of using a symlink.
<source lang="sh">
$ populate_downloads.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
Outside the container
Outside the container
Line 282: Line 290:
<li><strong>Update the symbolic links</strong>
<li><strong>Update the symbolic links</strong>
<source lang="sh">$ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
<source lang="sh">
$ populate_downloads.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
$ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
Line 296: Line 304:
<source lang="sh">$ generate-cgcs-tis-repo
<source lang="sh">
$ generate-cgcs-tis-repo
==Build StarlingX ISO==
==Build StarlingX ISO==

Revision as of 04:55, 5 September 2018

This section contains the steps for building a StarlingX ISO from Master branch.


The recommended minimum requirements include:

Hardware Requirements

A workstation computer with:

  • Processor: x86_64 is the only supported architecture
  • Memory: At least 32GB RAM
  • Hard Disk: 500GB HDD
  • Network: Network adapter with active Internet connection

Software Requirements

A workstation computer with:

Development Environment Setup

This section describes how to set up a StarlingX development system on a workstation computer. After completing these steps, you will be able to build a StarlingX ISO image on the following Linux distribution:

  • Ubuntu 16.04 LTS 64-bit

Update Your Operating System

Before proceeding with the build, ensure your OS is up to date. You’ll first need to update the local database list of available packages:

$ sudo apt-get update

Installation Requirements and Dependencies


  1. Install the required packages in an Ubuntu host system with:
    $ sudo apt-get install make git curl
  2. Make sure to setup your identity
    $ git config --global user.name "Name LastName"
    $ git config --global user.email "Email Address"
  3. Docker CE

  4. Install the required Docker CE packages in an Ubuntu host system. See Get Docker for more information.

    Android Repo Tool

  5. <p>Install the required Android Repo Tool in an Ubuntu host system. Follow the 2 steps in "Installing Repo" section from Installing Repo to have Andriod Repo Tool installed.

  6. Install Public SSH Key

    1. Follow these instructions on GitHub to Generate a Public SSH Key and then upload your public key to your GitHub and Gerrit account profiles:

    Install stx-tools project

    1. Under your $HOME directory, clone the <stx-tools> project
      $ cd $HOME
      $ git clone https://git.starlingx.io/stx-tools

    Create a Workspace Directory

    1. Create a starlingx workspace directory on your workstation computer. Usually, you’ll want to create it somewhere under your user’s home directory.
      $ mkdir -p $HOME/starlingx/

    Build the CentOS Mirror Repository

    This section describes how to build the CentOS Mirror Repository.

    Setup Repository Docker Container

    Run the following commands under a terminal identified as "One".

    1. Navigate to the <$HOME/stx-tools>/centos-mirror-tool project directory:
      $ cd $HOME/stx-tools/centos-mirror-tools/
    2. Temporal! Only Branch Master Check if the following Gerrit Reviews are merged, if not, cherry pick them
    3. Temporal! Only Branch m/2018.07 Check if the following Gerrit Reviews are merged, if not, cherry pick them
    4. If necessary you might have to set http/https proxy in your Dockerfile before building the docker image.
      ENV http_proxy " http://your.actual_http_proxy.com:your_port "
      ENV https_proxy " https://your.actual_https_proxy.com:your_port "
      ENV ftp_proxy " http://your.actual_ftp_proxy.com:your_port "
      RUN echo " proxy=http://your-proxy.com:port " >> /etc/yum.conf
    5. Build your <user>:<tag> base container image with e.g. user:centos-mirror-repository
      $ docker build -t <user>:centos-mirror-repository -f Dockerfile .
    6. Launch a <user> docker container using previously created Docker base container image <user>:<tag> e.g. <user>-centos-mirror-repository. As /localdisk is defined as the workdir of the container, the same folder name should be used to define the volume. The container will start to run and populate a logs and output folders in this directory. The container shall be run from the same directory where the other scripts are stored.
      $ docker run -it --name <user>-centos-mirror-repository -v $(pwd):/localdisk <user>:centos-mirror-repository

    Download Packages

    1. Inside Repository Docker container, enter the following command to download the required packages to populate the CentOS Mirror Repository:
      # bash download_mirror.sh
    2. Monitor the download of packages until it is complete. When download is complete, the following message is displayed:
      totally 17 files are downloaded!
      step #3: done successfully
      IMPORTANT: The following 3 files are just bootstrap versions. Based on them, the workable images
      for StarlingX could be generated by running "update-pxe-network-installer" command after "build-iso"
          - out/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img
          - out/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img
          - out/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz

    Verify Packages

    1. Verify there are no missing or failed packages:
      # cat output/3rd-centos_rpms_missing_L1.txt output/3rd_rpms_missing_L1.txt output/3rd_srpms_missing_L1.txt output/centos_rpms_missing_L1.txt output/centos_srpms_missing_L1.txt
      # cat output/3rd_rpms_fail_move_L1.txt output/3rd_srpms_fail_move_L1.txt output/centos_rpms_fail_move_L1.txt output/centos_srpms_fail_move_L1.txt
    2. In case there are missing or failed ones due to network instability (or timeout), you should download them manually, to assure you get all RPMs listed in "rpms_from_3rd_parties.lst" and "rpms_from_centos_repo.lst".

    3. Temporal! None

      $ None

    Create CentOS Mirror Repository

    Outside your Repository Docker container, in another terminal identified as "Two", run the following commands:

    1. From terminal identified as "Two", create a mirror/CentOS directory under your starlingx workspace directory:
      $ mkdir -p $HOME/starlingx/mirror/CentOS/
    2. Copy the built CentOS Mirror Repository built under <stx-tools>/centos-mirror-tool to the $HOME/starlingx/mirror/ workspace directory.
      $ cp -r $HOME/stx-tools/centos-mirror-tools/output/stx-r1/ $HOME/starlingx/mirror/CentOS/

    Create StarlingX Packages

    Setup Building Docker Container

    1. From terminal identified as "Two", Workspace Setup
      $ mkdir -p $HOME/starlingx/workspace
    2. Navigate to the <$HOME/stx-tools> project directory:
      $ cd $HOME/stx-tools
    3. Copy your git options
      $ cp ~/.gitconfig toCOPY
    4. Create a <localrc> file
      $ nano localrc
    5. # tbuilder localrc
    6. If necessary you might have to set http/https proxy in your Dockerfile.centos73 before building the docker image.
      ENV http_proxy  "http://your.actual_http_proxy.com:your_port"
      ENV https_proxy "https://your.actual_https_proxy.com:your_port"
      ENV ftp_proxy "http://your.actual_ftp_proxy.com:your_port"
      RUN echo "proxy=http://your-proxy.com:port" >> /etc/yum.conf
    7. Base container setup If you are running in fedora system, you will see " .makeenv:88: *** missing separator. Stop. " error, to continue :
      • delete the functions define in the .makeenv ( module () { ... } )
      • delete the line 19 in the Makefile and ( NULL := $(shell bash -c "source buildrc ... ).
      $ make base-build
    8. Build container setup
      $ make build
    9. Verify environment variables
      $ bash tb.sh env
    10. Build container run
      $ bash tb.sh run
    11. Execute the built container:
      $ bash tb.sh exec

    Download Source Code Repositories

    1. From terminal identified as "Two", now inside the Building Docker container, Internal environment
      $ eval $(ssh-agent)
      $ ssh-add
    2. Repo init
      $ cd $MY_REPO_ROOT_DIR
      $ repo init -u https://git.starlingx.io/stx-manifest.git -m default.xml
    3. Repo sync
      $ repo sync
    4. Temporal! Only Branch Master Check if the following Gerrit Reviews are merged, if not, cherry pick them
    5. Tarballs Repository
      $ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/

      Alternatively you can run the populate_downloads.sh script to copy the tarballs instead of using a symlink.

      $ populate_downloads.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
    6. Outside the container

    7. From another terminal identified as "Three", Mirror Binaries
      $ mkdir -p $HOME/starlingx/mirror/CentOS/tis-installer
      $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img $HOME/starlingx/mirror/CentOS/tis-installer/initrd.img-stx-0.2
      $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz $HOME/starlingx/mirror/CentOS/tis-installer/vmlinuz-stx-0.2
      $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img $HOME/starlingx/mirror/CentOS/tis-installer/squashfs.img-stx-0.2

    Build Packages

    1. Back to the Building Docker container, terminal identified as "Two"
    2. Temporal! Build-Pkgs Errors Be prepared to have some missing / corrupted rpm and tarball packages generated during StarlingX/Developer_Guide#Build_the_CentOS_Mirror_Repository which will make the next step to fail, if that happens please download manually those missing / corrupted packages.
    3. Update the symbolic links
      $ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
    4. Build-Pkgs
      $ build-pkgs
    5. Optional! Generate-Cgcs-Tis-Repo
      This step is optional but but will improve performance on subsequent builds. The cgcs-tis-repo has the dependency information that sequences the build order; To generate or update the information the following command needs to be executed after building modified or new packages.
    6. $ generate-cgcs-tis-repo

    Build StarlingX ISO

    1. Build-Iso
      $ build-iso