Difference between revisions of "StarlingX/DebianBuildEnvironment"
(→Install Minikube and Helm) |
|||
Line 19: | Line 19: | ||
# Build packages/ISO creation. | # Build packages/ISO creation. | ||
− | == | + | == Configure build environment == |
− | + | We need to create and start the build containers, which requires some additional configuration described below. | |
− | build | ||
− | == Install Minikube and Helm == | + | === Install Minikube and Helm === |
Install Minikube to support the local k8s framework for building. Install Helm tools to manage the Helm Charts required to | Install Minikube to support the local k8s framework for building. Install Helm tools to manage the Helm Charts required to | ||
Line 30: | Line 29: | ||
that Docker is available in your environment. | that Docker is available in your environment. | ||
− | + | Install minikube (https://minikube.sigs.k8s.io/docs/start/): | |
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 | curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 | ||
sudo install minikube-linux-amd64 /usr/local/bin/minikube | sudo install minikube-linux-amd64 /usr/local/bin/minikube | ||
− | + | Alternatively, we can also use a third-party Minikube binary: | |
curl -LO http://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/releases/v1.20.0/minikube-linux-amd64 | curl -LO http://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/releases/v1.20.0/minikube-linux-amd64 | ||
sudo install minikube-linux-amd64 /usr/local/bin/minikube | sudo install minikube-linux-amd64 /usr/local/bin/minikube | ||
− | Install Helm | + | Install Helm -- you can select the version listed here or the latest released version: |
curl -LO https://get.helm.sh/helm-v3.6.2-linux-amd64.tar.gz | curl -LO https://get.helm.sh/helm-v3.6.2-linux-amd64.tar.gz | ||
Line 46: | Line 45: | ||
sudo mv linux-amd64/helm /usr/local/bin/ | sudo mv linux-amd64/helm /usr/local/bin/ | ||
− | + | Add your user account to docker group: | |
− | + | sudo usermod -aG docker $(id -un) && newgrp docker | |
− | + | === Clone build tools and set up workspace === | |
− | |||
− | + | Clone build tools: | |
+ | export TOOL_HOME=~/DebianBuild | ||
+ | mkdir -p $TOOL_HOME | ||
+ | cd $TOOL_HOME | ||
+ | git clone https://opendev.org/starlingx/tools | ||
− | + | Create a workspace directory; it will be mapped into build container. | |
− | + | export WORKSPACE_HOME=~/DebianBuildWorkspace | |
− | + | mkdir -p $WORKSPACE_HOME | |
+ | sudo mkdir -p /localdisk | ||
+ | sudo ln -sf $WORKSPACE_HOME /localdisk/$(id -nu) | ||
− | + | === Source the environment === | |
− | |||
− | + | The build tools comes with a script, import-stx, which sets up your PATH and other environment as necessary. This script must be sourced before attempting to use any tools: | |
− | |||
− | |||
+ | There's a number of environment variables you can set prior to sourcing this file, please fee free to review that script for a full list. | ||
+ | |||
+ | '''WARNING''': minikube can't work if your $HOME directory points to an NFS location, we need to point it to some other local file system by defining ``MINIKUBE_HOME`` in the environment before sourcing ``import-stx``: | ||
+ | |||
+ | # Necessary if your $HOME is on NFS | ||
+ | export MINIKUBE_HOME=/localdisk/$(id -nu) | ||
+ | # Source the environment | ||
+ | cd $TOOL_HOME/tools | ||
+ | source import-stx | ||
+ | |||
+ | === Configure build containers === | ||
+ | |||
+ | The build expects a configuration file, ``stx.conf`` ([https://opendev.org/starlingx/tools/src/branch/master/stx.conf example]) to exist at the root of the build tools working directory. It is a key/value file containing various build options. The ``stx config`` command may be used to add/modify entries in the file. | ||
+ | |||
+ | # source the environment | ||
+ | cd $TOOL_HOME/tools | ||
+ | source ./import-stx | ||
+ | |||
# Align the builder container to use your user/UID | # Align the builder container to use your user/UID | ||
stx config --add builder.myuname $(id -un) | stx config --add builder.myuname $(id -un) | ||
stx config --add builder.uid $(id -u) | stx config --add builder.uid $(id -u) | ||
+ | |||
# Embedded in ~/localrc of the build container | # Embedded in ~/localrc of the build container | ||
stx config --add project.gituser "First Last" | stx config --add project.gituser "First Last" | ||
stx config --add project.gitemail <your email address> | stx config --add project.gitemail <your email address> | ||
+ | |||
# This will be included in the name of your build container and the basename for $MY_REPO_ROOT_DIR | # This will be included in the name of your build container and the basename for $MY_REPO_ROOT_DIR | ||
stx config --add project.name stx-deb-bld-1 | stx config --add project.name stx-deb-bld-1 | ||
stx config --add project.proxy false | stx config --add project.proxy false | ||
− | |||
# Show all the settings | # Show all the settings | ||
stx config --show | stx config --show | ||
− | + | # Show usage information | |
− | + | stx config --help | |
− | |||
− | |||
− | |||
− | |||
− | stx- | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | === Create build containers === | |
− | |||
− | |||
− | |||
− | + | The ``stx-init-env`` script will download or re-create build (docker) containers, and start them: | |
− | + | cd $TOOL_HOME/tools | |
+ | bash stx-init-env | ||
− | + | The script pulls build containers from DockerHub by default, where a new version is built once per day (ie default container images may be slightly out of date when you pull them). You can force a local re-build as follows: | |
− | + | cd $TOOL_HOME/tools | |
+ | bash stx-init-env --rebuild | ||
+ | Once docker images are available locally, you can start & stop them using the ``stx`` tool: | ||
− | + | stx control start # start builder PODs if not running | |
+ | stx control status # display POD status | ||
+ | stx control stop # stop PODs | ||
− | + | '''WARNING''': any changes to ``stx.conf`` or (``stx config add`` etc) requires that the PODs are re-started. f you want to make changes to the environment in the build container, use ‘stx control stop’, then ‘stx config’ to adjust the variables, and re-start the containers. | |
− | + | stx control stop | |
+ | stx config add <...> | ||
+ | stx control start | ||
− | + | == Entering & controlling Pods == | |
− | + | Once the containers are running, one can enter them (think ``docker exec <...> /bin/bash). While there are 4 containers, most build tasks are driven from the "builder" container, which is the default when using the ``stx`` tool: | |
− | |||
− | |||
− | |||
− | |||
− | |||
+ | # enter the "builder" container | ||
stx control enter | stx control enter | ||
− | + | you can enter other containers as follows | |
− | |||
stx control enter --dockername [builder|pkgbuilder|lat|repomgr] | stx control enter --dockername [builder|pkgbuilder|lat|repomgr] | ||
− | Use ``exit`` command to exit from the node to host | + | Use ``exit`` command to exit from the node to host environment. |
− | + | You can use the ``stx control`` command to start/stop & monitor builder POD status: | |
− | + | # control the Pods | |
+ | stx control start | ||
+ | stx control stop | ||
+ | stx control status | ||
− | stx control | + | # more info |
+ | stx control --help | ||
− | + | The ``status`` command will include Helm status, including deployments and the pods. You can use that information to manually enter or troubleshoot POds using munikube or kubectl. | |
− | manually enter | ||
− | == | + | === Every time you start/restart Pods === |
− | + | Execute these mandatory steps: | |
− | + | sudo apt-get update | |
+ | sudo apt-get install less | ||
+ | git config --global user.name "First Last" | ||
+ | git config --global user.email your@email.com | ||
+ | '''NOTE''': you may see the following errors from apt. You can ignore this and continue. | ||
+ | E: Failed to fetch http://stx-stx-repomgr:80/deb-local-source/dists/bullseye/main/source/Sources 404 Not Found [IP: 10.102.135.193 80] | ||
+ | E: Some index files failed to download. They have been ignored, or old ones used instead. | ||
== Build packages/ISO creation == | == Build packages/ISO creation == | ||
Line 249: | Line 256: | ||
build-image | build-image | ||
+ | ls -al /localdisk/deploy/*.iso | ||
+ | |||
+ | == Log files == | ||
+ | |||
+ | While inside the build container, log files may be found here: | ||
+ | |||
+ | * /localdisk/builder.log /localdisk/pkgbuilder.log - top-level build controller log files | ||
+ | * ${MY_WORKSPACE}/<std or rt>/<package name>/*.build' - individual package build logs |
Revision as of 22:03, 24 January 2022
Contents
StarlingX Build Tools
The Debian build is completed using a set of containers designed to run in a Kubernetes environment. To facilitate this we are currently making use of Minikube and Helm, later on we will provide versions of the Helm Charts to allow for running builds directly on Kubernetes or StarlingX.
There are four containers (stx-builder|stx-pkgbuilder|stx-repomgr| stx-lat-tool) required to complete a build:
- stx-builder: main developer build container.
- stx-pkgbuilder: Debian package builder (uses sbuild).
- stx-repomgr: Debian local repository archive (uses aptly)
- stx-lat-tool: Debian image builder
At a high level the StarlingX ISO image creation flow involves the following general steps (assuming you have already configured Docker on your system).
- Install Minikube and Helm.
- Build or download the StarlingX k8s development environment.
- Enter the stx-builder pod/container to triger the building task.
- Build packages/ISO creation.
Configure build environment
We need to create and start the build containers, which requires some additional configuration described below.
Install Minikube and Helm
Install Minikube to support the local k8s framework for building. Install Helm tools to manage the Helm Charts required to start/stop/upgrade the pods or the deployments for the StarlingX Building system. Before installing these components please make sure that Docker is available in your environment.
Install minikube (https://minikube.sigs.k8s.io/docs/start/):
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 sudo install minikube-linux-amd64 /usr/local/bin/minikube
Alternatively, we can also use a third-party Minikube binary:
curl -LO http://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/releases/v1.20.0/minikube-linux-amd64 sudo install minikube-linux-amd64 /usr/local/bin/minikube
Install Helm -- you can select the version listed here or the latest released version:
curl -LO https://get.helm.sh/helm-v3.6.2-linux-amd64.tar.gz tar xvf helm-v3.6.2-linux-amd64.tar.gz sudo mv linux-amd64/helm /usr/local/bin/
Add your user account to docker group:
sudo usermod -aG docker $(id -un) && newgrp docker
Clone build tools and set up workspace
Clone build tools:
export TOOL_HOME=~/DebianBuild mkdir -p $TOOL_HOME cd $TOOL_HOME git clone https://opendev.org/starlingx/tools
Create a workspace directory; it will be mapped into build container.
export WORKSPACE_HOME=~/DebianBuildWorkspace mkdir -p $WORKSPACE_HOME sudo mkdir -p /localdisk sudo ln -sf $WORKSPACE_HOME /localdisk/$(id -nu)
Source the environment
The build tools comes with a script, import-stx, which sets up your PATH and other environment as necessary. This script must be sourced before attempting to use any tools:
There's a number of environment variables you can set prior to sourcing this file, please fee free to review that script for a full list.
WARNING: minikube can't work if your $HOME directory points to an NFS location, we need to point it to some other local file system by defining ``MINIKUBE_HOME`` in the environment before sourcing ``import-stx``:
# Necessary if your $HOME is on NFS export MINIKUBE_HOME=/localdisk/$(id -nu) # Source the environment cd $TOOL_HOME/tools source import-stx
Configure build containers
The build expects a configuration file, ``stx.conf`` (example) to exist at the root of the build tools working directory. It is a key/value file containing various build options. The ``stx config`` command may be used to add/modify entries in the file.
# source the environment cd $TOOL_HOME/tools source ./import-stx # Align the builder container to use your user/UID stx config --add builder.myuname $(id -un) stx config --add builder.uid $(id -u) # Embedded in ~/localrc of the build container stx config --add project.gituser "First Last" stx config --add project.gitemail <your email address> # This will be included in the name of your build container and the basename for $MY_REPO_ROOT_DIR stx config --add project.name stx-deb-bld-1 stx config --add project.proxy false # Show all the settings stx config --show
# Show usage information stx config --help
Create build containers
The ``stx-init-env`` script will download or re-create build (docker) containers, and start them:
cd $TOOL_HOME/tools bash stx-init-env
The script pulls build containers from DockerHub by default, where a new version is built once per day (ie default container images may be slightly out of date when you pull them). You can force a local re-build as follows:
cd $TOOL_HOME/tools bash stx-init-env --rebuild
Once docker images are available locally, you can start & stop them using the ``stx`` tool:
stx control start # start builder PODs if not running stx control status # display POD status stx control stop # stop PODs
WARNING: any changes to ``stx.conf`` or (``stx config add`` etc) requires that the PODs are re-started. f you want to make changes to the environment in the build container, use ‘stx control stop’, then ‘stx config’ to adjust the variables, and re-start the containers.
stx control stop stx config add <...> stx control start
Entering & controlling Pods
Once the containers are running, one can enter them (think ``docker exec <...> /bin/bash). While there are 4 containers, most build tasks are driven from the "builder" container, which is the default when using the ``stx`` tool:
# enter the "builder" container stx control enter
you can enter other containers as follows
stx control enter --dockername [builder|pkgbuilder|lat|repomgr]
Use ``exit`` command to exit from the node to host environment.
You can use the ``stx control`` command to start/stop & monitor builder POD status:
# control the Pods stx control start stx control stop stx control status
# more info stx control --help
The ``status`` command will include Helm status, including deployments and the pods. You can use that information to manually enter or troubleshoot POds using munikube or kubectl.
Every time you start/restart Pods
Execute these mandatory steps:
sudo apt-get update sudo apt-get install less git config --global user.name "First Last" git config --global user.email your@email.com
NOTE: you may see the following errors from apt. You can ignore this and continue.
E: Failed to fetch http://stx-stx-repomgr:80/deb-local-source/dists/bullseye/main/source/Sources 404 Not Found [IP: 10.102.135.193 80] E: Some index files failed to download. They have been ignored, or old ones used instead.
Build packages/ISO creation
The six builder is the container where you will perform most of the actions, such as launching the task of building packages and images.
stx control enter
Initialize the source tree
The StarlingX source tree consists of multiple git repositories. The tool ‘repo’ is used to sync these repositories locally, below config Is minimally required to config to make ‘repo’ work:
cd $MY_REPO_ROOT_DIR repo init -u https://opendev.org/starlingx/manifest -m default.xml repo sync
After the ‘repo sync’ is done, check the below directory:
$ ls $MY_REPO $ ls $MY_REPO/stx $ ls $MY_REPO_ROOT_DIR/stx-tools
Before running 'build-pkgs':
Run below command to download the sources of all buildable packages by scanning the repo root $MY_REPO/stx
the download directory is: $STX_MIRROR/sources
$ downloader -s
All the below lists with build types will be scanned in the repo root $MY_REPO/stx:
debian_pkg_dirs
debian_pkg_dirs_rt
debian_pkg_dirs_installer
Before running 'build-image':
Run below command to download the debian binary packages (distribution: bullseye) into directory $STX_MIRROR/binaries:
$ downloader -b
All the below lists of binary packages will be downloaded:
$MY_REPO_ROOT_DIR/stx-tools/debian-mirror-tools/config/debian/common/base-bullseye.lst
$MY_REPO_ROOT_DIR/stx-tools/debian-mirror-tools/config/debian/<layer>/os-std.lst
$MY_REPO_ROOT_DIR/stx-tools/debian-mirror-tools/config/debian/<layer>/os-rt.lst
Also run below command to download both sources and binaries:
$ downloader -b -s
$ downloader --help usage: downloader [-h] [-b] [-s] [-c] downloader helper optional arguments: -h, --help show this help message and exit -b, --download_binary download binary debs -s, --download_source download stx source -c, --clean_mirror clean the whole mirror and download again, be careful to use
Current the apt sources used to download packages is in '/etc/apt/sources.list' of builder container.
Build packages
To bulld an individual package:
build-pkgs -p <name of package>
To build all of the packages available
build-pkgs -a
Build ISO
Once you have built all of the packages you can build the iso by running the following command:
build-image ls -al /localdisk/deploy/*.iso
Log files
While inside the build container, log files may be found here:
- /localdisk/builder.log /localdisk/pkgbuilder.log - top-level build controller log files
- ${MY_WORKSPACE}/<std or rt>/<package name>/*.build' - individual package build logs