Jump to: navigation, search

Difference between revisions of "StarlingX/Developer Guide Master"

(Build Packages)
(Replaced content with "Please use * Developer Guide")
 
(16 intermediate revisions by the same user not shown)
Line 1: Line 1:
This section contains the steps for building a StarlingX ISO from Master branch.
+
Please use * [[StarlingX/Developer Guide|Developer Guide]]
 
 
== Requirements ==
 
 
 
The recommended minimum requirements include:
 
 
 
=== Hardware Requirements ===
 
 
 
A workstation computer with:
 
 
 
* Processor: x86_64 only supported architecture
 
* Memory: At least 32GB RAM
 
* Hard Disk: 500GB HDD
 
* Network: Network adapter with active Internet connection
 
 
 
=== Software Requirements ===
 
 
 
A workstation computer with:
 
 
 
* Operating System: Ubuntu 16.04 LTS 64-bit
 
* Docker
 
* Android Repo Tool
 
* Proxy Settings Configured (If Required)
 
** See http://lists.starlingx.io/pipermail/starlingx-discuss/2018-July/000136.html for more details
 
* Public SSH Key
 
 
 
== Development Environment Setup ==
 
 
 
This section describes how to set up a StarlingX development system on a workstation computer. After completing these steps, you will be able to build a StarlingX ISO image on the following Linux distribution:
 
 
 
* Ubuntu 16.04 LTS 64-bit
 
 
 
=== Update Your Operating System ===
 
 
 
Before proceeding with the build, ensure your OS is up to date. You’ll first need to update the local database list of available packages:
 
 
 
<source lang="sh">$ sudo apt-get update
 
</source>
 
=== Installation Requirements and Dependencies ===
 
 
 
<ol start="1">
 
==== Git ====
 
<li>Install the required packages in an Ubuntu host system with:
 
<source lang="sh">$ sudo apt-get install git
 
</source></li>
 
==== Docker CE ====
 
<li><p>Install the required Docker CE packages in an Ubuntu host system. See [https://docs.docker.com/install/ Get Docker] for more information.</p></li>
 
==== Android Repo Tool ====
 
<li><p>Install the required Android Repo Tool in an Ubuntu host system. Follow the 2 steps in "Installing Repo" section from [https://source.android.com/setup/build/downloading#installing-repo Installing Repo] to have Andriod Repo Tool installed.</p></li></ol>
 
 
 
=== Install Public SSH Key===
 
 
 
# Follow these instructions on GitHub to [https://help.github.com/articles/connecting-to-github-with-ssh Generate a Public SSH Key] and then upload your public key to your GitHub and Gerrit account profiles:
 
#* [https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account Upload to Github]
 
#* [https://review.openstack.org/#/settings/ssh-keys Upload to Gerrit]
 
 
 
=== Install stx-tools project ===
 
 
 
<ol start="1"><li>Under your $HOME directory, clone the &lt;stx-tools&gt; project
 
 
 
<source lang="sh">$ cd $HOME
 
$ git clone git://git.openstack.org/openstack/stx-tools
 
</source></li></ol>
 
 
 
=== Create a Workspace Directory ===
 
 
 
<ol start="1"><li>Create a ''starlingx'' workspace directory on your workstation computer. Usually, you’ll want to create it somewhere under your user’s home directory.
 
 
 
<source lang="sh">$ mkdir -p $HOME/starlingx/
 
</source></li></ol>
 
 
 
== Build the CentOS Mirror Repository ==
 
 
 
This section describes how to build the CentOS Mirror Repository.
 
 
 
=== Setup Repository Docker Container ===
 
 
 
Run the following commands under a terminal identified as "one". <br>
 
 
 
<ol start="1"><li>Navigate to the ''&lt;$HOME/stx-tools&gt;/centos-mirror-tool'' project directory:
 
 
 
<source lang="sh">$ cd $HOME/stx-tools/centos-mirror-tools/
 
</source></li>
 
 
 
<li><strong> Temporal! </strong> Check if the following Gerrit Reviews are merged, if not, cherry pick them <br>
 
https://review.openstack.org/#/c/584538/4 <br>
 
https://review.openstack.org/#/c/584156/ <br>
 
https://review.openstack.org/#/c/584752/ <br>
 
</li>
 
 
 
<li>If necessary you might have to set http/https proxy in your Dockerfile before building the docker image.
 
 
 
<source lang="sh">ENV http_proxy  "http://your.actual_http_proxy.com:your_port" && \
 
    https_proxy "https://your.actual_https_proxy.com:your_port" && \
 
    ftp_proxy "http://your.actual_ftp_proxy.com:your_port"
 
RUN echo "proxy=http://your-proxy.com:port" >> /etc/yum.conf
 
</source></li>
 
<li>Build your ''&lt;user&gt;:&lt;tag&gt;'' base container image with '''e.g.''' ''user:centos-mirror-repository''
 
 
 
<source lang="sh">$ docker build -t user:centos-mirror-repository -f Dockerfile .
 
</source></li>
 
<li>Launch a ''&lt;user&gt;'' docker container using previously created Docker base container image ''&lt;user&gt;:&lt;tag&gt;'' '''e.g.''' ''user-centos-mirror-repository''. As /localdisk is defined as the workdir of the container, the same folder name should be used to define the volume. The container will start to run and populate a logs and output folders in this directory. The container shall be run from the same directory where the other scripts are stored.
 
 
 
<source lang="sh">$ docker run -it --name user-centos-mirror-repository -v $(pwd):/localdisk user:centos-mirror-repository
 
</source></li>
 
 
 
=== Download Packages ===
 
 
 
<ol start="1"><li>Enter the following command to download the required packages to populate the CentOS Mirror Repository:
 
 
 
<source lang="none"># bash download_mirror.sh
 
</source></li>
 
<li>Monitor the download of packages until it is complete. When download is complete, the following message is displayed:
 
 
 
<source lang="none">totally 17 files are downloaded!
 
step #3: done successfully
 
IMPORTANT: The following 3 files are just bootstrap versions. Based on them, the workable images
 
for StarlingX could be generated by running "update-pxe-network-installer" command after "build-iso"
 
    - out/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img
 
    - out/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img
 
    - out/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz
 
</source></li></ol>
 
 
 
=== Verify Packages ===
 
 
 
<ol start="1"><li>Verify there are no missing or failed packages:
 
 
 
<source lang="none"># cat output/3rd-centos_rpms_missing_L1.txt output/3rd_rpms_missing_L1.txt output/3rd_srpms_missing_L1.txt output/centos_rpms_missing_L1.txt output/centos_srpms_missing_L1.txt
 
# cat output/3rd_rpms_fail_move_L1.txt output/3rd_srpms_fail_move_L1.txt output/centos_rpms_fail_move_L1.txt output/centos_srpms_fail_move_L1.txt
 
</source></li>
 
<li><p>In case there are missing or failed ones due to network instability (or timeout), you should download them manually, to assure you get all RPMs listed in &quot;rpms_from_3rd_parties.lst&quot; and &quot;rpms_from_centos_repo.lst&quot;.</p></li>
 
 
 
<li><p>After all packages were succesfully downloaded, remove all i686 RPMs packages and change ''output'' directory ownership:</p>
 
 
 
<source lang="none"># find ./output -name "*.i686.rpm" | xargs rm -f
 
# chown 751:751 -R ./output
 
</source></li></ol>
 
 
 
=== Create CentOS Mirror Repository ===
 
 
 
Run the following commands under a terminal identified as "two".
 
 
 
<ol start="1"><li>From terminal "two", create a ''mirror/CentOS'' directory under your ''starlingx'' workspace directory:
 
 
 
<source lang="sh">$ mkdir -p $HOME/starlingx/mirror/CentOS/
 
</source></li>
 
<li>Copy the built CentOS Mirror Repository built under ''&lt;stx-tools&gt;/centos-mirror-tool'' to the ''$HOME/starlingx/mirror/CentOS'' workspace directory.
 
 
 
<source lang="sh">$ cp -r $HOME/stx-tools/centos-mirror-tools/output/stx-r1/ $HOME/starlingx/mirror/CentOS/
 
</source></li></ol>
 
 
 
==Create StarlingX Packages==
 
===Setup Building Docker Container===
 
 
 
<ol start="1"><li>From terminal "two", Workspace Setup
 
 
 
<source lang="sh">$ mkdir -p $HOME/starlingx/workspace
 
</source></li>
 
 
 
<li>Navigate to the ''&lt;$HOME/stx-tools&gt;'' project directory:
 
 
 
<source lang="sh">$ cd $HOME/stx-tools
 
</source></li>
 
 
 
<li>Copy your git options
 
 
 
<source lang="sh">$ cp ~/.gitconfig toCOPY
 
</source></li>
 
 
 
<li>Create a ''&lt;localrc&gt;'' file
 
 
 
<source lang="sh">$ nano localrc
 
</source></li>
 
 
 
<source lang="sh">
 
# tbuilder localrc
 
MYUNAME=user
 
PROJECT=starlingx
 
HOST_PREFIX=$HOME/starlingx/workspace
 
HOST_MIRROR_DIR=$HOME/starlingx/mirror
 
</source></li>
 
 
 
<li>If necessary you might have to set http/https proxy in your Dockerfile.centos73 before building the docker image.
 
 
 
<source lang="sh">ENV http_proxy  "http://your.actual_http_proxy.com:your_port" && \
 
    https_proxy "https://your.actual_https_proxy.com:your_port" && \
 
    ftp_proxy "http://your.actual_ftp_proxy.com:your_port"
 
RUN echo "proxy=http://your-proxy.com:port" >> /etc/yum.conf
 
</source></li>
 
 
 
<li>Base container setup
 
If you are running in fedora system, you will see  " .makeenv:88: *** missing separator.  Stop. " error, to continue :
 
 
 
* delete the functions define in the .makeenv ( module () { ... } )
 
* delete the line 19 in the Makefile and ( NULL := $(shell bash -c "source buildrc ... ).
 
 
 
 
 
<source lang="sh">$ make base-build
 
</source></li>
 
 
 
<li>Build container setup
 
 
 
<source lang="sh">
 
$ make build
 
</source></li>
 
 
 
<li>Verify environment variables
 
 
 
<source lang="sh">$ bash tb.sh env
 
</source></li>
 
 
 
<li>Build container run
 
 
 
<source lang="sh">$ bash tb.sh run
 
</source></li>
 
 
 
<li>Execute the built container:
 
 
 
<source lang="sh">$ bash tb.sh exec
 
</source></li></ol>
 
 
 
===Download Source Code Repositories===
 
 
 
<ol start="1"><li>Internal environment
 
 
 
<source lang="sh">$ eval $(ssh-agent)
 
$ ssh-add
 
</source></li>
 
 
 
<li>Repo init
 
 
 
<source lang="sh">$ cd $MY_REPO_ROOT_DIR
 
$ repo init -u git://git.openstack.org/openstack/stx-manifest.git -m default.xml
 
</source></li>
 
 
 
<li>Repo sync
 
 
 
<source lang="sh">$ repo sync
 
</source></li>
 
 
 
<li> cgcs-centos-repo
 
 
 
<source lang="sh">$ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
 
</source></li>
 
 
 
<li> cgcs-dl
 
 
 
<source lang="sh">$ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/
 
</source></li>
 
 
 
Outside the container, use another terminal
 
 
 
<li>Mirror Binaries
 
 
 
<source lang="sh">$ mkdir -p $HOME/starlingx/mirror/CentOS/tis-installer
 
$ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img $HOME/starlingx/mirror/CentOS/tis-installer/initrd.img-stx-0.2
 
$ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz $HOME/starlingx/mirror/CentOS/tis-installer/vmlinuz-stx-0.2
 
$ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img $HOME/starlingx/mirror/CentOS/tis-installer/squashfs.img-stx-0.2
 
</source></li></ol>
 
 
 
===Build Packages===
 
 
 
<ol start="1"><li><strong>Temporal!</strong> Build-Pkgs Errors
 
Be prepared to have some missing / corrupted packages generated during [[StarlingX/Developer_Guide#Build_the_CentOS_Mirror_Repository]] which will make the next step to fail, if that happens please download manually those missing / corrupted packages. <br>
 
</li>
 
 
 
<li><strong>Temporal!</strong> "Adding comps.xml and mock.proto default files." <br>
 
https://review.openstack.org/#/c/584214/
 
</li>
 
 
 
<li><strong>Update the symbolic links</strong>
 
<source lang="sh">$ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
 
</source></li>
 
 
 
<li>Build-Pkgs --Serial
 
 
 
<source lang="sh">$ cd $MY_WORKSPACE
 
$ build-pkgs --serial
 
</source></li>
 
 
 
<li> Generate-Cgcs-Tis-Repo
 
 
 
The cgcs-tis-repo has the dependency information that sequences the build order; To generate or update the information the following command needs to be executed after building modified or new packages.
 
<source lang="sh">$ generate-cgcs-tis-repo
 
</source></li>
 
 
 
<li>Build-Srpms
 
 
 
<source lang="sh">$ build-srpms --installer
 
</source></li>
 
 
 
<li>Build-Rpms
 
 
 
<source lang="sh">$ build-rpms --installer
 
</source></li></ol>
 
 
 
==Build StarlingX ISO==
 
 
 
<ol start="1"><li><strong>Temporal!</strong> Depedency Error, remove by commenting temporarely all kubernetes packages + dockers-ce + selinux-policy under image.inc
 
 
 
<source lang="sh">$ vi $MY_REPO/build-tools/build_iso/image.inc
 
...
 
#selinux-policy
 
...
 
#dockers-ce
 
#kubernetes
 
#kubernetes-client
 
#kubernetes-kubeadm
 
#kubernetes-master
 
#kubernetes-node
 
...
 
</source></li>
 
 
 
<li>Build-Iso
 
 
 
<source lang="sh">$ build-iso
 
</source></li></ol>
 

Latest revision as of 16:14, 6 August 2018

Please use * Developer Guide