|
|
(5 intermediate revisions by the same user not shown) |
Line 1: |
Line 1: |
− | This section contains the steps for building a StarlingX ISO from Master branch.
| + | Please use * [[StarlingX/Developer Guide|Developer Guide]] |
− | | |
− | == Requirements ==
| |
− | | |
− | The recommended minimum requirements include:
| |
− | | |
− | === Hardware Requirements ===
| |
− | | |
− | A workstation computer with:
| |
− | | |
− | * Processor: x86_64 only supported architecture
| |
− | * Memory: At least 32GB RAM
| |
− | * Hard Disk: 500GB HDD
| |
− | * Network: Network adapter with active Internet connection
| |
− | | |
− | === Software Requirements ===
| |
− | | |
− | A workstation computer with:
| |
− | | |
− | * Operating System: Ubuntu 16.04 LTS 64-bit
| |
− | * Docker
| |
− | * Android Repo Tool
| |
− | * Proxy Settings Configured (If Required)
| |
− | ** See http://lists.starlingx.io/pipermail/starlingx-discuss/2018-July/000136.html for more details
| |
− | * Public SSH Key
| |
− | | |
− | == Development Environment Setup ==
| |
− | | |
− | This section describes how to set up a StarlingX development system on a workstation computer. After completing these steps, you will be able to build a StarlingX ISO image on the following Linux distribution:
| |
− | | |
− | * Ubuntu 16.04 LTS 64-bit
| |
− | | |
− | === Update Your Operating System ===
| |
− | | |
− | Before proceeding with the build, ensure your OS is up to date. You’ll first need to update the local database list of available packages:
| |
− | | |
− | <source lang="sh">$ sudo apt-get update
| |
− | </source>
| |
− | === Installation Requirements and Dependencies ===
| |
− | | |
− | <ol start="1">
| |
− | ==== Git ====
| |
− | <li>Install the required packages in an Ubuntu host system with:
| |
− | <source lang="sh">$ sudo apt-get install git
| |
− | </source></li>
| |
− | ==== Docker CE ====
| |
− | <li><p>Install the required Docker CE packages in an Ubuntu host system. See [https://docs.docker.com/install/ Get Docker] for more information.</p></li>
| |
− | ==== Android Repo Tool ====
| |
− | <li><p>Install the required Android Repo Tool in an Ubuntu host system. Follow the 2 steps in "Installing Repo" section from [https://source.android.com/setup/build/downloading#installing-repo Installing Repo] to have Andriod Repo Tool installed.</p></li></ol>
| |
− | | |
− | === Install Public SSH Key===
| |
− | | |
− | # Follow these instructions on GitHub to [https://help.github.com/articles/connecting-to-github-with-ssh Generate a Public SSH Key] and then upload your public key to your GitHub and Gerrit account profiles:
| |
− | #* [https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account Upload to Github]
| |
− | #* [https://review.openstack.org/#/settings/ssh-keys Upload to Gerrit]
| |
− | | |
− | === Install stx-tools project ===
| |
− | | |
− | <ol start="1"><li>Under your $HOME directory, clone the <stx-tools> project
| |
− | | |
− | <source lang="sh">$ cd $HOME
| |
− | $ git clone git://git.openstack.org/openstack/stx-tools
| |
− | </source></li></ol>
| |
− | | |
− | === Create a Workspace Directory ===
| |
− | | |
− | <ol start="1"><li>Create a ''starlingx'' workspace directory on your workstation computer. Usually, you’ll want to create it somewhere under your user’s home directory.
| |
− | | |
− | <source lang="sh">$ mkdir -p $HOME/starlingx/
| |
− | </source></li></ol>
| |
− | | |
− | == Build the CentOS Mirror Repository ==
| |
− | | |
− | This section describes how to build the CentOS Mirror Repository.
| |
− | | |
− | === Setup Repository Docker Container ===
| |
− | | |
− | Run the following commands under a terminal identified as "one". <br>
| |
− | | |
− | <ol start="1"><li>Navigate to the ''<$HOME/stx-tools>/centos-mirror-tool'' project directory:
| |
− | | |
− | <source lang="sh">$ cd $HOME/stx-tools/centos-mirror-tools/
| |
− | </source></li>
| |
− | | |
− | <li><strong> Temporal! Only Branch Master </strong> Check if the following Gerrit Reviews are merged, if not, cherry pick them <br>
| |
− | https://review.openstack.org/#/c/584752/ <br>
| |
− | </li>
| |
− | | |
− | <li><strong> Temporal! Only Branch m/2018.07 </strong> Check if the following Gerrit Reviews are merged, if not, cherry pick them <br>
| |
− | https://review.openstack.org/#/c/585041/ <br>
| |
− | https://review.openstack.org/#/c/585042/ <br>
| |
− | https://review.openstack.org/#/c/585046/ <br>
| |
− | </li>
| |
− | | |
− | <li>If necessary you might have to set http/https proxy in your Dockerfile before building the docker image.
| |
− | | |
− | <source lang="sh">ENV http_proxy "http://your.actual_http_proxy.com:your_port" && \
| |
− | https_proxy "https://your.actual_https_proxy.com:your_port" && \
| |
− | ftp_proxy "http://your.actual_ftp_proxy.com:your_port"
| |
− | RUN echo "proxy=http://your-proxy.com:port" >> /etc/yum.conf
| |
− | </source></li>
| |
− | <li>Build your ''<user>:<tag>'' base container image with '''e.g.''' ''user:centos-mirror-repository''
| |
− | | |
− | <source lang="sh">$ docker build -t user:centos-mirror-repository -f Dockerfile .
| |
− | </source></li>
| |
− | <li>Launch a ''<user>'' docker container using previously created Docker base container image ''<user>:<tag>'' '''e.g.''' ''user-centos-mirror-repository''. As /localdisk is defined as the workdir of the container, the same folder name should be used to define the volume. The container will start to run and populate a logs and output folders in this directory. The container shall be run from the same directory where the other scripts are stored.
| |
− | | |
− | <source lang="sh">$ docker run -it --name user-centos-mirror-repository -v $(pwd):/localdisk user:centos-mirror-repository
| |
− | </source></li>
| |
− | | |
− | === Download Packages ===
| |
− | | |
− | <ol start="1"><li>Enter the following command to download the required packages to populate the CentOS Mirror Repository:
| |
− | | |
− | <source lang="none"># bash download_mirror.sh
| |
− | </source></li>
| |
− | <li>Monitor the download of packages until it is complete. When download is complete, the following message is displayed:
| |
− | | |
− | <source lang="none">totally 17 files are downloaded!
| |
− | step #3: done successfully
| |
− | IMPORTANT: The following 3 files are just bootstrap versions. Based on them, the workable images
| |
− | for StarlingX could be generated by running "update-pxe-network-installer" command after "build-iso"
| |
− | - out/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img
| |
− | - out/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img
| |
− | - out/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz
| |
− | </source></li></ol>
| |
− | | |
− | === Verify Packages ===
| |
− | | |
− | <ol start="1"><li>Verify there are no missing or failed packages:
| |
− | | |
− | <source lang="none"># cat output/3rd-centos_rpms_missing_L1.txt output/3rd_rpms_missing_L1.txt output/3rd_srpms_missing_L1.txt output/centos_rpms_missing_L1.txt output/centos_srpms_missing_L1.txt
| |
− | # cat output/3rd_rpms_fail_move_L1.txt output/3rd_srpms_fail_move_L1.txt output/centos_rpms_fail_move_L1.txt output/centos_srpms_fail_move_L1.txt
| |
− | </source></li>
| |
− | <li><p>In case there are missing or failed ones due to network instability (or timeout), you should download them manually, to assure you get all RPMs listed in "rpms_from_3rd_parties.lst" and "rpms_from_centos_repo.lst".</p></li>
| |
− | | |
− | === Create CentOS Mirror Repository ===
| |
− | | |
− | Run the following commands under a terminal identified as "two".
| |
− | | |
− | <ol start="1"><li>From terminal "two", create a ''mirror/CentOS'' directory under your ''starlingx'' workspace directory:
| |
− | | |
− | <source lang="sh">$ mkdir -p $HOME/starlingx/mirror/CentOS/
| |
− | </source></li>
| |
− | <li>Copy the built CentOS Mirror Repository built under ''<stx-tools>/centos-mirror-tool'' to the ''$HOME/starlingx/mirror/CentOS'' workspace directory.
| |
− | | |
− | <source lang="sh">$ cp -r $HOME/stx-tools/centos-mirror-tools/output/stx-r1/ $HOME/starlingx/mirror/CentOS/
| |
− | </source></li></ol>
| |
− | | |
− | ==Create StarlingX Packages==
| |
− | ===Setup Building Docker Container===
| |
− | | |
− | <ol start="1"><li>From terminal "two", Workspace Setup
| |
− | | |
− | <source lang="sh">$ mkdir -p $HOME/starlingx/workspace
| |
− | </source></li>
| |
− | | |
− | <li>Navigate to the ''<$HOME/stx-tools>'' project directory:
| |
− | | |
− | <source lang="sh">$ cd $HOME/stx-tools
| |
− | </source></li>
| |
− | | |
− | <li>Copy your git options
| |
− | | |
− | <source lang="sh">$ cp ~/.gitconfig toCOPY
| |
− | </source></li>
| |
− | | |
− | <li>Create a ''<localrc>'' file
| |
− | | |
− | <source lang="sh">$ nano localrc
| |
− | </source></li>
| |
− | | |
− | <source lang="sh">
| |
− | # tbuilder localrc
| |
− | MYUNAME=user
| |
− | PROJECT=starlingx
| |
− | HOST_PREFIX=$HOME/starlingx/workspace
| |
− | HOST_MIRROR_DIR=$HOME/starlingx/mirror
| |
− | </source></li>
| |
− | | |
− | <li>If necessary you might have to set http/https proxy in your Dockerfile.centos73 before building the docker image.
| |
− | | |
− | <source lang="sh">ENV http_proxy "http://your.actual_http_proxy.com:your_port" && \
| |
− | https_proxy "https://your.actual_https_proxy.com:your_port" && \
| |
− | ftp_proxy "http://your.actual_ftp_proxy.com:your_port"
| |
− | RUN echo "proxy=http://your-proxy.com:port" >> /etc/yum.conf
| |
− | </source></li>
| |
− | | |
− | <li>Base container setup
| |
− | If you are running in fedora system, you will see " .makeenv:88: *** missing separator. Stop. " error, to continue :
| |
− | | |
− | * delete the functions define in the .makeenv ( module () { ... } )
| |
− | * delete the line 19 in the Makefile and ( NULL := $(shell bash -c "source buildrc ... ).
| |
− | | |
− | | |
− | <source lang="sh">$ make base-build
| |
− | </source></li>
| |
− | | |
− | <li>Build container setup
| |
− | | |
− | <source lang="sh">
| |
− | $ make build
| |
− | </source></li>
| |
− | | |
− | <li>Verify environment variables
| |
− | | |
− | <source lang="sh">$ bash tb.sh env
| |
− | </source></li>
| |
− | | |
− | <li>Build container run
| |
− | | |
− | <source lang="sh">$ bash tb.sh run
| |
− | </source></li>
| |
− | | |
− | <li>Execute the built container:
| |
− | | |
− | <source lang="sh">$ bash tb.sh exec
| |
− | </source></li></ol>
| |
− | | |
− | ===Download Source Code Repositories===
| |
− | | |
− | <ol start="1"><li>Internal environment
| |
− | | |
− | <source lang="sh">$ eval $(ssh-agent)
| |
− | $ ssh-add
| |
− | </source></li>
| |
− | | |
− | <li>Repo init
| |
− | | |
− | <source lang="sh">$ cd $MY_REPO_ROOT_DIR
| |
− | $ repo init -u git://git.openstack.org/openstack/stx-manifest.git -m default.xml
| |
− | </source></li>
| |
− | | |
− | <li>Repo sync
| |
− | | |
− | <source lang="sh">$ repo sync
| |
− | </source></li>
| |
− | | |
− | <li> cgcs-dl
| |
− | | |
− | <source lang="sh">$ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/
| |
− | </source></li>
| |
− | | |
− | Outside the container, use another terminal
| |
− | | |
− | <li>Mirror Binaries
| |
− | | |
− | <source lang="sh">$ mkdir -p $HOME/starlingx/mirror/CentOS/tis-installer
| |
− | $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img $HOME/starlingx/mirror/CentOS/tis-installer/initrd.img-stx-0.2
| |
− | $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz $HOME/starlingx/mirror/CentOS/tis-installer/vmlinuz-stx-0.2
| |
− | $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img $HOME/starlingx/mirror/CentOS/tis-installer/squashfs.img-stx-0.2
| |
− | </source></li></ol>
| |
− | | |
− | ===Build Packages===
| |
− | | |
− | <ol start="1"><li><strong>Temporal!</strong> Build-Pkgs Errors
| |
− | Be prepared to have some missing / corrupted packages generated during [[StarlingX/Developer_Guide#Build_the_CentOS_Mirror_Repository]] which will make the next step to fail, if that happens please download manually those missing / corrupted packages. <br>
| |
− | </li>
| |
− | | |
− | <li><strong>Temporal! Only Branch m/2018.07</strong> "Adding comps.xml and mock.proto default files." <br>
| |
− | https://review.openstack.org/#/c/584214/
| |
− | </li>
| |
− | | |
− | <li><strong>Update the symbolic links</strong>
| |
− | <source lang="sh">$ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
| |
− | </source></li>
| |
− | | |
− | <li>Build-Pkgs --Serial
| |
− | | |
− | <source lang="sh">$ cd $MY_WORKSPACE
| |
− | $ build-pkgs --serial
| |
− | </source></li>
| |
− | | |
− | <li> Generate-Cgcs-Tis-Repo
| |
− | | |
− | The cgcs-tis-repo has the dependency information that sequences the build order; To generate or update the information the following command needs to be executed after building modified or new packages.
| |
− | <source lang="sh">$ generate-cgcs-tis-repo
| |
− | </source></li>
| |
− | | |
− | <li>Build-Srpms
| |
− | | |
− | <source lang="sh">$ build-srpms --installer
| |
− | </source></li>
| |
− | | |
− | <li>Build-Rpms
| |
− | | |
− | <source lang="sh">$ build-rpms --installer
| |
− | </source></li></ol>
| |
− | | |
− | ==Build StarlingX ISO==
| |
− | | |
− | <ol start="1"><li>Build-Iso
| |
− | | |
− | <source lang="sh">$ build-iso
| |
− | </source></li></ol>
| |