Jump to: navigation, search

Difference between revisions of "StarlingX/MultiOS/Debian"

(How to build packages in Debian)
(How to contribute to the project: Starling X build in Debian)
 
(3 intermediate revisions by the same user not shown)
Line 2: Line 2:
  
 
=== Scope of the project ===
 
=== Scope of the project ===
 +
 +
This sub-project of Starling X will extend the current Flock services build artifacts structure to support more operating systems apart from the current CentOS OS supported.
 +
 +
The current directory structure is based around the build working with CentOS OS and the current build tooling. The flock services git repositories provide a directory call centos with the spec files to build RPMs distribution packages. In order to support building the flock services packages for new, different OSes is necessary to provide build artifacts where the community can have a place to contribute with build scripts such as control and rules files (for Debian based operating systems). This wiki describes how to build a package from the Starling X project into a Debian base OS
  
 
=== How to build packages in Debian ===
 
=== How to build packages in Debian ===
Line 11: Line 15:
 
This tutorial is based on [1], to begin these are the steps:
 
This tutorial is based on [1], to begin these are the steps:
  
# Open your /etc/apt/sources.list file and check if you have one or more lines that start with deb-src.
+
* Open your /etc/apt/sources.list file and check if you have one or more lines that start with deb-src.
# Update:  
+
* Update:  
 
     sudo apt-get update
 
     sudo apt-get update
# In the directory of your preference create a directory to host the source deb packages:  
+
* In the directory of your preference create a directory to host the source deb packages:  
 
     mkdir -p src/debian/; cd src/debian  
 
     mkdir -p src/debian/; cd src/debian  
# Install the ackage you want to modify
+
* Install the ackage you want to modify
 
     sudo apt-get install fdupes
 
     sudo apt-get install fdupes
# Install the source code of the package you want to modify
+
* Install the source code of the package you want to modify
 
     apt-get source fdupes
 
     apt-get source fdupes
# You have now downloaded the 3 files (.dsc, .tar.gz, .diff.gz), composing the: '''Debian source package'''
+
* You have now downloaded the 3 files (.dsc, .tar.gz, .diff.gz), composing the: '''Debian source package'''
# Once the package is downloaded, you can check the directory where you are, and you'll find that apart from the 3 files that were downloaded you also have a directory, called fdupes-1.50-PR2. This is the unpacked source of the '''Debian package'''.
+
* Once the package is downloaded, you can check the directory where you are, and you'll find that apart from the 3 files that were downloaded you also have a directory, called fdupes-1.50-PR2. This is the unpacked source of the '''Debian package'''.
# Every Debian (or Debian derivative) source package includes a debian directory, where all the information related to the Debian build artifacts for the package are stored. Anything that's outside of that directory, is the upstream code, i.e. the original code released by whoever programmed the software.
+
* Every Debian (or Debian derivative) source package includes a debian directory, where all the information related to the Debian build artifacts for the package are stored. Anything that's outside of that directory, is the upstream code, i.e. the original code released by whoever programmed the software.
# This is the directory that the package maintainer has added to the source code to build the package.
+
* This is the directory that the package maintainer has added to the source code to build the package.
# In this directory you'll usually find lots of files related to Debian's version of the program, Debian specific patches, manpages, documentation, and so on
+
* In this directory you'll usually find lots of files related to Debian's version of the program, Debian specific patches, manpages, documentation, and so on
## rules file is the script file that we will be running in order to build the package.
+
** rules file is the script file that we will be running in order to build the package.
## in the patches directory, there are also a number of patches applied by the maintainer
+
** in the patches directory, there are also a number of patches applied by the maintainer
# Get the build dependencies with: sudo apt-get build-dep fdupes
+
* Get the build dependencies with: sudo apt-get build-dep fdupes
# Rebuild without changes with
+
* Rebuild without changes with
 
     debuild -b -uc -us
 
     debuild -b -uc -us
# Install this file with:
+
* Install this file with:
 
     sudo dpkg -i ../fdupes_1.50-PR2-3_<your arch>.deb
 
     sudo dpkg -i ../fdupes_1.50-PR2-3_<your arch>.deb
 
At this point, you might be wondering what is the difference between:  
 
At this point, you might be wondering what is the difference between:  
** debuild - build a Debian package
+
** [https://manpages.debian.org/jessie/devscripts/debuild.1.en.html debuild - build a Debian package]
https://manpages.debian.org/jessie/devscripts/debuild.1.en.html
+
** [https://manpages.debian.org/jessie/dpkg-dev/dpkg-buildpackage.1.en.html dpkg-buildpackage - build binary or source packages from sources]
** dpkg-buildpackage - build binary or source packages from sources
 
https://manpages.debian.org/jessie/dpkg-dev/dpkg-buildpackage.1.en.html
 
  
 
The debuild command is a wrapper script of the dpkg-buildpackage command to
 
The debuild command is a wrapper script of the dpkg-buildpackage command to
 
build the Debian binary package under the proper environment variables.
 
build the Debian binary package under the proper environment variables.
  
## How to build a package from tar.gz ( simple example )  
+
====How to build a package from tar.gz ( simple example ) ====
 
 
### Big picture
 
  
 
The big picture for building a single non-native Debian package from the upstream tarball debhello-0.0.tar.gz can be summarized as:
 
The big picture for building a single non-native Debian package from the upstream tarball debhello-0.0.tar.gz can be summarized as:
Line 70: Line 70:
 
After debmake debuild command build the package
 
After debmake debuild command build the package
  
### Commands to build:
+
===== Commands to build =====
  
 
Here is a summary of commands similar to the debuild command.
 
Here is a summary of commands similar to the debuild command.
Line 91: Line 91:
 
* The git-pbuilder command is another wrapper script to build the Debian binary package under the proper chroot environment with the proper environment variables. This provides an easier command line UI to switch among different build environments.
 
* The git-pbuilder command is another wrapper script to build the Debian binary package under the proper chroot environment with the proper environment variables. This provides an easier command line UI to switch among different build environments.
  
## Example
+
===== Example =====
  
 
Here is an example of creating a simple Debian package from a simple C source using the Makefile as its build system.
 
Here is an example of creating a simple Debian package from a simple C source using the Makefile as its build system.
  
 
Structure of directory:
 
Structure of directory:
 
  
 
     mkdir debhello-0.0
 
     mkdir debhello-0.0
Line 105: Line 104:
 
Source code:
 
Source code:
  
$ cat src/hello.c
+
    $ cat src/hello.c
#include <stdio.h>
+
    #include <stdio.h>
int
+
    int main(){
main()
 
{
 
 
         printf("Hello, world!\n");
 
         printf("Hello, world!\n");
 
         return 0;
 
         return 0;
}
+
    }
  
 
Makefile:
 
Makefile:
  
$ cat Makefile
+
    $ cat Makefile
prefix = /usr/local
+
    prefix = /usr/local
 
+
    all: src/hello
all: src/hello
+
    src/hello: src/hello.c
 
+
        @echo "CFLAGS=$(CFLAGS)" | \
src/hello: src/hello.c
+
            fold -s -w 70 | \
@echo "CFLAGS=$(CFLAGS)" | \
+
            sed -e 's/^/# /'
fold -s -w 70 | \
+
        $(CC) $(CPPFLAGS) $(CFLAGS) $(LDCFLAGS) -o $@ $^
sed -e 's/^/# /'
+
    install: src/hello
$(CC) $(CPPFLAGS) $(CFLAGS) $(LDCFLAGS) -o $@ $^
+
        install -D src/hello \
 
+
            $(DESTDIR)$(prefix)/bin/hello
install: src/hello
+
    clean:
install -D src/hello \
+
        -rm -f src/hello
$(DESTDIR)$(prefix)/bin/hello
+
    distclean: clean
 
+
    uninstall:
clean:
+
        -rm -f $(DESTDIR)$(prefix)/bin/hello
-rm -f src/hello
+
    .PHONY: all install clean distclean uninstall
 
 
distclean: clean
 
 
 
uninstall:
 
-rm -f $(DESTDIR)$(prefix)/bin/hello
 
 
 
.PHONY: all install clean distclean uninstall
 
  
 
Create tarball:  
 
Create tarball:  
Line 145: Line 135:
 
     tar -czf debhello-0.0.tar.gz debhello-0.0/
 
     tar -czf debhello-0.0.tar.gz debhello-0.0/
  
 +
After that is possible to start with packaging:
  
After that is possible to start with packaging:
 
 
     cd debhello-0.0/
 
     cd debhello-0.0/
 
     debmake
 
     debmake
 
  
 
Adjsut the override_dh_usrlocal:
 
Adjsut the override_dh_usrlocal:
```
 
vim debian/rules
 
#!/usr/bin/make -f
 
# You must remove unused comment lines for the released package.
 
export DH_VERBOSE = 1
 
#export DEB_BUILD_MAINT_OPTIONS = hardening=+all
 
export DEB_CFLAGS_MAINT_APPEND  = -Wall -pedantic
 
export DEB_LDFLAGS_MAINT_APPEND = -Wl,--as-needed
 
 
%:
 
dh $@
 
  
override_dh_usrlocal:
+
    cat debian/rules
 +
    #!/usr/bin/make -f
 +
    # You must remove unused comment lines for the released package.
 +
    export DH_VERBOSE = 1
 +
    #export DEB_BUILD_MAINT_OPTIONS = hardening=+all
 +
    export DEB_CFLAGS_MAINT_APPEND  = -Wall -pedantic
 +
    export DEB_LDFLAGS_MAINT_APPEND = -Wl,--as-needed
 +
    %:
 +
        dh $@
 +
    override_dh_usrlocal:
 +
    #override_dh_auto_install:
 +
    #    dh_auto_install -- LIBDIR=/usr/local/lib
 +
    #override_dh_install:
 +
    #    dh_install --list-missing -X.pyc -X.pyo
  
#override_dh_auto_install:
+
Build
# dh_auto_install -- LIBDIR=/usr/local/lib
 
  
#override_dh_install:
+
    dpkg-buildpackage -B
# dh_install --list-missing -X.pyc -X.pyo
 
```
 
Build
 
dpkg-buildpackage -B
 
  
 
Check that all binaries are installed in DEB
 
Check that all binaries are installed in DEB
Line 180: Line 166:
 
     sudo dpkg -c debhello_0.0-1_amd64.deb
 
     sudo dpkg -c debhello_0.0-1_amd64.deb
 
     hello
 
     hello
 
In the end the only file that is necesary is the DSC file
 
DSC = Debian source packages' control file format
 
Details of each filed on the DSC file at [ubuntu-manpages](http://manpages.ubuntu.com/manpages/bionic/man5/dsc.5.html)
 
 
  
 
If you have the DSC file you just need to:  
 
If you have the DSC file you just need to:  
Line 193: Line 174:
  
 
     dpkg-buildpackage -rfakeroot -b
 
     dpkg-buildpackage -rfakeroot -b
 
  
  
 
More info at :  
 
More info at :  
https://www.debian.org/doc/manuals/debmake-doc/ch04.en.html
 
  
## How to build a package from tar.gz ( more complex example )
+
[https://www.debian.org/doc/manuals/debmake-doc/ch04.en.html debmake-doc]
  
More instructions at:
+
====How to build a package from tar.gz ( more complex example ) ====
  
https://www.debian.org/doc/manuals/debmake-doc/ch08.en.html
+
[https://www.debian.org/doc/manuals/debmake-doc/ch08.en.html debmake-complex]
  
## Where does the debian repositories live:  
+
==== Where does the debian repositories live: ====
  
 
Examples:  
 
Examples:  
Line 212: Line 191:
 
* GCC:    https://salsa.debian.org/toolchain-team/gcc-cross  
 
* GCC:    https://salsa.debian.org/toolchain-team/gcc-cross  
  
### Links
+
==== Links ====
  
 
     https://wiki.debian.org/BuildingTutorial
 
     https://wiki.debian.org/BuildingTutorial
  
 
=== How to contribute to the project: Starling X build in Debian ===
 
=== How to contribute to the project: Starling X build in Debian ===
 +
 +
Starling X multi-os packaging is a side github project with examples of Debian rules and control files.
 +
 +
An example of the control, install and rules files can be seen here:
 +
 +
* https://github.com/marcelarosalesj/x.stx-fault/tree/debian/fm-common/debian
 +
 +
=== Directory layout specification ===
 +
 +
[https://opendev.org/starlingx/specs/src/branch/master/specs/2019.03/approved/multi-os-2004891-directory-layout.rst StarlingX: Define directory layout for MultiOS]

Latest revision as of 20:43, 31 May 2019

Starling X build in Debian

Scope of the project

This sub-project of Starling X will extend the current Flock services build artifacts structure to support more operating systems apart from the current CentOS OS supported.

The current directory structure is based around the build working with CentOS OS and the current build tooling. The flock services git repositories provide a directory call centos with the spec files to build RPMs distribution packages. In order to support building the flock services packages for new, different OSes is necessary to provide build artifacts where the community can have a place to contribute with build scripts such as control and rules files (for Debian based operating systems). This wiki describes how to build a package from the Starling X project into a Debian base OS

How to build packages in Debian

These are examples to document how to build Debian files

How to build a package that already exists in Debian

This tutorial is based on [1], to begin these are the steps:

  • Open your /etc/apt/sources.list file and check if you have one or more lines that start with deb-src.
  • Update:
   sudo apt-get update
  • In the directory of your preference create a directory to host the source deb packages:
   mkdir -p src/debian/; cd src/debian 
  • Install the ackage you want to modify
   sudo apt-get install fdupes
  • Install the source code of the package you want to modify
   apt-get source fdupes
  • You have now downloaded the 3 files (.dsc, .tar.gz, .diff.gz), composing the: Debian source package
  • Once the package is downloaded, you can check the directory where you are, and you'll find that apart from the 3 files that were downloaded you also have a directory, called fdupes-1.50-PR2. This is the unpacked source of the Debian package.
  • Every Debian (or Debian derivative) source package includes a debian directory, where all the information related to the Debian build artifacts for the package are stored. Anything that's outside of that directory, is the upstream code, i.e. the original code released by whoever programmed the software.
  • This is the directory that the package maintainer has added to the source code to build the package.
  • In this directory you'll usually find lots of files related to Debian's version of the program, Debian specific patches, manpages, documentation, and so on
    • rules file is the script file that we will be running in order to build the package.
    • in the patches directory, there are also a number of patches applied by the maintainer
  • Get the build dependencies with: sudo apt-get build-dep fdupes
  • Rebuild without changes with
   debuild -b -uc -us
  • Install this file with:
   sudo dpkg -i ../fdupes_1.50-PR2-3_<your arch>.deb

At this point, you might be wondering what is the difference between:

The debuild command is a wrapper script of the dpkg-buildpackage command to build the Debian binary package under the proper environment variables.

How to build a package from tar.gz ( simple example )

The big picture for building a single non-native Debian package from the upstream tarball debhello-0.0.tar.gz can be summarized as:

  • The maintainer obtains the upstream tarball debhello-0.0.tar.gz and untars its contents to the debhello-0.0 directory.
  • The debmake command debianizes the upstream source tree by adding template files only in the debian directory.
    • The debhello_0.0.orig.tar.gz symlink is created pointing to the debhello-0.0.tar.gz file.
    • The maintainer customizes template files.
  • The debuild command builds the binary package from the debianized source tree.
    • debhello-0.0-1.debian.tar.xz is created containing the debian directory.
   $ tar -xzmf debhello-0.0.tar.gz
   $ cd debhello-0.0
   $ debmake
       manual customization
   $ debuild

The debmake command is the helper script for the Debian packaging.

  • It always sets most of the obvious option states and values to reasonable defaults.
  • It generates the upstream tarball and its required symlink if they are missing.
  • It doesn’t overwrite the existing configuration files in the debian/ directory.
  • It supports the multiarch package.
  • It creates good template files such as the debian/copyright file compliant with DEP-5.

After debmake debuild command build the package

Commands to build

Here is a summary of commands similar to the debuild command.

  • The debian/rules file defines how the Debian binary package is built.
  • The dpkg-buildpackage command is the official command to build the Debian binary package. For normal binary build, it executes roughly:
    • “dpkg-source --before-build” (apply Debian patches, unless they are already applied)
    • “fakeroot debian/rules clean”
    • “dpkg-source --build” (build the Debian source package)
    • “fakeroot debian/rules build”
    • “fakeroot debian/rules binary”
    • “dpkg-genbuildinfo” (generate a *.buildinfo file)
    • “dpkg-genchanges” (generate a *.changes file)
    • “fakeroot debian/rules clean”
    • “dpkg-source --after-build” (unapply Debian patches, if they are applied during --before-build)
    • “debsign” (sign the *.dsc and *.changes files)
  • The debuild command is a wrapper script of the dpkg-buildpackage command to build the Debian binary package under the proper environment variables.
  • The pdebuild command is a wrapper script to build the Debian binary package under the proper chroot environment with the proper environment variables.
  • The git-pbuilder command is another wrapper script to build the Debian binary package under the proper chroot environment with the proper environment variables. This provides an easier command line UI to switch among different build environments.
Example

Here is an example of creating a simple Debian package from a simple C source using the Makefile as its build system.

Structure of directory:

   mkdir debhello-0.0
   ├── Makefile
   └── src
          └── hello.c

Source code:

   $ cat src/hello.c
   #include <stdio.h>
   int main(){
       printf("Hello, world!\n");
       return 0;
   }

Makefile:

   $ cat Makefile
   prefix = /usr/local
   all: src/hello
   src/hello: src/hello.c
       @echo "CFLAGS=$(CFLAGS)" | \
           fold -s -w 70 | \
           sed -e 's/^/# /'
       $(CC) $(CPPFLAGS) $(CFLAGS) $(LDCFLAGS) -o $@ $^
   install: src/hello
       install -D src/hello \
           $(DESTDIR)$(prefix)/bin/hello
   clean:
       -rm -f src/hello
   distclean: clean
   uninstall:
       -rm -f $(DESTDIR)$(prefix)/bin/hello
   .PHONY: all install clean distclean uninstall

Create tarball:

   tar -czf debhello-0.0.tar.gz debhello-0.0/

After that is possible to start with packaging:

   cd debhello-0.0/
   debmake

Adjsut the override_dh_usrlocal:

   cat debian/rules
   #!/usr/bin/make -f
   # You must remove unused comment lines for the released package.
   export DH_VERBOSE = 1
   #export DEB_BUILD_MAINT_OPTIONS = hardening=+all
   export DEB_CFLAGS_MAINT_APPEND  = -Wall -pedantic
   export DEB_LDFLAGS_MAINT_APPEND = -Wl,--as-needed
   %:
       dh $@
   override_dh_usrlocal:
   #override_dh_auto_install:
   #    dh_auto_install -- LIBDIR=/usr/local/lib
   #override_dh_install:
   #    dh_install --list-missing -X.pyc -X.pyo

Build

   dpkg-buildpackage -B

Check that all binaries are installed in DEB

   cd ..
   sudo dpkg -c debhello_0.0-1_amd64.deb
   hello

If you have the DSC file you just need to:

   dpkg-source -x foo_version.dsc

change into the extracted source directory. Then run:

   dpkg-buildpackage -rfakeroot -b


More info at :

debmake-doc

How to build a package from tar.gz ( more complex example )

debmake-complex

Where does the debian repositories live:

Examples:

Links

   https://wiki.debian.org/BuildingTutorial

How to contribute to the project: Starling X build in Debian

Starling X multi-os packaging is a side github project with examples of Debian rules and control files.

An example of the control, install and rules files can be seen here:

Directory layout specification

StarlingX: Define directory layout for MultiOS