HowTo: An in depth Guide to package building for the VL repository



Introduction:

For those of you new to the GNU/Linux world, you may have noticed that the way software installation and maintenance is dealt with here is very different from the way it is handled on proprietary platforms such as Microsoft Windows.

Instead of surfing the web in search of Freeware*, Shareware or cracked software (“warez”) as is frequent on Windows, a GNU/Linux user can download software directly from a server provided by the distribution in a faster and safer manner.

This software server, or repository, hosts software packages that are tailored for that particular distribution and contains information that will allow a user to easily install or upgrade software in a seamless and automated way.

This is a great concept and any seasoned Linux user will tell you that after using this software management system for a while, you will wonder how you ever did without it!

But how does the software end up on that server? Who creates those packages?
Like with most Free Software/Open Source projects, this is done largely by volunteer effort.

One of the first concerns a potential user has before trying/adopting a distro is whether the software repository has lots of packages.
Because of this we need all the help we can get ;)

What this HowTo is going to talk about:

This HowTo should set you up with the basic tools and knowledge needed to create packages for the VectorLinux repository.
These topics will be discussed:



  1. Preparing the package building environment

In order to create good packages, you need a “clean” build environment. Having a clean build environment will ensure that the packages you build don't have unnecessary dependencies.

One example of a “dirty” build environment is one that has OpenOffice installed. Packages built in that environment may report needing OpenOffice installed in order to work, which in most cases is not true.

This means that when creating packages, you should use a version with the least amount of extra software, such as VectorLinux Light.

There are several ways to create a clean build environment:

The first method is fairly straight forward. Use a partitioning tool such as Gparted to create a separate partition where all the packaging will take place. That way you can easily format the partition when it becomes too “dirty” and start over.

The second method is somewhat more advanced, so to put it simply, it involves creating a fully working operating system within a directory of your current system. This can be a lot more convenient once you get the hang of it, since you can quickly erase the self-contained system and create a new one without ever formatting are re-partitioning your hard-drive.

To easily create a chroot jail, you can use a script from this this thread:  http://www.vectorlinux.com/forum2/index.php?topic=5152.0

Once you have setup your build environment, you need to set some important environment variables that will be used by the compiler later on. VectorLinux has chosen to compile its software so it can be run on pre-Pentium II chips, the i586 family. When compiling source code, GCC (the compiler) will look at the $CFLAGS and $CXXFLAGS variables to determine how the compiled code should be optimized. To set these optimizations to i586, we need to set the above mentioned variables.
Open a terminal, and logon as root. Then open /etc/profile.d/vasm.sh and make sure that this bit of text is presnt at the end of the file:

export CFLAGS=”-O2 -mtune=i686 -march=i586”

export CXXFLAGS=”$CFLAGS”

Those variables will now be set at boot time. You are almost ready to start packaging.


  1. The basics of package building

Before we get to how packages need to be made specifically for VectorLinux, we will cover the basic steps that are necessary to create any simple Slackware Package.

This task can essentially be subdivided into 7 steps:

    a) getting and extracting the source code:

First of all, you need to get the source code for the program or libraries that you are going to package.

Once again, Google is your friend, so fire up your browser and search for the name of the program you would like to package, and download the source code from the website into a directory on your hard drive.

The source code is usually compressed with Gzip (.tar.gz extension) or Bzip2 (.tar.bz2 extension), so you can use the tar command to decompress either format like this:

tar xvf filename.tar.bz2

After extraction, in almost all cases there is a directory with the name and version of the program that contains the source code. Enter it with this command,

cd name-version

and them move on:


    b) Configuring the source code


What does “configuring the source code” really mean?

Well, is consists of creating a file which will include all the options and instructions the compiler needs to create the binary the way you want it.

This file is called the Makefile. The problem is that with very large scale projects, the Makefile can be massive and would take ages to write and tweak by hand. So generaly, a script called “configure” is provided to automate the source code configuration task.

Of you simply run the configure script like this:

./configure

it will create a Makefile with the default values and configurations.

The most basic option of the configure script is the prefix option. The prefix indicates the base directory where the files will be installed. The default value for the prefix option is /usr/local, which means that all files will be installed to that directory (libraries to /usr/local/lib and executable binaries to /usr/local/bin).

However, this is not the correct prefix when building packages. To adhere to Slackware's standard, files in that package must be prefixed to /usr.

To view more options available in the configure script, run it with the help flag:

./configure –help

That will show you the options you can use to tweak the Makefile before compiling.


This is the way you should run the configure script to include the most basic options for a Slackware compatible build:

./configure –prefix=/usr --sysconfdir=/etc –with-included-gettext –mandir=/usr/man

As you can see, the prefix is set to /usr. --sysconfdir specifies that any configuration files must go in /etc; --with-included-gettext indicates that the compiler should include Gettext (for internationalization purposes) in the build if possible; and –mandir specifies the location where the manual pages should be installed (/usr/share/man would be the default, but on Slackware and Slackware based systems such as VectorLinux, the correct location is /usr/man).

While that script runs, it will check for all the required dependencies and print that information in the terminal. If it fails, it will indicate what dependency is missing.


    c) compiling the source

If there were no errors during the congfigure step then you can simply run

make

to start compiling the source code.
After that finishes successfully, you need to install the compiled source code to a temporary location so it can be packaged.


    d) installing the compiled source into the package dir

If there were no errors during the make step then you can simply run

make install DESTDIR=/tmp/name-version

to install the files into the package directory

the $DESTDIR variable indicates where to place the files temporarily. After running that command, all the files that would be installed to /usr end up in /tmp/name-version.

Now, enter the temporary install location, and run the ls command

ls

you will see that there is a directory hierarchy that mimics what you would see in the filesystem's root directory (/).


    e) make the package description

Next, you need to create the package description. This file is called slack-desc and must be placed in a directory called install.

mkdir install

mcedit install/slack-desc

in the package description file, the beginning of every line must start with the package name followed by as colon, like so:

uberpackage: Uberpackage contains an uber program!
uberpackage:
uberpackage: This uberprogram will increase your productivity
uberpackage: by 1000%.
uberpackage: It is also bery flexible and extensible, and can also
uberpackage: perform your every-day chores, like taking out the
uberpackage: garbage, washing dishes and scrubbing the toilet.
uberpackage:
uberpackage: Website: http://www.ubersite.com/
uberpackage: License: License Name and Version here
uberpackage:

The description file cant be longer than 11 lines in total. The first line should include a very brief indication of what is in the package. After leaving a line, write a more thorough description of what is in the package. Last, you MUST indicate the website and the license under which the program is distrbuted.

The license information is generally included with the source code in a file called COPYING. Because not all software is licensed under the GNU GPL or the BSD License, never be too lazy to open that file and see what the license is. Branding a package with the incorrect license could bring legal problems.


    f) Find program dependencies

Great, so now the last thing you need to do before the package can be built is to create a dependency list. This will tell the package manager later on exactly which other packages this one depends on. The tool of choice is called requiredbuilder. It may take a while to run, but it is generaly very accurate.

requiredbuilder -v -y ./

The -v option tells requiredbuilder to determinate the dependency version; -y forces requiredbuilder to dump the gathered information into a file called slack-required inside the install directory.


    g) make the package

To create the package, run the makepkg command:

makepkg -l y -c n name-version-arch-release.tlz

The -l y option tells makepkg to remove any symbolic links it may find and store that information in install/doinst.sh. -c n lets makepkg know that is must not change the permissions of the files in the packge.

You can use the .tgz extension instead of the .tlz extension if you wish to usr Gzip compression instead of LZMA compression. Gzip compression is faster but achieves lower compression rates. LZMA is a little slower, but compresses the files a lot more.

And that is how a package is created “manually”.


    3) The automated way


There are three main problems with building packages manually.

The first is that when you need to build many of them, the steps become very repetitive. Computers are there to do the repetitive work for us, right?

Secondly, after building a package that requires a lot of effort and tweaking, one tends to forget all the steps necessary to build that package again in the future (say a newer version of a program), and its harder for other to figure out exactly how you built your package.

Last but not least, there are licensing requirements. The most popular open source license in use to day is the GNU General Public License, which requires a distributor of the software to also provide the source code. This means that its not enough to just give the original source code for the program in the package, you also have to distribute any modifications made (patches, addons, etc), and make the configuration of the program available (configure script flags, compiler flags).

What a pain in the butt, eh?

Well, here is where using the shell (command line) shows its power.

Instead of manually running all those commands by hand and writing a text file indicating what needed to be done to build the package, you can simply place all those commands you use in a shell script.

The shell then interprets the script and runs all the commands in it. You get your computer to do all the heavy lifting, there is no GPL infringement and the end-user gets (and other packagers) get to see how the package was built.

Your package and script can then be uploaded to the repository.

Since it can take some time to become familiar with scripting and because your script will most likely contain bugs and need improvements that others have already worked on, we provide you with a template script that you can use to build your packages. This script will allow you to build a package with the very basic configurations, so its still up to you to add configure-time tweaks to it.

Download it here: http://vectorlinux.osuosl.org/docs/packaging/default.SlackBuild

Read through it, and most importantly, understand what it does.

After downloading it, create a directory named after the package that you are going to build. Inside it, create another directory with the source version as its name. Then Inside it, create another directory called src.

In there, place the script (renamed to the name of the package you are going to build, with a .SlackBuild extension; ie: PackageName.SlackBuild) and the compressed source code.
So for the source package openbox you would have a directory structure like this:
openbox/
-- 3.4.6.1
   `-- src
       |-- openbox-3.4.6.1.tar.gz
       `-- openbox.SlackBuild

Make the necessary adjustments to the script, and then make it executable:

chmod +x PackageName.SlackBuild

After running the script and if it builds the package successfully, then the package will end up in the directory you created for it like this:
openbox/
`-- 3.4.6.1
    |-- openbox-3.4.6.1-i586-1vl59.tlz
    |-- slack-desc
    |-- slack-required
    `-- src
        |-- openbox-3.4.6.1.tar.gz
        `-- openbox.SlackBuild


Now all you need to do is use an FTP program such as GFTP to upload the directory to your contributors account.


  1. Some rules and “ethics”


If a package you need already exists in the repository, then install it and use it. Don't re-invent the wheel. Any duplicate packages will not be accepted.

Before embarking on a large scale or a more difficult packaging project, open a thread in the packager's forum stating what your intentions are and asking if anyone else may already be working on packages that will overlap with yours.