Package Dependency Tracking

Enabling Package Dependency Tracking

Package Dependency Tracking during build is optional, and can be enabled by setting KX_SYSTEMTAP="yes" in custom_definitions.shinc.

How Dependency Tracking Works

In order to track package dependencies during build, the systemtap script master/chroot_scripts/kx_open.stp is used.

The KaarPux build scripts will run each step under systemtap with kx_open.stp enabled.: file:kx_open.stp will output the path of every file used by the build script. The output is piped through through grep and awk, in order to list the relevant files into kaarpux/flist/*.rw.

Using Package Dependency Tracking

It might be possible to use Package Dependency Tracking on hosts other than KaarPux (e.g. Ubuntu). However, it is recommended to use Package Dependency Tracking using a KaarPux host (i.e. building a new KaarPux target system on an existing:program:KaarPux host system), because the Package Dependency Tracking setup is readily available on KaarPux.

On KaarPux, bootstrap_8 installs systemtap, and runs the script master/chroot_scripts/install_kx_open_stp.sh to install kx_open.stp. Hence, on a KaarPux host, using KX_SYSTEMTAP="yes" should work like a charm.

/lib/modules

When installing a linux kernel, the kernel modules are installed into /lib/modules/KERNELVERSION. The files needed for normal operation are copied into this directory. However, to compile new kernel modules (such as master/chroot_scripts/kx_open.stp), the kernel build tree needs to be left in place (under kaarpux/bootstrap/build/bootstrap_8/linux-RELEASE or:file:kaarpux/linux/build/linux/linux-RELEASE), as /lib/modules/KERNELVERSION/build and /lib/modules/KERNELVERSION/source are symbolic links to the buld tree. master/chroot_scripts/kx_open.stp is compiled into /lib/modules/KERNELVERSION/systemtap/kx_open.ko.

Bootstrap vs. Linux modules

When building KaarPux with KX_SYSTEMTAP="yes", the running Linux kernel version and release must match the Linux kernel version and release used to build the modules in /lib/modules/KERNELVERSION. This is ensured “automagically” when using a KaarPux system as host.

You can get the linux release with uname -r, returning something like 3.2.9-kx-4eefd8aa, which also designates the subdirectory of /lib/modules/ used for kernel mdules.

You can get the linux version with uname -v, returning something like #3 SMP Wed Jun 20 22:00:00 CEST 2012,

During bootstrap, we use the /lib/modules/KERNELVERSION on the host system (which should be consistent on a KaarPux host). During linux/opt, whe use the /lib/modules/KERNELVERSION build during bootstrap (hence also consistent).

Pre-processing the Package Dependency Tracking Information

Run the command kaarpux/tools/kx_flist.pl. This command will:

  1. Process all the kaarpux/flist/*.rw files to create maps in the kaarpux/dependencies directory.
  2. Process libraries and executables with ldd to create maps in the kaarpux/dependencies directory.

kaarpux/tools/kx_flist.pl includes some woodoo to handle files which are written by many packages, such as logfiles and databases.

Using the Package Dependency Tracking Information

Now dependencies can be viewed with kaarpux/tools/kx_dependencies.pl. For options see: kaarpux/tools/kx_dependencies.pl –man

Since there are more than 600 packages with more than 25.000 dependencies, the output of kaarpux/tools/kx_dependencies.pl may be incomprehensible.

However, kaarpux/tools/kx_dependencies.pl includes the option --merge-file. Using this option, you can specify file(s) with merge information. See: kaarpux/tools/kx_dependencies.pl –man For example, to show all packages under xorg-apps as just one item called xorg-apps, you may specify:

  • xorg-apps
  • xclock
  • xwud
  • xfd
  • xfontsel
  • xeyes
  • twm

To create a customized --merge-file file you may start out with:

{
sed -n -e'/- bootstrap_4:/,$p' master/master.yaml |
sed
-e's/#.*//'
-e's/  - //'
-e's/pkg: //'
-e's/sudo: //'
-e's/parallel://'
-e's/://'
-e's/- .*//'
-e'/^ \\+sh /d'
-e's/script \\+//'
-e's/perl_module \\+/PERLMODULE_/'
-e'/^\\s*$/d'
;
for f in $(grep -l 'xorg:' master/packages/x/\*.yaml); do
basename -s.yaml "$f"
grep -E '^\\s*[\\.P1]\\s*[0-9a-z]{32,64} +' $f |
sed -e 's/^\\s*[.P1]\\s*[0-9a-z]\\+ \\+\\(\[-a-zA-Z0-9.]\\+\\)-\[0-9.]*\\.tar.*$/  \\1/' ;
done
}

and edit the output as required.