Table of Contents
JHBuild is a tool designed to make building collections of source packages (also known as modules). It uses “module set” files to describe the modules available to build. These files include dependency information that allows JHBuild to work out what modules need to be built and in what order to build what the user requested.
JHBuild was originally written for building Gnome, but has since been extended to make it usable with other projects. A “module set” file can be hosted on a web server, allowing people to provide build rules independent of the JHBuild.
JHBuild can build modules from a variety of sources, including:
JHBuild is not intended as a replacement for the distribution's package management system. Instead, it makes it easy to build everything into a separate install prefix so that it doesn't interfere with the rest of the system.
JHBuild takes a bit of work to set up on a system. As well as installing JHBuild's prerequisites, it is necessary to install the prerequisite tools needed to build the software in CVS (or where ever else it is stored).
Before downloading JHBuild, you should make sure you have a copy of Python >= 2.0 installed on your system. It is also essential that the Expat XML parser extension is installed. This will be the case if you are using Python >= 2.3, or had expat installed when building Python. You can check whether this is the case by running the following simple command from the Python interpreter:
>>> import xml.parsers.expat >>>
If this completes without an exceptiopn, then it is installed correctly.
At the moment, the only way to download JHBuild is via CVS. This can be achieved with the following commands. They should be run in the directory where jhbuild will be installed (for example, ~/cvs/gnome2).
$ cvs -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login Logging in to :pserver:anonymous@anoncvs.gnome.org:2401/cvs/gnome CVS password: press enter $ cvs -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome checkout jhbuild $
This will download JHBuild into a jhbuild folder under the current directory. Now to build and install it:
$ cd jhbuild $ make ... $ make install ... $
If these steps complete successfully, a small shell script should be installed in ~/bin to start JHBuild. If this directory is not in the PATH, it will need to be added (possibly by editing ~/.profile or ~/.bashrc).
Before JHBuild can be run, it will be necessary to set up a ~/.jhbuildrc file that configures how JHBuild will behave.
The ~/.jhbuildrc file uses Python syntax to set a number of configuration variables for JHBuild. A minimal configuration file might look something like this:
moduleset = 'gnome-2.10'
modules = [ 'meta-gnome-desktop' ]
checkoutroot = os.path.join(os.environ['HOME'], 'cvs', 'gnome2')
prefix = os.path.join(os.environ['HOME'], 'prefix')
os.environ['INSTALL'] = os.path.join(os.environ['HOME'],
'bin', 'install-check')
This will get JHBuild to build the meta-gnome-desktop module (and its dependencies) from the gnome-2.10 module set. It will unpack source trees to ~/cvs/gnome2 and install modules to ~/prefix. It also sets the INSTALL environment variable to a program that handles installation of headers specially in order to decrease the work during a rebuild.
Some of configuration variables available include:
| moduleset | A string giving the name of the module set to build. If it is a fully qualified URL, then the module set will be cached locally and regularly updated. |
| modules | A list of strings giving the modules you want to build. The list of modules actually built will be recursively expanded to include all the dependencies. |
| checkoutroot | The base directory where all source modules should be unpacked. |
| prefix | The directory prefix to install modules to. |
| os.environ | A dictionary representing the environment. This can be used to set or get environment variable values as seen in the example above. |
| skip | A list of module names that should not be included when deciding what to build. One use of this variable is to use the version of a package included with the distribution instead of building it |
| autogenargs | A string listing arguments that should be passed to the autogen.sh or configure scripts for modules. |
| makeargs | A string listing arguments that should be passed to make. |
| cvsroots | By default, JHBuild will check out code from CVS using an anonymous CVS root. This dictionary is used to tell JHBuild to use an alternative CVS root for a particular repository (a developer would probably want to do this). This variable is a dictionary where the keys are short repository names (for example, gnome.org is used for the Gnome CVS repository), and the values are the alternative CVS root strings. |
| svnroots | Similar to cvsroots but for Subversion repositories. |
Before any modules can be built, it is necessary to have certain build tools installed. These include the GNU auto tools (autoconf, automake, libtool and gettext), pkg-config nad Python.
JHBuild can check if your distro has installed these tools using the sanitycheck command:
$ jhbuild sanitycheck
If this command prints any messages, these can be fixed in one of two ways:
The bootstrap command can be invoked like so:
$ jhbuild bootstrap
This will download and install all the build prerequisites. Once it is finished, the sanitycheck command should be rerun to verify that everything is in place.
The bootstrap command does not build all the packages required by these tools. If the OS does not provide those packages, then they will need to be built separately.
Some packages to check for include m4, perl and a C compiler.
Now that everything is set up, JHBuild can be used to build some software. To build all the modules selected in the ~/.jhbuildrc file, run the following command:
$ jhbuild build
This will download, configure, compile and install each of the modules. If an error occurs at any stage, JHBuild will present a menu asking the user what to do. These choices include dropping to a shell to fix the error, rerunning the build stage, giving up on the module (which will also cause any modules depending on it to fail), or ignore the error and continue.
It is also possible to build a different set of modules (and their dependencies) by passing their names as arguments to the build command:
$ jhbuild build gtk+
If you exit JHBuild part way through a build for some reason, it is possible to pick up a build at a particular package using the --start-at option:
$ jhbuild build --start-at=pango
To build one or more modules, without their dependencies, the buildone command can be used:
$ jhbuild buildone gtk+
To get a list of the modules jhbuild will build, and the order they will be built in, use the list command:
$ jhbuild list
To get information about a particular module, the info command can be used:
$ jhbuild info gtk+
If your internet bandwidth varies, you can get JHBuild to download or update all the software it will build in one go without actually building it:
$ jhbuild update
Later on, you can tell JHBuild to build everything without downloading or updating:
$ jhbuild build --no-network
If you want to run a particular command with the same environment variables set that JHBuild uses, use the run command:
$ jhbuild run program
To start a shell with that environment, use the shell command:
$ jhbuild shell
JHBuild uses a command line syntax similar to tools like CVS:
jhbuild [global-options] command [command-arguments]
The global jhbuild options are:
Command specific options are listed below.
The bootstrap command is used to install a set of build utilities required to build most modules (eg. autoconf, automake, etc).
jhbuild bootstrap
Internally bootstrap is implemented using the same code as build, using the bootstrap.modules moduleset.
The build command is used to build one or more packages, including their dependencies.
jhbuild build [--autogen] [--clean] [--no-network] [--skip=module...] [--start-at=module] [-D date] [module...]
If no module names are given on the command line, then the module list found in the configuration file will be used.
The buildone command is similar to build, but it does not use dependency information to expand the module list. It is useful for quickly rebuilding one or more modules.
jhbuild buildone [--autogen] [--clean] [--no-network] [-D date] module...
The --autogen, --clean, --no-network and -D options are processed the same as for build.
Unlike build, at least one module must be listed on the command line.
The dot command generates a file describing the directed graph formed by the dependencies between a set of modules. This file can then be processed using the GraphViz software to produce a nice diagram.
jhbuild dot [module...]
If no module names are given on the command line, then the module list found in the configuration file will be used.
The output of this command can easily be piped to the dot utility to generate a postscript file:
$ jhbuild dot modules | dot -Tps > dependencies.ps
The info command is used to display information about one or more modules.
jhbuild info module...
The command prints the module name, type, dependencies, dependent packages, and the time it was last installed with JHBuild. It may also print some information specific to the module type, such as the CVS repository or download URL.
The list command is used to show the expanded list of modules the build command would build.
jhbuild list [--show-revision] [module...]
If no module names are given on the command line, then the module list found in the configuration file will be used.
The run command is used to run an arbitrary command using the same environment as JHBuild uses when building modules.
jhbuild run program [argument...]
If using JHBuild to build Gnome, this command can be useful in X startup scripts.
The sanitycheck command performs a number of checks to see whether the build environment is okay.
jhbuild sanitycheck
Some of the checks include:
The shell command starts the user's shell with the same environment as JHBuild uses when building modules.
jhbuild shell
This command is roughly equivalent to the following:
$ jhbuild run $SHELL
The tinderbox command is similar to build, but writes all terminal output to HTML files suitable for publishing on a website. It can be used to set up systems similar to Mozilla's Tinderbox, or Debian's Buildd.
jhbuild tinderbox [--autogen] [--clean] [--no-network] [--output=directory] [--skip=module...] [--start-at=module] [-D date] [module...]
The --autogen, --clean, --no-network, --skip, --start-at and -D options are processed the same as for build.
The update command is similar to build, but only performs the download or update stage for modules without building them.
jhbuild update [--skip=module...] [--start-at=module] [-D date] [module...]
The --skip, --start-at and -D options are processed the same as for build.
The updateone command is similar to update, but it does not use dependency information to expand the module list. It is useful for quickly updating one or more modules.
jhbuild updateone [-D date] module...
The -D option is processed the same as for update.
Unlike update, at least one module must be listed on the command line.
The ~/.jhbuildrc file uses standard Python syntax. The file is run, and the resulting variables defined in the namespace are used to control how JHBuild acts. A set of default values are inserted into the namespace before running the user's configuration file.
| alwaysautogen | If set to True, then always run autogen.sh before make even if a makefile exists. This is equivalent to passing --always-autogen option to JHBuild. Defaults to False. |
| autogenargs | A string containing arguments passed to the autogen.sh script of all modules. Can be overriden for particular modules using the module_autogenargs dictionary. |
| builddir_pattern | A printf style formatting pattern used to generate build directory names. This is only used when using separate source and build trees. The %s in the format string will be replaced with the source directory name. Defaults to '%s'. |
| buildroot | A string giving the parent directory to place build trees. Defaults to None, which causes builds to be performed within the source trees. |
| checkoutroot | A string giving the directory to unpack source trees to. Unless buildroot is set, builds will occur in this directory too. Defaults to ~/cvs/gnome2. |
| cvsroots | A dictionary that can be used to change the CVS roots used to check out source code. If you have a CVS account for a particular project, you can set the associated key to use that account rather than the anonymous account. For example, you might want to set 'gnome.org' to ':ext:username@cvs.gnome.org:/cvs/gnome'. |
| branches | A dictionary that can be used to override the branch used for a particular module. This is useful if you are doing some changes on a branch of a module and want JHBuild to build that branch instead of the one listed in the module set. |
| makeargs | A string listing additional arguments to be passed to make. Defaults to ''. |
| makecheck | A boolean value specifying whether to run make check after make. This might be useful in a tinderbox-style setup. Defaults to False. |
| makeclean | A boolean value specifying whether to run make clean before make. Defaults to False. |
| module_autogenargs | A dictionary mapping module names to strings giving arguments to be passed to autogen.sh. If a particular module isn't listed in the dictionary, the global autogenargs will be used instead. |
| module_makeargs | A dictionary mapping module names to strings giving arguments to be passed to make. If a particular module isn't listed in the dictionary, the global makeargs will be used instead. |
| modules | A list of module names to build. This list will be expanded using the dependency information found in the module set. Defaults to [ 'meta-gnome-desktop' ]. |
| moduleset | A string giving the name of the module set to use. This can either be a short string to refer to one of JHBuild's included module sets, or a full HTTP URL to refer to an externally managed module set. Currently defaults to 'gnome-2.10', but is usually updated as Gnome development progresses. |
| nonetwork | A boolean value saying whether to access the network or not. This affects checking out or updating CVS modules, downloading tarballs and updating module sets. Setting this to True is equivalent to passing the --no-network option to JHBuild. Defaults to False. |
| prefix | A string giving the prefix to install modules to. Defaults to '/opt/gnome2'. |
| pretty_print | A boolean value that can be used to disable pretty printing of subprocess output. Currently there is only support for pretty printing CVS output. You probably only want to disable this if the pretty printing causes problems. Defaults to True. |
| skip | A list of modules to skip when expanding the list of modules to build. This is similar to the --skip option (in fact, the --skip option extends this list). This list is empty by default. |
| sticky_date | If set, JHBuild will attempt to check out modules as they existed at the given date. The date should be given in the form 'yyyy-mm-dd'. Defaults to None. |
| svnroots | Similar to cvsroots but for Subversion repositories. |
| tarballdir | If set, tarballs will be downloaded to this directory instead of checkoutroot. This is useful if you have multiple JHBuild environments or regularly blow away your checkoutroot and want to reduce bandwidth usage. |
| tinderbox_outputdir | A string giving the directory to store jhbuild tinderbox output. This string can be overridden by the --output option. Defaults to None, so either the command line option must be used or this variable must be set in the configuration file. |
| use_lib64 | A boolean value that specifies whether to install libraries to lib64 directories. If this is set, --libdir=\${exec_prefix}/lib64 will be passed to configure. Defaults to True if running on x86_64, ppc64 or s390x Linux, and False on other systems. |
In addition to the above variables, there are some other things that can be set in the configuration file:
This is dictionary represents the environment of the process (which also gets passed on to processes that JHBuild spawns).
Some environment variables you may want to set include CFLAGS, INSTALL (to use the more efficient install-check program included with JHBuild) and LDFLAGS.
This will add a directory to a PATH-style environment variable. It will correctly handle the case when the environment variable is initially empty (having a stray colon at the beginning or end of an environment variable can have unexpected consequences).
This function has special handling for the ACLOCAL_FLAGS environment variable, which expects paths to be listed in the form -I pathname.
After processing the configuration file, JHBuild will alter some paths based on variables such as prefix (eg. adding $prefix/bin to the start of PATH).
The prependpath function works like addpath, except that the environment variable is modified after JHBuild has made its changes to the environment.
JHBuild uses simple XML files to describe the dependencies between modules. A RELAX-NG schema and Document Type Definition are included with JHBuild in the modulesets/ directory. The RELAX-NG schema makes it trivial to edit module set files using nxml-mode in Emacs.
The toplevel element in a module set file is moduleset element. Currently no XML namespace is used, but in the future this might change. The elements below the toplevel come in three types: module sources, include statements and module definitions.
Rather than listing the full location of every module, a number of "module sources" are listed in the module set, and then referenced by name in the module definitions. As well as reducing the amount of redundant information in the module set, it makes it easy for a user to specify an alternative source for those modules (for CVS and Subversion, it is common for developers and users to use different repository access methods).
The cvsroot element is used to describe a CVS repository.
<cvsroot name="rootname"
[ default="yes|no" ]
root="anon-cvsroot"
password="anon-password"/>
The name attribute should be a unique identifier for the CVS repository.
If default attribute says whether this is the default module source for this module set file.
The root attribute lists the CVS root used for anonymous access to this repository, and the password attribute gives the password used for anonymous access.
The svnroot element is used to describe a Subversion repository.
<svnroot name="rootname"
[ default="yes|no" ]
href="anon-svnroot"/>
The name attribute should be a unique identifier for the Subversion repository.
If default attribute says whether this is the default module source for this module set file.
The href attribute lists the base URL for the repository. This will probably be either a http, https or svn URL.
The arch-archive element is used to describe a GNU Arch archive.
<arch-archive name="archivename"
[ default="yes|no" ]
href="mirror-url"/>
The name attribute should be the Arch archive name.
If default attribute says whether this is the default module source for this module set file.
The href attribute lists a public mirror URL for the archive.
JHBuild allows one module set to include the contents of another by reference using the include element.
<include href="uri"/>
The href is a URI reference to the module set to be included, relative to the file containing the include element.
Only module definitions are imported from the referenced module set — module sources are not. Multiple levels of includes are allowed, but include loops are not (there isn't any code to handle loops at the moment).
There are various types of module definitions that can be used in a module set file, and the list can easily be extended. Only the most common ones will be mentioned here.
The cvsmodule element is used to define a module that is to be built from CVS.
<cvsmodule module="modulename"
[ revision="branch-or-tag" ]
[ checkoutdir="directory" ]
[ root="rootname" ]
[ autogenargs="autogenargs" ]
[ makeargs="makeargs" ]
[ supports-non-srcdir-builds="yes|no" ]>
<dependencies>
<dep package="modulename"/>
...
</dependencies>
<suggests>
<dep package="modulename"/>
...
</suggests>
</cvsmodule>
The module, revision and root attributes identify the module to check out from CVS. The checkoutdir attribute can be used to specify an alternative directory to check out to (by default, the value of module is used).
The autogenargs, makeargs and supports-non-srcdir-builds attributes are common to many different module types. The autogenargs attribute lists additional arguments to be passed to autogen.sh, and makeargs lists additional arguments to be passed to make. The supports-non-srcdir-builds attribute is used to mark modules that can't be cleanly built using a separate source directory.
The dependencies and suggests elements are used to declare the dependencies of the module. Any modules listed in the dependencies element will be added to the module list for jhbuild build if it isn't already included, and make sure the dependent modules are built first.
After generating the modules list, the modules listed in the suggests element will be used to further sort the modules list (although it will not pull any additional modules). This is intended for cases where a module has an optional dependency on another module.
The svnmodule element is used to define a module that is to be built from Subversion.
<svnmodule module="modulename"
[ checkoutdir="directory" ]
[ root="rootname" ]
[ autogenargs="autogenargs" ]
[ makeargs="makeargs" ]
[ supports-non-srcdir-builds="yes|no" ]>
<dependencies>
<dep package="modulename"/>
...
</dependencies>
<suggests>
<dep package="modulename"/>
...
</suggests>
</svnmodule>
The module attribute gives the path of the module relative to the repository URI. All other options for this element are processed as for cvsmodule.
The archmodule element is used to define a module that is to be built from a GNU Arch archive.
<archmodule version="modulename"
[ checkoutdir="directory" ]
[ root="rootname" ]
[ autogenargs="autogenargs" ]
[ makeargs="makeargs" ]
[ supports-non-srcdir-builds="yes|no" ]>
<dependencies>
<dep package="modulename"/>
...
</dependencies>
<suggests>
<dep package="modulename"/>
...
</suggests>
</archmodule>
The version attribute gives the version to be checked out from the archive specified by root. All other options for this element are processed as for cvsmodule.
The tarball element is used to define a module that is to be built from a tarball.
<tarball id="modulename"
[ version="version" ]
[ autogenargs="autogenargs" ]
[ makeargs="makeargs" ]
[ supports-non-srcdir-builds="yes|no" ]>
<source href="source-url"
[ size="source-size" ]
[ md5sum="source-md5sum" ]/>
<patches>
<patch file="filename" strip="level"/>
...
</patches>
<dependencies>
<dep package="modulename"/>
...
</dependencies>
<suggests>
<dep package="modulename"/>
...
</suggests>
</tarball>
The id and version attributes are used to identify the module.
The source element specifies the file to download and compile. The href attribute is mandatory, while the size and md5sum attributes are optional. If the last two attributes are present, they are used to check that the source package was downloaded correctly.
The patches element is used to specify one or more patches to apply to the source tree after unpacking. The patch files are looked up in the jhbuild/patches/ directory, and the strip attribute says how many levels of directories to prune when applying the patch.
The other attributes and the dependencies and suggests sub-elements are processed as for cvsmodule.
The metamodule element defines a module that doesn't actually do anythin. The only purpose of a module of this type is its dependencies.
<metamodule id="modulename">
<dependencies>
<dep package="modulename"/>
...
</dependencies>
<suggests>
<dep package="modulename"/>
...
</suggests>
</metamodule>
The id attribute gives the name of the module. The child elements are handled as for cvsmodule.
1. General JHBuild Questions | |
| Q: | The wget command can't download any tarballs. How do I get it to work with my firewall? |
| A: | This can be fixed by creating or editing a ~/.wgetrc file. If you need to go through an HTTP proxy to access FTP sites, add a line like the following to the file: ftp_proxy = http://hostname:port/ If you just need to use passive FTP connections (sometimes needed with NAT firewalls), add the following line: passive_ftp = on This will fix all uses of the wget command. |
| Q: | Building stuff is slow. Is there any way I can speed it up? |
| A: | Other than buying a faster CPU, hard disk or getting more memory, you might want to install CCache, which can cache compilation results for you. It is available with most distributions. After installing CCache, set the cache size with the following command: ccache -M 2G (replace 2G with the size you want for your cache). Then create symlinks to CCache for your compiler in ~/bin: cd ~/bin for cmd in cc gcc c++ g++; do ln -s /usr/bin/ccache $cmd done You can check the status of your cache (such as cache hit rates) with the following command: ccache -s |
| Q: | Is there a better way to monitor the status of the build than looking at terminal window? |
| A: | If you have Zenity >= 2.9 installed on your system, JHBuild will display an icon in the system tray. The icon will display the current build stage, and the tooltip will show the last message from JHBuild. In the future, the icon may get support for popping up a balloon on error. |
2. Building Gnome | |
| Q: | What other prerequisites are needed to build Gnome with JHBuild? |
| A: | Some of the packages you will need on your system include:
If you are installing distro packages, you may need to install corresponding "dev" or "devel" packages. Note that this list is just a starting point — not a comprehensive list. |
| Q: | I've built Gnome with JHBuild. How do I run it? |
| A: | You will want to create a ~/.xsession file, which is run by the display manager when you log in. This file should look something like this: #!/bin/sh exec jhbuild run gnome-session Finally, you should make sure that the ~/.xsession file is executable. |
| Q: | I built Gnome using JHBuild with prefix set to /usr, and now my system is broken. What should I do? |
| A: | Don't set prefix to /usr. |
| Q: | How do I get gnome-volume-manager to work when running in a prefix? |
| A: | The gnome-volume-manager program reacts to messages from hald over the system message bus, which must be running as root. Assuming that your distribution comes with HAL, the main problem is getting gnome-volume-manager to talk to the system message bus. Since communication is done over a UNIX domain socket, the easiest way to do this is to create a symlink from /var/run/dbus to $prefix/var/run/dbus: mkdir -p $prefix/var/run cd $prefix/var/run ln -s /var/run/dbus dbus You may also have trouble building HAL with JHBuild, since it tries to install some things outside of its build root. Running make -k install in the hal directory might help here. |