Next Previous Contents

2. Installation and configuration

This chapter details a fresh installation of DOMjudge. The first section is a Quick Installation Reference, but that should only be used by those already acquainted with the system. A detailed guide follows after that.

2.1 Quick installation

Note: this is not a replacement for the thorough installation instructions below, but more a cheat-sheet for those who've already installed DOMjudge before and need a few hints. When in doubt, always consult the full installation instruction.

External software:

DOMjudge:

On the domserver host: On the judgehosts: It should be done by now. As a check that (almost) everything works, the set of test sources can be submitted:
cd tests
make check
./check-judgings

The check-judgings script automatically verifies most of the test sources, except for a few with multiple possible outcomes; these have to be verified by hand. Read the sources for a description of what should (not) happen.

Optionally:

2.2 Concepts

This manual assumes you are aware of some of the concepts used within DOMjudge. Here's an overview.

DOMjudge discerns three different kinds of hosts:

Team computer

Workstation for a team, where they develop their solutions and from which they submit them to the jury system. The only part of DOMjudge that runs here is the optional command line submit client; all other interaction by teams is done with a browser via the web interface.

DOMjudge server

A host that receives the submissions, runs the database and serves the web pages. This host will run Apache, and MySQL. Optionally these tasks can be further split out to separate machines, but that's normally not necessary and not supported out of the box.

Judgehosts

A number of hosts, at least one, that will retrieve submitted solutions from the DOMjudge server, compile and run them and send the results back to the server. Since this is computationally intensive, there should ideally be at least a couple of these. They will run the judgedaemon from DOMjudge. For security and performance reasons it is highly recommended not to use the server as a judgehost.

Note that the judges (persons) are not required and not recommended to work on any of the DOMjudge server or judgehosts. They can just access the system via the jury web interface and working e.g. on judgehosts can interfere with system stability.

2.3 Requirements

System requirements

The requirements for the deployment of DOMjudge are:

Software requirements

The following software is required for running DOMjudge.

The following items are optional, but may be required to use certain functionality.

Software required for building DOMjudge from distributed sources.

Additional software required for building DOMjudge from a Subversion checkout.

Requirements for team workstations

In the most basic setup the team workstations only need (next to the tools needed for program development) a web browser. The web interface fully works with any known browser, with the exception of notification of new clarifications in the menu bar. That can be updated without reloading the page by using AJAX. This is supported by any reasonably current browser with JavaScript enabled.

Debian installation command

For your convenience, the following command will install needed software on the DOMjudge server as mentioned above when using Debian GNU/Linux, or one of its derivate distributions. Most systems will have the bulk of these packages installed already.


apt-get install gcc g++ make libcurl4-gnutls-dev mysql-server \
        apache2 php5 php5-cli libapache2-mod-php5 php5-mysql \
        ntp sudo procps sharutils \
        phpmyadmin xsltproc libboost-regex-dev libgmp3-dev \
        linuxdoc-tools transfig texlive-latex-recommended texlive-latex-extra

This is for Debian 5.0 "Lenny", for Debian 4.0 "Etch", replace libcurl4-gnutls-dev with libcurl3-dev.

On a judgehost, the following should be sufficient. The last line shows some example compilers to install for C, C++, Java (GNU), Java (Sun), Haskell and Pascal; change the list as appropriate.


apt-get install make sudo php5-cli php5-mysql ntp xsltproc procps sharutils \
        gcc g++ gcj openjdk-6-jre ghc fp-compiler

2.4 Installation system

The DOMjudge build/install system consists of a configure script and makefiles, but when installing it, some more care has to be taken than simply running './configure && make && make install'. DOMjudge needs to be installed both on the server and on the judgehosts. These require different parts of the complete system to be present and can be installed separately. Within the build system these parts are referred to as domserver, judgehost and additionally docs for all documentation.

When installing from a Subversion checkout, the configure/build system first has to be bootstrapped. This can be done by running make dist, which creates the configure script and generates documentation from SGML/LaTeX sources. Note that this requires additional software as specified in the software requirements.

There are three different methods for installing DOMjudge:

Single directory tree

With this method all DOMjudge related files and programs are installed in a single directory tree which is specified by the prefix option of configure, like


./configure --prefix=$HOME/domjudge

This will install each of the domserver, judgehost, docs parts in a subdirectory $HOME/domjudge/domserver etc. Note that these subdirectories can be overridden from the defaults with options like --with-domserver_root=DIR, see configure --help for a complete list. The prefix defaults to /opt/domjudge.

Besides the installed files, there will also be directories for logging, temporary files, submitted sources and judging data:

log

contains all log files.

tmp

contains temporary files.

submissions

(optionally) on the domserver contains all correctly submitted files: as backup only, the database is the authoritative source. Note that this directory must be writable by the web server for this feature to work.

judgings

location on judgehosts where submissions are tested, each in its own subdirectory. The system needs root access to this directory! (for chroot and mounting of proc-fs).

This method of installation is the default and probably most practical for normal purposes as it keeps all files together, hence easily found.

FHS compliant

This method installs DOMjudge in directories according to the Filesystem Hierarchy Standard. It can be enabled by passing the option --enable-fhs to configure and in this case the prefix defaults to /usr/local. Files will be placed e.g. in PREFIX/share/domjudge, PREFIX/bin, /var/log, /tmp, /etc/domjudge.

Maintainer install

The last installation method is meant for developers/maintainers of DOMjudge and does an in-place installation within the source tree. This allows one to immediately see effects when modifying code.

This method requires some special steps which can most easily be run via makefile rules as follows:


make maintainer-conf [CONFIGURE_FLAGS=<extra options for ./configure>]
make maintainer-install

Note that these targets have to be executed separately and the latter requests root privileges via su.

After running the configure script, the system can be built and installed. Each of the domserver, judgehost, docs parts can be built and installed separately, respectively by:


make domserver && sudo make install-domserver
make judgehost && sudo make install-judgehost
make docs && make install-docs

Note that even when installing e.g. in your own home directory, root privileges are still required for domserver and judgehost installation, because user and group ownership of password files, some directories and the setuid-root program runguard have to be set. One should not run DOMjudge programs under the root user however, but under a normal user: runguard is specifically installed setuid-root to make this unnecessary and running as root will give rise to problems, see runguard: root privileges not dropped in the common problems section.

For a list of basic make targets, run make in the source root directory without arguments.

2.5 Configuration

Configuration of the judge system is mostly done by editing the configuration file(s) in etc: domserver-config.php, judgehost-config.php, common-config.php for the configuration options of the domserver, judgehost and shared configuration options respectively. The latter should be synchronised between domserver and judgehosts. Descriptions of settings are included in these files.

Besides these settings, there are a few other places where changes can be made to the system, see other configurable scripts.

2.6 Configuration of languages

Configuration of the compilers of the supported languages should be done separately. For each supported language a shell-script named compile_<lang>.sh should be created and placed in lib/judge on the judgehosts, where <lang> is the ID of the language as specified in the database. For more information, see for example compile_c.sh and test_solution.sh in lib/judge for syntax. Note that compile scripts are included for the most common languages already.

Interpreted languages and non-statically linked binaries can in principle also be used, but then the option USE_CHROOT should be disabled (or all dependencies be added to the chroot environment). Interpreted languages do not generate an executable and in principle do not need a compilation step. However, to be able to use interpreted languages (also Sun's Java), a script must be generated during the compilation step, which will function as the executable: the script must run the interpreter on the source. See compile_perl.sh and compile_java_javac.sh in lib/judge for examples.

DOMjudge supports the use of Sun Java within a chroot environment. For this, a chroot environment which includes the Sun Java libraries must first be built. This can be accomplished with the included script dj_make_chroot: run this as root and pass as arguments the target directory to build the chroot environment in and as second argument the target machine architecture. Start the script without arguments for usage information. See also sections Installation of a judgehost and Problems: Java & chroot.

2.7 Configuration of special run and compare programs

To allow for problems that do not fit within the standard scheme of fixed input and/or output, DOMjudge has the possibility to change the way submissions are run and checked for correctness.

The back-end script (test_solution.sh) that handles the compilation, running and checking of submissions, calls separate programs for running and comparison of the results. These can be specialised and adapted to the requirements per problem. For this, one has to create programs or scripts run_<some-tag> and/or compare_<some-tag> in the lib/judge directory (see run and compare for examples and usage information). Then one must specify this <some-tag> in the special_run and/or special_compare fields of the problem entry in the MySQL database (empty means that the default script should be used).

Implementing a special compare script, also called a validator, can be done in two ways: either write a program that is called directly (by test_solution.sh) or use (a copy of) the compare_program.sh script. The latter generates the XML result file and handles redirection of input/output for you. When using this wrapper (the easiest method), the jury should write a checker program which can be called as


$CHECK_PROGRAM <testdata.in> <program.out> <testdata.out>

and this program should write some kind of difference to stdout. No output results in a correct verdict and a nonzero exitcode in an internal (system) error. The script compare_program.sh as shipped is configured to call check_float, which compares floating point numbers. For more details, see the comments at the top of the files test_solution.sh, compare_program.sh and (when not using the wrapper) the appendix on the ICPC validator interface.

The shipped test_solution.sh has support for enabling presentation-error to be returned as a judging result. This is counted as an incorrect submission. For this to work, you need to uncomment the marked lines near the bottom of this script. The validator used should also support returning presentation error results; the shipped compare supports this and reports presentation-error when the output only differs in whitespace from the reference output. The script compare_program.sh does not support presentation error results.

2.8 Alerting system

DOMjudge includes an alerting system. This allows the administrator to receive alerts when important system events happen, e.g. an error occurs, or a submission or judging is made.

These alerts are passed to a plugin script alert which can easily be adapted to fit your needs. The default script emits different beeping sounds for the different messages when the beep program is available, but it could for example also be modified to send a mail on specific issues, connect to monitoring software like Nagios, etc. For more details, see the script lib/alert.

2.9 Other configurable scripts

There are a few more places where some configuration of the system can be made. These are sometimes needed in non-standard environments.

2.10 Submission methods

DOMjudge supports two submission methods: via the command line submit program and via the web interface. From experience, both methods have users that prefer the one above the other.

The command line submit client can send submissions by either using the web interface internally (http protocol, the default), or using a special command line submit protocol, called Dolstra. The latter has some special features but is not usually needed. See Submitdaemon and the Dolstra protocol for details on this.

Using the http protocol with the submit client requires the libcURL library development files at compile time (the submit client is statically linked to libcURL to avoid a runtime dependency).

The database is the authoritative version for submission sources; file system storage is available as an easy way to access the source files and as backup. The program bin/restore_sources2db is available to recover the submission table in the database from these files. The command line daemon will automatically store sources on the file system; the web server needs write permissions on SUBMITDIR and ignores file system storage if these permissions are not set.

2.11 Database installation

DOMjudge uses a MySQL database server for information storage.

The database structure and privileges are included in MySQL dump files in the sql subdirectory. The default database name is domjudge. This can be changed manually in the etc/dbpasswords.secret file: the database name as specified for the jury user will be used when installing.

Installation of the database is done with bin/dj-setup-database. For this, you need an installed and configured MySQL server and administrator access to it. Run


dj-setup-database [-u <admin_user>] [-p <password>|-r] install

to create the database, users and insert some default/example data into the domjudge database. The option -r will prompt for a password; when no user is specified, the mysql client will try to read credentials from $HOME/.my.cnf as usual. The command uninstall can be passed to dj-setup-database to remove the DOMjudge database and users; this deletes all data!

The domjudge database contains a number of tables, some of which need to be manually filled with data before the contest can be run. See the database section of Contest setup for details.

Fine tuning settings

It may be desirable or even necessary to fine tune some MySQL default settings:

Setting up replication or backups

The MySQL server is the central place of information storage for DOMjudge. Think well about what to do if the MySQL host fails or loses your data.

A very robust solution is to set up a replicating MySQL server on another host. This will be a hot copy of all data up to the second, and can take over immediately in the event of failure. The MySQL manual has more information about setting this up.

Alternatively, you can make regular backups of your data to another host, for example with mysqldump, or use a RAID based system.

Replication can also be used to improve performance, by directing all select-queries to one or more replicated slave servers, while updates will still be done to the master. This is not supported out of the box, and will require making changes to the DOMjudge source.

2.12 Web server configuration

For the web interface, you need to have a web server (e.g. Apache) installed on the jury system and made sure that PHP correctly works with it. Refer to the documentation of your web server and PHP for details.

You should turn PHP's magic_quotes_* options off. We also recommend to turn off register_globals. If you want to upload large testcases in the jury web interface, it may be necessary to raise some PHP limits or you'll get "memory exhausted" errors. Make sure that the parameters memory_limit, upload_max_filesize and post_max_size in php.ini are all well above the size of your largest testcase.

To configure the web server for DOMjudge, use the Apache configuration snippet from etc/apache.conf. It contains examples for configuring the DOMjudge pages with an alias directive, or as a virtualhost, optionally with SSL; it also contains PHP and security settings. Reload the web server for changes to take effect.

See also section Security: webserver privileges for some details on file permissions for the etc/dbpasswords.secret and etc/htpasswd-{jury,plugin} files.

2.13 Logging & debugging

All DOMjudge daemons and web interface scripts support logging and debugging in a uniform manner via functions in lib.error.*. There are three ways in which information is logged:

Each script also defines a default threshold level for messages to be logged to stderr (VERBOSE: defaults to LOG_INFO in daemons and LOG_ERROR in the web interface) and for log file/syslog (LOGLEVEL: defaults to LOG_DEBUG).

In case of problems, it is advisable to check the logs for clues. Extra debugging information can be obtained by setting the config option DEBUG to a bitwise-or of the available DEBUG_* flags in etc/common-config.php, to e.g. generate extra SQL query and timing information in the web interface.

2.14 Installation of a judgehost

A few extra steps might need to be taken to completely install and configure a judgehost.

For running solution programs under a non-privileged user, a user has to be added to the system(s) that act as judgehost. This user does not need a home-directory or password, so the following command would suffice to add a user `domjudge-run' with minimal privileges.

On RedHat:

useradd -d /nonexistent -g nobody -M -n -s /bin/false domjudge-run
On Debian:
useradd -d /nonexistent -g nogroup -s /bin/false domjudge-run

For other systems check the specifics of your useradd command. This user must also be configured as the user under which programs run via configure --enable-runuser=USER; the default is domjudge-run.

When the chroot setting is enabled (default), a static POSIX shell has to be available for copying it to the chroot environment. For Linux i386, a static Dash shell is included, which works out of the box. For other architectures or operating systems, a shell has to be added manually. Then simply point the lib/sh-static symlink to this file.

If you use the default chroot-startstop.sh script, then the following lines must be added to /etc/sudoers:

domjudge ALL=(root) NOPASSWD: /bin/mount -n -t proc --bind /proc proc
domjudge ALL=(root) NOPASSWD: /bin/umount /*/proc
domjudge ALL=(root) NOPASSWD: /bin/mount --bind <chrootdir>/*
domjudge ALL=(root) NOPASSWD: /bin/umount JUDGEDIR/*
Here the user domjudge must be replaced by the user you intend to run the judgedaemon as, <chrootdir> by the path the chroot environment was installed to and JUDGEDIR by the value of judgehost_judgedir specified by configure. Note that <chrootdir> is different from CHROOTDIR as specified in configure; the first is the tree from which bind-mounts are made when Sun Java is used, the latter the directory under which judgings are allowed to be executed in a chroot environment, and this path is by default set to judgehost_judgedir.

2.15 Building and installing the submit client

The submit client can be built with make submitclient. There is no make target to install the submit client, as its location will very much depend on the environment. You might e.g. want to copy it to all team computers or make it available on a network filesystem. Note that if the team computers run a different (version of the) operating system than the jury systems, then you need to build the submit client for that OS.

The submit client needs to know the address of the domserver. This can be passed as a command line option or environment variable. The latter option makes for easier usage. A sample script submit_wrapper.sh is included, which sets this variable. See that script for more details on how to set this up.

The submit client under Windows/Cygwin

The submit client can also be built under Windows when the Cygwin environment is installed. First the Cygwin setup.exe program must be downloaded and installed with GCC, curl-devel and maybe some more packages included.

When Cygwin is correctly installed with all necessary development tools, the submit binary can be created by running configure followed by make submit.exe in the submit directory.

2.16 (Re)generating documentation and the team manual

There are three sets of documentation available under the doc directory in DOMjudge:

the admin-manual

for administrators of the system (this document),

the judge-manual

for judges, describing the jury web interface and giving some general information about this system,

the team-manual

for teams, explaining how to use the system and what restrictions there are.

The team manual is only available in PDF format and must be built from the LaTeX sources in doc/team after configuration of the system. A prebuilt team manual is included, but note that it contains default/example values for site-specific configuration settings such as the team web interface URL and judging settings such as the memory limit. We strongly recommend rebuilding the team manual to include site-specific settings and also to revise it to reflect your contest specific environment and rules.

Besides a standard LaTeX installation, the team manual requires the svn and expdlist packages. These are available in TeX Live in the texlive-latex-extra package in any modern Linux distribution. Alternatively, you can download and install them manually from their respective subdirectories in http://mirror.ctan.org/macros/latex/contrib.

When the docs part of DOMjudge is installed and site-specific configuration set, the team manual can be generated with the command genteammanual. The PDF document will be placed in the current directory or a directory given as argument. The following should do it on a Debian-like system:


sudo apt-get install make transfig texlive-latex-extra texlive-latex-recommended
./genteammanual

The team manual is currently available in two languages: English and Dutch. We welcome any translations to other languages.

The administrator's and judge's manuals are available in PDF and HTML format and prebuilt from SGML sources. Rebuilding these is not normally necessary. To rebuild them on a Debian-like system, the following commands should do it:


sudo apt-get install linuxdoc-tools make transfig texlive-latex-recommended
make -C doc/admin docs
make -C doc/judge docs

2.17 Optional features

Source code syntax highlighting

To support coloured display of submitted source code in the jury interface, the PEAR Text_Highlighter class needs to be available in your PHP include path. This means that you can install this system wide with the PEAR-provided tools, like this: pear install Text_Highlighter.

Alternatively you can download the source code from the Text_Highlighter website and unpack that under the www/jury directory. Rename the resulting Text_Highlighter-x.y.z directory to just Text.

NTP time synchronisation

We advise to install an NTP-daemon (Network Time Protocol) to make sure the time between jury computer and judgehost (and team computers) is in sync.

The plugin web interface

Next to the public, team and jury web interfaces, DOMjudge also provides a plugin web interface. This web interface is still in development so subject to change. The interface provides contest data from DOMjudge in XML format and is meant to provide external programs (plugins) with data on the contest. This allows for all kinds of extensions beyond the core functionality of DOMjudge such as providing a fancy scoreboard with more statistics, aggregation of scoreboard data for a final presentation during the prize ceremony.

As we are still thinking about possible uses and thus the data to be provided, the exact specification of this interface may change. Also, we are especially interested in feedback and ideas.

There are currently two data-sets provided within the plugin subdirectory of the DOMjudge web interface, both in XML format:

scoreboard.php

This page provides a representation of the scoreboard. Additionally it includes legend tables for problems, languages, affiliations and team categories. It does not accept any arguments.

event.php

This page provides a representation of events that happened during the contest, including submissions, judgings, contest state changes and general clarifications. This page accepts two arguments fromid and toid to limit the output to events with event ID in that range.

See these pages or the accompanying xsd-files for the exact structure.

2.18 Upgrading

Upgrading DOMjudge to a newer version is not (yet) well supported. In case you want to upgrade, be aware of the following problems.

There might be database layout incompatibility problems. The first place to check for this is the ChangeLog. There is not yet functionality to automatically upgrade to a new layout.

Upgrading is probably best done by installing the new version of DOMjudge in a separate place and transferring the configuration settings from the old version.


Next Previous Contents