Search Google Appliance

Home >> Advanced Research Computing >> Application Guide - Java

Application Guide - Java

Introduction

Java one of the most popular programming languages, particularly for client-server web applications, often preferred over other options for faster code development, higher code reliability and easy portability.  Although not often used in scientific programming, Java is nevertheless the language in which some applications are written (e.g. Beast, a Bayesian MCMC analysis application for molecular sequences) or in which researchers choose to develop their own codes.

This guide presents two solutions for running Java code on parallel resources.  These are:

  • running Java using multiple threads (suitable for running on multi-core machines, such as departmental workstations or cluster nodes) and
  • running Java using a distributed-computing library (suitable for running on a collection of cluster nodes).

Multi-threaded Java applications

Java applications can benefit from faster execution on modern multi-core machines through using multiple execution threads, and Java provides built-in support for multithreaded programming.  A multithreaded program contains two or more threads (each defining a separate path of execution) that can run concurrently.  Multi-threaded programming in Java is beyond the scope of this guide and users are invited to start with one of the numerous web tutorials available on the matter, e.g. this one.  Once an application is ready, it can be run using a PBS job script similar to the following one:

#!/bin/bash
#PBS -V
#PBS -l nodes=1
#PBS -l walltime=0:10:00
#PBS -N threadedTest
cd $PBS_O_WORKDIR
myApplication -threads 8 -input myAppInput.txt -output myAppOutput.txt

 

The above example runs on a single cluster node (#PBS -l nodes=1) using the 8 cores available on each (by specifying a 8-way threaded execution).  It is assumed that the Java code myApplication takes the number of threads as a command line argument, along with the name of the input and output files.

Distributed Java applications

Although Java does not have a built-in or "official" mechanism for distributed computing, several research groups have developed bindings for MPI (the de-facto standard for distributed computing programming), to various degrees of success.  One of the first such attempts was mpiJava, essentially a set of wrappers around C MPI library functions, which proved to be of limited interest because of its relatively low portability.  The same project however defined the mpiJava API, a very successful API for Java that closely mimics the equivalent C++ bindings.  Yet another project was the MPJ API, which is less used than the mpiJava API but was designed to be more object oriented that the latter.

The guide shows how to run distributed Java applications, written using MPJ Express.  MPJ Express is an open-source Java message-passing library designed to enable programmers to develop distributed parallel applications that run on multicore processors as well as on clusters and cloud resources.  Programming documentation can be found here.

First, you need to load the module for the MPJ Express package, making the executables available in the path and the adjustments to the shell environment needed to run and automatically loading the appropriate java module:

module load mpjexpress

 

Then, create a directory call test to run an example, and copy the example files

mkdir test
cd test
cp /system/software/examples/java/mpjexpress/* .

 

Two files are copied; a PBS script submit.sh and a java source file integration.java.

This source integration.java is a simple example that computes the value of a function integral using the trapezium rule and distributed MPI-style computing, a classical example for parallel programming.  Compile the source using the command

javac -cp .:$MPJ_HOME/lib/mpj.jar integration.java

This creates the file integration.class, which can be run through the MPJ Express launcher.

Finally, you can run a job using the PBS submission script provided, whch looks like this:

#!/bin/bash
#PBS -V
#PBS -l nodes=2:ppn=8
#PBS -l walltime=0:10:00
#PBS -N mpj-express
# go to directory where job was submitted
cd $PBS_O_WORKDIR
# prepare hostname file
sort $PBS_NODEFILE | uniq > hosts-$PBS_JOBID
# count the number of java processes to launch
NUM_PROCS=` wc -l $PBS_NODEFILE | awk '{printf $1}' `
# boot up the daemon ring
mpjboot hosts-$PBS_JOBID
# launch the application
mpjrun.sh -np $NUM_PROCS -machinesfile hosts-$PBS_JOBID -dev niodev integration
# halt daemons
mpjhalt hosts-$PBS_JOBID

 

The script requests two cluster nodes for execution, writes the host names to the file hosts-$PBS_JOBID using the PBS node file information.  The host name file is uniquely identified by the job number, which is useful in case the above script is started as an array job, in which the runs have to be managed (daemons started and stopped) independently.  The script starts the MPI daemon ring on the specified hosts and launches the application into execution through the launcher script mpjrun.sh.  The daemon is stopped at the end of the execution.

Service area: 

Written by IT Services. Latest revision 28 March 2017