Running batch jobs on the system

The new systems use the Platform Computing LSF batch queuing system to run batch jobs. LSF lets you submit batch jobs to a queue and will run jobs when requested resources are available. You submit jobs with bsub to one of the available queues. Options to bsub can be specified either as a line in a script (known as an embedded option) or on the command line. If embedded options are used, the script must be submitted using the following format: 
   user@gridur> bsub < jobscript
where jobscript contains job submission options (lines beginning with #BSUB) as well as command lines to execute. For some of the most used bsub options see sample batch script below. You can also submit batch jobs with options to bsub via the command line, e.g. 
   user@gridur> bsub -o job.out -e job.err a.out
will submit a.out to the (default) normal queue, and store standard output and standard error in the files job.out and job.err respectively. However, we strongly recommend that you submit batch jobs using a job script.

To insure agreement between the number of processors you specify for your job using the -n option to bsub and the number of processors during execution you are adviced to use the LSB_MCPU_HOSTS environment variable. This is set at job submission and contains the number of CPUs requested from LSF. Using this variable (see sample batch script below) you only have to change the number specified at the option line (#BSUB -n ...) when later running your job with a different number of processors.

To make I/O more efficient always copy your inputfiles to the /work/$LOGNAME directory. Also temporary store outputfiles at this location. After job execution move the outputfiles to your home directory. Notice that files which have not been accessed for a certain amount of time, normally 14 days, will be automatically deleted, this is subject to change but it is not expected to be less than a week.

For a complete description of the bsub command and its options refer to the man page.

Sample batch script

#!/bin/ksh
#BSUB -J myjob        # Assigns the specified name to the job
#BSUB -q normal       # Submits the job to the specified queue
#BSUB -o job.out      # Appends the standard output to the specified file
#BSUB -e job.err      # Appends the standard error to the specified file
#BSUB -N              # Sends the job report by mail when the job finishes
#BSUB -n 4            # Specify the number of processors

set -e

# Get the number of processors requested from LSF
nprocs=$(echo $LSB_MCPU_HOSTS | sed s'/.* //')              # ksh syntax

# Create (if necessary) and move to my working directory
w=$WORKDIR/$LOGNAME
if [ ! -d $w ]; then mkdir $w; fi
cd $w

# Copy inputfiles to working directory
cp $HOME/what_ever_path/inputfiles .

# Run my MPI program
mpirun -cpr -np $nprocs $HOME/what_ever_path/my_mpi_program

# Move results back to home directory
mv outputfiles $HOME/what_ever_path/

exit 0