Automatic parallelisation

The IBM Fortran, C, and C++ compilers support automatic parallelization of code. To invoke this, compile and link your files like this: (Fortran77)

xlf_r  -qsmp=auto  [other flags]  file.f

Note that it is necessary to append '_r' to the compiler name to specify the thread-safe version of the compiler.

To see which loops are parallelised automatically by the compiler, use the option -qreport=smplist. This option returns brief information of all the loops that are parallelized. A more detailed listing can be found in files with suffix .lst. One such file is generated for each source file.

To test a small program, use a small number of threads (e.g. less than 4) as follows: (assuming the shell is tcsh)

setenv OMP_NUM_THREADS 2

This is the way to set the number of threads for all SMP-type programs. Then execute your program. If your job requires a large number of threads, a large amount of memory, or long running time, you must submit the job to the batch system. (Running jobs interactively kills performance for both you and the other users on the machine.)

Manual SMP parallelisation

Manual parallelisation of a program is achieved by inserting so-called compiler directives into the source code. Such directives are simple instructions that tell the compiler how to distribute work over the different threads. All directives start with a "c" in F77, a "!" in F90 and "#pragma" in C, and therefore the files containing directives can also be compiled as sequential programs (without the multiprocessing option).

On the Regatta, manual SMP parallelisation can be achieved using directives from OpenMP or IBM. OpenMP has become the new standard for SMP-parallelisation. Therefore, we recommend you to use the IBM specific directives sparingly, if at all, if you want your codes to be easily portable and future proof.

To compile and link a program with OpenMP directives (assuming Fortran95):

xlf95_r  -qsmp=omp:opt  [other flags]  file.f

Here we instruct the compiler to use the OpenMP directives, and also allow the compiler to optimize code inside parallel regions. When debugging, you might want to remove the :opt part.

Online manuals:

• OpenMP Specifications (well written and useful for learning OpenMP)

For more language specific info, see our separate page for programming languages.

A strategy for combining automatic and manual parallelization

1. Compile and link your files with the options -qsmp -qreport=smplist
A listingfile (a file ending with .lst) is produced for each source file.

2. Use the screen output and listingfiles to find the loops in your program which have been parallelized automatically by the compiler. These loops do not contain any data dependencies and the individual iterations of such a loop can safely be run in parallel.

It may be advantagous to help the compiler by adding directives for the parallelization of these loops. It is, however, a quite common mistake to add directives uncritically. This often leads to poor performance of your parallelized program. Keep in mind that the creation of parallel regions and threads costs time. This overhead is justified only when the (wall-clock) time required for the threads to execute the parallelized loop is significantly less than the time required for one (sequential) thread to execute the unparallelized loop.

3. Focus your effort on the routines/loops that are most time-consuming. You can detect these by profiling your code. You can assist the compiler in parallelising your code, by carefully adding directives to expensive loops which were not parallelized automatically. After adding the directive(s), recompile the program, check the listingfile(s), and the correctness of the result after execution of your program. Continue this way with the second most time-consuming routine/loop, and so on, until you have parallelized the most important parts of your program.

4. It is not always possible (unsafe) to insert directives and re-compile with -qsmp. This occurs for example when a loop contains data dependencies between successive iterations. In such cases, you must examine the loops more closely to see whether you can make algorithmic changes in order to remove these dependencies.