Shared Memory Programming

Shared memory computing makes it possible to take a sequential program and parallelize it step by step. Parallel execution is achieved either by distributing the iterations in a program loop over several threads, or by executing code sections in parallel. For this to be possible, the order of the loop iterations or code sections should not influence the answer.

A program can be made parallel by using techniques such as manual parallelization , automatic parallelization, or a combination of both. Get a quick intro on this page.

General advice

1. Make sure that your program is optimized and runs well sequentially. That is, compile your program without SMP parallellization options and run the executable on one processor.

2. Make sure that your program is optimized and runs well on one thread. That is, compile your program with SMP parallellization options and run the executable on one thread.

3. Concentrate your parallelization efforts on the most time-consuming loops. Finding these hot-spots requires a technique called profiling. Several tools for profiling are provided on the Regatta. See the page on profiling for more info.

4. Be aware that running a program on many processors may actually slow it down. The more processes are used, the more overhead is needed (for communication, synchronization, thread creation, etc.) and eventually these overheads can be so large that they outweigh the gain from executing the loops in parallel. Therefore an advice is to start with only 1 processor and then repeatedly increment the number of processors by one until you no longer see a significant reduction in (wall-clock) time.

5. The number of processors (threads) is specified with an environment variable :

setenv OMP_NUM_THREADS <number>   (for C shell 'csh')
export OMP_NUM_THREADS=<number>   (for Korn shell 'ksh')

This can also be done by typing (in C shell 'csh'):

mp <number>

After specifying the number of threads, you can submit the job to the batch system (or better, specify the number of threads in the job script that you give to the batch system).

Runtime environment variables

Environment variables that may improve the runtime behaviour of SMP/OpenMP programs are (for Korn shell):

export XLSMPOPTS="stack=10000000:spins=0:yields=0"
export AIXTHREAD_SCOPE=S
export SPINLOOPTIME=50000
export YIELDLOOPTIME=50000
export MALLOCMULTIHEAP=true

We note that these variables do not guarantee performance improvement. They may have an opposite/negative effect and should therefore be used with care. More information can be found here, here, here, and here.

The default stack space for (OpenMP) threads is 4 Megabyte. If exceeded, this will result in a segmentation fault without proper messages being returned. Use environment variable XLSMPOPTS to increase the default value:

export XLSMPOPTS="stack=67108864"

The above value 67108864 (64 Megabyte) is the maximum value.