Last update: April 27, 2007
Questions and comments to: support-uio@notur.no

Hardware

The 64 processor HP Superdome, magnum.uio.no, is the largest SMP resource at the University of Oslo. magnum.uio.no is a shared memory computer, which means that any process running on the system can, in theory, use all the available memory (128GB). Magnum has 3.5 TB Fiber channel disk system and a 6 TB FC/Sata disk system. The batch system is PBS/moab, and is designed to run with optimal throughput based on a "greedy algorithm" approach.

Contents Intel Xeon cluster (titan.uio.no) 1390 CPUs, 1 - 4 GB memory per CPU (core).
HP Integrity Superdome (magnum.uio.no) 64 CPUs, 128GB memory.
HP rx2600 IA64 (femto.uio.no) 2 CPUs, 4GB memory.

HP rx5670 IA64 (yngel.uio.no) 4CPUs, 8GB memory.
Intel Xeon cluster (pamaster.uio.no) 88CPU, 88GB memory.

Intel Xeon cluster titan.uio.no is a 1380 CPU cluster composed of:
  • 224 4-core SUN X2200 AMD 1u nodes, total: 896 CPUs
  • 180 2-CPU DELL 1425 Intel 1u nodes, total: 360 CPUs
The AMD-nodes run at 2.6 GHz and the Intel-nodes at 3.4 GHz. Even though the AMD-nodes have a slower clock frequency, they usually perform 10-30% better than the 800MHz faster Intel CPUs. The AMD nodes have 8GB of memory per node, while the Intel nodes are equipped with 2-4GB per node. 96 AMD nodes (384 CPUs) and 82 Intel nodes (164 CPUs) are connected through high speed InfiniBand interconnects allowing 940 MB/s point to point communication between each node at 1.5 ms latency.

TITAN servers:

  • 4 16-way AMD nodes with 32 GB memory
  • 1 16-way AMD nodes with 64 GB memory
  • 3 2-way Intel nodes with 4GB memory
  • 3 4-way AMD nodes with 8GB memory
  • 6 4-way AMD fileservers with FC mass storage interconnect
  • 4 2-way Intel fileservers with FC mass storage interconnect
The theoretical peak performance of the cluster is: 10TFlops.
The TITAN cluster will be upgraded to about 15TFlops Q2 2007 and to about 20TFlops by the end of 2007.

Xanadu, the TITAN mass storage is composed of:

  • 100 TB EVA8000 disk (140 TB raw)
  • 15 TB SATAbeast disk
  • 6 TB SATAblade disk
A central GPFS based storage capacity of 130 TB are connected to all nodes and application servers.

HP Superdome

The 64 processor HP Superdome magnum.uio.no is one of the most powerful computational resource available at the University of Oslo. magnum is a shared memory computer, which means that any process running on the system can, in theory, use all the available memory (128GB). Magnum also has some storage space, 3.5TB. Not all of this space can be used by the batch jobs running on the system.


Design
The HP Superdome CPUs are 64 Itanium2 processors running at 1.5GHz and with 6 Mb cache. The processors are all superscalar, capable of 4 instructions per cycle. Thus, the Theoretical peak performance of Magnum is; 4 instructions * 1.5GHz * 64 processors = 384 GFlops.

The Superdome has a hierarchical structure with processors, cells, and ccNUMA systems being the basic building blocks. ccNUMA (cache-coherent Non Uniform Memory Access) provides the capability that any processor on any node can directly address any byte of memory on any node, by processor issued load or store instructions. Cache coherence is maintained by a directory-based coherence scheme where each cache-line in the memory systems is expanded by a set of tab-bits, which are set and used by the hardware and the operating system.

HP rx2600 IA64 The 2 processor rx2600, femto.uio.no. is being used as a test resource for magnum. femto is a shared multiprocessor machine, and it can best be thought of as magnums small cousin. The processors installed in Femto are Itanium2, running at 1 GHz.

HP rx5670 IA64 yngel.uio.no is a Hewlet-Packard machine with four 1.3 GHz Itanium2 64 bit CPUs and with 8Gbytes memory. yngel runs HP-UX and is connected to a 2.3 TB ATAboy2f disk system. The machine belong to SGP grpup.

Intel Xeon cluster pamaster is the front end for the 86CPU Intel Xeon cluster used for ParAlign and Blast runs. The nodes run at 2,4 GHz, having 1GB of memory per CPU.