EventScope: Configurable On-line Monitoring of Parallel and Distributed Applications

16.12.2002

Abstract

Making parallel applications scale is a difficult task. Running several applications on a set of nodes makes it even more difficult. For improved scalability, on-line adaption can be used. For adaption, on-line monitoring is necessary. This thesis describes EventScope, a framework used by on-line monitors analyzing the communication behavior of parallel applications.

Monitors using the data need to gather large amounts of data, at a high sample rate with minimal perturbation of the monitored application. Monitors also have different data, performance and perturbation
requirements.

Applications are instrumented in a configurable manner by inserting sensors into their communication paths. Sensors collect data in a low-intrusive manner, and store it in 'trace elements' located in EventScope, from which it can be collected by monitors. Monitors use a hierarchical 'gather tree' to collect data from a set of trace elements. The gather tree is implemented using configurable collective operations, that can be mapped to the available resources, enabling to make trade-offs between performance and perturbation. Filtering, sorting, conversion and reduction functions can be added to the tree. The functions are then mapped to the available nodes to reduce or filter data close to the source, or in order to use available resources on the computation nodes to preprocess the gathered data. Similar operations can also be performed before storing data in EventScope.

A parallel wind-tunnel application was monitored using several monitors, to evaluate the amount of data that can be gathered at different sample rates. In particular a hierarchical barrier tree was monitored in great detail. For most monitors the slowdown introduced by monitoring could not be measured. For monitors where slowdown was a problem, reconfiguration reduced it from 1.17 to 1.08.