Trine Dahl:

Lexical Cohesion-Based Text Condensation: An Evaluation of Automatically Produced Summaries of Scientific Articles by Comparison with Author-Written Abstracts


Key words: lexical cohesion, text condensation, scientific text, text evaluation

The aim of the present study is twofold. Firstly, it aims at contributing to a description of how scientific text is structured or organised. Secondly, it aims at evaluating computer-generated summaries of research articles with a view to using them as condensed versions in place of the author-written abstracts.

My text-linguistic point of departure is the theory of text developed by Hoey (1991). His method for (manually) analysing non-narrative text by means of lexical patterns in the text has been computer implemented by Dr Maria Helena Tostevin, a (former) researcher with the Artificial Intelligence Group at the University of Surrey. The program, called Tele-pattan II, is a development from a prototype text-processing system, Tele-pattan (TExt and LExical PATTerns ANalysis), developed by Benbrahim (1996). The program is based on pattern-matching, through which significant (so-called bonded) sentences are extracted from the full text. The program, which has been fine-tuned during the process of writing this thesis, is fast and very robust.

The material consists of 15 research articles from the field of corrosion, taken from four different, refereed, journals. The articles typically report on experimental work. All the texts come equipped with an informative author-written abstract. I propose to see the author abstract as a template against which the computer-generated summaries may be compared and evaluated. The comparison is made by means of a frame-based analysis developed to analyse empirical abstracts (Liddy 1988), with slots for information on Background, Purpose, Methods, Results and Conclusions, where the information found in the abstract is compared with that found in the computer condensation of each text. In addition, since the computer summaries are composed of single sentences presented without any post-editing, the readability of these summaries is also considered.

The results indicate that even though human text condensation involves complex cognitive processes, a so-called shallow approach to automatic text condensation, here represented by Tele-pattan II, may yield acceptable output. It turns out that the content of the computer-generated summaries matches the content of the author abstracts quite well and that readability is generally good. It seems likely that for the majority of the texts investigated we may claim that the computer summaries may replace the abstracts for an expert readership.

This result is seen as empirical support for Hoey’s (1991) claim concerning the importance of lexis in text organisation.