Research

Data analysis is of immense importance in society. By data, we mean any object or quantity represented electronically, usually as a string of numbers. Common situations where data must be analyzed occur for example in face detection in digital cameras, in modern spam filters, in internet search, in bioinformatics, in remote sensing etc. In all such applications, mathematical and statistical methods are required to transform the information provided by the raw data into knowledge about the structure, or patterns, of the data, e.g. concerning different classes or groups of objects, the ranking of such objects, or the prediction of future values.

Machine learning is a modern approach to pattern recognition, where the parameters of a system, or machine, are computed in an iterative process in order to capture the patterns of the data by optimizing a function for extracting knowledge from the data.

The properties of the cost function used determines what kind of data patterns one might capture. My own research has for a large part evolved around the development of information theoretic cost functions for machine learning. This approach is called information theoretic learning (ITL). A common property of these methods is that so-called Parzen windowing for density estimation is implicitly used to compute the ITL measures. My involvement in ITL started in the period 2002-2004, when I had two different research stays at the University of Florida, working with Prof. J. C. Principe and his co-workers. ITL has had great success. See also the recent book by Principe (I am co-author on two chapters.)

Some highlights of my recent research include:

• Kernel Entropy Component Analysis: This method, abbreviated kernel ECA, performs data transformation and dimensionality reduction by preserving as much as possible of the entropy of the original data set. Perhaps surprisingly, this amounts to projections onto certain kernel PCA axes, which may not necessarily correspond to the top eigenvalues of the kernel matrix, in some cases leading to drastically different results than kernel PCA. This method has also recently been used in cloud screening in remote sensing, and in face recognition.
• Information Cut for Clustering: Interestingly, divergence measures between probability density functions may in the ITL approach be shown to correspond to graph cuts. The information cut bridges divergences and graph cuts, and is in this work optimized for clustering using Lagrange theory. This leads to a fixed point iteration based on gradients, obtaining good results.
• Kernel Perspectives: In the course of working with ITL, it has become clear that ITL and the celebrated kernel methods have many similarities and common properties. This is also the case for ITL and some graph approaches. Several papers I have authored have investigated such common points.
• Scatter SVM: Spending a year at TU Berlin, working with K.-R. Muller and co-workers, I became interested in the extension of SVMs to multiple classes. This work extends the dual of binary SVMs, interpreted as scatter, as opposed to extending the primal. This keeps the number of variables and constraints under control, and leads to a fast multi-class algorithm.