Beyond the curse of multidimensionality: high dimensional clustering in text mining

Data accumulate and there is a growing need of automated systems for partitioning data into groups, in order to describe, organize and retrieve information. Dealing with documental databases, one of the main aims is text categorization, consisting in identifying documents with similar topics. In the usual Vector space mode, thel documents are represented as points in the high dimensional space spanned by words. Obstacle to an efficient performance of algorithms is the curse of dimensionality: while dimensions increase, the space where individuals are represented becomes sparse. Classical statistical methods lack their properties and interest is devoted to find dense areas in lower dimensional spaces. Aim of the paper is reviewing the basic literature on the topic, focusing attention on dimensionality reduction and double clustering.