A topic detection method for high dimensional datasets

Topics extraction from documents has become increasingly important due to its effectiveness in many tasks, including information retrieval, information filtering and organization of document collections in digital libraries. The Topic Detection consists to find the most significant topics within a document corpus. In this paper we explore the adoption of a methodology of feature ex- Traction and reduction to underline the most significant topics within a corpus. We used an approach based on a clustering algorithm (X-means) over the tf - idf matrix calculated starting from the corpus, by which we describe the frequency of terms, represented by the columns, that occur in each document, represented by a row. To extract the topics, we build n binary problems, where n is the numbers of clusters produced by an unsupervised clustering approach and we operate a supervised feature selection over them considering the top features as the topic descriptors. We will show the results obtained on two different corpora. Both collections are expressed in Italian: The first collection consists of documents of the University of Naples Federico II, the second one consists in a col- lection of medical records. Copyright © (2014) by Universita Reggio Calabria & Centro di Competenza (ICT-SUD) All rights reserved.