Evaluating classifier predictive performance in multi-class problems with balanced and imbalanced data sets

A major issue in classification problems arises when dealing with class imbalance, which requires the adoption of a suitable performance measure able to handle imbalanced data sets. This paper introduces the Balanced AC1 and its weighted version Balanced AC2 as classifier performance measures suitable for both balanced and imbalanced data sets. The performances of the proposed measures are compared against those of other well-known performance measures through an empirical comparison using several algorithms and data sets. Moreover, the applicability of Balanced AC1 is showcased through an illustrative example dealing with steel plate faults classifications, where class imbalance typically occurs due to non-common defects which, though rare, may seriously impact steel quality.