Advances in error estimation and multi-dimensional supervised classification

Abstract

The first contribution is a novel decomposition of the variance of classification error estimators taking into account its different variance sources. We analyze the statistical properties (bias and variance) of the most popular error estimators. A general framework to analyze the decomposition of the variance considering the nature of the variance (reducible/irreducible) and the different sources of sensitivity (internal/external sensitivity) is presented. An extensive empirical study has been performed and, based on the obtained results, we propose the most appropriate error estimators for model selection under different experimental conditions.The second contribution is a novel method for learning multi-dimensional Bayesian network classifiers via a multi-objective evolutionary algorithm. The multi-objective strategy considers the accuracy of each class variable separately as the functions to optimize. In order to evaluate the proposed learning approach , this dissertation includes a study that compares it with the main alternatives to deal with multi-dimensional classification.Finally, a medical application of multi-dimensional Bayesian network classifiers is presented for Multiple Sclerosis. The application tries to help a physician to predict the expected progression of the disease and to plan the most suitable treatment.