Hybrid data mining model for managerial classification : feature weighting and classifier combination

Abstract

The task of classification permeates all walks of life, from business and economics to science and engineering. In this context, nonlinear techniques from artificial intelligence have often proven to be more effective than the methods of classical statistics. In many managerial classification problems, the volume of accumulated data is relatively small compared to applications in science and engineering. Furthermore, the classifier’s capability to handle small datasets and to explain classification result is an important advantage. In this context, lazy learning algorithms (LLAs) provide an effective approach. Furthermore, an LLA is often effective even where human expertise is unavailable or even nonexistent. This thesis proposes a series of effective feature weighting algorithms for LLAs based on elementary data mining methodologies. One approach involves the use of mutual information arising from an inductive decision tree as the weight for a feature. A second method relies on the judicious interpretation of weight connections from a trained neural network as a measure of the importance of a feature. The two basic approaches to feature weighting are compared to the selection of features by a genetic algorithm (GA). Subsequently, a combined approach is suggested as a cost-effective solution. In relation to the LLA-based combination of multiple classifiers (CMC), we propose an efficient meta-classifier architecture based on the assumption that uncorrelated subsets of features will generate independent classifiers toward an oracle. To overcome the limitations of kNN-based boosting using a straightforward approach, we propose an improved feature-weighted boosting algorithm for the kNN classifier. Moreover, we propose a kNN-directed noise injection scheme to enhance the effect of the boosting procedure by expanding the original dataset in a fortuitous fashion. Finally, we discuss a decision flipping approach. The methodology takes an advantage of the correlated e...