Classification and variable selection algorithms using signomial function

Abstract

Data mining techniques extract useful information from large databases. The techniques can be categorized as being either descriptive or predictive. In this thesis, we focus on classification, the predictive data mining used for discrete target variables, and variable selection for classification. We propose classification algorithms for multi-class classification problems, and variable selection algorithms for binary classification and multi-class classification using signomial function. Specifically, this research contributes to the field of classification and variable selection by:

1. Constructing a multi-class classifier directly by solving a single optimization problem to be capable of capturing the correlations among classes;

2. Obtaining classifiers which are sparse and can be explicitly described in original space, which facilitates interpretation;

3. Determining a subset of variables that is desirable for predicting the output, considering nonlinear interactions of variables;

4. Performing variable selection for multi-class classification by treating multiple classes jointly to select a small common subset of variables.

First, we propose two multi-class classification methods using signomial function. Each of them directly constructs a multi-class classifier by solving a single optimization problem. Since the number of possible signomial terms is huge, we propose a column generation method that iteratively generates good signomial terms. The both methods obtain better or comparable classification accuracies and give more sparse classifiers than the existing methods.

Next, we propose two embedded variable selection methods using signomial function. We attempt to select, among a set of the input variables, those that lead to the best performance of the classifier. One method repeatedly removes variables based on backward selection, and the other method directly select a set of the variables by so...