Support vector-based feature selection using Fisher's linear discriminant and Support Vector Machine

Abstract

The problem of feature selection is to find a subset of features for optimal classification. A critical part of feature selection is to rank features according to their importance for classification. The Support Vector Machine (SVM) has been applied to a number of applications, such as bioinformatics, face recognition, text categorization, handwritten digit recognition, and so forth. Based on the success of the SVM, several feature selection algorithms that use it have recently been proposed. This paper proposes a new feature-ranking algorithm based on support vectors (SVs). Support vectors refer to those sample vectors that lie around the decision boundary between two different classes. Although SV-based feature ranking can be applied to any discriminant analysis, two linear discriminants are considered here: Fisher's linear discriminant and the Support Vector Machine. Features are ranked based on the weight associated with each feature or as determined by recursive feature elimination. The experiments show that our feature-ranking algorithms are competitive in accuracy with the existing methods and much faster.