Feature selection and pattern classification based on piecewise linear decision boundary

Abstract

Nowadays, many pattern recognition methods are practically utilized in various areas. In this dissertation, two novel pattern recognition methodologies based on piecewise linear decision boundary (PLDB) are proposed. First, a novel feature selection scheme for a given decision boundary is proposed. We construct a PLDB on the original feature space and find a feature sub-space which produces similar decision results from the constructed PLDB. By this way, we obtain a feature sub-space minimizing the degradation of recognition accuracy with reduced computational cost. To show the efficiency of the proposed scheme, we present an analytical expression of a PDLB and derive a process of optimal feature selection for a given PLDB. We also propose elemental direction preserving discriminant analysis (EDPDA) which produces a near-optimal result for the nearest neighbor classifier. Unlike other feature selection methods based on sample distances, EDPDA produces better recognition performance owing to its decision boundary oriented nature. In our experiments managed with popular face image databases, the proposed scheme shows better recognition accuracy than other methods. Second, a novel classifier named geometrically constructed piecewise linear machine (GCPLM) is introduced. The proposed GCPLM generates geometrically a PLDB which minimizes the number of misclassified training samples. In the proposed method, a PLDB is obtained by modifying sequentially and iteratively each linear function constituting the PLDB. Since GCPLM constructs a PLDB directly rather than generating discriminant functions, it is more advantageous when the class-conditional probability distributions of patterns are harder to estimate. GCPLM is appropriate when the number of training samples is large enough to achieve high recognition accuracy by minimizing the number of misclassified training samples. Especially, GCPLM is valuable when rapid classification is needed since it classifies a sampl...