(A) complexity-reduced and performance-improved technique for feature extraction with kernel discriminant analysis

Abstract

The linear discriminant analysis (LDA), as one of the most fundamental and powerful feature extraction methods, has been successfully applied in many pattern recognition (PR) problems. Although the LDA and LDA-related methods can cope well with linearly separable PR problems, they generally result in degraded performance for linearly unseparable problems. In order to deal with linearly unseparable problems more effectively, the kernel discriminant analysis (KDA) has been proposed by extending the LDA into the kernel space. The KDA, aiming at maximizing the ratio of the kernel betweenclass distance to the kernel within-class distance of data, generally provides good PR performance for most of PR problems including facial recognition, text recognition, and image retrieval. Due to the eigen-decomposition technique adopted, however, the original scheme for the feature extraction with the KDA suffers from a high complexity burden. In this dissertation, we derive a transformation of the KDA into a linear equation problem, and propose a novel scheme for the feature extraction with the KDA. The proposed scheme is shown to provide us with a reduction of complexity without degradation of PR performance. In addition, to enhance the PR performance further, we address the incorporation of regularization into the proposed scheme.