Parameter optimization for kernel-based pattern classification and agglomerative clustering

Abstract

In order to facilitate parameter optimization process in designing assistive robotic systems, such as EMG signal-based walking phase recognition system and human behavior patterns prediction system, in this paper is suggested a generalized parameter optimization methods for kernel-based pattern classification and agglomerative clustering. First, a criterion for kernel parameter optimization is derived, and several theoretical results are reported. Specifically, a Fisher criterion function in a kernel feature space is derived, and then its relation to the ideal kernel and the empirical kernel target alignment is reported. For more profound perspective on kernel parameter optimization, a notion of kernel is expanded; a generalized kernel function and some theoretical researches in the framework of a generalized kernel function are conducted. Specifically, it is shown that the generalized kernel function involves another nonlinear mapping that can be controlled by a parameter set of the corresponding kernel function. In conjunction with the Fisher criterion function, an application to kernel parameter optimization is presented. Experimental results on artificial datasets, benchmark datasets, and EMG datasets show that the method is promising. Second, a study on agglomerative clustering is conducted. It is remarked that, for outlier detection, there was an attempt of Bayesian interpretation for IAFC, the clustering techniques in which a fuzzy concept is incorporated into competitive learning scheme. The partial interpretation on the decision process, however, was not based on a concrete theoretical basis. And, it was found that the assumption of conditional pdf in a complicated form makes the interpretation rather trivial. To fix this problem, Iterative Bayesian Fuzzy Clustering (IBFC) is proposed, and its Bayesian interpretation is conducted. It is noted that the decision and learning process of IBFC follows Bayesian minimum classification rule. Furthermore,...